Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-4510

PowerOfTwoBufferPool leaks permits when compression is enabled

    • Fully Compatible
    • Not Needed

      Summary

      InternalStreamConnection doesn't release buffers returned by PowerOfTwoBufferPool when compression is enabled. It leads to permits leakage. After Integer.MAX_VALUE buffer retrievals buffer pool is fully exhausted and all following requests would be blocked indefinitely unless thread get interrupted. 

      How to Reproduce

      Code below simulate the problem but require significant amount of time to exhaust the pool. I would recommend to run it for some time and take a heap dump. Search for ConcurrentPool instances in heap dump and check for permits number. It will be equal to Integer.MAX_VALUE minus number of requests made. See attachments for an example.

              MongoClientSettings settings = MongoClientSettings.builder()
                      .compressorList(singletonList(MongoCompressor.createZlibCompressor()))
                      .build();
      
              MongoClient mongo = MongoClients.create(settings);
              MongoDatabase db = mongo.getDatabase("test");
              MongoCollection<Document> collection = db.getCollection("JAVA4510");
              
              ExecutorService executorService = Executors.newFixedThreadPool(100);
              List<Future<?>> tasks = new ArrayList<>();
              BasicDBObject searchQuery = new BasicDBObject();
              searchQuery.put("_id", 1);
              for (long i = 0; i <= Integer.MAX_VALUE; i += 1_000) {
                  for (long j = 0; j < 1_000; j++) {
                      tasks.add(executorService.submit(() -> {
                          collection.find(searchQuery).first();
                      }));
                      for (Future<?> task : tasks) {
                          task.get();
                      }
                      tasks.clear();
                  }
              }
      
              executorService.shutdownNow();
              collection.find(searchQuery);
      

            Assignee:
            valentin.kovalenko@mongodb.com Valentin Kavalenka
            Reporter:
            klabun@indeed.com Konstantin Labun
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: