Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-43096

Pre-allocate smaller buffer for "distinct" commands

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.2.1, 4.3.1
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • v4.2, v4.0
    • Query 2019-10-07, Query 2019-10-21

      We allocate a 16 MB buffer to hold the results of every "distinct" command, no matter how big or small the result is. This can create a performance problem: every buffer is allocated directly from and returned directly to the tcmalloc pageheap, which is moderately expensive, and is a serial bottleneck.

      Also, tcmalloc decommits memory (returns it to the o/s) at a rate more-or-less proportional to the rate at which memory is returned to the pageheap, so this high rate of memory returned to the pageheap can create a high rate of decommitting and the re-commiting memory, which is a very expensive operation.

      We could consider pre-allocating a much smaller buffer here and allowing it to grow as needed.

            Assignee:
            james.wahlin@mongodb.com James Wahlin
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            16 Start watching this issue

              Created:
              Updated:
              Resolved: