Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-62094

Incorrect memory usage calculation in $top/$bottom/$topN/$bottomN window functions due to Document field caching

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 5.3.0, 5.2.0-rc3
    • Affects Version/s: 5.2.0-rc1
    • Component/s: None
    • None
    • Fully Compatible
    • ALL
    • v5.2
    • Hide

      use one of these accumulators in $setWindowFields over a collection with documents containing large strings. The accumulator spec should use the large string field as part of the sortBy expression. This will cause the memory usage to eventually be negative

      Show
      use one of these accumulators in $setWindowFields over a collection with documents containing large strings. The accumulator spec should use the large string field as part of the sortBy expression. This will cause the memory usage to eventually be negative
    • QE 2021-12-27
    • 150

      The Document class has a cache to speed up accessing fields on large documents that can increase the value of getApproximateSize as fields are accessed. The outer window function code greedily calls fillCache() on input documents so that their field caches are stable and remove() calls subtracts the same amount of memory as the corresponding add() call removed. However these accumulators still run into the issue since they generate a new Document Value when they calculate sortKeys and it starts out with an empty cache. remove() searches through the internal map structure to remove the value provided, so by the time we do the memory calculation the sortKey's field cache is already fully populated, thus causing remove() calls to subtract more memory than add() added.

            Assignee:
            mickey.winters@mongodb.com Mickey Winters
            Reporter:
            mickey.winters@mongodb.com Mickey Winters
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: