Long-running queries should not cause a build-up of unused ChunkManager objects

XMLWordPrintableJSON

    • Fully Compatible
    • ALL
    • v4.0, v3.6
    • Sharding 2019-05-06, Sharding 2019-05-20, Sharding 2019-06-03
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      In SERVER-35666 we saw that keeping a query open for a long time can lead to memory buildup due to ChunkManager objects not being cleaned up. This is because here, when we go to expire old ChunkManager objects, we iterate through the queue of metadata objects from oldest to newest, and if we find an object with a usageCounter > 0, we stop, so that we don't delete an object used by an ongoing query.

      However, this is overly conservative. It's possible that the oldest object in the queue could have one query attached to it (usageCounter = 1), while many newer metadata objects have usageCounter = 0 and thus could be deleted. I believe we could also delete even metadata objects that have orphans on them, if the metadata object is never used by the range deleter.

            Assignee:
            Janna Golden
            Reporter:
            Matthew Saltz (Inactive)
            Votes:
            3 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated:
              Resolved: