Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-103337

Fix memory tracker propagation on killCursor and $exchange

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Integration
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      This is a fix for two bugs that showed up in testing when hooking up the memory tracker that reports statistics upstream to an actual DocumentSource. 

      1. Move memory tracker back to OpCtx when killCursor is issued.

      When we have getMore that must return early (e.g. assertion failure) and the cursor indicates that it needs to be deleted, we need to move the memory tracker from the cursor back onto the opCtx. This ensures that we are deleting the cursor's resources are deleted in the correct order and lifetimes are managed correctly.

      This showed up in testing via bugs where a sequence like this happens: we want the cursor to be deleted, we indicate that the pipeline should be deleted, and then we try to reset the memory tracker to zero. The memory tracker had already been deleted, so this was causing segfaults.

      2. Do not report memory metrics if the pipeline has an $exchange stage

      Essentially $exchange can produce more than one cursor and this caused a use-after-free bug with the new memory tracking mechanism. The memory tracker is only compatible with stages with one cursor – because it is a unique_ptr by design, the memory tracker is freed when the first cursor is exhausted, and when the next cursor(s) try to reference the memory tracker, we end up with a use-after-free.

      To do this, we should not create the OperationMemoryUsageTracker when we have an exchange stage. We can set a flag on the expression context to indicate that this pipeline is incompatible for memory tracking. To set the value of this flag, we should use the method on AggregateCommandRequest to tell if we have an exchange stage in the command.

      Something like this: 

       

      if (!feature_flags::gFeatureFlagQueryMemoryTracking.isEnabled() || expCtx->getInRouter() || !expCtx->isIncompatibleForMemoryTracking()) { // create memory tracker that does not report upstream } 

            Assignee:
            erin.liang@mongodb.com Erin Liang
            Reporter:
            erin.liang@mongodb.com Erin Liang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              None
              None
              None
              None