Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-79862

Exponential batch size growth in DocumentSourceCursor

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 7.3.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • Fully Compatible
    • QE 2023-11-13
    • 135

      as part of the migration of $search to SBE, we will be using DocumentSourceCursor together with PlanExecutorSBE. Due to how DocumentSourceCursor batching works, there is a potential risk that the performance of some queries will significantly degrade. Specifically queries like:
      {{[$search, $stageNotSupportedBySBE, $unwind, $limit] }}  will result in DocumentSourceCursor attempting to unnecessarily populate potentially a large batch of documents (4MB), which could potentially take multiple roundtrips to mongot, and then $limit may discard majority of that.This issue will not appear for "nicer" queries like:
      {{[$search, $stagesSupportedBySBE..., $limit] }}  -> here $limit will be pushed down below DocumentSourceCursor and the application of $limit will be done eficiently.
      [$search, $stageNotSupportedBySBEButReorderableWithSkipLimit..., $limit]  -> here despite $limit being outside of supported prefix, the $search stages will infer the limit and apply it properly (at least after https://jira.mongodb.org/browse/SERVER-78351 is fixed).This problem should go away once SBE supports all stages, however until then there is a risk that some queries may become really really slow. Is this a blocker for shipping the search on SBE?

      Use exponential batch size growth in DocumentSourceCursor to avoid performance issue for particular queries in SBE.

            Assignee:
            zixuan.zhuang@mongodb.com Zixuan Zhuang
            Reporter:
            zixuan.zhuang@mongodb.com Zixuan Zhuang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: