Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-53764

Support spilling to disk in sbe::HashAggStage

    • Type: Icon: Task Task
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Querying
    • None
    • Query Execution

      The slot-based execution (SBE) engine currently supports spilling to disk for large blocking sort operations, but we still need to extend the hash-based aggregation operation to be able to spill pieces of its hash table. The HashAggStage has limited uses in plans constructed for find commands, but when we start using SBE to execute entire aggregation pipelines, we may choose a plan that implements MQL $group using sbe::HashAggStage. When the application sets allowDiskUse to true, the system should spill to disk when its reaches its allotted memory for the hash-aggregation rather than failing the query.

      Note that as part of this work, we need to extend SERVER-52793. Namely, the HashAggStage must report execution stats about how many keys it sorted (if we spill the hash table sorted by key) and about how many times it spilled data to disk.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            david.storch@mongodb.com David Storch
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: