Please refer to the discussion here: https://www.mongodb.com/community/forums/t/addtoset-slower-hanging-on-large-collections-since-mongodb-6/192395/3
In short, there appear to be two issues with the SBE:
- It too aggressively resorts to disk for $group processing. In my example above, disk isn't used with the classic engine on 6.0, or on 4.0.
- When resorting to disk, it is incredibly slow.
The only solution was to force the use of the classic query engine, which doesn't seem like a desirable approach.
- is depended on by
-
SERVER-71680 $operationMetrics fails to account for "keysSorted" or "sorterSpills" when $group is pushed down to SBE
- Closed
-
SERVER-73311 Report group spill stats in serverStatus for SBE
- Closed
- related to
-
SERVER-76321 Buffer overrun while deserializing compound group key spilled to disk in SBE hash agg implementation
- Closed
-
SERVER-71684 $operationMetrics accounts for reading/writing to TemporaryRecordStore for spilling in query execution
- Closed
-
SERVER-74291 Investigate whether new SBE HashAggStage spilling algorithm should be improved to avoid random access into spill table
- Open
-
SERVER-74293 Improve spilling algorithm for HashLookupStage
- Open