-
Type: Bug
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Aggregation Framework
-
None
-
Query Execution
-
ALL
-
(copied to CRM)
Generally every stage which builds some large data structure (think $sort or $group) will use estimated sizes to cap that memory usage to 100MB. Nothing like this exists for expressions or accumulators, but something like a $push can end up allocating lots of memory.
For example, the pipeline:
[{$group: {_id: null, all: {$push: "$$ROOT"}}}]
will push everything to one giant array. This actually will error if spilling to disk is not enabled, since $group will approximate the size of all accumulators. But once we spill to disk and then begin merging back together there is no memory usage tracking. Because $push tracks just a vector<Value>, that array itself isn't going to grow very large (each Value is just 16 bytes). However each of those Values indicates a sub-tree of memory that can add up to lots and lots of memory in the worst case.
SERVER-44174 tracks the work to fix $push and $addToSet, while this ticket tracks the work to create a more general solution for all expressions and accumulators.
- is related to
-
SERVER-44174 $push and $addToSet should restrict memory usage
- Closed
- related to
-
SERVER-96350 $push, $addToSet, $setUnion, $concatArrays window functions should respect memory limits
- Needs Scheduling