-
Type: Bug
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Execution
-
ALL
tl;dr: After a long investigation into DocumentSourceLookup::_fromExpCtx, I found some things that I think will guard against future bugs. My recommendations are in a comment, which unfortunately, does not have a tl;dr.
Although _fromExpCtx's _memory is reference counted via intrusive_ptr, all ExpressionContext objects also need an owner that is responsible for detaching and reattaching the ExpressionContext::opCtx pointer when pipeline execution spans multiple batches (each one necessitating a new opCtx).
Normally, this responsibility belongs to a Pipeline object, which has detachFromOperationContext and reattachToOperationContext member functions for this purpose. One-to-one PipelinetoExpressionContext ownership does not exactly work for the DocumentSourceLookUp's sub-pipeline, however, because depending on execution state, the ExpressionContexts used by the sub-pipelines can be referenced by between 0--2 Pipeline objects.
- In the common case, there is a DocumentSourceLookUp::resolvedIntrospectionPipeline that holds a reference to DocumentSourceLookUp::_fromExpCtx, but this pipeline's detachFromOperationContext and reattachToOperationContext methods do not get called, because the introspection pipeline is a reference-only pipeline that does not execute and _usually does not need an up-to-date opCtx. (See
SERVER-91362for exceptions.) Instead, the DocumentSourceLookUp directly performs the attach/ detach operations itself. - However, while the DocumentSourceLookUp is serving the results of a subsumed $unwind operation, it maintains an active Pipeline (stored in DocumentSourceLookUp::_pipeline) that also references _fromExpCtx. During this time, _pipeline becomes the effective owner of the detach/ reattach operations.
- When the $lookup foreign collection is a view on a sharded collection, it may be possible for DocumentSourceLookUp::buildPipelineFromViewDefinition() to replace DocumentSourceLookUp::_fromExpCtx with a new ExpressionContext that is different from the one that _resolvedIntrospectionPipeline and _pipeline reference. This new ExpressionContext has no detach/reattach owner and may maintain a dangling reference to an old opCtx. That operation happens here.
Edit: I tried super hard to format all of this nicely, but Jira keeps randomly changing the formatting around .
- related to
-
SERVER-91362 Performance: Do not copy JS "scope" object if a cached JsExecution exists
- Closed