-
Type: Task
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Querying
When I tried to run the "word_count.js" benchmark (from https://github.com/10gen/workloads) and with SBE mode enabled, mongod crashed due to an invariant failure at line 1539 at "src/mongo/db/query/sbe_stage_builder_expression.cpp":
invariant(it != _context->environment.end());
I attached gdb to mongod (debug build) and then I ran the "work_count.js" benchmark with SBE mode enabled to reproduce the crash, and here was the stack trace I got:
#0 raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:51 .. #3 0x0000564379faf11d in mongo::invariantWithLocation<bool> (testOK=@0x7fca449670e8: false, expr=0x56437e6eedc8 "it != _context->environment.end()", file=0x56437e6ee668 "src/mongo/db/query/sbe_stage_builder_expression.cpp", line=1539) at src/mongo/util/invariant.h:69 #4 0x000056437b83ecea in mongo::stage_builder::(anonymous namespace)::ExpressionPostVisitor::visit ( this=0x7fca44967370, expr=0x7fca15da4f20) at src/mongo/db/query/sbe_stage_builder_expression.cpp:1539 #5 0x000056437c328da0 in mongo::ExpressionFieldPath::acceptVisitor (this=0x7fca15da4f20, visitor=0x7fca44967370) at src/mongo/db/pipeline/expression.h:1441 #6 0x000056437b84a45e in mongo::stage_builder::(anonymous namespace)::ExpressionWalker::postVisit ( this=0x7fca449673a0, expr=0x7fca15da4f20) at src/mongo/db/query/sbe_stage_builder_expression.cpp:2728 #7 0x000056437b84bd84 in mongo::expression_walker::walk<mongo::stage_builder::(anonymous namespace)::ExpressionWalker> (walker=0x7fca449673a0, expression=0x7fca15da4f20) at src/mongo/db/pipeline/expression_walker.h:62 #8 0x000056437b84bd63 in mongo::expression_walker::walk<mongo::stage_builder::(anonymous namespace)::ExpressionWalker> (walker=0x7fca449673a0, expression=0x7fca3d11dd80) at src/mongo/db/pipeline/expression_walker.h:59 #9 0x000056437b84bd63 in mongo::expression_walker::walk<mongo::stage_builder::(anonymous namespace)::ExpressionWalker> (walker=0x7fca449673a0, expression=0x7fca15da5520) at src/mongo/db/pipeline/expression_walker.h:59 #10 0x000056437b84aaf6 in mongo::stage_builder::generateExpression (opCtx=0x7fca2586f720, expr=0x7fca15da5520, stage=..., slotIdGenerator=0x7fc9d50b6f48, frameIdGenerator=0x7fc9d50b6f58, rootSlot=2, env=0x7fc9fe7a9120, planNodeId=2, relevantSlots=0x7fca44967848) at src/mongo/db/query/sbe_stage_builder_expression.cpp:2797 #11 0x000056437b87d8f3 in mongo::stage_builder::(anonymous namespace)::ProjectionTraversalPostVisitor::visit (this=0x7fca44967750, node=0x7fc9cc5010e0) at src/mongo/db/query/sbe_stage_builder_projection.cpp:243 #12 0x000056437c0feb18 in mongo::projection_ast::ExpressionASTNode::acceptVisitor (this=0x7fc9cc5010e0, visitor=0x7fca44967750) at src/mongo/db/query/projection_ast.h:293 #13 0x000056437b87f298 in mongo::stage_builder::(anonymous namespace)::ProjectionTraversalWalker::postVisit (this=0x7fca44967760, node=0x7fc9cc5010e0) at src/mongo/db/query/sbe_stage_builder_projection.cpp:382 #14 0x000056437b87fb20 in mongo::tree_walker::walk<true, mongo::projection_ast::ASTNode, mongo::stage_builder::(anonymous namespace)::ProjectionTraversalWalker> (node=0x7fc9cc5010e0, walker=0x7fca44967760) at src/mongo/db/query/tree_walker.h:70 #15 0x000056437b87faff in mongo::tree_walker::walk<true, mongo::projection_ast::ASTNode, mongo::stage_builder::(anonymous namespace)::ProjectionTraversalWalker> (node=0x7fc9eb38d358, walker=0x7fca44967760) at src/mongo/db/query/tree_walker.h:67 #16 0x000056437b87f4c7 in mongo::stage_builder::generateProjection (opCtx=0x7fca2586f720, projection=0x7fc9eb38d358, stage=..., slotIdGenerator=0x7fc9d50b6f48, frameIdGenerator=0x7fc9d50b6f58, inputVar=2, env=0x7fc9fe7a9120, planNodeId=2) at src/mongo/db/query/sbe_stage_builder_projection.cpp:413 #17 0x000056437b8138ef in mongo::stage_builder::SlotBasedStageBuilder::buildProjectionDefault ( this=0x7fc9d50b6f20, root=0x7fc9eb38d2e0, reqs=...) at src/mongo/db/query/sbe_stage_builder.cpp:708 .. #25 0x000056437b816af6 in mongo::stage_builder::SlotBasedStageBuilder::build (this=0x7fc9d50b6f20, root=0x7fc9eb38d2e0, reqs=...) at src/mongo/db/query/sbe_stage_builder.cpp:1052 #26 0x000056437b80ea57 in mongo::stage_builder::SlotBasedStageBuilder::build (this=0x7fc9d50b6f20, root=0x7fc9eb38d2e0) at src/mongo/db/query/sbe_stage_builder.cpp:159 #27 0x000056437b88a4c3 in mongo::stage_builder::buildSlotBasedExecutableTree (opCtx=0x7fca2586f720, collection=..., cq=..., solution=..., yieldPolicy=0x7fca2445f160, needsTrialRunProgressTracker=false) at src/mongo/db/query/stage_builder_util.cpp:74 #28 0x000056437b7b8421 in mongo::(anonymous namespace)::SlotBasedPrepareExecutionHelper::buildExecutableTree (this=0x7fca449688f0, solution=..., needsTrialRunProgressTracker=false) at src/mongo/db/query/get_executor.cpp:956 #29 0x000056437b7b7d86 in mongo::(anonymous namespace)::SlotBasedPrepareExecutionHelper::buildExecutableTree (this=0x7fca449688f0, solution=...) at src/mongo/db/query/get_executor.cpp:904 #30 0x000056437b7c4e87 in mongo::(anonymous namespace)::PrepareExecutionHelper<std::pair<std::unique_ptr<mongo::sbe::PlanStage, std::default_delete<mongo::sbe::PlanStage> >, mongo::stage_builder::PlanStageData>, mongo::(anonymous namespace)::SlotBasedPrepareExecutionResult>::prepare (this=0x7fca449688f0) at src/mongo/db/query/get_executor.cpp:672 #31 0x000056437b7b8ebb in mongo::(anonymous namespace)::getSlotBasedExecutor (opCtx=0x7fca2586f720, collection=0x7fca44969938, cq=..., requestedYieldPolicy=mongo::PlanYieldPolicy::YieldPolicy::YIELD_AUTO, plannerOptions=0) at src/mongo/db/query/get_executor.cpp:1068 #32 0x000056437b7b9660 in mongo::getExecutor (opCtx=0x7fca2586f720, collection=0x7fca44969938, canonicalQuery=..., yieldPolicy=mongo::PlanYieldPolicy::YieldPolicy::YIELD_AUTO, plannerOptions=0) at src/mongo/db/query/get_executor.cpp:1116 .. #36 0x000056437b791def in mongo::PipelineD::prepareExecutor (expCtx=..., collection=..., nss=..., pipeline=0x7fc9f55427c0, sortStage=..., rewrittenGroupStage=..., unavailableMetadata=..., queryObj=..., skipThenLimit=..., aggRequest=0x7fc96b543058, matcherFeatures=@0x56437e6a8198: 57, hasNoRequirements=0x7fca44969042) at src/mongo/db/pipeline/pipeline_d.cpp:788 ..
I came up with a small repro for this crash - see the "Steps To Reproduce" section.
I also tried out running a modified version of the small repro where I replaced "$$word" with 1, like so:
db.adminCommand({setParameter:1, internalQueryEnableSlotBasedExecutionEngine:true}); db.foo.insert({ "t" : "blah" }); db.foo.aggregate([{$project: {emits: {$map: {input: {$split: ["$t", " "]}, as: "word", in: {k: 1, v: 1}}}}}]);
When I ran the commands above, mongod did not crash, but I got the following error message:
"Expression is not supported in SBE: $map"
Thus it looks like even if the crash were fixed, the "word_count.js" benchmark would still fail because SBE doesn't support the $map aggregation pipeline operator yet.
The goal of this task is: (1) to implement SBE support for the $map aggregation pipeline operator; and (2) to fix the "invariant(it != _context->environment.end())" crash.
- duplicates
-
SERVER-51823 Use classic engine to evaluate queries containing expressions not supported in SBE
- Closed
-
SERVER-53036 [SBE] Investigate why several sys-perf benchmarks fail
- Closed
- is related to
-
SERVER-51655 Investigate sys-perf benchmark performance in SBE
- Closed