Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-58954

Refactor and unify the code to optimize expressions in projections

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution

      As part of the work in SERVER-57749 we looked into cleaning up / standardizing code to optimize projection expressions, but found that doing so would involve a large overhaul of the existing code and that it is beyond the scope of the original ticket (which was about a late call to optimize() on a projection in SBE). 

      To fix the issue of a late optimization as well as avoid the issue with 'find' in current state producing sub-optimal plans for some queries, we instead suggest to do it earlier when building the projection AST. However, because we currently have an optimize() method on a ProjectionNode, this causes there to be two ways in which we can potentially optimize a projection:

      1. Parse the projection
        Build ProjectionExecutor
        Optimize ProjectionExecutor

       
      (new method)

      1. Parse projection
        optimize it
        Build ProjectionExecutor

      The goal is to clean this up by ideally removing the optimize() method on the ProjectionNode, however, this is difficult since we rely on optimizations in certain cases with $lookup. In this situation we need to make sure that $lookup inner pipelines are not being optimized too early, as that might lead to incorrect positioning of the caching stage if certain variables have been optimized away. See this comment for this description:

       

      One proposed approach that could be implemented as suggested by Ian is to include a new "prepare" method to DocumentSource that would mostly do nothing but in the case of DocumentSourceSingleDocumentTransformation can be used for the optimization purposes. This then would also require us to plumb through a call to optimize to the 'TransformerInterface' of this DocumentSource for projections.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            andrii.dobroshynski@mongodb.com Andrii Dobroshynski (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: