Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-65884

$lookup from time-series can place $sequentialCache after correlated $match

    • Fully Compatible
    • ALL
    • v6.0, v5.3, v5.0
    • QO 2022-05-16, QO 2022-05-30, QO 2022-06-13, QO 2022-06-27, QO 2022-07-11, QO 2022-07-25
    • 205

      When $lookup's foreign collection is time-series, and the inner pipeline has a correlated $match stage, it can incorrectly place a $sequentialCache after the $match.

      For example, suppose we start with:

      db.someCollection.aggregate([
         {$lookup: {
          from: 'timeseriesColl',
          let: {myJoinVar: ...},
          pipeline: [
            {$match: ... myJoinVar ...}
          ]
        }}
      ])
      

      When $lookup optimizes a pipeline, it first resolves the views and tentatively adds a $sequentialCache stage.

      {$_internalUnpackBucket: ...}
      {$match ... myJoinVar ...}
      {$sequentialCache ...}
      

      Then it calls optimizeAt left-to-right. Since the $match does not depend on any measurement fields in the inner pipeline, it correctly swaps with $_internalUnpackBucket:

      {$match ... myJoinVar ...}
      {$_internalUnpackBucket: ...}
      {$sequentialCache ...}
      

      Next we optimize the $_internalUnpackBucket stage. Like $lookup, this stage does something special by calling optimize recursively in its own optimize method. It zooms in on the pipeline suffix:

      {$sequentialCache ...}
      

      Now we optimize $sequentialCache, and it doesn't see the other stages, so it thinks this is an uncorrelated pipeline. It does not remove itself.

      Once all the calls to optimize return we're left with this incorrect pipeline:

      {$match ... myJoinVar ...}
      {$_internalUnpackBucket: ...}
      {$sequentialCache ...}
      

      We should have either removed the $sequentialCache, or positioned it before the correlated $match stage.

            Assignee:
            ruslan.abdulkhalikov@mongodb.com Ruslan Abdulkhalikov (Inactive)
            Reporter:
            david.percy@mongodb.com David Percy
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: