Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-68687

Update Global Index fetcher pipeline

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Cluster Scalability

      Pipeline should look like:

      [
      {
        // only if resuming
        $match: {
          $expr: {$gte: ["$_id", {$literal: <last seen _id>}]}} 
        }
      },
      {
        $sort: {_id: 1}
      },
      {
        // Use replaceRoot to guarantee the order of output fields.
        $repacleRoot: {
          newRoot: {
            _id: { 
              $indexKey: {doc: "$$ROOT", spec: <object containing global index spec>}
            },
            documentKey: {
      
              // Use arrayToObject as a work around to output valid shard key patterns with
              // dotted field names.
              $arrayToObject: [[
                {k: "_id", v: "$_id"},
      
                // Explicitly output null value for missing fields, otherwise $arrayToObject complains:
                // "$arrayToObject requires an object keys of 'k' and 'v'. Found incorrect number of keys:1"
                {k: <sourceShardKey1>, v: {$ifNull: ["$sourceShardKey1", null]}},
                …
              ]]
            }
        }
      },
      {
        // unwind is needed because $indexKey outputs an array to handle multikey 
        $unwind: "$_id"
      },
      {
        $project: {
          _id: 0,
          shard: {
             $owningShard: {shardKeyVal: {_id: "$_id"}, ns: <globalIndexNs>, uuid: <globalIndexUUID>},
          },
          indexData: "$$ROOT"
        }
      },
      {
        $match: {shard: <recipientShardId>}
      },
      {
        $replaceRoot: {
          newRoot: "$indexData"
        }
      }
      ]
      

            Assignee:
            backlog-server-cluster-scalability [DO NOT USE] Backlog - Cluster Scalability
            Reporter:
            randolph@mongodb.com Randolph Tan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: