Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-63328

Relax checks for write lock to support spilling to temp WT record store on secondaries

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.0.0-rc0, 5.2.2
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • ALL
    • v5.2
    • QE 2022-02-21

      Repro:

      (function() {
          "use strict";
      
          const replTest = new ReplSetTest({
              nodes: 3,
          });
      
          replTest.startSet();
          replTest.initiate();
      
          let primary = replTest.getPrimary();
          const insertColl = primary.getDB("test").foo;
          for (let i = 0; i < 500; ++i) {
              insertColl.insert({a: i, string: "test test test"});
          }
      
          let secondary = replTest.getSecondary();
          secondary.adminCommand({setParameter: 1, internalQuerySlotBasedExecutionHashAggApproxMemoryUseInBytesBeforeSpill: 1});
      
          let readColl = secondary.getDB("test").foo;
      
          const pipeline = [{$group: {_id: '$a', s: {$addToSet: '$string'}, p: {$push: '$a'}}}];
          
          let res = readColl.aggregate(pipeline, {allowDiskUse: true,
                                                  readConcern: {level: "majority"},
                                                  writeConcern:{"w": "majority"}
                                                 }).toArray();
      
          // Never reached.
          replTest.stopSet();
      })();
      

      In the SBE HashAgg stage, we take a global IX lock before spilling to disk.

      This will cause the wasGlobalLockTakenForWrite() function to return true at the end of command processing, and we will end up calling waitForWriteConcern here, even if the operation only wrote to a local temporary RecordStore for spilling.

      As Max H as pointed out, the SpillableCache used in window functions likely suffers the same issue, since it also takes an IX lock here.

      The user facing impact of this is that $group queries which spill, when run on secondaries with a writeConcern stronger than w:1 are broken.

            Assignee:
            eric.cox@mongodb.com Eric Cox (Inactive)
            Reporter:
            ian.boros@mongodb.com Ian Boros
            Votes:
            1 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: