Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16750

Document never matching query predicate incorrectly returned by query if concurrently updated (WiredTiger)

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • 3.0.0-rc6
    • Affects Version/s: 2.8.0-rc4
    • Component/s: Querying, Storage
    • None
    • Fully Compatible
    • ALL

      When a query run against a WiredTiger-enabled mongod yields (or is saved for a getMore), the query's storage-layer snapshot is dropped and later replaced by a new snapshot. For queries that perform multiple index lookups for a single document, this creates an opportunity for index key data saved by the query stage tree to be stale by the time the new snapshot is created. As a result, queries can return documents that never match the given predicate, if they partially match the predicate in an old snapshot and partially match the predicate in a new shapshot. For example, the below script shows a query with predicate {a: 1, b: 1} returning a document that matches predicates {a: 1} and {b: 1} at separate times, but never at the same time.

      This issue does not affect mongod configured with the MMAPv1 storage engine, as the write invalidation framework forces queries to perform a full predicate re-evaluation on documents that are mutated over the course of the query's lifetime.

      To reproduce, run the following shell script against a mongod configured with the WiredTiger storage engine:

      db.foo.drop();
      for (var i=0; i<1000; i++) {
          assert.writeOK(db.foo.insert({_id: i, a: 2, b: 1}));
      }
      assert.writeOK(db.foo.insert({_id: -1, a: 1, b: 2}));
      assert.commandWorked(db.foo.ensureIndex({a: 1}));
      assert.commandWorked(db.foo.ensureIndex({b: 1}));
      
      // Add artificial score boost to AND_SORTED plans.
      assert.commandWorked(db.adminCommand({setParameter: 1, internalQueryForceIntersectionPlans: true}));
      
      // Increase frequency of yielding.
      assert.commandWorked(db.adminCommand({setParameter: 1, internalQueryExecYieldIterations: 2}));
      
      // In a parallel shell, continuously flop the "a" and "b" values of the {_id: -1} document.
      startParallelShell(
          "while (true) { " +
          "   assert.writeOK(db.foo.update({_id: -1}, {$set: {a: 2, b: 1}})); " +
          "   assert.writeOK(db.foo.update({_id: -1}, {$set: {a: 1, b: 2}})); " +
          "}"
      );
      while (true) {
          // Run an AND_SORTED intersection query on a predicate that matches no documents.
          // This below assertion fails when the query incorrectly returns the {_id: -1} document.
          assert.eq(0, db.foo.find({a: 1, b: 1}, {_id: 0, a: 1, b: 1}).itcount());
      }
      

            Assignee:
            david.storch@mongodb.com David Storch
            Reporter:
            rassi J Rassi
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: