Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-88375

Initial sync / secondary oplog application don't update _nextId for recordId

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • ALL
    • Hide

      On the primary, perform the following:

      assert.commandWorked(primDB[collName].insertMany([
    {_id: 1, a: 1},                 // recordId: 1
    {$recordId: 12, _id: 2, a: 2},  // recordId: 2
    {_id: 3, $recordId: 13, a: 3},  // recordId: 3
    {_id: 4, a: 4}                  // recordId: 4
]));

// Remove recordId: 1 and then insert the doc again, which now gets recordId: 5.
assert.commandWorked(primDB[collName].remove({_id: 1}));
assert.commandWorked(primDB[collName].insert({_id: 1, a: 1}));

// There are now a total of 4 documents on the primary, but the recordIds are
// [2, 3, 4, 5].

      Add a new node and let it initial sync. Make the new node primary. As there are only 4 documents, on the new node, _ndextId is currently 5 (4 + 1). Therefore the next document inserted on it will get recordId(5) which collides with the doc with {_id: 1, a: 1}.

      Insert doc on new primary.

      assert.commandWorked(primDB[collName].insert({_id: 6, a: 6}));

      We see that the old doc was overwritten:

      > db.rrid.find().showRecordId()
{ "_id" : 2, "$recordId" : NumberLong(2), "a" : 2 }
{ "_id" : 3, "$recordId" : NumberLong(3), "a" : 3 }
{ "_id" : 4, "a" : 4, "$recordId" : NumberLong(4) }
{ "_id" : 6, "$recordId" : NumberLong(5) } // Overwrote the old doc!
      Show
      On the primary, perform the following: assert .commandWorked(primDB[collName].insertMany([ {_id: 1, a: 1}, // recordId: 1 {$recordId: 12, _id: 2, a: 2}, // recordId: 2 {_id: 3, $recordId: 13, a: 3}, // recordId: 3 {_id: 4, a: 4} // recordId: 4 ])); // Remove recordId: 1 and then insert the doc again, which now gets recordId: 5. assert .commandWorked(primDB[collName].remove({_id: 1})); assert .commandWorked(primDB[collName].insert({_id: 1, a: 1})); // There are now a total of 4 documents on the primary, but the recordIds are // [2, 3, 4, 5]. Add a new node and let it initial sync. Make the new node primary. As there are only 4 documents, on the new node, _ndextId is currently 5 (4 + 1). Therefore the next document inserted on it will get recordId(5) which collides with the doc with { _id: 1, a: 1 }. Insert doc on new primary. assert .commandWorked(primDB[collName].insert({_id: 6, a: 6})); We see that the old doc was overwritten: > db.rrid.find().showRecordId() { "_id" : 2, "$recordId" : NumberLong(2), "a" : 2 } { "_id" : 3, "$recordId" : NumberLong(3), "a" : 3 } { "_id" : 4, "a" : 4, "$recordId" : NumberLong(4) } { "_id" : 6, "$recordId" : NumberLong(5) } // Overwrote the old doc!

      This was temporarily solved through SERVER-88523.

      When the recordId is provided (as in colls with recordIdsReplicated:true), on secondaries / during initial sync we don't take any additional steps to update _nextId so that it points to the highest of the recordIds provided (i.e. something like _nextId = max(_nextId, providedRecordId)), and therefore, _nextId can fall behind the highest recordId on the node.

      As a result, if the former secondary / initial syncing node becomes the primary, it will start allocating recordIds from the value of _nextId it has, which can mean it may use recordIds that are already in use, as _nextId might be smaller than the highest recordId on the node.

      This will result in the bug SERVER-88309.

      It's a surprise we haven't hit this dassert yet either.

            Assignee:
            Unassigned Unassigned
            Reporter:
            vishnu.kaushik@mongodb.com Vishnu Kaushik
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: