Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-65553

Atomic applyOps generates a change stream pre-image on secondaries but not on the primary

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • ALL
    • Hide 
      db.getSiblingDB('test').createCollection('c', {changeStreamPreAndPostImages: {enabled: true}})
db.getSiblingDB('test').c.insertOne({_id:1, a:"a"})
db.getSiblingDB('test').runCommand({applyOps:[{op:'u', ns:'test.c', o2: {_id:1}, o: {$set:{a:"b"}}}]})

      Results:

      MongoDB Enterprise replset:PRIMARY> db.getSiblingDB('config').system.preimages.find().itcount()
0

      MongoDB Enterprise replset:SECONDARY> db.getSiblingDB('config').system.preimages.find()
{ "_id" : { "nsUUID" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669"), "ts" : Timestamp(1649864240, 1), "applyOpsIndex" : NumberLong(0) }, "operationTime" : ISODate("2022-04-13T15:37:20.364Z"), "preImage" : { "_id" : 1, "a" : "a" } }

      Oplog:

      { "op" : "c", "ns" : "test.$cmd", "o" : { "applyOps" : [ { "op" : "u", "ns" : "test.c", "o2" : { "_id" : 1 }, "o" : { "$set" : { "a" : "b" } }, "ui" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669") } ], "lsid" : { "id" : UUID("d2de497f-e3a4-4c07-979b-629bda64c9e9") }, "$clusterTime" : { "clusterTime" : Timestamp(1649864239, 2), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "$db" : "test" }, "ts" : Timestamp(1649864240, 1), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-04-13T15:37:20.364Z") }
{ "op" : "i", "ns" : "test.c", "ui" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669"), "o" : { "_id" : 1, "a" : "a" }, "o2" : { "_id" : 1 }, "ts" : Timestamp(1649864239, 2), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-04-13T15:37:19.392Z") }
{ "op" : "c", "ns" : "test.$cmd", "ui" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669"), "o" : { "create" : "c", "changeStreamPreAndPostImages" : { "enabled" : true }, "idIndex" : { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_" } }, "ts" : Timestamp(1649864239, 1), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-04-13T15:37:19.356Z") }
      Show
      db.getSiblingDB('test').createCollection('c', {changeStreamPreAndPostImages: {enabled: true}}) db.getSiblingDB('test').c.insertOne({_id:1, a:"a"}) db.getSiblingDB('test').runCommand({applyOps:[{op:'u', ns:'test.c', o2: {_id:1}, o: {$set:{a:"b"}}}]}) Results: MongoDB Enterprise replset:PRIMARY> db.getSiblingDB('config').system.preimages.find().itcount() 0 MongoDB Enterprise replset:SECONDARY> db.getSiblingDB('config').system.preimages.find() { "_id" : { "nsUUID" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669"), "ts" : Timestamp(1649864240, 1), "applyOpsIndex" : NumberLong(0) }, "operationTime" : ISODate("2022-04-13T15:37:20.364Z"), "preImage" : { "_id" : 1, "a" : "a" } } Oplog: { "op" : "c", "ns" : "test.$cmd", "o" : { "applyOps" : [ { "op" : "u", "ns" : "test.c", "o2" : { "_id" : 1 }, "o" : { "$set" : { "a" : "b" } }, "ui" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669") } ], "lsid" : { "id" : UUID("d2de497f-e3a4-4c07-979b-629bda64c9e9") }, "$clusterTime" : { "clusterTime" : Timestamp(1649864239, 2), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "$db" : "test" }, "ts" : Timestamp(1649864240, 1), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-04-13T15:37:20.364Z") } { "op" : "i", "ns" : "test.c", "ui" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669"), "o" : { "_id" : 1, "a" : "a" }, "o2" : { "_id" : 1 }, "ts" : Timestamp(1649864239, 2), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-04-13T15:37:19.392Z") } { "op" : "c", "ns" : "test.$cmd", "ui" : UUID("efa189d2-9da8-40b8-9de2-baa08cae4669"), "o" : { "create" : "c", "changeStreamPreAndPostImages" : { "enabled" : true }, "idIndex" : { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_" } }, "ts" : Timestamp(1649864239, 1), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-04-13T15:37:19.356Z") }

      By design, atomic applyOps does not generate a change stream pre-image (kApplyOpsCmd is excluded here). However, the secondaries replicate the applyOps statements without knowing whether they originated from an applyOps command, and they generate a change stream pre-image. This can result in replica set inconsistencies, as the secondaries may hold content in config.system.preimages that is not existent on the primary.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            josef.ahmad@mongodb.com Josef Ahmad
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: