Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38265

Change Stream Events on sharded clusters are not returned in real time

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.0.4
    • Component/s: Performance, Sharding
    • None
    • ALL
    • Hide

      1. Setup sharding as shown below:

      • Replicaset (shard servers) (rs0)
        • mongo1.example.com:27018, mongo2.example.com:27018, mongo3.example.com:27018 
      • Replicaset (shard servers) (rs1)
        • mongo4.example.com:27018, mongo5.example.com:27018, mongo6.example.com:27018
      • Config servers
        • mongo1.example.com:27019, mongo2.example.com:27019, mongo3.example.com:27019
      • Mongos servers:
        • mongo1.example.com:27017, mongo2.example.com:27017, mongo3.example.com:27017
      --- Sharding Status ---
  sharding version: {
   "_id" : 1,
   "minCompatibleVersion" : 5,
   "currentVersion" : 6,
   "clusterId" : ObjectId("5beadc55eb955fb607ecd4e7")
  }
  shards:
        {  "_id" : "rs0db",  "host" : "rs0db/mongo1.example.com:27018,mongo2.example.com:27018,mongo3.example.com:27018",  "state" : 1,  "tags" : [ "rs0zone" ] }
        {  "_id" : "rs1db",  "host" : "rs1db/mongo4.example.com:27018,mongo5.example.com:27018,mongo6.example.com:27018",  "state" : 1,  "tags" : [ "rs1zone" ] }
  active mongoses:
        "4.0.3" : 3
        "4.0.4" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                No recent migrations

      2. Run the snippet as shown below:

      • if ran against mongos
        • it will print changes in "batches" (oldest change can be 10seconds delayed, latest change can be 0 seconds delayed)
      • if ran against mongod (shard)
        • it will print changes "live" (delay=0)
      var changeStream = db.getMongo().watch();
while(!changeStream.isExhausted()){
 if(changeStream.hasNext()){
  doc = changeStream.next();
  c = new Date().getTime() - new Date(doc.clusterTime.getTime()*1000).getTime();
  c = Math.floor(c/1000).toString();
  print(c); //Print delay for each received change
 }
}

       

       

      Show
      1. Setup sharding as shown below: Replicaset (shard servers) (rs0) mongo1.example.com:27018, mongo2.example.com:27018, mongo3.example.com:27018  Replicaset (shard servers) (rs1) mongo4.example.com:27018, mongo5.example.com:27018, mongo6.example.com:27018 Config servers mongo1.example.com:27019, mongo2.example.com:27019, mongo3.example.com:27019 Mongos servers: mongo1.example.com:27017, mongo2.example.com:27017, mongo3.example.com:27017 --- Sharding Status ---   sharding version: {   "_id" : 1,   "minCompatibleVersion" : 5,   "currentVersion" : 6,   "clusterId" : ObjectId("5beadc55eb955fb607ecd4e7")   }   shards:         {  "_id" : "rs0db",  "host" : "rs0db/mongo1.example.com:27018,mongo2.example.com:27018,mongo3.example.com:27018",  "state" : 1,  "tags" : [ "rs0zone" ] }         {  "_id" : "rs1db",  "host" : "rs1db/mongo4.example.com:27018,mongo5.example.com:27018,mongo6.example.com:27018",  "state" : 1,  "tags" : [ "rs1zone" ] }   active mongoses:         "4.0.3" : 3         "4.0.4" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations 2. Run the snippet as shown below: if ran against mongos it will print changes in "batches" (oldest change can be 10seconds delayed, latest change can be 0 seconds delayed) if ran against mongod (shard) it will print changes "live" (delay=0) var changeStream = db.getMongo().watch(); while (!changeStream.isExhausted()){ if (changeStream.hasNext()){ doc = changeStream.next(); c = new Date().getTime() - new Date(doc.clusterTime.getTime()*1000).getTime(); c = Math .floor(c/1000).toString(); print(c); //Print delay for each received change } }    

      When using a sharded setup, trying to tail changestream via mongos (as suggested by the docs), returns changes in "batches" instead of live.

      If you tail the changestream via the replicaset directly (which is not the point, because you want to use sharding), then it works as expected i.e updates come in "live" not in "batches".

      You can see if an update is coming in "live" or "delayed" by printing the time difference between now & the clusterTime (from the received change).

      This is tested this against:

      • mongo client
      • a database with sharding enabled
      • a database without sharding enabled

      They all reproduce the same effect.

            Assignee:
            eric.sedor@mongodb.com Eric Sedor
            Reporter:
            kurteknikk Kurt Agius
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: