Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-101312

Incorrect ns.db reported for change streams on Flex clusters

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 8.0.4
    • Component/s: None
    • ALL
    • Hide

      The code below produces two events on the changestream.
      The first one has `ns:

      { db: '67b83e86cd20730f1e766dde_ps', coll: 'test' }

      `, while the second one has the correct `

      { db: 'ps', coll: 'test' }

      `.

      import * as mongo from 'mongodb';

const client = new mongo.MongoClient(process.env.MONGO_URI);
const dbName = process.env.MONGO_DB ?? 'ps';

const db = client.db(dbName);

const startAfter = new mongo.Timestamp({ t: Math.floor(Date.now() / 1_000) - 10, i: 0 });

console.log('Inserting document');
await db.collection('test').insertOne({ name: 'test1' });
console.log('Starting stream at', startAfter);

const stream = db.watch([{ $match: { ns: { $in: [{ db: dbName, coll: 'test' }] } } }], {
  startAtOperationTime: startAfter
});

// These two variations also reproduce the same issue:
// const stream = db.watch([], {
//   startAtOperationTime: startAfter
// });
// const stream = db.collection('test').watch([], {
//   startAtOperationTime: startAfter
// });

// This variation always gives the correct ns.db:
// const stream = client.watch([{ $match: { ns: { $in: [{ db: dbName, coll: 'test' }] } } }], {
//   startAtOperationTime: startAfter
// });

let didInsertSecond = false;
for await (let item of stream) {
  console.log(item);

  if (!didInsertSecond) {
    await db.collection('test').insertOne({ name: 'test2' });
    didInsertSecond = true;
  }
}
      Show
      The code below produces two events on the changestream. The first one has `ns: { db: '67b83e86cd20730f1e766dde_ps', coll: 'test' } `, while the second one has the correct ` { db: 'ps', coll: 'test' } `. import * as mongo from 'mongodb' ; const client = new mongo.MongoClient(process.env.MONGO_URI); const dbName = process.env.MONGO_DB ?? 'ps' ; const db = client.db(dbName); const startAfter = new mongo.Timestamp({ t: Math .floor(Date.now() / 1_000) - 10, i: 0 }); console.log( 'Inserting document' ); await db.collection( 'test' ).insertOne({ name: 'test1' }); console.log( 'Starting stream at' , startAfter); const stream = db.watch([{ $match: { ns: { $ in : [{ db: dbName, coll: 'test' }] } } }], { startAtOperationTime: startAfter }); // These two variations also reproduce the same issue: // const stream = db.watch([], { // startAtOperationTime: startAfter // }); // const stream = db.collection( 'test' ).watch([], { // startAtOperationTime: startAfter // }); // This variation always gives the correct ns.db: // const stream = client.watch([{ $match: { ns: { $ in : [{ db: dbName, coll: 'test' }] } } }], { // startAtOperationTime: startAfter // }); let didInsertSecond = false ; for await (let item of stream) { console.log(item); if (!didInsertSecond) { await db.collection( 'test' ).insertOne({ name: 'test2' }); didInsertSecond = true ; } }
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      On Flex clusters specifically, change streams can report an `ns.db` value of for example `67b83e86cd20730f1e766dde_ps` instead of `ps`. This generally only happens when changes were made while the client was offline, and then the client catches up using `resumeToken` or `startAtOperationTime` - then the events made while offline has the incorrect `ns.db` value, while the new ones have the correct value.

      This only happens when watching on a single database. When watching on multiple databases, I could not reproduce the same issue. So it does have a simple workaround at least since I know what the database should be, but it does still appear to be a bug.

      This also appears to be specific to Flex clusters. I have not been able to reproduce on dedicated or self-hosted clusters. I haven't tested this with free clusters yet.

      In my specific example here I'm watching for events on a single collection, so the ns value is not important. However, I used this in production for a use cases that could potentially get events on multiple different databases and collections, where the exact value is important.

            Assignee:
            chris.kelly@mongodb.com Chris Kelly
            Reporter:
            ralf@journeyapps.com Ralf Kistner
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved:
              None
              None
              None
              None