Uploaded image for project: 'Realm JavaScript SDK'
  1. Realm JavaScript SDK
  2. RJS-821

Alternative MongoDB Watch API

      We had a discussion on the PR adding the `AsyncIterator` implementation of the MongoDB watch API.

      For any of the routes below I think it'll be in the order of a week to add an implementation and update / add the tests.

      These are some of our alternatives to the current implementation:

      Alternatives

      0. Keep the AsyncIterator as is.

      Whatever we do, I believe it might make sense to keep this API around for expert users. I do see the advantages of the pull-based iteration which this API enables and I think it could be valuable for an advanced user of the library.

      1. A simple listener pattern, like the Node.js driver ChangeStream API

      One alternative is the simplified listener pattern which the Node.js driver is currently exposing: https://mongodb.github.io/node-mongodb-native/3.6/api/ChangeStream.html

      This is probably my favorite, since it's simpler to implement and it mimics the Node.js driver enabling transfer of knowledge between drivers and SDKs.

      const stream = collection.watch();
      stream.onNext(change => {
        // Do whatever with the change event ...
      });
      // After some time, when we're done with the stream
      stream.close()
      

      2. A full-on listener pattern through an API similar to (but perhaps a bit more limited than) EventEmitter with separate event names per operation type

      const emitter = collection.watch();
      // Register callbacks to react to changes
      // Could also be the more verbose `emitter.addListener("insert", event => {`
      emitter.on("insert", event => {
        const document = event.fullDocument;
        // Stop listening conditionally based on some value in a document inserted in the collection
        if (document.stop === true) {
          emitter.close();
        }
      });
      // Register a callback to react to updates
      emitter.on("update", event => {
        const document = event.fullDocument;
        // Stop listening conditionally based on some value in a document inserted in the collection
        if (document.stop === true) {
          emitter.close();
        }
      });
      // Register a callback to react to errors
      // This includes errors from establishing the connection as well as errors parsing the event stream, etc.
      emitter.on("error", console.error);
      // Potentially a way to register a callback to react to any type of event
      emitter.on(event => {
        if (event.operationType === "insert") {
          const document = event.fullDocument;
          // Do whatever with the document ...
        }
      });
      // After some time - close the emitter when we don't care about events anymore
      emitter.close();
      

      Alternatives for extending the current API

      If we choose to add an event emitting variant to this API, I see four options

      0) Return a ChangeStream with a an Symbol.asyncIterator property

      This would allow user to either call the API to attach a listener or use the object returned from watch as an interator.

      This is probably my favorite, since we would still support an event emitter and it wouldn't feel as "second class" as option 1 and 2 (below).

      Using it as an Async iterator

      // Using it as an Async iterator
      const stream = collection.watch({ filter });
      for await (const event of stream) {
        // Do whatever with event
      }
      // Or - use it as an emitter by registering a listener
      stream.on(event => {
        // Do whatever with event
      });
      // When you're done, call close
      stream.close();
      

      1) Provide a mode to watch

      Users could provide an additional optional option when calling collection.watch.

      Using an Async iterator

      Probably the default to limit breaking changes and nudge users into using this pattern.

      for await (const event of collection.watch({ filter, mode: "iterator" })) {
        // Do whatever with event
      }
      

      Using an Event Emitter

      const emitter = collection.watch({ filter, mode: "emitter" });
      // Register a listener
      emitter.on(event => {
        // Do whatever with event
      });
      // When you're done, call close
      emitter.close();
      

      2) Adding a method with a different name

      The easiest in terms of development time would be to add a future alternative as a separate method.
      Ideas for names could be: collection.watchEmitting(), collection.watchWithEmitter(), etc.

      3) Break the API

      Simply break the API, only returning an event emitter and abandoning the async iterator all together.

            Assignee:
            Unassigned Unassigned
            Reporter:
            kraen.hansen@mongodb.com Kræn Hansen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: