Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-3976

.shift() usage in nextDocument greatly slows down with large batchSize

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None

      https://github.com/mongodb/node-mongodb-native/blob/a7a3b99645ec76e4d653537b79263f00b3587e87/src/cursor/abstract_cursor.ts#L630 contains:

      const doc = cursor[kDocuments].shift();

      In V8, Array.prototype.shift() potentially copies the entirety of the array forward by one element, leading to bad performance for large batch sizes (which we tried using as part of investigating COMPASS-5455 – see attached CPU profile which points to unreasonably much time being spent in nextDocument). Specifically, fetching documents happens in O(nDocuments · batchSize) rather than the expectable O(nDocuments).

      If this is adjusted to perform plain array accesses instead of .shift() calls, performance is as expected.

        1. Screenshot from 2022-02-08 17-56-30.png
          34 kB
          Anna Henningsen

            Assignee:
            Unassigned Unassigned
            Reporter:
            anna.henningsen@mongodb.com Anna Henningsen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: