Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-1885

Bulk write can error when upserting across batches

      The buggy behaviour comes into play for a bulk write operation when:

      1) Number of bulk writes is greater than internal batch size + 1 (i.e. > 1000 + 1 = 1002 or above)
      2) Writes are upsert operations. In our case we were using UpdateOneModel with $set and $setOnInsert with upsert = true
      2) Bulk operations are interleaved operations of different types e.g. writes and delete e.g. alternate write and deletes (1002 each or 2004 in total)
      3) Writes are unordered (we have not tested the ordered writes)

      The problem does not occur if:
      1) The documents being upserted are already there i.e. if you repeat the bulk write for same set of documents, then you will not see the issue.

      The error thrown is MongoInternalException "no mapping found for index 0"

      The error seems to be caused when com.mongodb.operation.MixedBulkWriteOperation.UnorderedRunGenerator -> iterator#next() adds a entry in Index map for second batch of upserts
      run.add(writeRequest, curIndex);

      Internally Index map converts the RangeBased map to HashBased map:

      else

      { IndexMap hashBasedMap = new HashBased(startIndex, count); hashBasedMap.add(index, originalIndex); return hashBasedMap; }

      But the initialization code in the constructor of HashBased() does not entet the for loop as the condition i <= count is not met.

      for (int i = startIndex; i <= count; i++)

      { indexMap.put(i - startIndex, i); }

      Thus the mapping for index 0 is not created

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            nikhilgargi NIkhilesh Gargi
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: