Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2993

Specify how to handle unacknowledged+(ordered|verbose|multi-batch) bulk writes

    • Type: Icon: Spec Change Spec Change
    • Resolution: Unresolved
    • Priority: Icon: Unknown Unknown
    • None
    • Component/s: Bulk API
    • None
    • Needed
    • Hide

      Summary of necessary driver changes

      Update behavior for MongoClient.bulkWrite:

      • Error if w:0 is used with ordered=true or verboseResults=true
      • Use w:0 for each batch for a multi-batch unacknowledged bulk write.

      Update and implement new tests.

      Commits for syncing spec/prose tests

      Show
      Summary of necessary driver changes Update behavior for MongoClient.bulkWrite: Error if w:0 is used with ordered=true or verboseResults=true Use w:0 for each batch for a multi-batch unacknowledged bulk write. Update and implement new tests. Commits for syncing spec/prose tests https://github.com/mongodb/specifications/commit/c84488cb9cf185e31869547a6378a701124b4665
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-5741 In Code Review
      CXX-3113 Backlog
      CSHARP-5300 Fixed 3.1.0
      GODRIVER-3364 In Code Review
      JAVA-5635 Ready for Work
      NODE-6409 Fixed 6.10.0
      MOTOR-1373 Duplicate
      PYTHON-4807 Fixed 4.11
      PHPLIB-1542 Backlog
      RUBY-3547 Backlog
      RUST-2047 Works as Designed
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-5741 In Code Review CXX-3113 Backlog CSHARP-5300 Fixed 3.1.0 GODRIVER-3364 In Code Review JAVA-5635 Ready for Work NODE-6409 Fixed 6.10.0 MOTOR-1373 Duplicate PYTHON-4807 Fixed 4.11 PHPLIB-1542 Backlog RUBY-3547 Backlog RUST-2047 Works as Designed

      Summary

      When a multi-batch ordered bulk write is performed, drivers need to inspect the result from each batch to determine whether they should continue sending batches or terminate the bulk write. This is at odds with unacknowledged (w:0) writes, for which drivers set the moreToCome OP_MSG bit and do not attempt to read a response from the server. Existing driver behavior differs for writes under these conditions: some drivers set w:1 for all of the non-final batches, and some drivers set w:0 but do not set the moreToCome bit for all of the non-final batches. The latter option is preferable because specifying w:0 (even without moreToCome) prevents the server from returning detailed results or errors. We should specify and test the desired behavior for ordered, unacknowledged, multi-batch writes. This issue is especially relevant for the client-level bulk write API, as using w:1 for the non-final batches may result in a cursor with a nonzero ID being returned that the driver would need to exhaust or kill.

      verboseResults cannot be satisfied with w:0.

      Ordered bulk writes are expected to stop at the first error. With w:0, drivers cannot determine if a server error occurred.

      Motivation

      Who is the affected end user?

      Users performing multi-batch, ordered, unacknowledged bulk writes.

      How does this affect the end user?

      There are not visible impacts for the user, but there may be a performance hit if drivers are attempting to exhaust cursors for non-final batches. Bad driver behavior may also leave cursors open on the server.

      How likely is it that this problem or use case will occur?

      Edge case - large enough bulk writes that would warrant a) driver-side batch-splitting and b) a nonzero cursor ID are likely rare. The unacknowledged + ordered option combination also may be less common.

      If the problem does occur, what are the consequences and how severe are they?

      Not severe.

      Is this issue urgent?

      No, but completing this soon will allow drivers to implement this behavior as part of DRIVERS-716.

      Is this ticket required by a downstream team?

      No.

      Is this ticket only for tests?

      No.

      Acceptance Criteria

      • Specify behavior with unacknowledged (w:0) writes for MongoClient.bulkWrite:
        • Return error when verboseResults is requested
        • Return error when ordered writes are requested
        • When a write requires multiple batches, use w:0 for each batch
      • Test to ensure that the expected write concern is sent, and/or errors are reported.

      Changing MongoCollection.bulkWrite is out-of-scope (likely backwards breaking for some drivers).

            Assignee:
            kevin.albertson@mongodb.com Kevin Albertson
            Reporter:
            isabel.atkinson@mongodb.com Isabel Atkinson
            Kevin Albertson Kevin Albertson
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: