Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-329

Cannot perform update operations on Time Series collections with Spark Connector

    • Type: Icon: New Feature New Feature
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 10.1.0
    • Affects Version/s: 3.0.1, 10.0.4
    • Component/s: Writes
    • Needed
    • Hide

      add note how to now write to time series collections in mongodb is using upsertDocument set to false

      Show
      add note how to now write to time series collections in mongodb is using upsertDocument set to false

      Scenario

      Update operations to time series collections fail due to the Updates and Deletes limitations limitations for these collection types.

      The update functionality for the Time Series collections, with the below limitations, was introduced in MongoDB 5.5.

      • The update command may not limit the number of documents to be updated. You must use an update command with multi: true or the updateMany() method.
      • The update command may not set upsert: true.

      As per internal discussion:

      • The spark connector does not allow to set the required `multi: true` option.
      • The spark connector implicitly sets `upsert: true` for bulk operations.

      Improvement Request

      Please add Spark Connector update methods which can meet the Time Series limitations. Alternatively please allow to modify the default operation options to meet the required limitations.

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            stas.baskin@mongodb.com Stas Baskin
            Ross Lawley Ross Lawley
            Robert Walters Robert Walters
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: