Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-367

New flag to control 'Insert' behavior for update and replace operations

    • Type: Icon: New Feature New Feature
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 10.1.0
    • Affects Version/s: 3.0.0, 2.4.4, 2.3.6, 2.2.10
    • Component/s: Writes
    • None

      Hi Team,

      MongoSpark write behavior can be controlled using 'forceInsert' and 'replaceDocument' (As of 3.0.x). But there is no flag to control if 'Inserts' are needed during a write.

      Use case: One use case would be if customer wants only to update/set the fields of matching documents containing an '_id' field. 

      I have created a pull request for above mentioned implementation of a new flag 'upsertDocument' as spark option. This flag is used in-house to fine tune Mongo Spark write behavior. https://github.com/mongodb/mongo-spark/pull/80

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            santhoshsuresh95@gmail.com Santhosh Suresh
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: