Hi Team,
MongoSpark write behavior can be controlled using 'forceInsert' and 'replaceDocument' (As of 3.0.x). But there is no flag to control if 'Inserts' are needed during a write.
Use case: One use case would be if customer wants only to update/set the fields of matching documents containing an '_id' field.
I have created a pull request for above mentioned implementation of a new flag 'upsertDocument' as spark option. This flag is used in-house to fine tune Mongo Spark write behavior. https://github.com/mongodb/mongo-spark/pull/80
- is related to
-
SPARK-329 Cannot perform update operations on Time Series collections with Spark Connector
- Closed