Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-394

Remove null value column when write

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 10.2.0
    • Affects Version/s: 10.1.0
    • Component/s: Writes
    • None
    • Needed
    • Hide

      Add ignoreNullValues to: https://www.mongodb.com/docs/spark-connector/current/configuration/write/

      1. What would you like to communicate to the user about this feature?

      New write configuration: ignoreNullValues
      When true writes will ignore any null values, even those within arrays or sub documents

      Defaults to: false

      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      n/a

      3. Which versions of the driver/connector does this apply to?
      10.2.0

      Show
      Add ignoreNullValues to: https://www.mongodb.com/docs/spark-connector/current/configuration/write/ 1. What would you like to communicate to the user about this feature? New write configuration: ignoreNullValues When true writes will ignore any null values, even those within arrays or sub documents Defaults to: false 2. Would you like the user to see examples of the syntax and/or executable code and its output? n/a 3. Which versions of the driver/connector does this apply to? 10.2.0

      Before v10 Spark connector, the Spark connector doesn't write null value column into MongoDB. This is changed after v10 and it increases the total memory consumption on our DB.

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            cheehong05090@gmail.com Vincent Chee
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: