Uploaded image for project: 'pymongoarrow'
  1. pymongoarrow
  2. ARROW-233

Add an optional bool flag to the write function to skip writing null fields

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 1.4
    • Affects Version/s: None
    • Component/s: None
    • None
    • Python Drivers
    • Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      Adds a new optional argument to the write() function, exclude_none. By default it is false. When it is set to true, empty values will not be written to the database.
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      Yes, for example:

              schema = {"a": int32(), "b": int32()}
              b_data = [i for i in range(10)] * 2
              b_data[2] = None
              data = Table.from_pydict(
                  {
                      "a": [i for i in range(10)] * 2,
                      "b": b_data,
                  },
                  ArrowSchema(schema),
              )
              coll.drop()
              write(coll, data)
              col_data = list(coll.find({}))
              assert "b" in col_data[2]
      
              coll.drop()
              write(coll, data, exclude_none=True)
              col_data = list(coll.find({}))
              assert "b" not in col_data[2]
      

      3. Which versions of the driver/connector does this apply to?
      PyMongoArrow 1.4

      Show
      1. What would you like to communicate to the user about this feature? Adds a new optional argument to the write() function, exclude_none . By default it is false. When it is set to true, empty values will not be written to the database. 2. Would you like the user to see examples of the syntax and/or executable code and its output? Yes, for example: schema = { "a" : int32(), "b" : int32()} b_data = [i for i in range(10)] * 2 b_data[2] = None data = Table.from_pydict( { "a" : [i for i in range(10)] * 2, "b" : b_data, }, ArrowSchema(schema), ) coll.drop() write(coll, data) col_data = list(coll.find({})) assert "b" in col_data[2] coll.drop() write(coll, data, exclude_none=True) col_data = list(coll.find({})) assert "b" not in col_data[2] 3. Which versions of the driver/connector does this apply to? PyMongoArrow 1.4

      Context

      Opened as https://github.com/mongodb-labs/mongo-arrow/issues/210.

      Definition of done

      Add the optional flag and add test.

      Pitfalls

      N/A

            Assignee:
            steve.silvester@mongodb.com Steve Silvester
            Reporter:
            steve.silvester@mongodb.com Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: