Uploaded image for project: 'pymongoarrow'
  1. pymongoarrow
  2. ARROW-134

Cannot encode pandas NA objects

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 0.7.0
    • Affects Version/s: None
    • Component/s: None
    • None

      Originally reported as https://github.com/mongodb-labs/mongo-arrow/issues/117.

       

      We can add a custom TypeCodec to handle the Pandas NA type.

      For example,

          codec_options = collections.codec_options
          if DataFrame is not None:
      
              from bson.codec_options import TypeRegistry, TypeCodec
              from pandas import NA
              class PandasNanCodec(TypeCodec):
                  python_type = NA.__class__
                  bson_type = None
                  def transform_python(self, value):
                      """Function that transforms a custom type value into a type
                      that BSON can encode."""
                      return None
      
                  def transform_bson(self, value):
                      """Function that transforms a vanilla BSON type value into our
                      custom type."""
                      return NA
      
              type_registry = TypeRegistry([PandasNanCodec()])
              codec_options = codec_options.with_options(type_registry=type_registry)
      

            Assignee:
            steve.silvester@mongodb.com Steve Silvester
            Reporter:
            steve.silvester@mongodb.com Steve Silvester
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: