-
Type: Improvement
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
In ARROW-15 we added support for treating BSON Decimal128 as string types.
This allows for a clean mapping with no data loss, at the expense of slowness and the awkwardness of having to convert the returned string value to {Decimal}.
We should look into also offering the ability to map BSON Decimal128 values to PyArrow decimal128 values. Care must be taken to ensure there is no data loss, since the two data structures are not directly compatible.
The BSON Decimal128 spec is defined here. It uses an IEEE 754-2008 decimal128 value.
The arrow type is a fundamentally different type unrelated to IEEE 754-2008. The arrow type is expecting a fixed scale for the data type, where BSON decimal128 values have a precision of 34 digits and a variable scale. The arrow type is meant for a set of data that has a consistent representation, like dollar amounts.
- is related to
-
INTPYTHON-15 Add support for BSON Decimal128 type
- Closed