Problem description

BSON can accurately represent IEEE doubles, including unusual values like "Inf", "-Inf", "NaN" and "-0.0". NaN can additionally include a "payload" value.

The shell and mongoexport handle these cases in problematic or inconsistent ways:

• Both the shell and mongoexport uses non-standard JSON for positive and negative infinity and NaN. These will cause JSON decoding errors on any strict JSON parser in the wild. They also have a slight difference in how they represent positive infinity:
  • The shell uses: Infinity, -Infinity and NaN
  • mongoexport uses: +Infinity, -Infinity and NaN

• The shell and mongoexport differ in their treatment of -0.0:
  • The shell shows it as 0
  • mongoexport shows it as -0.0
  • Interestingly, the shell will read -0.0 and correctly insert a negative zero into the database: db.foo.insertOne({a:-0.0})

• Neither the shell nor mongoexport have any way to represent an NaN payload. For a BSON document with an NaN payload, both merely show it as NaN

Details of the test documents may be found at the end of the Description field.

Thoughts and Recommendations

Representing doubles in JSON is inherently non-standard. I think our goal, particularly with mongoexport, should be to maximize portability. Within the shell, portability is not a constraint and we should aim for consistency and usability.

NaN payloads add a degree of complexity and may be rare in the wild. We may or may not want to support them.

Therefore, I recommend the following:

• We create a new $numberDouble extended JSON type for use with mongoexport with the string representation of special double values. E.g. {"$numberDouble": "-Infinity"}. We should use Infinity instead of +Infinity for consistency with Javascript.

• When parsing $numberDouble:
  • string literal doubles should be allowed (e.g. "123.45" or "-1.23e-45")
  • "+Infinity" should be allowed
  • If desired, we could allow NaN payloads with a special syntax. Various strtod implementations take either NaN(XXX) or NaNXXX, so we can consider those "standard" approaches. We could choose one or support both.

• The shell should show negative zero as -0.0 so that its output can be cut/pasted as input without loss of information

• If we choose to support NaN payloads in mongoexport, then the shell should add a NumberDouble function with the same semantics as $numberDouble. It should output NumberDouble for NaNs with a payload (similarly to how the shell outputs ObjectID for object IDs).

• Documentation should be updated to reflect how to represent and interpret special doubles.

Details of test documents

Documents used for testing are shown below. The double tested is given first on a # line, followed by the hex mongodump output, the shell output, and the mongoexport output. (The "type" field was my own annotation of which document was which without having to cross reference Object IDs.)

# Inf: 00 00 00 00 00 00 f0 7f
Dump: 2f 00 00 00 07 5f 69 64 00 56 ea de b3 ea 79 f2 2d ad 6b 30 c9 02 74 79 70 65 00 04 00 00 00 49 6e 66 00 01 61 00 00 00 00 00 00 00 f0 7f 00
Shell: { "_id" : ObjectId("56eadeb3ea79f22dad6b30c9"), "type" : "Inf", "a" : Infinity }
Export: {"_id":{"$oid":"56eadeb3ea79f22dad6b30c9"},"type":"Inf","a":+Infinity}

# Negative Inf: 00 00 00 00 00 00 f0 ff
Dump: 30 00 00 00 07 5f 69 64 00 56 ea de d0 ea 79 f2 2d ad 6b 30 cd 02 74 79 70 65 00 05 00 00 00 4e 49 6e 66 00 01 61 00 00 00 00 00 00 00 f0 ff 00
Shell: { "_id" : ObjectId("56eaded0ea79f22dad6b30cd"), "type" : "NInf", "a" : -Infinity }
Export: {"_id":{"$oid":"56eaded0ea79f22dad6b30cd"},"type":"NInf","a":-Infinity}

# NaN: 00 00 00 00 00 00 f8 7f
Dump: 2f 00 00 00 07 5f 69 64 00 56 ea de bb ea 79 f2 2d ad 6b 30 cb 02 74 79 70 65 00 04 00 00 00 4e 61 4e 00 01 61 00 00 00 00 00 00 00 f8 7f 00
Shell: { "_id" : ObjectId("56eadebbea79f22dad6b30cb"), "type" : "NaN", "a" : NaN }
Export: {"_id":{"$oid":"56eadebbea79f22dad6b30cb"},"type":"NaN","a":NaN}

# Negative Zero: 00 00 00 00 00 00 00 80
Dump: 31 00 00 00 07 5f 69 64 00 56 ea de be ea 79 f2 2d ad 6b 30 cc 02 74 79 70 65 00 06 00 00 00 4e 5a 65 72 6f 00 01 61 00 00 00 00 00 00 00 00 80 00
Shell: { "_id" : ObjectId("56eadebeea79f22dad6b30cc"), "type" : "NZero", "a" : 0 }
Export: {"_id":{"$oid":"56eadebeea79f22dad6b30cc"},"type":"NZero","a":-0.0}

# Special NaN (0x15): 15 00 00 00 00 00 f8 7f
Dump: 30 00 00 00 07 5f 69 64 00 56 ea e6 14 ea 79 f2 3f 35 4d e5 f1 02 74 79 70 65 00 05 00 00 00 53 4e 61 4e 00 01 61 00 15 00 00 00 00 00 f8 7f 00
Shell: { "_id" : ObjectId("56eadeb8ea79f22dad6b30ca"), "type" : "SNaN", "a" : NaN }
Export: {"_id":{"$oid":"56eadeb8ea79f22dad6b30ca"},"type":"SNaN","a":NaN}