Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-62485

Match the SBE $add behavior to the classic engine's or the other way around

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • ALL
    • Hide

      See description.

      Show
      See description.
    • QE 2022-04-04, QE 2022-02-07, QE 2022-02-21, QE 2022-03-07, QE 2022-03-21, QE 2022-01-24, QE 2022-04-18
    • 146

      1st repro

      > db.t.insert({a: NumberDecimal("0.0")});
> db.t.find({}, {_id: 0, o: {$add: [{$exp: NumberLong("4")}, "$a"]}});
{ "o" : NumberDecimal("54.5981500331442") }
> db.adminCommand({setParameter: 1, internalQueryForceClassicEngine: true});
> db.t.find({}, {_id: 0, o: {$add: [{$exp: NumberLong("4")}, "$a"]}});
{ "o" : NumberDecimal("54.59815003314423620395245961844921") }

      2nd repro:

      > db.t3.find();
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cb"), "a" : NumberDecimal("0.0"), "b" : 1e-15 }
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cc"), "a" : NumberDecimal("0.0"), "b" : 2e-15 }
> db.t3.find({}, {o: {$exp: NumberLong("4")}});
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cb"), "o" : 54.598150033144236 }
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cc"), "o" : 54.598150033144236 }
> db.t3.find({}, {o: {$add: [{$exp: NumberLong("4")}, "$a", "$b"]}});
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cb"), "o" : NumberDecimal("54.59815003314420100000000000000") }
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cc"), "o" : NumberDecimal("54.59815003314420200000000000000") }
> db.adminCommand({setParameter: 1, internalQueryForceClassicEngine: true});
{ "was" : false, "ok" : 1 }
> db.t3.find({}, {o: {$add: [{$exp: NumberLong("4")}, "$a", "$b"]}});
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cb"), "o" : NumberDecimal("54.59815003314423720395245961844929") }
{ "_id" : ObjectId("61ddd0403dea1cd22d1fb3cc"), "o" : NumberDecimal("54.59815003314423820395245961844937") }

      Last repro:
      Data:

      > db.t1.find();
{ "_id" : ObjectId("61d7310c6fce2cdbb8498b8e"), "a" : 1e+34, "b" : NumberDecimal("0.1"), "c" : -1e+34, "d" : NumberDecimal("0.1") }
{ "_id" : ObjectId("61d732ce6fce2cdbb8498b8f"), "a" : 1e+34, "b" : 0.1, "c" : -1e+34, "d" : 0.1 }
{ "_id" : ObjectId("61d734236fce2cdbb8498b90"), "a" : NumberDecimal("1.0E+34"), "b" : NumberDecimal("0.1"), "c" : NumberDecimal("-1.0E+34"), "d" : NumberDecimal("0.1") }

      SBE:

      > db.t1.find({}, {o: {$add: ["$a", "$b", "$c", "$d"]}});
{ "_id" : ObjectId("61d7310c6fce2cdbb8498b8e"), "o" : NumberDecimal("0.1") }
{ "_id" : ObjectId("61d732ce6fce2cdbb8498b8f"), "o" : 0.2 }
{ "_id" : ObjectId("61d734236fce2cdbb8498b90"), "o" : NumberDecimal("0.1") }

      Classic:

      > db.t1.find({}, {o: {$add: ["$a", "$b", "$c", "$d"]}});
{ "_id" : ObjectId("61d7310c6fce2cdbb8498b8e"), "o" : NumberDecimal("0.2") }
{ "_id" : ObjectId("61d732ce6fce2cdbb8498b8f"), "o" : 0.2 }
{ "_id" : ObjectId("61d734236fce2cdbb8498b90"), "o" : NumberDecimal("0.1") }

      This issue is caused by the difference of how we $add operations in classic engine and SBE.

      There are two sources of differences:

      1. The SBE’s builtinDoubleDoubleSum algorithm is slightly different from the classic engine’s in the aspect that the SBE’s computes the expected result type first and then do real addition. If the expected result type is not Decimal128, it does not use DoubleDoubleSummation at all and all input values are converted first to Decimal128 and added to the partial Decimal128 result. But the classic engine’s always uses DoubleDoubleSummation for non-decimal values and add Decimal128 results and DoubleDoubleSummation result.
      2. As in 1., when the SBE’s builtinDoubleDoubleSum converts a double input value to Decimal128 or when the SBE's genericAdd for binary $add converts a double input value to Decimal128, it uses numericCast<Decimal128> and 15 digit rounding and the classic engine 34 digit rounding.
      3. The SBE genericAdd promotes overflowed long addition into Decimal128, which is very reasonable in that it can keep all digits but the SBE's builtinDoubleDoubleSum and the classical engine's ExpressionAdd::evaluate promote it to double.
      4. The $sum accumulator in both the SBE and the classical engine promote overflowed long into a double.

      I think we can match the SBE behavior to the classic engine’s behavior.

      1. We always use DoubleDoubleSummation for non-decimal values just like the classic engine. For this, we need to update builtinDoubleDoubleSum() function.
      2. We use 34 digit rounding when converting double input into Decimal128. Otherwise, we may increase error. For this, we need to update numericCast<Decimal128> so that it can support rounding option and genericAdd calls numericCast<Decimal128> with the rounding option.
      3. Always promotes overflowed long to Decimal128 across the board: binary $add, 3 or more arg $add and $sum accumulator.

      Update:

      • There was a typo in differences. Fixed it.
      • Added another difference that when the SBE's genericAdd for binary $add converts a double input value to Decimal128, it uses numericCast<Decimal128> and 15 digit rounding
      • Added the 2nd repro.
        1/12/2021
      • Updated differences and the proposed fix.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            yoonsoo.kim@mongodb.com Yoon Soo Kim
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: