Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-26193

conversion from double to decimal rounds 0.999999999999999 incorrectly

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.4.0-rc0
    • Affects Version/s: None
    • Component/s: Shell
    • None
    • Fully Compatible
    • ALL
    • Hide

      > x=NumberDecimal(0.999999999999999)
      NumberDecimal("1.00000000000000")

      Show
      > x=NumberDecimal(0.999999999999999) NumberDecimal("1.00000000000000")
    • Integration 2016-10-10

      When determining the number of digits to round to, the existing code first uses a possible overestimate of the decimal place to round at, and corrects it if the final number of digits is not 15. However, this is incorrect when the rounding causes the decade to change: 1.00000000000000 has the same number of digits as 0.999999999999999. So, first try rounding at the smaller decimal place instead.

            Assignee:
            geert.bosch@mongodb.com Geert Bosch
            Reporter:
            geert.bosch@mongodb.com Geert Bosch
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: