Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1875

Improper parsing of json $date without time-offset or time-secfrac

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.9
    • Affects Version/s: 3.7.2
    • Component/s: Error Handling
    • None
    • Environment:
      Installed via Anaconda
    • Fully Compatible

      The Issue

      The error message produced by `bson.json_util.loads` when in incorrect date format is provided is unhelpful and, probably, incorrect.

      To Reproduce

      from bson import json_util
      json_util.loads('{"$date": "2019-01-01T01:02:03"}')
      

      This produces the following error (full traceback below):

          ValueError: time data '2019-01-01T01' does not match format '%Y-%m-%dT%H:%M:%S'

      This indicates that the format should be "%Y-%m-%dT%H:%M:%S and suggests that the value that was passed in is six characters shorter than the true input value.  In reality, the format should be "%Y-%m-%dT%H:%M:%S.%L" in MongoDB's format syntax or "%Y-%m-%dT%H:%M:%S.%f" in the syntax used by Python's datetime package.

      Full Traceback

      In [4]: json_util.loads('\{"$date": "2019-01-01T01:02:03"}') 
      ---------------------------------------------------------------------------
      ValueError Traceback (most recent call last)
      <ipython-input-4-85a7366e7304> in <module>
      ----> 1 json_util.loads('\{"$date": "2019-01-01T01:02:03"}')
      
      ~/apps/anaconda3/lib/python3.6/site-packages/bson/json_util.py in loads(s, *args, **kwargs)
       436 else:
       437 kwargs["object_hook"] = lambda obj: object_hook(obj, json_options)
      --> 438 return json.loads(s, *args, **kwargs)
       439 
       440
      
      ~/apps/anaconda3/lib/python3.6/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw) 
       365 if parse_constant is not None:
       366 kw['parse_constant'] = parse_constant
      --> 367 return cls(**kw).decode(s)
      
      ~/apps/anaconda3/lib/python3.6/json/decoder.py in decode(self, s, _w)
       337 
       338 """
      --> 339 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
       340 end = _w(s, end).end()
       341 if end != len(s):
      
      ~/apps/anaconda3/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
       353 """
       354 try:
      --> 355 obj, end = self.scan_once(s, idx)
       356 except StopIteration as err:
       357 raise JSONDecodeError("Expecting value", s, err.value) from None
      
      ~/apps/anaconda3/lib/python3.6/site-packages/bson/json_util.py in <lambda>(pairs)
       433 if _HAS_OBJECT_PAIRS_HOOK:
       434 kwargs["object_pairs_hook"] = lambda pairs: object_pairs_hook(
      --> 435 pairs, json_options)
       436 else:
       437 kwargs["object_hook"] = lambda obj: object_hook(obj, json_options)
      
      ~/apps/anaconda3/lib/python3.6/site-packages/bson/json_util.py in object_pairs_hook(pairs, json_options)
       455 
       456 def object_pairs_hook(pairs, json_options=DEFAULT_JSON_OPTIONS):
      --> 457 return object_hook(json_options.document_class(pairs), json_options)
       458 
       459
      
      ~/apps/anaconda3/lib/python3.6/site-packages/bson/json_util.py in object_hook(dct, json_options)
       464 return _parse_canonical_dbref(dct)
       465 if "$date" in dct:
      --> 466 return _parse_canonical_datetime(dct, json_options)
       467 if "$regex" in dct:
       468 return _parse_legacy_regex(dct)
      
      ~/apps/anaconda3/lib/python3.6/site-packages/bson/json_util.py in _parse_canonical_datetime(doc, json_options)
       598 
       599 aware = datetime.datetime.strptime(
      --> 600 dt, "%Y-%m-%dT%H:%M:%S").replace(microsecond=microsecond,
       601 tzinfo=utc)
       602
      
      ~/apps/anaconda3/lib/python3.6/_strptime.py in _strptime_datetime(cls, data_string, format)
       563 """Return a class cls instance based on the input string and the
       564 format string."""
      --> 565 tt, fraction = _strptime(data_string, format)
       566 tzname, gmtoff = tt[-2:]
       567 args = tt[:6] + (fraction,)
      
      ~/apps/anaconda3/lib/python3.6/_strptime.py in _strptime(data_string, format)
       360 if not found:
       361 raise ValueError("time data %r does not match format %r" %
      --> 362 (data_string, format))
       363 if len(data_string) != found.end():
       364 raise ValueError("unconverted data remains: %s" %
      
      ValueError: time data '2019-01-01T01' does not match format '%Y-%m-%dT%H:%M:%S'
      

            Assignee:
            shane.harvey@mongodb.com Shane Harvey
            Reporter:
            jsolbrig Jeremy Solbrig
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: