Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1374

Optimize json_util performance

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.7
    • Affects Version/s: None
    • Component/s: JSON, Performance
    • None
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      While working on PYTHON-1277 I've come to realize just how slow json_util is. We can definitely improve things:

      • Allow applications to use simplejson instead of json. We currently only attempt to import simplejson with Python 2.6 for ordered dictionary support, but simplejson is often faster than the standard library json module.
      • Provide dump and load helpers to go along with dumps and loads
      • Use a single dispatch pattern with _type_marker to speed up json_util.default for BSON types.
      • Use set intersection to quickly find the $ key (if any) for a BSON type in json_util.object_hook.

      There are other fast JSON libraries for Python (cjson, ujson, rapidjson, etc.), but none of them provide the features we need for json_util (default, object_hook, object_pairs_hook, etc.).

            Assignee:
            Unassigned Unassigned
            Reporter:
            bernie@mongodb.com Bernie Hackett
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: