Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-3729

Speed up C BSON encoding by using PyObject_GetAttr instead of PyObject_GetAttrString

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 4.5
    • Affects Version/s: None
    • Component/s: neweng
    • None

      PYTHON-3717 speeds up BSON encoding by caching the "_type_marker" string and translating this:

      PyObject_GetAttrString(object, "_type_marker");
      

      to this:

      PyObject_GetAttr(object, _type_marker_str);
      

      This is faster because we don't need to create/destroy a "_type_marker" string on every lookup. For reference here's the C implement of PyObject_GetAttrString:

      PyObject *
      PyObject_GetAttrString(PyObject *v, const char *name)
      {
          PyObject *w, *res;
      
          if (Py_TYPE(v)->tp_getattr != NULL)
              return (*Py_TYPE(v)->tp_getattr)(v, (char*)name);
          w = PyUnicode_FromString(name);
          if (w == NULL)
              return NULL;
          res = PyObject_GetAttr(v, w);
          Py_DECREF(w);
          return res;
      }
      

      We should be able to apply this same pattern to speed up other uses of PyObject_GetAttrString:

      git grep PyObject_GetAttrString
      bson/_cbsonmodule.c:187:    error = PyObject_GetAttrString(errors, name);
      bson/_cbsonmodule.c:443:        imported = PyObject_GetAttrString(module, object_name);
      bson/_cbsonmodule.c:460:    *object = PyObject_GetAttrString(module, object_name);
      bson/_cbsonmodule.c:541:     * arbitrary types for a call to PyObject_GetAttrString. For example
      bson/_cbsonmodule.c:568:    registry->encoder_map = PyObject_GetAttrString(registry_obj, "_encoder_map");
      bson/_cbsonmodule.c:574:    registry->decoder_map = PyObject_GetAttrString(registry_obj, "_decoder_map");
      bson/_cbsonmodule.c:580:    registry->fallback_encoder = PyObject_GetAttrString(registry_obj, "_fallback_encoder");
      bson/_cbsonmodule.c:708:    py_flags = PyObject_GetAttrString(value, "flags");
      bson/_cbsonmodule.c:717:    py_pattern = PyObject_GetAttrString(value, "pattern");
      bson/_cbsonmodule.c:838:            subtype_object = PyObject_GetAttrString(value, "subtype");
      bson/_cbsonmodule.c:886:            PyObject* pystring = PyObject_GetAttrString(value, "binary");
      bson/_cbsonmodule.c:915:            PyObject* scope = PyObject_GetAttrString(value, "scope");
      bson/_cbsonmodule.c:958:            obj = PyObject_GetAttrString(value, "inc");
      bson/_cbsonmodule.c:971:            obj = PyObject_GetAttrString(value, "time");
      bson/_cbsonmodule.c:1006:            PyObject* pystring = PyObject_GetAttrString(value, "bid");
      bson/_cbsonmodule.c:1443:    bytes_obj = PyObject_GetAttrString(raw, "raw");
      bson/_cbsonmodule.c:1619:        raw_bson_document_bytes_obj = PyObject_GetAttrString(dict, "raw");
      bson/_cbsonmodule.c:2101:            replace = PyObject_GetAttrString(naive, "replace");
      bson/_cbsonmodule.c:2136:                astimezone = PyObject_GetAttrString(value, "astimezone");
      pymongo/_cmessagemodule.c:49:    error = PyObject_GetAttrString(errors, name);
      pymongo/_cmessagemodule.c:369:    max_bson_size_obj = PyObject_GetAttrString(ctx, "max_bson_size");
      pymongo/_cmessagemodule.c:376:    max_write_batch_size_obj = PyObject_GetAttrString(ctx, "max_write_batch_size");
      pymongo/_cmessagemodule.c:383:    max_message_size_obj = PyObject_GetAttrString(ctx, "max_message_size");
      pymongo/_cmessagemodule.c:670:    max_bson_size_obj = PyObject_GetAttrString(ctx, "max_bson_size");
      pymongo/_cmessagemodule.c:682:    max_write_batch_size_obj = PyObject_GetAttrString(ctx, "max_write_batch_size");
      pymongo/_cmessagemodule.c:692:    max_split_size_obj = PyObject_GetAttrString(ctx, "max_split_size");
      pymongo/_cmessagemodule.c:965:    c_api_object = PyObject_GetAttrString(_cbson, "_C_API");
      

            Assignee:
            iris.ho@mongodb.com Iris Ho
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: