Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1228

Improve performance of WT_SESSION::open_cursor

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.6.4, 3.7.3, WT3.1.0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Engines
    • Storage 2017-03-27, Storage 2017-04-17, Storage 2017-05-08, Storage 2017-05-29, Storage 2017-06-19, Storage 2017-07-10, Storage 2017-07-31, Storage 2017-08-21, Storage 2017-09-11, Storage 2017-10-02, Storage 2017-10-23, Storage 2017-11-13, Storage 2017-12-04, Storage 2017-12-18, Storage 2018-01-01, Storage 2018-01-15, Storage 2018-01-29, Storage 2018-02-12, Storage 2018-02-26

      MongoDB currently maintains a cache of cursors for each session because it improves performance for some workloads. Let's re-run the workloads we used to tune the MongoDB cursor cache with caching disabled (i.e., releaseCursor always closes the cursor rather than adding it to the cache). Assuming that is slower than the default behavior, measure where the time is going and try to improve things in WiredTiger enough that the MongoDB cursor cache is no longer required.

      Original description:

      Opening cursors is quite expensive. I suspect that many applications will open/close cursors with matching configurations regularly.

      It could make sense to keep a cache of "stateless" cursors in each session. It would work like:

      • Each session starts with an empty set of cursors.
      • Each time cursor->close is called, the cursor position and any transactional resources would be released. The cursor would be added to a set of cursors.
      • When a new cursor is opened, we first check the set of available cursors to see if there is one for a matching table/configuration on the list.
      • When a schema level operation is run (drop, rename, etc). We go through and (actually) close the cursors for that table in each of the sessions.

      We need to be a bit careful to:

      • Limit the size of the cursor cache
      • Make searching for cursors efficient (based on table/config). Maybe we could make it a map based on a hash of the name/config strings.

            Assignee:
            donald.anderson@mongodb.com Donald Anderson
            Reporter:
            alexander.gorrod@mongodb.com Alexander Gorrod
            Votes:
            2 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: