-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines
-
(copied to CRM)
-
Storage 2017-03-27, Storage 2017-04-17, Storage 2017-05-08, Storage 2017-05-29, Storage 2017-06-19, Storage 2017-07-10, Storage 2017-07-31, Storage 2017-08-21, Storage 2017-09-11, Storage 2017-10-02, Storage 2017-10-23, Storage 2017-11-13, Storage 2017-12-04, Storage 2017-12-18, Storage 2018-01-01, Storage 2018-01-15, Storage 2018-01-29, Storage 2018-02-12, Storage 2018-02-26
MongoDB currently maintains a cache of cursors for each session because it improves performance for some workloads. Let's re-run the workloads we used to tune the MongoDB cursor cache with caching disabled (i.e., releaseCursor always closes the cursor rather than adding it to the cache). Assuming that is slower than the default behavior, measure where the time is going and try to improve things in WiredTiger enough that the MongoDB cursor cache is no longer required.
Original description:
Opening cursors is quite expensive. I suspect that many applications will open/close cursors with matching configurations regularly.
It could make sense to keep a cache of "stateless" cursors in each session. It would work like:
- Each session starts with an empty set of cursors.
- Each time cursor->close is called, the cursor position and any transactional resources would be released. The cursor would be added to a set of cursors.
- When a new cursor is opened, we first check the set of available cursors to see if there is one for a matching table/configuration on the list.
- When a schema level operation is run (drop, rename, etc). We go through and (actually) close the cursors for that table in each of the sessions.
We need to be a bit careful to:
- Limit the size of the cursor cache
- Make searching for cursors efficient (based on table/config). Maybe we could make it a map based on a hash of the name/config strings.
- is duplicated by
-
SERVER-27909 OOM and possible memory leaks
- Closed
-
SERVER-27345 reclaim inactive WT sessions on calling closeAllCursors
- Closed
-
WT-3813 Cursor caching: implement/test basic caching
- Closed
- related to
-
SERVER-28172 Renaming collection causes permanent performance issue on secondary
- Closed
-
WT-3882 Cursor caching: add per-session hash table for cached cursors
- Closed
-
WT-3883 Cursor caching: clean up stale cursors in session
- Closed
-
WT-3935 Enable cursor caching by default in WiredTiger.
- Closed