In SERVER-37795 we observed excess allocated memory associated with cursors due to WT "modify" operations in 3.6.8. This should not occur in 4.0 because improved cursor caching should limit the number of cursors, but we still see excess allocated memory:
The excess memory in 4.0.3 is more than double that of 3.6.8. Note that the rate of updates is somewhat higher in 4.0.3 than 3.6.8 (probably SERVER-36221), but that doesn't appear to be enough larger to account for the >2x larger excess memory allocated in 4.0.3.
Possibly related: "cached cursor count" seems to grow indefinitely, although we only have that information for the oplog collection and not globally.
Notes:
- Verified that the memory is associated with cursors by disabling cursor caching in 3.6.8 and the excess memory disappeared.
- Uesed the heap profiler to confirmed that the excess allocated memory is coming from the same allocation sites in 4.0.3 as 3.6.8.
Repro:
load("/home/bdlucas/mongodb/git/mongo/jstests/libs/parallelTester.js") size = 1000000 bunch = 100 // start a bunch of threads inserting to a bunch of collections // in 3.6.8 this creates a big bunch of cursors (bunch^2, maybe) // but not in 4.0.3 threads = [] for (var t=0; t<bunch; t++) { thread = new ScopedThread(function(t) { c = db['c' + t] while (true) c.insert({}) }, t) threads.push(thread) thread.start() } // while that's going on do some small updates to large documents // this uses WT "modify" triggering the issue in SERVER-37795 in 3.6.8 // that shouldn't happen in 4.0.3 but it does for (i = 0; i < 100; i++) { c = db['c' + i] x = 'x'.repeat(size) c.insert({_id: 0, x: x}) } while (true) { for (i = 0; i < 100; i++) { c = db['c' + i] c.update({_id: 0}, {$inc: {i: 1}}) c.find({_id: 0}).itcount() } }
- duplicates
-
WT-4418 Don't keep key/value memory buffers allocated for cached cursors
- Closed
- is related to
-
SERVER-37795 WiredTiger again uses way more memory than instructed by cache limit in 3.6
- Closed