Heap profiler shows memory usage increasing over time on the secondary. The profile is characteristic of a buffer that grows by doubling each time.
Allocated here:
heapProfile stack155: { 0: "tcmalloc::allocate_full_cpp_throw_oom", 1: "absl::container_internal::raw_hash_set<absl::container_internal::FlatHashMapPolicy<std::__cxx11::basic_string<char, std::char_traits<char>, std::alloc...", 2: "mongo::Top::record", 3: "mongo::AutoStatsTracker::~AutoStatsTracker", 4: "0x5623f753bc58", 5: "0x5623f753caae", 6: "mongo::createCollectionForApplyOps", 7: "0x5623f74c7cc6", 8: "0x5623f74c7ff2", 9: "mongo::repl::applyCommand_inlock", 10: "0x5623f6fa06a4", 11: "mongo::repl::SyncTail::syncApply", 12: "mongo::repl::multiSyncApply", 13: "std::_Function_handler<mongo::Status ", 14: "0x5623f6f9fb5d", 15: "mongo::ThreadPool::_doOneTask", 16: "mongo::ThreadPool::_consumeTasks", 17: "mongo::ThreadPool::_workerThreadBody", 18: "0x5623f87e0a1f", 19: "0x7fa6cc9d26db", 20: "clone" }
Reproduced by a high rate of collection creates and drops using mapReduce directed to the primary:
function repro() {
db.c.insert({})
nthreads = 10
threads = []
for (var t=0; t<nthreads; t++) {
thread = new ScopedThread(function(t) {
count = 1000000
for (var i=0; i<count; i++) {
db.c.mapReduce(
function () {},
function () {},
{out: {merge: "d"}}
)
}
}, t)
threads.push(thread)
thread.start()
}
for (var t = 0; t < nthreads; t++)
threads[t].join()
}
In this repro the mapReduce commands are done on the primary, but the memory increase only seems to happen on the secondary.
I don't know whether it is specific to collections created and dropped by mapReduce, and I don't know if it's a 4.2 regression (but I note there appear to have been some considerable code changes in this area).
- is related to
-
SERVER-45855 The 'Top' command managing its own data in the UsageMap is error prone which causes memory leaks
-
- Backlog
-
- related to
-
SERVER-46196 Failed collection creation leaves an orphaned entry inside Top data structure
-
- Closed
-
-
SERVER-44230 Move top_drop.js from noPassthrough to Core
-
- Closed
-