Heap profiler shows memory usage increasing over time on the secondary. The profile is characteristic of a buffer that grows by doubling each time.
Allocated here:
heapProfile stack155: { 0: "tcmalloc::allocate_full_cpp_throw_oom", 1: "absl::container_internal::raw_hash_set<absl::container_internal::FlatHashMapPolicy<std::__cxx11::basic_string<char, std::char_traits<char>, std::alloc...", 2: "mongo::Top::record", 3: "mongo::AutoStatsTracker::~AutoStatsTracker", 4: "0x5623f753bc58", 5: "0x5623f753caae", 6: "mongo::createCollectionForApplyOps", 7: "0x5623f74c7cc6", 8: "0x5623f74c7ff2", 9: "mongo::repl::applyCommand_inlock", 10: "0x5623f6fa06a4", 11: "mongo::repl::SyncTail::syncApply", 12: "mongo::repl::multiSyncApply", 13: "std::_Function_handler<mongo::Status ", 14: "0x5623f6f9fb5d", 15: "mongo::ThreadPool::_doOneTask", 16: "mongo::ThreadPool::_consumeTasks", 17: "mongo::ThreadPool::_workerThreadBody", 18: "0x5623f87e0a1f", 19: "0x7fa6cc9d26db", 20: "clone" }
Reproduced by a high rate of collection creates and drops using mapReduce directed to the primary:
function repro() { db.c.insert({}) nthreads = 10 threads = [] for (var t=0; t<nthreads; t++) { thread = new ScopedThread(function(t) { count = 1000000 for (var i=0; i<count; i++) { db.c.mapReduce( function () {}, function () {}, {out: {merge: "d"}} ) } }, t) threads.push(thread) thread.start() } for (var t = 0; t < nthreads; t++) threads[t].join() }
In this repro the mapReduce commands are done on the primary, but the memory increase only seems to happen on the secondary.
I don't know whether it is specific to collections created and dropped by mapReduce, and I don't know if it's a 4.2 regression (but I note there appear to have been some considerable code changes in this area).
- is related to
-
SERVER-45855 The 'Top' command managing its own data in the UsageMap is error prone which causes memory leaks
- Backlog
- related to
-
SERVER-46196 Failed collection creation leaves an orphaned entry inside Top data structure
- Closed
-
SERVER-44230 Move top_drop.js from noPassthrough to Core
- Closed