-
Type: Question
-
Resolution: Incomplete
-
Priority: Major - P3
-
None
-
Affects Version/s: 3.6.8
-
Component/s: Admin
-
None
Due to a problem with the query in the program, running mongo memory and cpu is high for a while. Long time running, mongo process crashes, resulting in the following error:
2019-09-27T01:16:26.637+0800 I NETWORK [listener] connection accepted from 5.138.16.27:61524 #2276 (1570 connections now open) 2019-09-27T01:16:26.770+0800 F - [PeriodicTaskRunner] Invariant failure (grantedModes & modeMask(mode)) == 0 src/mongo/db/concurrency/lock_manager.cpp 243 2019-09-27T01:16:26.775+0800 F - [PeriodicTaskRunner] ***aborting after invariant() failure 2019-09-27T01:16:27.038+0800 I NETWORK [conn2276] received client metadata from 5.138.16.27:61524 conn2276: \{ driver: { name: "mongo-java-driver", version: "3.8.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "3.10.0-862.14.1.2.h239.eulerosv2r7.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_191-b12" } 2019-09-27T01:16:27.039+0800 I ACCESS [conn2276] authenticate db: $external \{ authenticate: 1, user: "CN=VCM,OU=Clients,O=HW,L=HZ,ST=ZJ,C=CN", mechanism: "MONGODB-X509", $db: "$external" } 2019-09-27T01:16:27.535+0800 I NETWORK [listener] connection accepted from 5.138.16.27:61544 #2277 (1571 connections now open) 2019-09-27T01:16:27.935+0800 I NETWORK [conn2277] received client metadata from 5.138.16.27:61544 conn2277: \{ driver: { name: "mongo-java-driver", version: "3.8.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "3.10.0-862.14.1.2.h239.eulerosv2r7.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_191-b12" } 2019-09-27T01:16:27.936+0800 I ACCESS [conn2277] authenticate db: $external \{ authenticate: 1, user: "CN=VCM,OU=Clients,O=HW,L=HZ,ST=ZJ,C=CN", mechanism: "MONGODB-X509", $db: "$external" } 2019-09-27T01:16:29.138+0800 F - [PeriodicTaskRunner] Got signal: 6 (Aborted). 0x562c99138a91 0x562c99137cc9 0x562c991381ad 0x7f5f93def100 0x7f5f93a50597 0x7f5f93a51c88 0x562c9788f4b2 0x562c98a60f80 0x562c98a60fcf 0x562c98a61181 0x562c98a64d03 0x562c9902a8f3 0x562c9902b9ae 0x562c9902a671 0x7f5f945d2ec0 0x7f5f93de7dc5 0x7f5f93b1550d ----- BEGIN BACKTRACE ----- {"backtrace":[\{"b":"562C96F57000","o":"21E1A91","s":"_ZN5mongo15printStackTraceERSo"},\{"b":"562C96F57000","o":"21E0CC9"},\{"b":"562C96F57000","o":"21E11AD"},\{"b":"7F5F93DE0000","o":"F100"},\{"b":"7F5F93A1B000","o":"35597","s":"gsignal"},\{"b":"7F5F93A1B000","o":"36C88","s":"abort"},\{"b":"562C96F57000","o":"9384B2","s":"_ZN5mongo22invariantFailedWithMsgEPKcS1_S1_j"},\{"b":"562C96F57000","o":"1B09F80","s":"_ZN5mongo8LockHead27migratePartitionedLockHeadsEv"},\{"b":"562C96F57000","o":"1B09FCF","s":"_ZN5mongo11LockManager27_cleanupUnusedLocksInBucketEPNS0_10LockBucketE"},\{"b":"562C96F57000","o":"1B0A181","s":"_ZN5mongo11LockManager18cleanupUnusedLocksEv"},\{"b":"562C96F57000","o":"1B0DD03"},\{"b":"562C96F57000","o":"20D38F3"},\{"b":"562C96F57000","o":"20D49AE"},\{"b":"562C96F57000","o":"20D3671","s":"_ZN5mongo13BackgroundJob7jobBodyEv"},\{"b":"7F5F94514000","o":"BEEC0"},\{"b":"7F5F93DE0000","o":"7DC5"},\{"b":"7F5F93A1B000","o":"FA50D","s":"clone"}],"processInfo":\{ "mongodbVersion" : "3.6.8", "gitVersion" : "nogitversion", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-862.14.1.2.h239.eulerosv2r7.x86_64", "version" : "#1 SMP Mon Jul 22 00:00:00 UTC 2019", "machine" : "x86_64" }, "somap" : [ \{ "b" : "562C96F57000", "elfType" : 3, "buildId" : "6C1188BA90C232DE633202500BCB78AE795FB4B2" }, \{ "b" : "7FFD464FB000", "elfType" : 3, "buildId" : "4FD55B3D8140F8BD093537637EE17ED8809CC581" }, \{ "b" : "7F5F95464000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "33CC61E1C3203B99DB184210060FADB4B582001D" }, \{ "b" : "7F5F94F40000", "path" : "/home/lib/libcrypto.so.1.1", "elfType" : 3, "buildId" : "56F785481886EB3B47E98BE9070DCCBA9805791F" }, \{ "b" : "7F5F94CA2000", "path" : "/home/lib/libssl.so.1.1", "elfType" : 3, "buildId" : "DF7EBC97F30C599502760CEBA6CAADA818A34D19" }, \{ "b" : "7F5F94A9E000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "926DBA873EFA28B2B12B5F42A5D56CC2785E36A4" }, \{ "b" : "7F5F94896000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "9BDBE5FE8313D6AC869AAECB70428E41AF76685F" }, \{ "b" : "7F5F94514000", "path" : "/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "3BCAF1CF6BBCC1C79A0A64B3E996BD365F11C71A" }, \{ "b" : "7F5F94212000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "B5FAA069E09135A3F1486DD8931E01B8976E42A7" }, \{ "b" : "7F5F93FFC000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "3B7E79322B4B032A5AE40B0E7A6C4297059C9FD8" }, \{ "b" : "7F5F93DE0000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "6DE18DE37DCBA5C51745D477FB07C64E2728606D" }, \{ "b" : "7F5F93A1B000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "054CA217723B9959C44A11E212A89A235F19BE61" }, \{ "b" : "7F5F9567E000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "5460CA1B85820630FE5171A049178C9774B1BB12" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x562c99138a91] mongod(+0x21E0CC9) [0x562c99137cc9] mongod(+0x21E11AD) [0x562c991381ad] libpthread.so.0(+0xF100) [0x7f5f93def100] libc.so.6(gsignal+0x37) [0x7f5f93a50597] libc.so.6(abort+0x148) [0x7f5f93a51c88] mongod(_ZN5mongo22invariantFailedWithMsgEPKcS1_S1_j+0x0) [0x562c9788f4b2] mongod(_ZN5mongo8LockHead27migratePartitionedLockHeadsEv+0x300) [0x562c98a60f80] mongod(_ZN5mongo11LockManager27_cleanupUnusedLocksInBucketEPNS0_10LockBucketE+0x2F) [0x562c98a60fcf] mongod(_ZN5mongo11LockManager18cleanupUnusedLocksEv+0x31) [0x562c98a61181] mongod(+0x1B0DD03) [0x562c98a64d03] mongod(+0x20D38F3) [0x562c9902a8f3] mongod(+0x20D49AE) [0x562c9902b9ae] mongod(_ZN5mongo13BackgroundJob7jobBodyEv+0x131) [0x562c9902a671] libstdc++.so.6(+0xBEEC0) [0x7f5f945d2ec0] libpthread.so.0(+0x7DC5) [0x7f5f93de7dc5] libc.so.6(clone+0x6D) [0x7f5f93b1550d] ----- END BACKTRACE -----