-
Type: Bug
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: 4.4.0
-
Component/s: Stability
-
Service Arch
-
ALL
-
Security 2020-11-30
-
(copied to CRM)
Running mondodb 4.4.0 on Ubuntu 20.04.1 one of our replicas crashed with an assertion. How do we diagnose this?
{"t":{"$date":"2020-11-10T14:07:13.371+00:00"},"s":"F", "c":"-", "id":23079, "ctx":"monitoring-keys-for-HMAC","msg":"Invariant failure","attr":{"expr":"opCtx != nullptr && _opCtx == nullptr","file":"src/mongo/db/client.cpp","line":126}} {"t":{"$date":"2020-11-10T14:07:13.371+00:00"},"s":"F", "c":"-", "id":23080, "ctx":"monitoring-keys-for-HMAC","msg":"\n\n***aborting after invariant() failure\n\n"} {"t":{"$date":"2020-11-10T14:07:13.373+00:00"},"s":"F", "c":"CONTROL", "id":4757800, "ctx":"monitoring-keys-for-HMAC","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31431, "ctx":"monitoring-keys-for-HMAC","msg":"BACKTRACE: {bt}","attr":{"bt":{"backtrace":[{"a":"556342B47811","b":"55633FE8C000","o":"2CBB811","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1E1"},{"a":"556342B48EB9","b":"55633FE8C000","o":"2CBCEB9","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"556342B466A6","b":"55633FE8C000","o":"2CBA6A6","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP9siginfo_tPv","s+":"66"},{"a":"7F8DEAD7C3C0","b":"7F8DEAD67000","o":"153C0","s":"funlockfile","s+":"60"},{"a":"7F8DEABBB18B","b":"7F8DEAB75000","o":"4618B","s":"gsignal","s+":"CB"},{"a":"7F8DEAB9A859","b":"7F8DEAB75000","o":"25859","s":"abort","s+":"12B"},{"a":"556340D8FBE1","b":"55633FE8C000","o":"F03BE1","s":"_ZN5mongo15invariantFailedEPKcS1_j","s+":"12C"},{"a":"556340D65270","b":"55633FE8C000","o":"ED9270","s":"_ZN5mongo6Client19setOperationContextEPNS_16OperationContextE.cold.135","s+":"18"},{"a":"556342A15BE9","b":"55633FE8C000","o":"2B89BE9","s":"_ZN5mongo14ServiceContext20makeOperationContextEPNS_6ClientE","s+":"129"},{"a":"556342A0A697","b":"55633FE8C000","o":"2B7E697","s":"_ZN5mongo6Client20makeOperationContextEv","s+":"27"},{"a":"556342809712","b":"55633FE8C000","o":"297D712","s":"_ZN5mongo21KeysCollectionManager14PeriodicRunner18_doPeriodicRefreshEPNS_14ServiceContextENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_8DurationISt5ratioILl1ELl1000EEEE","s+":"162"},{"a":"55634280B6C3","b":"55633FE8C000","o":"297F6C3","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_21KeysCollectionManager14PeriodicRunner5startEPNS3_14ServiceContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS3_8DurationISt5ratioILl1ELl1000EEEEEUlvE_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"93"},{"a":"556342CF78FF","b":"55633FE8C000","o":"2E6B8FF","s":"execute_native_thread_routine","s+":"F"},{"a":"7F8DEAD70609","b":"7F8DEAD67000","o":"9609","s":"start_thread","s+":"D9"},{"a":"7F8DEAC97293","b":"7F8DEAB75000","o":"122293","s":"clone","s+":"43"}],"processInfo":{"mongodbVersion":"4.4.0","gitVersion":"563487e100c4215e2dce98d0af2a6a5a2d67c5cf","compiledModules":[],"uname":{"sysname":"Linux","release":"5.4.0-1025-aws","version":"#25-Ubuntu SMP Fri Sep 11 09:37:24 UTC 2020","machine":"x86_64"},"somap":[{"b":"55633FE8C000","elfType":3,"buildId":"77B6A138746C90015067F12B963853BD51DAA5A6"},{"b":"7F8DEAD67000","path":"/lib/x86_64-linux-gnu/libpthread.so.0","elfType":3,"buildId":"4FC5FC33F4429136A494C640B113D76F610E4ABC"},{"b":"7F8DEAB75000","path":"/lib/x86_64-linux-gnu/libc.so.6","elfType":3,"buildId":"F3FF3FDA80B817C464A56EED59FF09DC864EAEB0"}]}}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556342B47811","b":"55633FE8C000","o":"2CBB811","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1E1"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556342B48EB9","b":"55633FE8C000","o":"2CBCEB9","s":"_ZN5mongo15printStackTraceEv","s+":"29"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556342B466A6","b":"55633FE8C000","o":"2CBA6A6","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP9siginfo_tPv","s+":"66"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"7F8DEAD7C3C0","b":"7F8DEAD67000","o":"153C0","s":"funlockfile","s+":"60"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"7F8DEABBB18B","b":"7F8DEAB75000","o":"4618B","s":"gsignal","s+":"CB"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"7F8DEAB9A859","b":"7F8DEAB75000","o":"25859","s":"abort","s+":"12B"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556340D8FBE1","b":"55633FE8C000","o":"F03BE1","s":"_ZN5mongo15invariantFailedEPKcS1_j","s+":"12C"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556340D65270","b":"55633FE8C000","o":"ED9270","s":"_ZN5mongo6Client19setOperationContextEPNS_16OperationContextE.cold.135","s+":"18"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556342A15BE9","b":"55633FE8C000","o":"2B89BE9","s":"_ZN5mongo14ServiceContext20makeOperationContextEPNS_6ClientE","s+":"129"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556342A0A697","b":"55633FE8C000","o":"2B7E697","s":"_ZN5mongo6Client20makeOperationContextEv","s+":"27"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556342809712","b":"55633FE8C000","o":"297D712","s":"_ZN5mongo21KeysCollectionManager14PeriodicRunner18_doPeriodicRefreshEPNS_14ServiceContextENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_8DurationISt5ratioILl1ELl1000EEEE","s+":"162"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"55634280B6C3","b":"55633FE8C000","o":"297F6C3","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_21KeysCollectionManager14PeriodicRunner5startEPNS3_14ServiceContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS3_8DurationISt5ratioILl1ELl1000EEEEEUlvE_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"93"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"556342CF78FF","b":"55633FE8C000","o":"2E6B8FF","s":"execute_native_thread_routine","s+":"F"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"7F8DEAD70609","b":"7F8DEAD67000","o":"9609","s":"start_thread","s+":"D9"}}} {"t":{"$date":"2020-11-10T14:07:13.480+00:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"monitoring-keys-for-HMAC","msg":" Frame: {frame}","attr":{"frame":{"a":"7F8DEAC97293","b":"7F8DEAB75000","o":"122293","s":"clone","s+":"43"}}}{"t":{"$date":"2020-11-10T14:13:25.412+00:00"},"s":"W", "c":"CONTROL", "id":20698, "ctx":"main","msg":"***** SERVER RESTARTED *****","tags":["startupWarnings"]}
- duplicates
-
SERVER-53537 MongoDB crashed after state change
- Closed
-
SERVER-53566 Investigate and reproduce "opCtx != nullptr && _opCtx == nullptr" invariant
- Closed
-
SERVER-53482 Invariant failure: opCtx != nullptr && _opCtx == nullptr in src/mongo/db/client.cpp, line: 126
- Closed
- is duplicated by
-
SERVER-53747 replica set down after setFeatureCompatibilityVersion 4.4
- Closed
- is related to
-
SERVER-53537 MongoDB crashed after state change
- Closed
-
SERVER-53566 Investigate and reproduce "opCtx != nullptr && _opCtx == nullptr" invariant
- Closed
-
SERVER-53482 Invariant failure: opCtx != nullptr && _opCtx == nullptr in src/mongo/db/client.cpp, line: 126
- Closed
-
SERVER-53747 replica set down after setFeatureCompatibilityVersion 4.4
- Closed
-
SERVER-54445 Compact in Secondary ReplicaSet 4.4.0 change member state
- Closed