-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: 4.4.4, 4.4.5
-
Component/s: Upgrade/Downgrade
-
None
-
ALL
-
Hi, i have a replica set of 3 nodes, with 4.2.13 version, i upgraded the secondary to 4.4.4 and started it. I see that this node becames SECONDARY and can comunicate with PRIMARY.
Then i go to the primary and i do rs.stepDown() to install the new version. The upgraded node becames PRIMARY and then dies.
Log of node 4.4.4 becaming primary:
{"t":{"$date":"2021-04-15T13:16:53.211+01:00"},"s":"I", "c":"COMMAND", "id":21581, "ctx":"conn5","msg":"Received replSetStepUp request"}
{"t":{"$date":"2021-04-15T13:16:53.211+01:00"},"s":"I", "c":"ELECTION", "id":4615661, "ctx":"conn5","msg":"Starting an election due to step up request"}
{"t":{"$date":"2021-04-15T13:16:53.211+01:00"},"s":"I", "c":"ELECTION", "id":21437, "ctx":"conn5","msg":"Skipping dry run and running for election","attr":{"newTerm":23}}
{"t":{"$date":"2021-04-15T13:16:53.212+01:00"},"s":"I", "c":"REPL", "id":21752, "ctx":"ReplCoord-1","msg":"Scheduling remote command request","attr":{"context":"vote request","request":"RemoteCommand 202 -- target:127.0.0.1:27017 db:admin cmd:{ replSetRequestVotes: 1, setName: \"sigo\", dryRun: false, term: 23, candidateIndex: 1, configVersion: 1396130, lastCommittedOp: { ts: Timestamp(1618489005, 1), t: 22 } }"}}
{"t":{"$date":"2021-04-15T13:16:53.213+01:00"},"s":"I", "c":"REPL", "id":21752, "ctx":"ReplCoord-1","msg":"Scheduling remote command request","attr":{"context":"vote request","request":"RemoteCommand 203 -- target:127.0.0.3:27018 db:admin cmd:{ replSetRequestVotes: 1, setName: \"sigo\", dryRun: false, term: 23, candidateIndex: 1, configVersion: 1396130, lastCommittedOp: { ts: Timestamp(1618489005, 1), t: 22 } }"}}
{"t":{"$date":"2021-04-15T13:16:53.213+01:00"},"s":"I", "c":"ELECTION", "id":51799, "ctx":"ReplCoord-0","msg":"VoteRequester processResponse","attr":{"term":23,"dryRun":false,"vote":"no","from":"127.0.0.3:27018","reason":"can see a healthy primary (127.0.0.1:27017) of equal or greater priority","message":{"term":23,"voteGranted":false,"reason":"can see a healthy primary (127.0.0.1:27017) of equal or greater priority","ok":1.0}}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"ELECTION", "id":51799, "ctx":"ReplCoord-2","msg":"VoteRequester processResponse","attr":{"term":23,"dryRun":false,"vote":"yes","from":"127.0.0.1:27017","message":{"term":23,"voteGranted":true,"reason":"","ok":1.0,"$clusterTime":{"clusterTime":{"$timestamp":{"t":1618489005,"i":1}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"operationTime":{"$timestamp":{"t":1618489005,"i":1}}}}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"ELECTION", "id":21450, "ctx":"ReplCoord-2","msg":"Election succeeded, assuming primary role","attr":{"term":23}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21358, "ctx":"ReplCoord-2","msg":"Replica set state transition","attr":{"newState":"PRIMARY","oldState":"SECONDARY"}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21106, "ctx":"ReplCoord-2","msg":"Resetting sync source to empty","attr":{"previousSyncSource":"127.0.0.1:27017"}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21359, "ctx":"ReplCoord-2","msg":"Entering primary catch-up mode"}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21215, "ctx":"ReplCoord-5","msg":"Member is in new state","attr":{"hostAndPort":"127.0.0.1:27017","newState":"SECONDARY"}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21364, "ctx":"ReplCoord-5","msg":"Caught up to the latest optime known via heartbeats after becoming primary","attr":{"targetOpTime":{"ts":{"$timestamp":{"t":1618489005,"i":1}},"t":22},"myLastApplied":{"ts":{"$timestamp":{"t":1618489005,"i":1}},"t":22}}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21363, "ctx":"ReplCoord-5","msg":"Exited primary catch-up mode"}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21107, "ctx":"ReplCoord-5","msg":"Stopping replication producer"}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"NETWORK", "id":20125, "ctx":"ReplCoordExtern-0","msg":"DBClientConnection failed to receive message","attr":{"connString":"127.0.0.1:27017","error":"HostUnreachable: Connection closed by peer"}}
{"t":{"$date":"2021-04-15T13:16:53.215+01:00"},"s":"I", "c":"REPL", "id":21275, "ctx":"ReplCoordExtern-0","msg":"Recreating cursor for oplog fetcher due to error","attr":{"lastOpTimeFetched":{"ts":{"$timestamp":{"t":1618489005,"i":1}},"t":22},"attemptsRemaining":1,"error":"HostUnreachable: Error while getting the next batch in the oplog fetcher :: caused by :: dbclient error communicating with server 127.0.0.1:27017 :: caused by :: Connection closed by peer"}}
{"t":{"$date":"2021-04-15T13:16:53.214+01:00"},"s":"I", "c":"REPL", "id":21239, "ctx":"ReplBatcher","msg":"Oplog buffer has been drained","attr":{"term":23}}
{"t":{"$date":"2021-04-15T13:16:53.215+01:00"},"s":"I", "c":"REPL", "id":21343, "ctx":"RstlKillOpThread","msg":"Starting to kill user operations"}
{"t":{"$date":"2021-04-15T13:16:53.215+01:00"},"s":"I", "c":"REPL", "id":21344, "ctx":"RstlKillOpThread","msg":"Stopped killing user operations"}
{"t":{"$date":"2021-04-15T13:16:53.215+01:00"},"s":"I", "c":"REPL", "id":21340, "ctx":"RstlKillOpThread","msg":"State transition ops metrics","attr":{"metrics":{"lastStateTransition":"stepUp","userOpsKilled":0,"userOpsRunning":0}}}
{"t":{"$date":"2021-04-15T13:16:53.215+01:00"},"s":"I", "c":"STORAGE", "id":20657, "ctx":"OplogApplier-0","msg":"IndexBuildsCoordinator::onStepUp - this node is stepping up to primary"}
{"t":{"$date":"2021-04-15T13:16:53.216+01:00"},"s":"I", "c":"REPL", "id":21331, "ctx":"OplogApplier-0","msg":"Transition to primary complete; database writes are now permitted"}
{"t":{"$date":"2021-04-15T13:16:53.216+01:00"},"s":"I", "c":"NETWORK", "id":20120, "ctx":"ReplCoordExtern-0","msg":"Trying to reconnect","attr":{"connString":"127.0.0.1:27017 failed"}}
{"t":{"$date":"2021-04-15T13:16:53.216+01:00"},"s":"I", "c":"NETWORK", "id":20121, "ctx":"ReplCoordExtern-0","msg":"Reconnect attempt failed","attr":{"connString":"127.0.0.1:27017 failed","error":""}}
{"t":{"$date":"2021-04-15T13:16:53.216+01:00"},"s":"I", "c":"-", "id":4615600, "ctx":"ReplCoordExtern-0","msg":"caught exception in destructor","attr":{"exception":"socket exception [CONNECT_ERROR] server [socket exception [FAILED_STATE] server [127.0.0.1:27017 failed]]","function":"kill"}}
{"t":{"$date":"2021-04-15T13:16:53.216+01:00"},"s":"I", "c":"REPL", "id":21094, "ctx":"BackgroundSync","msg":"Replication producer stopped after oplog fetcher finished returning a batch from our sync source. Abandoning this batch of oplog entries and re-evaluating our sync source"}
{"t":{"$date":"2021-04-15T13:16:55.213+01:00"},"s":"F", "c":"-", "id":23079, "ctx":"conn6","msg":"Invariant failure","attr":{"expr":"opTime.opTime.isNull() || opTime.wallTime > Date_t()","file":"src/mongo/db/repl/member_data.cpp","line":175}}
{"t":{"$date":"2021-04-15T13:16:55.213+01:00"},"s":"F", "c":"-", "id":23080, "ctx":"conn6","msg":"\n\n***aborting after invariant() failure\n\n"}
{"t":{"$date":"2021-04-15T13:16:55.213+01:00"},"s":"F", "c":"CONTROL", "id":4757800, "ctx":"conn6","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}
{"t":{"$date":"2021-04-15T13:16:55.213+01:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:37538","connectionId":14,"connectionCount":4}}
{"t":{"$date":"2021-04-15T13:16:55.213+01:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn14","msg":"client metadata","attr":{"remote":"127.0.0.1:37538","client":"conn14","doc":{"driver":{"name":"NetworkInterfaceTL","version":"4.2.13"},"os":{"type":"Linux","name":"Red Hat Enterprise Linux Server release 6.8 (Santiago)","architecture":"x86_64","version":"Kernel 2.6.32-642.11.1.el6.x86_64"}}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31431, "ctx":"conn6","msg":"BACKTRACE: {bt}","attr":{"bt":{"backtrace":[{"a":"7FEE165E5101","b":"7FEE138DC000","o":"2D09101","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1E1"},{"a":"7FEE165E6739","b":"7FEE138DC000","o":"2D0A739","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"7FEE165E3F96","b":"7FEE138DC000","o":"2D07F96","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP7siginfoPv","s+":"66"},{"a":"7FEE11F257E0","b":"7FEE11F16000","o":"F7E0","s":"_L_unlock_16","s+":"2D"},{"a":"7FEE11BB45E5","b":"7FEE11B82000","o":"325E5","s":"gsignal","s+":"35"},{"a":"7FEE11BB5DC5","b":"7FEE11B82000","o":"33DC5","s":"abort","s+":"175"},{"a":"7FEE147F6AB3","b":"7FEE138DC000","o":"F1AAB3","s":"_ZN5mongo15invariantFailedEPKcS1_j","s+":"12C"},{"a":"7FEE145489D0","b":"7FEE138DC000","o":"C6C9D0","s":"_ZN5mongo4repl10MemberData35advanceLastDurableOpTimeAndWallTimeENS0_17OpTimeAndWallTimeENS_6Date_tE.cold.338","s+":"18"},{"a":"7FEE14C6CC33","b":"7FEE138DC000","o":"1390C33","s":"_ZN5mongo4repl19TopologyCoordinator13setLastOptimeERKNS0_18UpdatePositionArgs10UpdateInfoENS_6Date_tEPx","s+":"2E3"},{"a":"7FEE14BBA0C5","b":"7FEE138DC000","o":"12DE0C5","s":"_ZN5mongo4repl26ReplicationCoordinatorImpl14_setLastOptimeENS_8WithLockERKNS0_18UpdatePositionArgs10UpdateInfoEPx","s+":"55"},{"a":"7FEE14BBA22F","b":"7FEE138DC000","o":"12DE22F","s":"_ZN5mongo4repl26ReplicationCoordinatorImpl28processReplSetUpdatePositionERKNS0_18UpdatePositionArgsEPx","s+":"AF"},{"a":"7FEE14B74777","b":"7FEE138DC000","o":"1298777","s":"_ZN5mongo4repl24CmdReplSetUpdatePosition3runEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_7BSONObjERNS_14BSONObjBuilderE","s+":"167"},{"a":"7FEE155500CA","b":"7FEE138DC000","o":"1C740CA","s":"_ZN5mongo37BasicCommandWithReplyBuilderInterface10Invocation3runEPNS_16OperationContextEPNS_3rpc21ReplyBuilderInterfaceE","s+":"11A"},{"a":"7FEE1554A60F","b":"7FEE138DC000","o":"1C6E60F","s":"_ZN5mongo14CommandHelpers20runCommandInvocationEPNS_16OperationContextERKNS_12OpMsgRequestEPNS_17CommandInvocationEPNS_3rpc21ReplyBuilderInterfaceE","s+":"7F"},{"a":"7FEE14CF1AC5","b":"7FEE138DC000","o":"1415AC5","s":"_ZN5mongo12_GLOBAL__N_114runCommandImplEPNS_16OperationContextEPNS_17CommandInvocationERKNS_12OpMsgRequestEPNS_3rpc21ReplyBuilderInterfaceENS_11LogicalTimeERKNS_23ServiceEntryPointCommon5HooksEPNS_14BSONObjBuilderERKNS_30OperationSessionInfoFromClientE","s+":"135"},{"a":"7FEE14CF4929","b":"7FEE138DC000","o":"1418929","s":"_ZN5mongo12_GLOBAL__N_119execCommandDatabaseEPNS_16OperationContextEPNS_7CommandERKNS_12OpMsgRequestEPNS_3rpc21ReplyBuilderInterfaceERKNS_23ServiceEntryPointCommon5HooksE","s+":"11B9"},{"a":"7FEE14CF5C10","b":"7FEE138DC000","o":"1419C10","s":"_ZN5mongo12_GLOBAL__N_116receivedCommandsEPNS_16OperationContextERKNS_7MessageERKNS_23ServiceEntryPointCommon5HooksE","s+":"650"},{"a":"7FEE14CF689D","b":"7FEE138DC000","o":"141A89D","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE","s+":"50D"},{"a":"7FEE14CE434C","b":"7FEE138DC000","o":"140834C","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE","s+":"3C"},{"a":"7FEE14CEEDD5","b":"7FEE138DC000","o":"1412DD5","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE","s+":"F5"},{"a":"7FEE14CECAE5","b":"7FEE138DC000","o":"1410AE5","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE","s+":"125"},{"a":"7FEE14CEDA76","b":"7FEE138DC000","o":"1411A76","s":"_ZNSt17_Function_handlerIFvvEZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS2_11ThreadGuardENS1_9transport15ServiceExecutor13ScheduleFlagsENS4_23ServiceExecutorTaskNameENS2_9OwnershipEEUlvE_E9_M_invokeERKSt9_Any_data","s+":"56"},{"a":"7FEE15FABBE2","b":"7FEE138DC000","o":"26CFBE2","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE","s+":"182"},{"a":"7FEE14CEBEFB","b":"7FEE138DC000","o":"140FEFB","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE","s+":"DB"},{"a":"7FEE14CED25D","b":"7FEE138DC000","o":"141125D","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE","s+":"6AD"},{"a":"7FEE14CED340","b":"7FEE138DC000","o":"1411340","s":"_ZN5mongo14future_details4callIRZNS_19ServiceStateMachine14_sourceMessageENS2_11ThreadGuardEEUlNS_10StatusWithINS_7MessageEEEE0_S6_EEDaOT_OT0_","s+":"60"},{"a":"7FEE14CED5E5","b":"7FEE138DC000","o":"14115E5","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE","s+":"145"},{"a":"7FEE14CECB5A","b":"7FEE138DC000","o":"1410B5A","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE","s+":"19A"},{"a":"7FEE14CEDA76","b":"7FEE138DC000","o":"1411A76","s":"_ZNSt17_Function_handlerIFvvEZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS2_11ThreadGuardENS1_9transport15ServiceExecutor13ScheduleFlagsENS4_23ServiceExecutorTaskNameENS2_9OwnershipEEUlvE_E9_M_invokeERKSt9_Any_data","s+":"56"},{"a":"7FEE15FAC248","b":"7FEE138DC000","o":"26D0248","s":"_ZNSt17_Function_handlerIFvvEZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIS0_ENS2_15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameEEUlvE0_E9_M_invokeERKSt9_Any_data","s+":"B8"},{"a":"7FEE162FC706","b":"7FEE138DC000","o":"2A20706","s":"_ZNSt17_Function_handlerIFvvEZN5mongo25launchServiceWorkerThreadESt8functionIS0_EEUlvE1_E9_M_invokeERKSt9_Any_data","s+":"56"},{"a":"7FEE162FC774","b":"7FEE138DC000","o":"2A20774","s":"_ZN5mongo12_GLOBAL__N_17runFuncEPv","s+":"14"},{"a":"7FEE11F1DAA1","b":"7FEE11F16000","o":"7AA1","s":"start_thread","s+":"D1"},{"a":"7FEE11C6AAAD","b":"7FEE11B82000","o":"E8AAD","s":"clone","s+":"6D"}],"processInfo":{"mongodbVersion":"4.4.4","gitVersion":"8db30a63db1a9d84bdcad0c83369623f708e0397","compiledModules":[],"uname":{"sysname":"Linux","release":"2.6.32-642.3.1.el6.x86_64","version":"#1 SMP Sun Jun 26 18:16:44 EDT 2016","machine":"x86_64"},"somap":[{"b":"7FEE138DC000","elfType":3,"buildId":"58BE9325EB37EA25F1A25D0963EAF5DB9C4F4A67"}]}}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE165E5101","b":"7FEE138DC000","o":"2D09101","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1E1"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE165E6739","b":"7FEE138DC000","o":"2D0A739","s":"_ZN5mongo15printStackTraceEv","s+":"29"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE165E3F96","b":"7FEE138DC000","o":"2D07F96","s":"_ZN5mongo12_GLOBAL__N_116abruptQuitActionEiP7siginfoPv","s+":"66"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE11F257E0","b":"7FEE11F16000","o":"F7E0","s":"_L_unlock_16","s+":"2D"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE11BB45E5","b":"7FEE11B82000","o":"325E5","s":"gsignal","s+":"35"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE11BB5DC5","b":"7FEE11B82000","o":"33DC5","s":"abort","s+":"175"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE147F6AB3","b":"7FEE138DC000","o":"F1AAB3","s":"_ZN5mongo15invariantFailedEPKcS1_j","s+":"12C"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE145489D0","b":"7FEE138DC000","o":"C6C9D0","s":"_ZN5mongo4repl10MemberData35advanceLastDurableOpTimeAndWallTimeENS0_17OpTimeAndWallTimeENS_6Date_tE.cold.338","s+":"18"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14C6CC33","b":"7FEE138DC000","o":"1390C33","s":"_ZN5mongo4repl19TopologyCoordinator13setLastOptimeERKNS0_18UpdatePositionArgs10UpdateInfoENS_6Date_tEPx","s+":"2E3"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14BBA0C5","b":"7FEE138DC000","o":"12DE0C5","s":"_ZN5mongo4repl26ReplicationCoordinatorImpl14_setLastOptimeENS_8WithLockERKNS0_18UpdatePositionArgs10UpdateInfoEPx","s+":"55"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14BBA22F","b":"7FEE138DC000","o":"12DE22F","s":"_ZN5mongo4repl26ReplicationCoordinatorImpl28processReplSetUpdatePositionERKNS0_18UpdatePositionArgsEPx","s+":"AF"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14B74777","b":"7FEE138DC000","o":"1298777","s":"_ZN5mongo4repl24CmdReplSetUpdatePosition3runEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_7BSONObjERNS_14BSONObjBuilderE","s+":"167"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE155500CA","b":"7FEE138DC000","o":"1C740CA","s":"_ZN5mongo37BasicCommandWithReplyBuilderInterface10Invocation3runEPNS_16OperationContextEPNS_3rpc21ReplyBuilderInterfaceE","s+":"11A"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE1554A60F","b":"7FEE138DC000","o":"1C6E60F","s":"_ZN5mongo14CommandHelpers20runCommandInvocationEPNS_16OperationContextERKNS_12OpMsgRequestEPNS_17CommandInvocationEPNS_3rpc21ReplyBuilderInterfaceE","s+":"7F"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CF1AC5","b":"7FEE138DC000","o":"1415AC5","s":"_ZN5mongo12_GLOBAL__N_114runCommandImplEPNS_16OperationContextEPNS_17CommandInvocationERKNS_12OpMsgRequestEPNS_3rpc21ReplyBuilderInterfaceENS_11LogicalTimeERKNS_23ServiceEntryPointCommon5HooksEPNS_14BSONObjBuilderERKNS_30OperationSessionInfoFromClientE","s+":"135"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CF4929","b":"7FEE138DC000","o":"1418929","s":"_ZN5mongo12_GLOBAL__N_119execCommandDatabaseEPNS_16OperationContextEPNS_7CommandERKNS_12OpMsgRequestEPNS_3rpc21ReplyBuilderInterfaceERKNS_23ServiceEntryPointCommon5HooksE","s+":"11B9"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CF5C10","b":"7FEE138DC000","o":"1419C10","s":"_ZN5mongo12_GLOBAL__N_116receivedCommandsEPNS_16OperationContextERKNS_7MessageERKNS_23ServiceEntryPointCommon5HooksE","s+":"650"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CF689D","b":"7FEE138DC000","o":"141A89D","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE","s+":"50D"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CE434C","b":"7FEE138DC000","o":"140834C","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE","s+":"3C"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CEEDD5","b":"7FEE138DC000","o":"1412DD5","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE","s+":"F5"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CECAE5","b":"7FEE138DC000","o":"1410AE5","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE","s+":"125"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CEDA76","b":"7FEE138DC000","o":"1411A76","s":"_ZNSt17_Function_handlerIFvvEZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS2_11ThreadGuardENS1_9transport15ServiceExecutor13ScheduleFlagsENS4_23ServiceExecutorTaskNameENS2_9OwnershipEEUlvE_E9_M_invokeERKSt9_Any_data","s+":"56"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE15FABBE2","b":"7FEE138DC000","o":"26CFBE2","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE","s+":"182"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CEBEFB","b":"7FEE138DC000","o":"140FEFB","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE","s+":"DB"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CED25D","b":"7FEE138DC000","o":"141125D","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE","s+":"6AD"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CED340","b":"7FEE138DC000","o":"1411340","s":"_ZN5mongo14future_details4callIRZNS_19ServiceStateMachine14_sourceMessageENS2_11ThreadGuardEEUlNS_10StatusWithINS_7MessageEEEE0_S6_EEDaOT_OT0_","s+":"60"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CED5E5","b":"7FEE138DC000","o":"14115E5","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE","s+":"145"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CECB5A","b":"7FEE138DC000","o":"1410B5A","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE","s+":"19A"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE14CEDA76","b":"7FEE138DC000","o":"1411A76","s":"_ZNSt17_Function_handlerIFvvEZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS2_11ThreadGuardENS1_9transport15ServiceExecutor13ScheduleFlagsENS4_23ServiceExecutorTaskNameENS2_9OwnershipEEUlvE_E9_M_invokeERKSt9_Any_data","s+":"56"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE15FAC248","b":"7FEE138DC000","o":"26D0248","s":"_ZNSt17_Function_handlerIFvvEZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIS0_ENS2_15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameEEUlvE0_E9_M_invokeERKSt9_Any_data","s+":"B8"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE162FC706","b":"7FEE138DC000","o":"2A20706","s":"_ZNSt17_Function_handlerIFvvEZN5mongo25launchServiceWorkerThreadESt8functionIS0_EEUlvE1_E9_M_invokeERKSt9_Any_data","s+":"56"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE162FC774","b":"7FEE138DC000","o":"2A20774","s":"_ZN5mongo12_GLOBAL__N_17runFuncEPv","s+":"14"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE11F1DAA1","b":"7FEE11F16000","o":"7AA1","s":"start_thread","s+":"D1"}}}
{"t":{"$date":"2021-04-15T13:16:55.376+01:00"},"s":"I", "c":"CONTROL", "id":31427, "ctx":"conn6","msg":" Frame: {frame}","attr":{"frame":{"a":"7FEE11C6AAAD","b":"7FEE11B82000","o":"E8AAD","s":"clone","s+":"6D"}}}
IP INFO, not the original ip's:
127.0.0.1: PRIMARY 4.2.13
127.0.0.2: SECONDARY 4.4.4
127.0.0.3: ABRITER 4.2.13
Replicaset conf:
{
"_id" : "myid",
"version" : 1396130,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 0,
"host" : "127.0.0.1:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 2,
"tags" : { },
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 1,
"host" : "127.0.0.2:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 2,
"tags" : { },
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 3,
"host" : "127.0.0.3:27018",
"arbiterOnly" : true,
"buildIndexes" : true,
"hidden" : false,
"priority" : 0,
"tags" : { },
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : { },
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
}
}
}
I already googled for a solution but now i am stuck. This is a quality enviroment and i want to upgrade PRD, but first i need to understand the problem.
- duplicates
-
-
- Closed
-