-
Type: Bug
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: 3.2.3
-
Component/s: None
-
None
-
ALL
2017-11-13T17:03:40.886+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.42:40336 #978 (943 connections now open) 2017-11-13T17:03:40.887+0000 E STORAGE [repl writer worker 4] WiredTiger (22) [1510592620:887915][10630:0x7fc141a4f700], file:index-31062--824032195446804458.wt, WT_SESSION.open_cursor: live.avail: m erge range 40960-53248 overlaps with existing range 49152-53248: Invalid argument 2017-11-13T17:03:40.887+0000 E STORAGE [repl writer worker 4] WiredTiger (-31804) [1510592620:887951][10630:0x7fc141a4f700], file:index-31062--824032195446804458.wt, WT_SESSION.open_cursor: the proce ss must exit and restart: WT_PANIC: WiredTiger library panic 2017-11-13T17:03:40.887+0000 I - [repl writer worker 4] Fatal Assertion 28558 2017-11-13T17:03:40.888+0000 I - [repl writer worker 4] ***aborting after fassert() failure 2017-11-13T17:03:40.888+0000 I ACCESS [conn936] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.890+0000 I - [repl writer worker 5] Fatal Assertion 28559 2017-11-13T17:03:40.890+0000 I - [repl writer worker 5] ***aborting after fassert() failure 2017-11-13T17:03:40.890+0000 I - [repl writer worker 6] Fatal Assertion 28559 2017-11-13T17:03:40.890+0000 I - [repl writer worker 6] ***aborting after fassert() failure 2017-11-13T17:03:40.890+0000 I - [repl writer worker 7] Fatal Assertion 28559 2017-11-13T17:03:40.890+0000 I - [repl writer worker 7] ***aborting after fassert() failure 2017-11-13T17:03:40.890+0000 I - [repl writer worker 8] Fatal Assertion 28559 2017-11-13T17:03:40.890+0000 I - [repl writer worker 8] ***aborting after fassert() failure 2017-11-13T17:03:40.891+0000 I - [repl writer worker 9] Fatal Assertion 28559 2017-11-13T17:03:40.891+0000 I - [repl writer worker 9] ***aborting after fassert() failure 2017-11-13T17:03:40.891+0000 I - [repl writer worker 10] Fatal Assertion 28559 2017-11-13T17:03:40.891+0000 I - [repl writer worker 10] ***aborting after fassert() failure 2017-11-13T17:03:40.891+0000 I - [repl writer worker 11] Fatal Assertion 28559 2017-11-13T17:03:40.891+0000 I - [repl writer worker 11] ***aborting after fassert() failure 2017-11-13T17:03:40.891+0000 I - [repl writer worker 3] Fatal Assertion 28559 2017-11-13T17:03:40.891+0000 I - [repl writer worker 3] ***aborting after fassert() failure 2017-11-13T17:03:40.891+0000 I - [repl writer worker 2] Fatal Assertion 28559 2017-11-13T17:03:40.891+0000 I - [repl writer worker 2] ***aborting after fassert() failure 2017-11-13T17:03:40.892+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42326 #979 (944 connections now open) 2017-11-13T17:03:40.893+0000 I ACCESS [conn962] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.895+0000 I ACCESS [conn971] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.896+0000 I ACCESS [conn967] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.901+0000 I ACCESS [conn970] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.902+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42328 #980 (945 connections now open) 2017-11-13T17:03:40.904+0000 I ACCESS [conn966] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.905+0000 I ACCESS [conn969] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.908+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42330 #981 (946 connections now open) 2017-11-13T17:03:40.914+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.42:40338 #982 (947 connections now open) 2017-11-13T17:03:40.916+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42336 #983 (948 connections now open) 2017-11-13T17:03:40.916+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42340 #984 (949 connections now open) 2017-11-13T17:03:40.916+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42338 #985 (950 connections now open) 2017-11-13T17:03:40.916+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42334 #986 (951 connections now open) 2017-11-13T17:03:40.917+0000 I ACCESS [conn977] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.918+0000 I ACCESS [conn899] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.918+0000 I ACCESS [conn913] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.918+0000 I ACCESS [conn907] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.920+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42344 #987 (952 connections now open) 2017-11-13T17:03:40.921+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42342 #988 (953 connections now open) 2017-11-13T17:03:40.922+0000 I ACCESS [conn904] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.923+0000 I ACCESS [conn927] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.923+0000 I ACCESS [conn972] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.925+0000 I NETWORK [initandlisten] connection accepted from 161.202.16.41:42346 #989 (954 connections now open) 2017-11-13T17:03:40.927+0000 I ACCESS [conn895] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.927+0000 I ACCESS [conn924] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.928+0000 I ACCESS [conn918] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.928+0000 I ACCESS [conn942] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.928+0000 I ACCESS [conn876] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.929+0000 I ACCESS [conn919] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.930+0000 I ACCESS [conn903] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.932+0000 I ACCESS [conn976] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.932+0000 I ACCESS [conn974] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.932+0000 I ACCESS [conn973] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.933+0000 I ACCESS [conn877] Successfully authenticated as principal admin on admin 2017-11-13T17:03:40.934+0000 F - [repl writer worker 4] Got signal: 6 (Aborted). 0x130e1e2 0x130d119 0x130d922 0x7fc15ab4c100 0x7fc15a7b05f7 0x7fc15a7b1ce8 0x1295b32 0x108b3e3 0x1a57e3c 0x1a57fdd 0x1a583c4 0x198f81d 0x19909a4 0x1990bf3 0x198dbd7 0x19919de 0x19ae1a8 0x19e2140 0x1a 56bfb 0x1a57117 0x1a5726b 0x19f02f8 0x1a54415 0x1a061a3 0x1a54299 0x1a54829 0x1085b14 0x1084460 0x106856b 0xc87055 0xbc96fa 0xbfefc4 0xe0c6d5 0xe0cd99 0xe0ce95 0xd17139 0xe9d024 0xf34c6e 0xf2ea60 0xf3 07e6 0xf313e6 0xf345ab 0x12a0651 0x12a0fb9 0x12a1b10 0x1ac9600 0x7fc15ab44dc5 0x7fc15a871c9d ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"F0E1E2","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"F0D119"},{"b":"400000","o":"F0D922"},{"b":"7FC15AB3D000","o":"F100"},{"b":"7FC15A77B000","o":"355F7"," s":"gsignal"},{"b":"7FC15A77B000","o":"36CE8","s":"abort"},{"b":"400000","o":"E95B32","s":"_ZN5mongo13fassertFailedEi"},{"b":"400000","o":"C8B3E3"},{"b":"400000","o":"1657E3C","s":"__wt_eventv"},{"b": "400000","o":"1657FDD","s":"__wt_err"},{"b":"400000","o":"16583C4","s":"__wt_panic"},{"b":"400000","o":"158F81D"},{"b":"400000","o":"15909A4","s":"__wt_block_extlist_read"},{"b":"400000","o":"1590BF3" ,"s":"__wt_block_extlist_read_avail"},{"b":"400000","o":"158DBD7","s":"__wt_block_checkpoint_load"},{"b":"400000","o":"15919DE"},{"b":"400000","o":"15AE1A8","s":"__wt_btree_open"},{"b":"400000","o":"1 5E2140","s":"__wt_conn_btree_open"},{"b":"400000","o":"1656BFB","s":"__wt_session_get_btree"},{"b":"400000","o":"1657117","s":"__wt_session_get_btree"},{"b":"400000","o":"165726B","s":"__wt_session_ge t_btree_ckpt"},{"b":"400000","o":"15F02F8","s":"__wt_curfile_open"},{"b":"400000","o":"1654415"},{"b":"400000","o":"16061A3","s":"__wt_curtable_open"},{"b":"400000","o":"1654299"},{"b":"400000","o":"1 654829"},{"b":"400000","o":"C85B14","s":"_ZN5mongo17WiredTigerSession9getCursorERKSsmb"},{"b":"400000","o":"C84460","s":"_ZN5mongo16WiredTigerCursorC1ERKSsmbPNS_16OperationContextE"},{"b":"400000","o" :"C6856B","s":"_ZNK5mongo21WiredTigerIndexUnique9newCursorEPNS_16OperationContextEb"},{"b":"400000","o":"887055","s":"_ZNK5mongo17IndexAccessMethod10findSingleEPNS_16OperationContextERKNS_7BSONObjE"}, {"b":"400000","o":"7C96FA","s":"_ZN5mongo11IDHackStage4workEPm"},{"b":"400000","o":"7FEFC4","s":"_ZN5mongo11UpdateStage4workEPm"},{"b":"400000","o":"A0C6D5","s":"_ZN5mongo12PlanExecutor11getNextImplEP NS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE"},{"b":"400000","o":"A0CD99","s":"_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE"},{"b":"400000","o":"A0CE95","s":"_ZN5mongo12PlanExecutor11e xecutePlanEv"},{"b":"400000","o":"917139","s":"_ZN5mongo6updateEPNS_16OperationContextEPNS_8DatabaseERKNS_13UpdateRequestEPNS_7OpDebugE"},{"b":"400000","o":"A9D024","s":"_ZN5mongo4repl21applyOperation _inlockEPNS_16OperationContextEPNS_8DatabaseERKNS_7BSONObjEb"},{"b":"400000","o":"B34C6E","s":"_ZNSt17_Function_handlerIFN5mongo6StatusEPNS0_16OperationContextEPNS0_8DatabaseERKNS0_7BSONObjEbEPS9_E9_M _invokeERKSt9_Any_dataS3_S5_S8_b"},{"b":"400000","o":"B2EA60"},{"b":"400000","o":"B307E6","s":"_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEbSt8functionIFNS_6StatusES3_PNS_8D atabaseES6_bEES7_IFS8_S3_S6_EES7_IFvvEE"},{"b":"400000","o":"B313E6","s":"_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEb"},{"b":"400000","o":"B345AB","s":"_ZN5mongo4repl14mul tiSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE"},{"b":"400000","o":"EA0651","s":"_ZN5mongo10ThreadPool10_doOneTaskEPSt11unique_lockISt5mutexE"},{"b":"400000","o":"EA0FB9","s":"_ZN5mongo10 ThreadPool13_consumeTasksEv"},{"b":"400000","o":"EA1B10","s":"_ZN5mongo10ThreadPool17_workerThreadBodyEPS0_RKSs"},{"b":"400000","o":"16C9600"},{"b":"7FC15AB3D000","o":"7DC5"},{"b":"7FC15A77B000","o":" F6C9D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.2.3", "gitVersion" : "b326ba837cf6f49d65c2f85e1b70f6f31ece7937", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.11 -23.53.amzn1.x86_64", "version" : "#1 SMP Wed Jun 1 22:22:50 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "B389239A9108B604FCA797D0F968E253E7BF9F5E" }, { "b" : "7FFDD2F89000", "elfType" : 3, "buildId" : "41B0E5BE826EA13B746A0EEA2F99E39A5E936B59" }, { "b" : "7FC15BD68000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "C9DB6FDB87F3CAAF 209AEACD2EC345B152605135" }, { "b" : "7FC15B982000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "09CACF79947C91081F28FCCAB5AF524D51751AF0" }, { "b" : "7FC15B77A000", "path" : "/lib6 4/librt.so.1", "elfType" : 3, "buildId" : "42833B65941483A611C40EA7D32F56EA83EA6E93" }, { "b" : "7FC15B576000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "6335077ACD51527BE9F2F18451A88E 2B7350C5B6" }, { "b" : "7FC15B271000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "0A90C35D3174805453EA67A785446D628E298B59" }, { "b" : "7FC15AF6F000", "path" : "/lib64/libm.so.6 ", "elfType" : 3, "buildId" : "BB312C4A65B8FD830C148612CBEACEACC8B08E4F" }, { "b" : "7FC15AD59000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "00FA2883FB47B1327397BBF167C52F51A723D01 3" }, { "b" : "7FC15AB3D000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "E5E575776DAD20ADE8FD0CAF17897C9D89020A87" }, { "b" : "7FC15A77B000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "7128B52741C2494951D285A80DAF25B2BC9C7A0D" }, { "b" : "7FC15BFD5000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "7B7BF8FEEF1A9C627EF90CA5C9188EFD4DA2DDD2" }, { "b" : "7FC15A52F000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "E203354E7F907ACC8C3028FE465541B666DCFBA0" }, { "b" : "7FC15A24A000", "path" : "/lib64/libkrb5.so.3", "elfTyp e" : 3, "buildId" : "D769C8FFAF8772FDA55031ABF2E167DF2207E378" }, { "b" : "7FC15A047000", "path" : "/usr/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "5C01209C5AE1B1714F19B07EB58F2A1274B69DC8" } , { "b" : "7FC159E15000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "6C2243D37143F7FD1E16ED1F6CE4D7F16C2D7EF1" }, { "b" : "7FC159BFF000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "89C6AF118B6B4FB6A73AE1813E2C8BDD722956D1" }, { "b" : "7FC1599F0000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "744272FEAAABCE629AB9E11FAA4A96AEBE8BC2B4" }, { " b" : "7FC1597ED000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "37A58210FA50C91E09387765408A92909468D25B" }, { "b" : "7FC1595D3000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "EE46CA824A1683662BCDA445CC1A4122BBE552BA" }, { "b" : "7FC1593B2000", "path" : "/usr/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "F5054DC94443326819FBF3065CFDF5E4726F57EE" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x130e1e2] mongod(+0xF0D119) [0x130d119] mongod(+0xF0D922) [0x130d922] libpthread.so.0(+0xF100) [0x7fc15ab4c100] libc.so.6(gsignal+0x37) [0x7fc15a7b05f7] libc.so.6(abort+0x148) [0x7fc15a7b1ce8] mongod(_ZN5mongo13fassertFailedEi+0x82) [0x1295b32] mongod(+0xC8B3E3) [0x108b3e3] mongod(__wt_eventv+0x40C) [0x1a57e3c] mongod(__wt_err+0x8D) [0x1a57fdd] mongod(__wt_panic+0x24) [0x1a583c4] mongod(+0x158F81D) [0x198f81d] mongod(__wt_block_extlist_read+0x394) [0x19909a4] mongod(__wt_block_extlist_read_avail+0x33) [0x1990bf3] mongod(__wt_block_checkpoint_load+0x3B7) [0x198dbd7] mongod(+0x15919DE) [0x19919de] mongod(__wt_btree_open+0xC68) [0x19ae1a8] mongod(__wt_conn_btree_open+0x140) [0x19e2140] mongod(__wt_session_get_btree+0xEB) [0x1a56bfb] mongod(__wt_session_get_btree+0x607) [0x1a57117] mongod(__wt_session_get_btree_ckpt+0xAB) [0x1a5726b] mongod(__wt_curfile_open+0x1E8) [0x19f02f8] mongod(+0x1654415) [0x1a54415] mongod(__wt_curtable_open+0x393) [0x1a061a3] mongod(+0x1654299) [0x1a54299] mongod(+0x1654829) [0x1a54829] mongod(_ZN5mongo17WiredTigerSession9getCursorERKSsmb+0x94) [0x1085b14] mongod(_ZN5mongo16WiredTigerCursorC1ERKSsmbPNS_16OperationContextE+0x50) [0x1084460] mongod(_ZNK5mongo21WiredTigerIndexUnique9newCursorEPNS_16OperationContextEb+0x15B) [0x106856b] mongod(_ZNK5mongo17IndexAccessMethod10findSingleEPNS_16OperationContextERKNS_7BSONObjE+0x25) [0xc87055] mongod(_ZN5mongo11IDHackStage4workEPm+0x11A) [0xbc96fa] mongod(_ZN5mongo11UpdateStage4workEPm+0x394) [0xbfefc4] mongod(_ZN5mongo12PlanExecutor11getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0x275) [0xe0c6d5] mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE+0x39) [0xe0cd99] mongod(_ZN5mongo12PlanExecutor11executePlanEv+0x55) [0xe0ce95] mongod(_ZN5mongo6updateEPNS_16OperationContextEPNS_8DatabaseERKNS_13UpdateRequestEPNS_7OpDebugE+0x179) [0xd17139] mongod(_ZN5mongo4repl21applyOperation_inlockEPNS_16OperationContextEPNS_8DatabaseERKNS_7BSONObjEb+0x924) [0xe9d024] mongod(_ZNSt17_Function_handlerIFN5mongo6StatusEPNS0_16OperationContextEPNS0_8DatabaseERKNS0_7BSONObjEbEPS9_E9_M_invokeERKSt9_Any_dataS3_S5_S8_b+0x1E) [0xf34c6e] mongod(+0xB2EA60) [0xf2ea60] mongod(_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEbSt8functionIFNS_6StatusES3_PNS_8DatabaseES6_bEES7_IFS8_S3_S6_EES7_IFvvEE+0x336) [0xf307e6] mongod(_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEb+0xE6) [0xf313e6] mongod(_ZN5mongo4repl14multiSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x9B) [0xf345ab] mongod(_ZN5mongo10ThreadPool10_doOneTaskEPSt11unique_lockISt5mutexE+0x121) [0x12a0651] mongod(_ZN5mongo10ThreadPool13_consumeTasksEv+0xA9) [0x12a0fb9] mongod(_ZN5mongo10ThreadPool17_workerThreadBodyEPS0_RKSs+0x100) [0x12a1b10] mongod(+0x16C9600) [0x1ac9600] libpthread.so.0(+0x7DC5) [0x7fc15ab44dc5] libc.so.6(clone+0x6D) [0x7fc15a871c9d] ----- END BACKTRACE ----- 2017-11-13T17:03:40.936+0000 I ACCESS [conn979] Successfully authenticated as principal admin on admin
- duplicates
-
SERVER-27942 mongodb lost databases after crash
- Closed