-
Type:
Bug
-
Resolution: Incomplete
-
Priority:
Critical - P2
-
None
-
Affects Version/s: 3.0.2, 3.0.4
-
Component/s: WiredTiger
-
None
-
Linux
My MongoDB server crashed and I can't start the mongod service. I'm using this command to try to repair database: mongod --dbpath . --storageEngine wiredTiger --repair
Mongod version was 3.0.2, but i tried to upgrade to 3.0.4 and repair again, but gave the same error.
I researched a lot of ISSUES and did so many things to try to recover my entire database but nothing works. I can't find ways to ignore corruption in collections (loss some data) but recover some part of them.
Error when I try to repair:
2015-07-06T13:49:53.180-0500 I INDEX [initandlisten] building index using bulk method
2015-07-06T13:49:53.578-0500 I STORAGE [initandlisten] Repairing collection database.users
2015-07-06T13:49:53.578-0500 I STORAGE [initandlisten] WiredTiger progress session.verify 6
2015-07-06T13:49:53.578-0500 I STORAGE [initandlisten] Verify succeeded on uri table:collection-40--7480132900646616609. Not salvaging.
2015-07-06T13:49:53.692-0500 I INDEX [initandlisten] build index on: database.users properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "database.users" }
2015-07-06T13:49:53.692-0500 I INDEX [initandlisten] building index using bulk method
2015-07-06T13:49:53.775-0500 I STORAGE [initandlisten] Repairing collection database.users_history
2015-07-06T13:49:53.775-0500 I STORAGE [initandlisten] WiredTiger progress session.verify 3
2015-07-06T13:49:53.775-0500 I STORAGE [initandlisten] Verify succeeded on uri table:collection-42--7480132900646616609. Not salvaging.
2015-07-06T13:49:53.880-0500 I INDEX [initandlisten] build index on: database.users_history properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "database.users_history" }
2015-07-06T13:49:53.880-0500 I INDEX [initandlisten] building index using bulk method
2015-07-06T13:49:53.972-0500 I STORAGE [initandlisten] repairDatabase datasource
2015-07-06T13:49:53.972-0500 I STORAGE [initandlisten] Repairing collection datasource.reviews
2015-07-06T13:49:53.972-0500 E STORAGE [initandlisten] WiredTiger (0) [1436208593:972487][16105:0x7fe889ce5bc0], file:collection-1595-7140502635356682714.wt, session.verify: read checksum error [4096B @ 9994240, 832067279 != 2155579310]
2015-07-06T13:49:53.972-0500 E STORAGE [initandlisten] WiredTiger (0) [1436208593:972512][16105:0x7fe889ce5bc0], file:collection-1595-7140502635356682714.wt, session.verify: collection-1595-7140502635356682714.wt: encountered an illegal file format or internal value
2015-07-06T13:49:53.972-0500 E STORAGE [initandlisten] WiredTiger (-31804) [1436208593:972525][16105:0x7fe889ce5bc0], file:collection-1595-7140502635356682714.wt, session.verify: the process must exit and restart: WT_PANIC: WiredTiger library panic
2015-07-06T13:49:53.972-0500 I - [initandlisten] Fatal Assertion 28558
2015-07-06T13:49:53.983-0500 I CONTROL [initandlisten]
0xf5e199 0xefd1b1 0xee1cb1 0xd87dda 0x1390e89 0x1391045 0x13914e4 0x12e228e 0x12e2728 0x12e54cf 0x12e5648 0x1310afa 0x138c16e 0x138c3a8 0x138c836 0xd70ed5 0xd71671 0xcf7250 0xbf9648 0x80a8f4 0x7d6b89 0x7fe8882aaec5 0x8080e7
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"B5E199"},{"b":"400000","o":"AFD1B1"},{"b":"400000","o":"AE1CB1"},{"b":"400000","o":"987DDA"},{"b":"400000","o":"F90E89"},{"b":"400000", "o":"F91045"},{"b":"400000","o":"F914E4"},{"b":"400000","o":"EE228E"},{"b":"400000","o":"EE2728"},{"b":"400000","o":"EE54CF"},{"b":"400000","o":"EE5648"},{"b":"400000", "o":"F10AFA"},{"b":"400000","o":"F8C16E"},{"b":"400000","o":"F8C3A8"},{"b":"400000","o":"F8C836"},{"b":"400000","o":"970ED5"},{"b":"400000","o":"971671"},{"b":"400000", "o":"8F7250"},{"b":"400000","o":"7F9648"},{"b":"400000","o":"40A8F4"},{"b":"400000","o":"3D6B89"},{"b":"7FE888289000","o":"21EC5"},{"b":"400000","o":"4080E7"}],"process Info":{ "mongodbVersion" : "3.0.4", "gitVersion" : "0481c958daeb2969800511e7475dc66986fa9ed5", "uname" : { "sysname" : "Linux", "release" : "3.13.0-37-generic", "versio n" : "#64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "32DC52072DB9385642CCB4D2AD2ACDA6E0 B87A27" }, { "b" : "7FFF57609000", "elfType" : 3, "buildId" : "0074678E5FFFF79F46C476077E67057161772F37" }, { "b" : "7FE8898B4000", "path" : "/lib/x86_64-linux-gnu/libp thread.so.0", "elfType" : 3, "buildId" : "9318E8AF0BFBE444731BB0461202EF57F7C39542" }, { "b" : "7FE889655000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfTyp e" : 3, "buildId" : "A20EFFEC993A8441FA17F2079F923CBD04079E19" }, { "b" : "7FE88927A000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "F000D29917E9B6E94A35A8F02E5C62846E5916BC" }, { "b" : "7FE889072000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "92FCF41EFE012D6186E31A5 9AD05BDBB487769AB" }, { "b" : "7FE888E6E000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "C1AE4CB7195D337A77A3C689051DABAA3980CA0C" }, { "b " : "7FE888B6A000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "4BF6F7ADD8244AD86008E6BF40D90F8873892197" }, { "b" : "7FE888864000" , "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "1D76B71E905CB867B27CEF230FCB20F01A3178F5" }, { "b" : "7FE88864E000", "path" : "/lib/x86_64-lin ux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "8D0AA71411580EE6C08809695C3984769F25725B" }, { "b" : "7FE888289000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elf Type" : 3, "buildId" : "30C94DC66A1FE95180C3D68D2B89E576D5AE213C" }, { "b" : "7FE889AD2000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9F00581 AB3C73E3AEA35995A0C50D24D59A01D47" } ] }}
mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf5e199]
mongod(_ZN5mongo10logContextEPKc+0xE1) [0xefd1b1]
mongod(_ZN5mongo13fassertFailedEi+0x61) [0xee1cb1]
mongod(+0x987DDA) [0xd87dda]
mongod(__wt_eventv+0x489) [0x1390e89]
mongod(__wt_err+0x95) [0x1391045]
mongod(__wt_panic+0x24) [0x13914e4]
mongod(__wt_block_extlist_read+0x6E) [0x12e228e]
mongod(__wt_block_extlist_read_avail+0x28) [0x12e2728]
mongod(+0xEE54CF) [0x12e54cf]
mongod(__wt_block_verify_start+0x108) [0x12e5648]
mongod(__wt_verify+0x4AA) [0x1310afa]
mongod(__wt_schema_worker+0x35E) [0x138c16e]
mongod(__wt_schema_worker+0x598) [0x138c3a8]
mongod(+0xF8C836) [0x138c836]
mongod(_ZN5mongo18WiredTigerKVEngine16_salvageIfNeededEPKc+0x45) [0xd70ed5]
mongod(_ZN5mongo18WiredTigerKVEngine11repairIdentEPNS_16OperationContextERKNS_10StringDataE+0x51) [0xd71671]
mongod(_ZN5mongo15KVStorageEngine17repairRecordStoreEPNS_16OperationContextERKSs+0xA0) [0xcf7250]
mongod(_ZN5mongo14repairDatabaseEPNS_16OperationContextEPNS_13StorageEngineERKSsbb+0x2A8) [0xbf9648]
mongod(_ZN5mongo13initAndListenEi+0xA44) [0x80a8f4]
mongod(main+0x139) [0x7d6b89]
libc.so.6(__libc_start_main+0xF5) [0x7fe8882aaec5]
mongod(+0x4080E7) [0x8080e7]
----- END BACKTRACE -----
2015-07-06T13:49:53.983-0500 I - [initandlisten]
***aborting after fassert() failure
- depends on
-
-
- Closed
-
- is related to
-
SERVER-19134 primary can't restart after being killed
-
- Closed
-