-
Type: Bug
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: 3.0.0
-
Component/s: WiredTiger
-
None
-
Fully Compatible
-
ALL
ISSUE SUMMARY
MongoDB running with the WiredTiger storage engine may hit a default limit and terminate with the following error message:
eviction-server: session 0x2b1a2c0: hazard pointer table full
The default limit for the number of hazard pointers is 1000, which may be hit if the database has a large number of collections and indexes.
USER IMPACT
The mongod process terminates and needs to be restarted.
WORKAROUNDS
Users affected by this issue can increase default limit for the size of the hazard pointer table to, for example, 10000, by using the following command line option:
--wiredTigerEngineConfigString="hazard_max=10000".
AFFECTED VERSIONS
MongoDB stable releases 3.0.0 and 3.0.1 are affected by this issue.
FIX VERSION
The fix is included in the 3.0.2 production release.
Original description
We're in the process of migrating our test environment to MongoDB 3.0. We spun up a new instance with 3.0, added it to the replica set and let it sync.
However, a few hours into the process it crashed with the following error:
2015-03-10T19:59:46.037+0000 I - [repl writer worker 11] Index Build: 196300/590825 33% 2015-03-10T19:59:46.272+0000 E STORAGE WiredTiger (0) [1426017586:270981][3822:0x7f38f7a89700], file:index-3720--4215350639811228100.wt, eviction-server: session 0x2ffe2c0: hazard pointer table full 2015-03-10T19:59:46.274+0000 E STORAGE WiredTiger (12) [1426017586:272380][3822:0x7f38f7a89700], eviction-server: cache eviction server error: Cannot allocate memory 2015-03-10T19:59:46.274+0000 E STORAGE WiredTiger (-31804) [1426017586:274241][3822:0x7f38f7a89700], eviction-server: the process must exit and restart: WT_PANIC: WiredTiger library panic 2015-03-10T19:59:46.274+0000 I - Fatal Assertion 28558 2015-03-10T19:59:46.274+0000 I - [repl writer worker 11] Fatal Assertion 28559 2015-03-10T19:59:46.288+0000 I NETWORK [initandlisten] connection accepted from X.Y.Z.X:58398 #1900 (5 connections now open) 2015-03-10T19:59:46.308+0000 I NETWORK [conn1897] end connection X.Y.Z.Y:58395 (4 connections now open) 2015-03-10T19:59:46.329+0000 I CONTROL 0xf5b229 0xefba41 0xee0de1 0xd71e16 0x1382a50 0x1382d15 0x13831b1 0x133a99f 0x7f38f9abfe9a 0x7f38f85962ed ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"B5B229"},{"b":"400000","o":"AFBA41"},{"b":"400000","o":"AE0DE1"},{"b":"400000","o":"971E16"},{"b":"400000","o":"F82A50"},{"b":"400000","o":"F82D15"},{"b":"400000","o":"F831B1"},{"b":"400000","o":"F3A99F"},{"b":"7F38F9AB8000","o":"7E9A"},{"b":"7F38F84A2000","o":"F42ED"}],"processInfo":{ "mongodbVersion" : "3.0.0", "gitVersion" : "a841fd6394365954886924a35076691b4d149168", "uname" : { "sysname" : "Linux", "release" : "3.2.0-75-virtual", "version" : "#110-Ubuntu SMP Tue Dec 16 19:24:01 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "5CFC121279316F2BC7FBB39F8268284EDF47E708" }, { "b" : "7FFF9A6FF000", "elfType" : 3, "buildId" : "975615E6DDB0F811039107F3F95DB4884097BE36" }, { "b" : "7F38F9AB8000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7F38F985A000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "E62302CCF011B0D1277B07E2B6170B678480F996" }, { "b" : "7F38F947F000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "DBF958ED97EEE7EAD42EC0F24EDB2E39ACFA2B83" }, { "b" : "7F38F9277000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7F38F9073000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7F38F8D73000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "B534DA725D06A04267EB2FEB92B9CC14C838B57B" }, { "b" : "7F38F8A77000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7F38F8861000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "ECF322A96E26633C5D10F18215170DD4395AF82C" }, { "b" : "7F38F84A2000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "DDD4987072B438ECB112023E07E432844CF89076" }, { "b" : "7F38F9CD5000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "E25AD1A11CCF57E734116B8EC9C69F643DCA9F18" }, { "b" : "7F38F828B000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "F695ECFCF3918D5D34989398A14B7ECDD9F46CD0" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf5b229] mongod(_ZN5mongo10logContextEPKc+0xE1) [0xefba41] mongod(_ZN5mongo13fassertFailedEi+0x61) [0xee0de1] mongod(+0x971E16) [0xd71e16] mongod(+0xF82A50) [0x1382a50] mongod(__wt_err+0x95) [0x1382d15] mongod(__wt_panic+0x21) [0x13831b1] mongod(+0xF3A99F) [0x133a99f] libpthread.so.0(+0x7E9A) [0x7f38f9abfe9a] libc.so.6(clone+0x6D) [0x7f38f85962ed] ----- END BACKTRACE ----- 2015-03-10T19:59:46.329+0000 I - ***aborting after fassert() failure
The database was running on an AWS instance with 7.5GB RAM, using the default WiredTiger config. We attempted again, on a larger instance (16GB RAM), but it still fails:
2015-03-11T13:18:44.503+0000 E STORAGE WiredTiger (0) [1426079924:501160][892:0x7f9799de0700], file:index-2117-6998163636744330624.wt, eviction-server: session 0x2b1a2c0: hazard pointer table full 2015-03-11T13:18:44.504+0000 E STORAGE WiredTiger (12) [1426079924:503483][892:0x7f9799de0700], eviction-server: cache eviction server error: Cannot allocate memory 2015-03-11T13:18:44.504+0000 E STORAGE WiredTiger (-31804) [1426079924:504371][892:0x7f9799de0700], eviction-server: the process must exit and restart: WT_PANIC: WiredTiger library panic 2015-03-11T13:18:44.505+0000 I - Fatal Assertion 28558 2015-03-11T13:18:44.506+0000 E STORAGE WiredTiger (-31804) [1426079924:506395][892:0x7f9797ddc700], session.checkpoint: metadata unroll update file:collection-1552-6998163636744330624.wt to allocation_size=4KB,app_metadata=(formatVersion=1),block_allocation=best,block_compressor=snappy,cache_resident=0,checkpoint=(WiredTigerCheckpoint.2=(addr="01c1e581e40bf68bf8c1e681e44df8a873c1e781e4c3e687b3808080e3226fc0e3217fc0",order=2,time=1426079809,size=2203648,write_gen=113)),checkpoint_lsn=(12784,6111872),checksum=uncompressed,collator=,columns=,dictionary=0,format=btree,huffman_key=,huffman_value=,id=14826,internal_item_max=0,internal_key_max=0,internal_key_truncate=,internal_page_max=4KB,key_format=q,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=1MB,memory_page_max=10m,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=0,prefix_compression_min=4,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,value_format=u,version=(major=1,minor=1): WT_PANIC: WiredTiger library panic 2015-03-11T13:18:44.506+0000 I - Fatal Assertion 28558 2015-03-11T13:18:44.506+0000 I - [repl writer worker 9] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 10] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 1] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 11] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 15] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 3] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 6] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 4] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 12] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 2] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 0] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 13] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 14] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 5] Fatal Assertion 28559 2015-03-11T13:18:44.506+0000 I - [repl writer worker 8] Fatal Assertion 28559 2015-03-11T13:18:44.507+0000 I - [repl writer worker 7] Fatal Assertion 28559 2015-03-11T13:18:44.591+0000 I CONTROL [repl writer worker 0] 0xf5b229 0xefba41 0xee0de1 0xd88fd9 0xd7d6b4 0xd7d70f 0xd7d775 0xd6c725 0xa9c8c5 0xa260c0 0xa522bc 0xbe50c4 0xbe5474 0xbe5ffd 0xb37531 0xc4e655 0xca4dfe 0xca78f8 0xef2a5b 0xfa7ce4 0x7f979be16e9a 0x7f979a8ed2ed ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"B5B229"},{"b":"400000","o":"AFBA41"},{"b":"400000","o":"AE0DE1"},{"b":"400000","o":"988FD9"},{"b":"400000","o":"97D6B4"},{"b":"400000","o":"97D70F"},{"b":"400000","o":"97D775"},{"b":"400000","o":"96C725"},{"b":"400000","o":"69C8C5"},{"b":"400000","o":"6260C0"},{"b":"400000","o":"6522BC"},{"b":"400000","o":"7E50C4"},{"b":"400000","o":"7E5474"},{"b":"400000","o":"7E5FFD"},{"b":"400000","o":"737531"},{"b":"400000","o":"84E655"},{"b":"400000","o":"8A4DFE"},{"b":"400000","o":"8A78F8"},{"b":"400000","o":"AF2A5B"},{"b":"400000","o":"BA7CE4"},{"b":"7F979BE0F000","o":"7E9A"},{"b":"7F979A7F9000","o":"F42ED"}],"processInfo":{ "mongodbVersion" : "3.0.0", "gitVersion" : "a841fd6394365954886924a35076691b4d149168", "uname" : { "sysname" : "Linux", "release" : "3.2.0-75-virtual", "version" : "#110-Ubuntu SMP Tue Dec 16 19:24:01 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "5CFC121279316F2BC7FBB39F8268284EDF47E708" }, { "b" : "7FFFDEC64000", "elfType" : 3, "buildId" : "975615E6DDB0F811039107F3F95DB4884097BE36" }, { "b" : "7F979BE0F000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7F979BBB1000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "E62302CCF011B0D1277B07E2B6170B678480F996" }, { "b" : "7F979B7D6000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "DBF958ED97EEE7EAD42EC0F24EDB2E39ACFA2B83" }, { "b" : "7F979B5CE000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7F979B3CA000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7F979B0CA000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "B534DA725D06A04267EB2FEB92B9CC14C838B57B" }, { "b" : "7F979ADCE000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7F979ABB8000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "ECF322A96E26633C5D10F18215170DD4395AF82C" }, { "b" : "7F979A7F9000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "DDD4987072B438ECB112023E07E432844CF89076" }, { "b" : "7F979C02C000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "E25AD1A11CCF57E734116B8EC9C69F643DCA9F18" }, { "b" : "7F979A5E2000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "F695ECFCF3918D5D34989398A14B7ECDD9F46CD0" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf5b229] mongod(_ZN5mongo10logContextEPKc+0xE1) [0xefba41] mongod(_ZN5mongo13fassertFailedEi+0x61) [0xee0de1] mongod(_ZN5mongo17wtRCToStatus_slowEiPKc+0x309) [0xd88fd9] mongod(_ZN5mongo22WiredTigerRecoveryUnit8_txnOpenEPNS_16OperationContextE+0x124) [0xd7d6b4] mongod(_ZN5mongo22WiredTigerRecoveryUnit10getSessionEPNS_16OperationContextE+0x1F) [0xd7d70f] mongod(_ZN5mongo16WiredTigerCursorC1ERKSsmbPNS_16OperationContextE+0x35) [0xd7d775] mongod(_ZNK5mongo21WiredTigerIndexUnique9newCursorEPNS_16OperationContextEi+0x55) [0xd6c725] mongod(_ZNK5mongo22BtreeBasedAccessMethod10findSingleEPNS_16OperationContextERKNS_7BSONObjE+0x25) [0xa9c8c5] mongod(_ZN5mongo11IDHackStage4workEPm+0xD0) [0xa260c0] mongod(_ZN5mongo11UpdateStage4workEPm+0x7C) [0xa522bc] mongod(_ZN5mongo12PlanExecutor18getNextSnapshottedEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0xA4) [0xbe50c4] mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE+0x34) [0xbe5474] mongod(_ZN5mongo12PlanExecutor11executePlanEv+0x3D) [0xbe5ffd] mongod(_ZN5mongo6updateEPNS_16OperationContextEPNS_8DatabaseERKNS_13UpdateRequestEPNS_7OpDebugE+0x111) [0xb37531] mongod(_ZN5mongo4repl21applyOperation_inlockEPNS_16OperationContextEPNS_8DatabaseERKNS_7BSONObjEbb+0x12B5) [0xc4e655] mongod(_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEb+0x2EE) [0xca4dfe] mongod(_ZN5mongo4repl21multiInitialSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x78) [0xca78f8] mongod(_ZN5mongo10threadpool6Worker4loopERKSs+0x2FB) [0xef2a5b] mongod(+0xBA7CE4) [0xfa7ce4] libpthread.so.0(+0x7E9A) [0x7f979be16e9a] libc.so.6(clone+0x6D) [0x7f979a8ed2ed] ----- END BACKTRACE ----- 2015-03-11T13:18:44.591+0000 I CONTROL [repl writer worker 6] 0xf5b229 0xefba41 0xee0de1 0xd88fd9 0xd7d6b4 0xd7d70f 0xd7d775 0xd6c725 0xa9c8c5 0xa260c0 0xa522bc 0xbe50c4 0xbe5474 0xbe5ffd 0xb37531 0xc4e655 0xca4dfe 0xca78f8 0xef2a5b 0xfa7ce4 0x7f979be16e9a 0x7f979a8ed2ed ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"B5B229"},{"b":"400000","o":"AFBA41"},{"b":"400000","o":"AE0DE1"},{"b":"400000","o":"988FD9"},{"b":"400000","o":"97D6B4"},{"b":"400000","o":"97D70F"},{"b":"400000","o":"97D775"},{"b":"400000","o":"96C725"},{"b":"400000","o":"69C8C5"},{"b":"400000","o":"6260C0"},{"b":"400000","o":"6522BC"},{"b":"400000","o":"7E50C4"},{"b":"400000","o":"7E5474"},{"b":"400000","o":"7E5FFD"},{"b":"400000","o":"737531"},{"b":"400000","o":"84E655"},{"b":"400000","o":"8A4DFE"},{"b":"400000","o":"8A78F8"},{"b":"400000","o":"AF2A5B"},{"b":"400000","o":"BA7CE4"},{"b":"7F979BE0F000","o":"7E9A"},{"b":"7F979A7F9000","o":"F42ED"}],"processInfo":{ "mongodbVersion" : "3.0.0", "gitVersion" : "a841fd6394365954886924a35076691b4d149168", "uname" : { "sysname" : "Linux", "release" : "3.2.0-75-virtual", "version" : "#110-Ubuntu SMP Tue Dec 16 19:24:01 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "5CFC121279316F2BC7FBB39F8268284EDF47E708" }, { "b" : "7FFFDEC64000", "elfType" : 3, "buildId" : "975615E6DDB0F811039107F3F95DB4884097BE36" }, { "b" : "7F979BE0F000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7F979BBB1000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "E62302CCF011B0D1277B07E2B6170B678480F996" }, { "b" : "7F979B7D6000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "DBF958ED97EEE7EAD42EC0F24EDB2E39ACFA2B83" }, { "b" : "7F979B5CE000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7F979B3CA000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7F979B0CA000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "B534DA725D06A04267EB2FEB92B9CC14C838B57B" }, { "b" : "7F979ADCE000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7F979ABB8000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "ECF322A96E26633C5D10F18215170DD4395AF82C" }, { "b" : "7F979A7F9000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "DDD4987072B438ECB112023E07E432844CF89076" }, { "b" : "7F979C02C000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "E25AD1A11CCF57E734116B8EC9C69F643DCA9F18" }, { "b" : "7F979A5E2000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "F695ECFCF3918D5D34989398A14B7ECDD9F46CD0" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf5b229] mongod(_ZN5mongo10logContextEPKc+0xE1) [0xefba41] mongod(_ZN5mongo13fassertFailedEi+0x61) [0xee0de1] mongod(_ZN5mongo17wtRCToStatus_slowEiPKc+0x309) [0xd88fd9] mongod(_ZN5mongo22WiredTigerRecoveryUnit8_txnOpenEPNS_16OperationContextE+0x124) [0xd7d6b4] mongod(_ZN5mongo22WiredTigerRecoveryUnit10getSessionEPNS_16OperationContextE+0x1F) [0xd7d70f] mongod(_ZN5mongo16WiredTigerCursorC1ERKSsmbPNS_16OperationContextE+0x35) [0xd7d775] mongod(_ZNK5mongo21WiredTigerIndexUnique9newCursorEPNS_16OperationContextEi+0x55) [0xd6c725] mongod(_ZNK5mongo22BtreeBasedAccessMethod10findSingleEPNS_16OperationContextERKNS_7BSONObjE+0x25) [0xa9c8c5] mongod(_ZN5mongo11IDHackStage4workEPm+0xD0) [0xa260c0] mongod(_ZN5mongo11UpdateStage4workEPm+0x7C) [0xa522bc] mongod(_ZN5mongo12PlanExecutor18getNextSnapshottedEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0xA4) [0xbe50c4] mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE+0x34) [0xbe5474] mongod(_ZN5mongo12PlanExecutor11executePlanEv+0x3D) [0xbe5ffd] mongod(_ZN5mongo6updateEPNS_16OperationContextEPNS_8DatabaseERKNS_13UpdateRequestEPNS_7OpDebugE+0x111) [0xb37531] mongod(_ZN5mongo4repl21applyOperation_inlockEPNS_16OperationContextEPNS_8DatabaseERKNS_7BSONObjEbb+0x12B5) [0xc4e655] mongod(_ZN5mongo4repl8SyncTail9syncApplyEPNS_16OperationContextERKNS_7BSONObjEb+0x2EE) [0xca4dfe] mongod(_ZN5mongo4repl21multiInitialSyncApplyERKSt6vectorINS_7BSONObjESaIS2_EEPNS0_8SyncTailE+0x78) [0xca78f8] mongod(_ZN5mongo10threadpool6Worker4loopERKSs+0x2FB) [0xef2a5b] mongod(+0xBA7CE4) [0xfa7ce4] libpthread.so.0(+0x7E9A) [0x7f979be16e9a] libc.so.6(clone+0x6D) [0x7f979a8ed2ed] ----- END BACKTRACE ----- 2015-03-11T13:18:44.591+0000 I - [repl writer worker 0] ***aborting after fassert() failure 2015-03-11T13:18:44.591+0000 I - [repl writer worker 6] ***aborting after fassert() failure
The database size on the 3.0.0 machine was ~50GB at this point. The master has ~220GB of data, but it's hard to know how far it had synced as it's using Snappy compression on the new node.
- is depended on by
-
WT-1901 Issues resolved in WiredTiger 2.5.3
- Closed
- is duplicated by
-
SERVER-17614 WT invariant crash with "hazard pointer table full" error
- Closed
-
SERVER-17814 eviction-server: the process must exit and restart: WT_PANIC: WiredTiger library panic
- Closed