-
Type: Bug
-
Resolution: Done
-
Priority: Critical - P2
-
Affects Version/s: 3.0.0-rc8
-
Component/s: Storage, WiredTiger
-
Fully Compatible
-
ALL
-
Using the 2/12 nightly build, pre-rc9 79492d9cc1885d74b31b5fe24194dbc227096d6e, found an invariant failure on Amazon Linux in EC2 (C3-8XL) upon shutdown.
Cache was full and checkpoints outstanding at time of shutdown (kill -15) but mixed read/write sysbench execute workload had completed.
2015-02-18T18:49:15.895+0000 I NETWORK [conn56] end connection 127.0.0.1:43053 (1 connection now open) 2015-02-18T18:49:26.135+0000 I NETWORK [conn10] end connection 127.0.0.1:43004 (0 connections now open) 2015-02-18T18:49:44.740+0000 I CONTROL [signalProcessingThread] got signal 15 (Terminated), will terminate after current cmd ends 2015-02-18T18:49:44.753+0000 I CONTROL [signalProcessingThread] now exiting 2015-02-18T18:49:44.753+0000 I NETWORK [signalProcessingThread] shutdown: going to close listening sockets... 2015-02-18T18:49:44.753+0000 I NETWORK [signalProcessingThread] closing listening socket: 6 2015-02-18T18:49:44.753+0000 I NETWORK [signalProcessingThread] closing listening socket: 7 2015-02-18T18:49:44.756+0000 I NETWORK [signalProcessingThread] removing socket file: /tmp/mongodb-27017.sock 2015-02-18T18:49:44.761+0000 I NETWORK [signalProcessingThread] shutdown: going to flush diaglog... 2015-02-18T18:49:44.761+0000 I NETWORK [signalProcessingThread] shutdown: going to close sockets... 2015-02-18T18:49:44.766+0000 I STORAGE [signalProcessingThread] WiredTigerKVEngine shutting down 2015-02-18T18:51:28.501+0000 E STORAGE [signalProcessingThread] WiredTiger (0) [1424285488:501446][30154:0x7f2de5bbf700], log-server: Connection has open file handles: sizeStorer.wt 2015-02-18T18:51:28.502+0000 E STORAGE WiredTiger (0) [1424285488:502494][30154:0x7f2de53be700], eviction-server: cache server: exiting with 2352566 pages in memory and 2352564 pages evicted 2015-02-18T18:51:28.502+0000 E STORAGE WiredTiger (0) [1424285488:502567][30154:0x7f2de53be700], eviction-server: cache server: exiting with 7026 bytes in memory 2015-02-18T18:51:28.502+0000 E STORAGE WiredTiger (0) [1424285488:502582][30154:0x7f2de53be700], eviction-server: cache server: exiting with 6777 bytes dirty and 1 pages dirty 2015-02-18T18:51:28.511+0000 I - [signalProcessingThread] Invariant failure: _conn->close(_conn, config) resulted in status UnknownError 16: Device or resource busy at src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 206 2015-02-18T18:51:28.596+0000 I CONTROL [signalProcessingThread] 0xf41659 0xeeb791 0xed176a 0xd56707 0xa69d1a 0xa99c09 0xf40506 0xf8ee04 0x7f2ded0b7f18 0x7f2dec1c9b2d ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"B41659"},{"b":"400000","o":"AEB791"},{"b":"400000","o":"AD176A"},{"b":"400000","o":"956707"},{"b":"400000","o":"669D1A"},{"b":"400000","o":"699C09"},{"b":"400000","o":"B40506"},{"b":"400000","o":"B8EE04"},{"b":"7F2DED0B0000","o":"7F18"},{"b":"7F2DEC0E7000","o":"E2B2D"}],"processInfo":{ "mongodbVersion" : "3.0.0-rc9-pre-", "gitVersion" : "79492d9cc1885d74b31b5fe24194dbc227096d6e", "uname" : { "sysname" : "Linux", "release" : "3.14.20-20.44.amzn1.x86_64", "version" : "#1 SMP Mon Oct 6 22:52:46 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFFCA0FE000", "elfType" : 3 }, { "b" : "7F2DED0B0000", "path" : "/lib64/libpthread.so.0", "elfType" : 3 }, { "b" : "7F2DECEA8000", "path" : "/lib64/librt.so.1", "elfType" : 3 }, { "b" : "7F2DECCA4000", "path" : "/lib64/libdl.so.2", "elfType" : 3 }, { "b" : "7F2DEC9A0000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F2DEC6A2000", "path" : "/lib64/libm.so.6", "elfType" : 3 }, { "b" : "7F2DEC48C000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F2DEC0E7000", "path" : "/lib64/libc.so.6", "elfType" : 3 }, { "b" : "7F2DED2CC000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf41659] mongod(_ZN5mongo10logContextEPKc+0xE1) [0xeeb791] mongod(_ZN5mongo17invariantOKFailedEPKcRKNS_6StatusES1_j+0xDA) [0xed176a] mongod(_ZN5mongo18WiredTigerKVEngine13cleanShutdownEv+0x107) [0xd56707] mongod(_ZN5mongo23GlobalEnvironmentMongoD34shutdownGlobalStorageEngineCleanlyEv+0x1A) [0xa69d1a] mongod(_ZN5mongo11exitCleanlyENS_8ExitCodeE+0x219) [0xa99c09] mongod(+0xB40506) [0xf40506] mongod(+0xB8EE04) [0xf8ee04] libpthread.so.0(+0x7F18) [0x7f2ded0b7f18] libc.so.6(clone+0x6D) [0x7f2dec1c9b2d] ----- END BACKTRACE ----- 2015-02-18T18:51:28.596+0000 I - [signalProcessingThread] ***aborting after invariant() failure
- is duplicated by
-
SERVER-17220 Invariant failure on shutdown: Device or resource busy at wiredtiger_kv_engine.cpp
- Closed