-
Type: Bug
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: 3.6.3
-
Component/s: None
-
Environment:OS: Windows Server 2008 R2 Enterprise with Service Pack 1 (64 bit)
CPU: Intel Xeon CPU E5-26xx v2 2.6GHz
RAM: 4 GB
disk: 100 GB disk partition for MongoDB.
MongoDB Replica Set: 3 instances, this instance is the Primary.
-
Storage Execution
Mongod crashed because of no space left on disk. This is the log when crash happened:
2018-04-09T16:13:33.568+0800 I COMMAND [conn484] command sessions.session_common command: findAndModify { findAndModify: "session_common", query:
Unknown macro: { sessionid}, update: { $inc:
Unknown macro: { maxmsgid}, $set:
Unknown macro: { activetime}}, new: true, $db: "sessions" } planSummary: IXSCAN
Unknown macro: { sessionid}keysExamined:1 docsExamined:1 nMatched:1 nModified:1 keysInserted:1 keysDeleted:1 numYields:0 reslen:417 locks:{ Global: { acquireCount:
Unknown macro: { r}}, Database: { acquireCount:
Unknown macro: { w}}, Collection: { acquireCount:
Unknown macro: { w}}, oplog: { acquireCount:
Unknown macro: { w}} } protocol:op_query 120ms 2018-04-09T16:13:33.876+0800 E STORAGE [thread3841] WiredTiger error (28) [1523261613:876714][3376:1995256704], log-server: D:\mongodb\data\journal\WiredTigerTmplog.0000001761: handle-set-end: SetEndOfFile: 磁盘空间不足。
: No space left on device 2018-04-09T16:13:33.876+0800 E STORAGE [thread3841] WiredTiger error (28) [1523261613:876714][3376:1995256704], log-server: log pre-alloc server error: No space left on device 2018-04-09T16:13:33.877+0800 E STORAGE [thread3841] WiredTiger error (28) [1523261613:877691][3376:1995256704], log-server: log server error: No space left on device 2018-04-09T16:13:33.877+0800 E STORAGE [thread3841] WiredTiger error (-31804) [1523261613:877691][3376:1995256704], log-server: the process must exit and restart: WT_PANIC: WiredTiger library panic 2018-04-09T16:13:33.886+0800 F - [conn486] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [conn475] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [conn483] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [conn478] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [WTOplogJournalThread] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [conn487] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [conn3774] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [thread3841] Fatal Assertion 28558 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 361 2018-04-09T16:13:33.886+0800 F - [conn470] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [conn486]
***aborting after fassert() failure
2018-04-09T16:13:33.886+0800 F - [conn466] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64 2018-04-09T16:13:33.886+0800 F - [conn475]
***aborting after fassert() failure
2018-04-09T16:13:33.886+0800 F - [conn483]
***aborting after fassert() failure
When I restarted the mongod the disk space was free.
It seems that journal files fill in the whole disk space. But my real dataset is not very big.
This is the database status in MongoDB:
Unknown macro: { "db" }Unknown macro: { "db" }Unknown macro: { "db" }
I notice that the number of WiredTigerPreplog file in journal directory always grows and doesn't stop. The total size of these files is even the majority of MongoDB data size.
This is the screenshot on the machine. Note number of WiredTigerPreplog file:
screenshot
- duplicates
-
WT-3985 Pre-allocated log files accumulate on Windows
- Closed