-
Type: Bug
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: 3.0.3
-
Component/s: WiredTiger
-
Fully Compatible
-
ALL
ISSUE SUMMARY
MongoDB running with the WiredTiger storage engine may, during large index builds, exceed the amount of memory allocated to the database cache.
This issue may prevent large index builds in some situations, such as during initial sync of new replica set members.
USER IMPACT
Excessive memory consumption may cause mongod to either abort with an out-of-memory condition, or be killed by the operating system's OOM killer, leading to a loss of availability of the affected node.
WORKAROUNDS
Lowering the amount of cache available to WiredTiger to the 1GB minimum may allow users affected by this issue to complete large index builds.
AFFECTED VERSIONS
MongoDB 3.0.0 through 3.0.4
FIX VERSION
The fix is included in the 3.0.5 production release.
Original description
This has been seen under somewhat different circumstances by a couple of customers.
Initial sync of a large db (multi TB, couple billion documents). Foreground build of _id index starts at A, and cache usage rises steadily to 6x configured maximum before being terminated by OOM.
Mongorestore of 100GB db. Multiple parallel background index builds begin at A; at B cache usage begins to grow until it reaches about 1.5x configured max and is terminated by OOM.
No complete repro yet, although may have got a partial repro: during initial sync of a 500 GB db cache usage briefly rose to about 120% of configured max.
- is depended on by
-
WT-1973 MongoDB changes for WiredTiger 2.7.0
- Closed
- is duplicated by
-
SERVER-18842 WiredTiger & indexing: "kernel: Out of memory: Kill process 32011 (mongod) score 966 or sacrifice child"
- Closed
-
SERVER-19066 Out Of Memory issues with WiredTiger during index build
- Closed
-
SERVER-19339 crash with this log
- Closed
-
SERVER-19620 Mongod killed because of OOM during index build Bottom up
- Closed
- is related to
-
SERVER-20159 Out of memory on index build during initial sync even with low cacheSize parameter
- Closed
-
SERVER-18674 Very low throughput during portion of checkpoint under WiredTiger
- Closed
-
SERVER-18677 Throughput drop during transaction pinned phase of checkpoints under WiredTiger (larger data set)
- Closed