-
Type: Improvement
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: Index Maintenance
-
None
-
Fully Compatible
-
v4.4
-
Execution Team 2020-04-06
-
18
Each index build (i.e. a single createIndexes command, which may build more than one index) is allowed to use up to 200MB of memory outside of the WT cache by default. This is controlled by maxIndexBuildMemoryUsageMegabytes. In an attempt to bound memory usage and reduce WT cache pressure, we introduced a limit the number of concurrent index builds started in SERVER-44984. This limit was bug-prone and also applied to index builds on secondaries. This ticket imposes a concurrent limit on index builds started by users on primaries, effectively the same observable behavior.
On primaries, any index builds started by a user over the limit will block until the number of concurrent index builds drops below the limit. The log message "Too many index builds running simultaneously, waiting until the number of active index builds is below the threshold" (ID 4715500) will be logged when the limit is reached and an index build has blocked.
This default limit can be raised with the maxNumActiveUserIndexBuilds setParameter. This can be changed at startup or runtime.
Original Description
Both the primary and secondary nodes will have an unlimited thread pool size. This is done to allow secondary nodes to startup as many index builders as necessary in order to prevent scheduling deadlocks during initial sync or oplog application.
When commands are run from user connections that need to create indexes, those commands will hang until there are less than 3 running index builder threads, or until the operation is interrupted.
- is depended on by
-
SERVER-46659 Make initial sync work with two phase index builds + commit quorum on
- Closed
- related to
-
SERVER-48617 Concurrent 'createIndexes' can block all write operations on a primary and standalone by exhausting write tickets.
- Closed
-
SERVER-59122 repeated blocked index build messages in logs
- Closed
-
SERVER-49215 Differentiate internal idents used for resumable index build information from other internal idents
- Closed
-
SERVER-44984 Reduce index thread pool size and reduce memory used per build
- Closed
-
SERVER-49948 [4.2] Introduce maxNumActiveUserIndexBuilds setParameter
- Closed