ISSUE DESCRIPTION
MongoDB 3.2.11 and 3.4.0 introduce a set of configurable parameters in mongos to help manage NetworkInterfaceASIO-based connection pools in sharded clusters. These parameters are:
TaskExecutorPoolSize
- The number of NetworkInterfaceASIO-based connection pools to use for a given mongos
- Defaults to 4 <= NUM_CORES <= 64
- When tuned down, this will reduce the number of open connections in the system at some performance cost under a heavier load
- A good default is generally either the actual default, or <16 if the the node running mongod has a large number of cores
ShardingTaskExecutorPoolHostTimeoutMS
- How long will mongos wait before dropping all connections to a host that it hasn't communicated with
- Default is 5 minutes
- Turning this up will prevent a mongos from entirely severing a connection to a mongod
- Longer durations will smooth connection spikes. Setting this to 3-4x the period of the connection spikes will generally prevent completely cold starts
ShardingTaskExecutorPoolMaxSize
- The maximum number of connections to hold to a given mongod at any time.
- Default is unlimited
- Giving this is a maximum value, may be useful to set a limit for this value if some mongod nodes in the system are suffering from connection floods
- Remember that setting this effectively limits the number of simultaneous operations from a mongos to a mongod at PoolMaxSize * TaskExecutorPoolSize
ShardingTaskExecutorPoolMinSize
- The minimum number of connections to hold to a given host at any time.
- Default is 1
- If a system is experiencing latency issues because of cold starts, users should consider increasing this value
- The cost is that mongos will keep these sessions open, which is a real fixed cost of TaskExecutorPoolSize * PoolMinSize * NUM_MONGOS per shard
ShardingTaskExecutorPoolRefreshRequirementMS
- How long will mongos wait before attempting to heartbeat an idle connection in the pool
- Default is 1 minute
- Setting this higher or lower will increase or decrease heartbeat traffic, which will increase/decrease idle load
- Lower values may reduce the number of transient network failures seen in a system (because non-user operations will see them earlier and transparently re-connect)
- It is unlikely that the default value needs to be modified
ShardingTaskExecutorPoolRefreshTimeoutMS
- How long will mongos wait before timing out a heartbeat
- Default 20 seconds
- If this value is too low, mongos won't be able to maintain connections in the pool
- Tuning this value up may help if heartbeats actually take 20 seconds
Original description
It would be nice if users could configure a outbound connection pools in a way as most of the drivers work. The use case is to do throttling of the amount of connections that hit the shards on the mongos level - e.g. if a number of requestes/active connections to mongos exceeds the specified threshold, some connections will have to wait for their turn to be served.
Implementing this would mean that we need to expose the standard connection pool-related options such as:
- maxPoolSize
- waitQueueMultiple
- waitQueueTimeoutMS
Given that mongos is based on a C++ driver I presume that should be doable.
—
Note on the implementation of this ticket
To clarify, this commit exposes the connection pooling options that currently exist in mongos, rather than driver equivalent options.
Added knobs include:
- ShardingTaskExecutorPoolHostTimeoutMS
- ShardingTaskExecutorPoolMaxSize,
- ShardingTaskExecutorPoolMinSize,
- ShardingTaskExecutorPoolRefreshRequirementMS
- ShardingTaskExecutorPoolRefreshTimeoutMS
- is duplicated by
-
SERVER-22082 Ability to specify minimum size for the connection pools from mongos -> mongod
- Closed
- is related to
-
SERVER-28564 How to maintain consistant connections on mongos
- Closed
- related to
-
SERVER-26740 Total connections is not stable and spiky when user authentication is enabled
- Closed