We've seen several incidents in versions >= 4.2 of frequent (every couple minutes) mongos-to-client latency spikes that last tens of seconds each. These latency spikes appear to originate on mongos (i.e. latency is not often or consistently observable at concurrently between mongos and mongod). The latency spikes are consistently associated with bursts of connection creation between mongos and mongod. We have a workaround that avoids this issue (increasing the minimum size of the ShardingTaskExecutorPool, the connection pool between mongos and the shards), but should find the root cause of this issue.

Acceptance criteria for this ticket: find the root cause of the latency spikes, and fix the issue if the fix is trivial; if not file a ticket to fix the issue.