-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Component/s: None
TLDR: Please validate that your driver's test suite does not depend on having two shards when testing against a sharded cluster.
The fix for SERVER-32088 has increased the latency for getting changes from a sharded ChangeStream. The basic gist is that if one shard is not performing any writes then the change stream will block until the shard's periodic no-op thread writes to the oplog. This increases the latency for changes from milliseconds to 10 seconds (the default periodicNoopIntervalSecs) or more. In total these extra 10 second pauses make PyMongo's ChangeStreams test suite to take 5 minutes instead of 5 seconds.
Some workarounds are to:
- Set the periodicNoopIntervalSecs startup parameter to 1 second.
- During change streams tests, periodically write to all the shards in the cluster.
- Only test on sharded clusters with a single shard.
Option (1) is a bit annoying since periodicNoopIntervalSecs has only existed since 3.6. We would need to make some mongo-orchestration changes to set this only on >=3.6 nodes or some other backwards compatible change.
Option (2) is even more annoying since every driver would need to do this in order to speed up their testing.
Option (3) is the simplest. All we need to do is remove the second shard from our sharded cluster configs in drivers-evergreen-tools.
Now my question for everyone is: does your driver rely on having two shards for testing or does a single shard suffice? Does anyone think we should keep testing with two shards for any other reason?