-
Type: Task
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Querying, Replication, Sharding
-
Query Optimization
-
(copied to CRM)
Currently change streams on sharded collections target all shards in the cluster, regardless of which shards actually have data for that collection. In order for change streams to target just the shards that have chunks, there are a few things we need.
First we need to be able to reconstruct the routing table at any point in the past, so that when we resume a stream we can know which shards had chunks at the time that the stream is resuming from.
Second, we need chunk migration commit to be a multi-document transaction. If the donor shard recorded a chunk migration commit operation with the same optime as the chunk migration commit operation on the config server, then when a change stream encounters that commit operation on the donor shard, it could use the multi-version routing table to tell whether the recipient shard had any other chunks for that collection as of the time of the commit and if not it could close its change stream cursor to force mongos to retarget. We could then remove the no-op oplog entry we currently log on the donor shard when we migrate a chunk to a shard that has no chunks for that collection.
- is related to
-
SERVER-80427 Avoid change streams latency caused by lack of writes on a shard
- Backlog
- related to
-
SERVER-42290 Target change streams to the primary shard when possible
- Backlog