-
Type: Bug
-
Resolution: Gone away
-
Priority: Major - P3
-
None
-
Affects Version/s: 7.0.0, 7.2.0-rc0, 7.1.0, 7.3.0-rc0, 8.0.0-rc0
-
Component/s: None
-
Catalog and Routing
-
ALL
-
CAR Team 2024-02-19, CAR Team 2024-03-04
-
2
In every balancing round, we retrieve per-shard storage statics. Since this is done serially, in a cluster with several shards this can be particularly slow, limiting considerably the balancing speed.
Back in SERVER-30060 we did an optimization to retrieve storage statistics only from shards that have maxSize configured.
Recently as part of SERVER-66297 we removed this optimization, so now we retrieve again statistics from all the shards serially at every round.
The implementation of these statistics retrieval is affected by multiple performance issues:
- Balancer retrieve storage statistics serially from all shards.
Proposal: Parallelize the retrieval.
- Balancer retrieve storage statistics multiple times in a single balancer round
Proposal: Do it only once per round and re-use the same ClusterStatistcs object
- depends on
-
SERVER-87063 Split currSizeBytes out of ShardStatistics
- Closed
- is caused by
-
SERVER-66297 Get rid of `maxSize` for shards
- Closed
- is depended on by
-
SERVER-34819 Optimize the sharding balancer's cluster statistics gathering
- Closed