-
Type: Task
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Catalog and Routing
-
CAR Team 2024-01-08, CAR Team 2024-01-22, CAR Team 2024-02-05, CAR Team 2024-02-19, CAR Team 2024-03-04, CAR Team 2024-04-15, CAR Team 2024-10-14, CAR Team 2024-10-28, CAR Team 2024-11-11, CAR Team 2024-11-25
-
3
Today, MongoD uses a single CatalogCache and CatalogCacheLoader for both it's shard-role responsibilities (i.e., updating the filtering metadata) and its bespoke router-role support (i.e., routing cross-shard $lookup queries).
As we give MongoD a proper routing support equivalent to the routing facilities MongoS provides, we want to allow the routing layer to use a separate CatalogCache that is backed by a ConfigServerCatalogCacheLoader (CSCCL), so that routing operations do not need to stall waiting for replication. However, the shard-role use of the CatalogCache needs to continue to use CatalogCache backed by ShardServerCatalogCacheLoader (SSCCL), to ensure that the routing table view is consistent between primaries and secondaries on refresh.
To achieve this goal, we'll support two CatalogCache and CatalogCacheLoader instances in each MongoD: one used for routing, and the other for shard-role activities. The use of the shard-role implementation should be very limited (essentially, only to filtering metadata updating). To implement this, we'll introduce a new CatalogCacheAndLoaderBackingShardRole object that encapsulates the shard-role state, and clarify that the CatalogCache available on the grid should be used exclusively for routing.
- depends on
-
SERVER-87391 CheckMetadataConsistency may fail to detect a RoutingTableRangeOverlap inconsistency (and return a ConflictingOperationInProgress error instead).
- Closed
-
SERVER-87693 RefineCollectionShardKeyCoordinator should not use filtering metadata cache utilities to fetch routing information
- Closed
-
SERVER-87796 The $_internalOwningShard expression should verify that the cached CollectionRoutingInfo has an initialised routing table before consuming it
- Closed
-
SERVER-85370 Move rename precondition checks away from generic utility API
- Closed
-
SERVER-85525 `collMod` coordinator can get the current shard ID without accessing the catalog cache
- Closed
-
SERVER-86805 Relax asserts on js tests profiling the execution of aggregations with nested lookups
- Closed
- has to be done before
-
SERVER-87552 Re-enable blacklisted tests in "replica set endpoint"-related suites
- Backlog
- has to be finished together with
-
SERVER-84247 Complete TODO listed in SERVER-82588
- Closed
- is depended on by
-
SERVER-86556 CheckMetadataConsistency should ignore deprecated index options
- Blocked
-
SERVER-83433 Add back the test coverage for running db hash and validation check on replica set that is fsync locked and has replica set endpoint enabled
- Backlog
-
SERVER-86800 Dedicated statistics for information routing and metadata filtering caches
- Blocked
-
SERVER-87131 Support stepdowns suites with the embedded router
- Blocked
-
SERVER-87555 Re-enable blacklisted tests under the sharding_auto_bootstrap suite
- Blocked
-
SERVER-87692 Concurrency/FSM replication state change tests for embedded router mode
- Blocked
-
SERVER-86163 Ensure upgrades from 7.x to 8.0 handle CatalogCacheLoader changes.
- Closed
-
SERVER-82707 Improve CatalogCache eviction policy
- Blocked
- is related to
-
SERVER-96903 Throwing when unyielding after sending a remote request may continuously swallow a StaleConfig error
- Open
-
SERVER-82588 Move CatalogCacheLoader to be a decoration on Service
- Closed
-
SERVER-82589 Move CatalogCache off of the Grid object and onto Service
- Closed
-
SERVER-87780 Dedicated catalog cache for config server's background activities
- Closed
-
SERVER-95392 Revisit all CatalogCache usages in the DDL coordinators
- Closed
-
SERVER-95393 Use a ConfigServerCatalogCacheLoader for the router-role and a ShardServerCatalogCacheLoader for the shard-role
- In Progress
-
SERVER-95390 Introduce FilteringMetadataCache class for shard_filtering_metadata_refresh.cpp
- Closed
-
SERVER-95391 Remove the CatalogCacheLoader as a decoration of the ServiceContext
- Closed
- related to
-
SERVER-84548 Using ShardServerCatalogCacheLoader on configsvr causes excessive WT data handles / memory usage
- Closed