Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-72530

$_internalOwningShard aggregation expression should not cause a refresh of the sharding filtering metadata

    • Sharding EMEA
    • Fully Compatible
    • Sharding EMEA 2023-01-23, Sharding EMEA 2023-06-12, Sharding EMEA 2023-06-26, Sharding EMEA 2023-07-10

      The $_internalOwningShard aggregation expression from SERVER-68686 is designed to be used to make routing decisions of the form [given an input document from collection Source, to what shard for collection Destination should the document be inserted into?]. Its usage assumes sharding_ddl_util::stopMigrations() was already run to ensure the placement information doesn't continue to change while the aggregation is being run. And, for the intended usage of the $_internalOwningShard aggregation expression in resharding a collection or building a global index, the split points + ownership for the new collection / global index container won't change until those operations entirely complete.

      The $_internalOwningShard aggregation expression therefore accepts a shard version to ensure the CatalogCache has been refreshed up until which provides the necessary causal consistency of the routing table information. In the case where the CatalogCache is stale, ExpressionInternalOwningShard throws a StaleConfig exception.

      // Invalidate catalog cache if the chunk manager version is stale.
      >if (chunkManager.getVersion().isOlderThan(shardVersion.placementVersion())) {
          boost::optional<CollectionIndexes> collIndexes;
          ShardVersion currentShardVersion(chunkManager.getVersion(), collIndexes);
          uasserted(StaleConfigInfo(ns,
                                    currentShardVersion,
                                    boost::none /* wanted */,
                                    ShardingState::get(opCtx)->shardId()),
                    str::stream()
                        << "Sharding information of collection " << ns
                        << " is currently stale and needs to be recovered from the config server");
      }
      

      While it may be the case that forcing a refresh of the sharding filter metadata induces a refresh of the routing information for the collection, this feels like an implementation detail that shouldn't be relied upon. It would be more direct and likely to continue being supported if ExpressionInternalOwningShard caused the routing information for the collection to be refreshed by doing

      1. Call catalogCache->invalidateShardOrEntireCollectionEntryForShardedCollection(ns, shardVersion, ShardId()).
      2. Throw a ShardCannotRefreshDueToLocksHeld exception to cause the shard to refresh (only) its CatalogCache.

            Assignee:
            pol.pinol@mongodb.com Pol Pinol
            Reporter:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: