Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-54020

ShardInvalidatedForTargeting thrown by resharding's getDestinedRecipient() not being retried by mongos

    • Fully Compatible
    • ALL
    • Hide

      Apply the following patch to delay when the donor shard performs its refresh of the temporary resharding collection.

      python buildscripts/resmoke.py run --suite=sharding repro_resharding_write_before_temp_ns_refresh.js
      
      Unable to find source-code formatter for language: diff. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      diff --git a/repro_resharding_write_before_temp_ns_refresh.js b/repro_resharding_write_before_temp_ns_refresh.js
      new file mode 100644
      index 0000000000..7012fae165
      --- /dev/null
      +++ b/repro_resharding_write_before_temp_ns_refresh.js
      @@ -0,0 +1,33 @@
      +(function() {
      +"use strict";
      +
      +load("jstests/sharding/libs/resharding_test_fixture.js");
      +
      +const reshardingTest = new ReshardingTest({numDonors: 2, numRecipients: 2, reshardInPlace: true});
      +reshardingTest.setup();
      +
      +const donorShardNames = reshardingTest.donorShardNames;
      +const sourceCollection = reshardingTest.createShardedCollection({
      +    ns: "reshardingDb.coll",
      +    shardKeyPattern: {oldKey: 1},
      +    chunks: [
      +        {min: {oldKey: MinKey}, max: {oldKey: 10}, shard: donorShardNames[0]},
      +        {min: {oldKey: 10}, max: {oldKey: MaxKey}, shard: donorShardNames[1]},
      +    ],
      +});
      +
      +const recipientShardNames = reshardingTest.recipientShardNames;
      +reshardingTest.withReshardingInBackground(  //
      +    {
      +        newShardKeyPattern: {newKey: 1},
      +        newChunks: [
      +            {min: {newKey: MinKey}, max: {newKey: 10}, shard: recipientShardNames[0]},
      +            {min: {newKey: 10}, max: {newKey: MaxKey}, shard: recipientShardNames[1]},
      +        ],
      +    },
      +    () => {
      +        assert.commandWorked(sourceCollection.insert({_id: 0, oldKey: 5, newKey: 15}));
      +    });
      +
      +reshardingTest.teardown();
      +})();
      diff --git a/src/mongo/db/s/resharding/resharding_donor_service.cpp b/src/mongo/db/s/resharding/resharding_donor_service.cpp
      index 2a7200f38e..6ba5aa0973 100644
      --- a/src/mongo/db/s/resharding/resharding_donor_service.cpp
      +++ b/src/mongo/db/s/resharding/resharding_donor_service.cpp
      @@ -263,6 +263,8 @@ void ReshardingDonorService::DonorStateMachine::
                   ->assertNoIndexBuildInProgForCollection(_donorDoc.getExistingUUID());
           }
      
      +    sleepsecs(10);
      +
           // Recipient shards expect to read from the donor shard's existing sharded collection
           // and the config.cache.chunks collection of the temporary resharding collection using
           // {atClusterTime: <fetchTimestamp>}. Refreshing the temporary resharding collection on
      
      Show
      Apply the following patch to delay when the donor shard performs its refresh of the temporary resharding collection. python buildscripts/resmoke.py run --suite=sharding repro_resharding_write_before_temp_ns_refresh.js Unable to find source-code formatter for language: diff. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml diff --git a/repro_resharding_write_before_temp_ns_refresh.js b/repro_resharding_write_before_temp_ns_refresh.js new file mode 100644 index 0000000000..7012fae165 --- /dev/ null +++ b/repro_resharding_write_before_temp_ns_refresh.js @@ -0,0 +1,33 @@ +(function() { + "use strict" ; + +load( "jstests/sharding/libs/resharding_test_fixture.js" ); + + const reshardingTest = new ReshardingTest({numDonors: 2, numRecipients: 2, reshardInPlace: true }); +reshardingTest.setup(); + + const donorShardNames = reshardingTest.donorShardNames; + const sourceCollection = reshardingTest.createShardedCollection({ + ns: "reshardingDb.coll" , + shardKeyPattern: {oldKey: 1}, + chunks: [ + {min: {oldKey: MinKey}, max: {oldKey: 10}, shard: donorShardNames[0]}, + {min: {oldKey: 10}, max: {oldKey: MaxKey}, shard: donorShardNames[1]}, + ], +}); + + const recipientShardNames = reshardingTest.recipientShardNames; +reshardingTest.withReshardingInBackground( // + { + newShardKeyPattern: {newKey: 1}, + newChunks: [ + {min: {newKey: MinKey}, max: {newKey: 10}, shard: recipientShardNames[0]}, + {min: {newKey: 10}, max: {newKey: MaxKey}, shard: recipientShardNames[1]}, + ], + }, + () => { + assert .commandWorked(sourceCollection.insert({_id: 0, oldKey: 5, newKey: 15})); + }); + +reshardingTest.teardown(); +})(); diff --git a/src/mongo/db/s/resharding/resharding_donor_service.cpp b/src/mongo/db/s/resharding/resharding_donor_service.cpp index 2a7200f38e..6ba5aa0973 100644 --- a/src/mongo/db/s/resharding/resharding_donor_service.cpp +++ b/src/mongo/db/s/resharding/resharding_donor_service.cpp @@ -263,6 +263,8 @@ void ReshardingDonorService::DonorStateMachine:: ->assertNoIndexBuildInProgForCollection(_donorDoc.getExistingUUID()); } + sleepsecs(10); + // Recipient shards expect to read from the donor shard's existing sharded collection // and the config.cache.chunks collection of the temporary resharding collection using // {atClusterTime: <fetchTimestamp>}. Refreshing the temporary resharding collection on
    • Sharding 2021-03-22, Sharding 2021-04-05, Sharding EMEA 2021-05-03
    • 39
    • 1

      For some reason, the ShardInvalidatedForTargeting exception is being propagated back to the client as a write error rather than being automatically retried by mongos.

      bool allowLocks = true;
      auto tempNssRoutingInfo = Grid::get(opCtx)->catalogCache()->getCollectionRoutingInfo(
          opCtx,
          constructTemporaryReshardingNss(sourceNss.db(), getCollectionUuid(opCtx, sourceNss)),
          allowLocks);
      
      uassert(ShardInvalidatedForTargetingInfo(sourceNss),
              "Routing information is not available for the temporary resharding collection.",
              tempNssRoutingInfo.getStatus() != ErrorCodes::StaleShardVersion);
      
      uassertStatusOK(tempNssRoutingInfo);
      

            Assignee:
            jordi.serra-torrens@mongodb.com Jordi Serra Torrens
            Reporter:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: