Problem Summary

We are proposing removing a collection X-lock in the MigrationSourceManager. This X-lock creates a barrier after which all active transactions on the collection have been committed. This barrier ensures that the migration's chunk cloner will not miss writes that started before the migration.

Without this barrier, the cloner will miss transaction writes that start before a migration starts. These writes will not be registered under the LogOpForShardingHandler.

Proposed Approach

Because of this inconsistency, we would like to switch how migrations handle observing CRUD operations in multi-document transactions.

After the change in SERVER-39017, we will be able to observe all transaction statements on commit.Accordingly, on the op observer onCommit handler, we can pass these statements to a new method on the OpObserverShardingImpl class. This new method is defined in the next paragraph.

A new method shardObserveTransactionCommit placed on the OpObserverShardingImpl class will do the following:

• Verify shard version (parity with other CRUD op observers).
• Lock the CollectionShardingRuntimeLock (parity with other CRUD op observers).
• Verify the existence of a migration (parity with other CRUD op observers).
• Iterate over transaction statements and apply each CRUD op individually.
• Verify that the chunk has not yet moved if in transaction (parity with other CRUD op observers).

The previously stated iteration process will do the following with a vector of ReplOperations.

• Check if the operation is CRUD type.
• Extract the document from the ReplOperation object.
• switch based on insert/update/delete and call the corresponding chunk cloner CRUD op observer.

NOTE: We previously stated that we needed an iterator class to analyze the list of replOperations. However, the iteration process is actually very simple – for now, I think an anonymous function in the same file as the OpObserverShardingImpl class will be sufficient.

As a result of the previously stated work, we will now be duplicating ops sent to the shard observer – we will be observing every individual CRUD operation, as well as those same operations if they are also inside a multi statement transaction. We don't want the migration to see the same write twice (since these updates are idempotent, it wouldn't be incorrect, just unnecessary). We should only send writes to the cloner if we are not in a multi statement transaction, and a simple check of an already-given boolean will do that.