-
Type: Task
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Sharding
-
v5.0
-
Sharding 2021-06-14, Sharding 2021-05-31
-
2
ReshardingOplogFetcher relies on {$_requestReshardingResumeToken: true} enabling QueryPlannerParams::ASSERT_MIN_TS_HAS_NOT_FALLEN_OFF_OPLOG and causing the aggregation pipeline to fail with an OplogQueryMinTsMissing error response if the donor's oplog rolls over. However, it is possible for the $graphLookup in the resharding oplog fetcher's aggregation pipeline to read from a later snapshot than the snapshot that was used for the collection scan to populate the DocumentSourceCursor. This means the following sequence could happen:
- ReshardingOplogFetcher runs [{$match: {ts: {$gte: Timestamp(10, 1)}}}, ..., {$graphLookup: ...}, ...].
- CollectionScan verifies the {ts: Timestamp(10, 1)} document is still present in the oplog. The {ts: Timestamp(10, 1)} document has {prevOpTime: {ts: Timestamp(5, 1), t: 1}}.
- DocumentSourceCursor buffers a batch of documents.
- Oplog truncation occurs and now {ts: Timestamp(6, 1)} is the oldest oplog entry.
- $graphLookup finds the {ts: Timestamp(10, 1)} document but fails to find a {ts: Timestamp(5, 1)} document.
The ReshardingOplogFetcher could confirm the prevOpTime value is {prevOpTime: {ts: Timestamp(0, 0), t: -1}} whenever it sees a new transaction. The aggregation pipeline that resharding's oplog fetching is using guarantees all of the oplog entries from the same multi-document transaction are contiguous so it only needs to track this for a single multi-document transaction at a time. We may as well verify the entire prevOpTime chain values for the later entries too.
- duplicates
-
SERVER-57303 Create transaction history iterator stage for Resharding
- Closed
- is related to
-
SERVER-49892 Continued testing for aggregation pipeline for oplog fetcher for resharding (transaction entries)
- Closed