Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-53544

Verify prevOpTime chain for transactions is intact in resharding oplog fetcher

    • v5.0
    • Sharding 2021-06-14, Sharding 2021-05-31
    • 2

      ReshardingOplogFetcher relies on {$_requestReshardingResumeToken: true} enabling QueryPlannerParams::ASSERT_MIN_TS_HAS_NOT_FALLEN_OFF_OPLOG and causing the aggregation pipeline to fail with an OplogQueryMinTsMissing error response if the donor's oplog rolls over. However, it is possible for the $graphLookup in the resharding oplog fetcher's aggregation pipeline to read from a later snapshot than the snapshot that was used for the collection scan to populate the DocumentSourceCursor. This means the following sequence could happen:

      1. ReshardingOplogFetcher runs [{$match: {ts: {$gte: Timestamp(10, 1)}}}, ..., {$graphLookup: ...}, ...].
      2. CollectionScan verifies the {ts: Timestamp(10, 1)} document is still present in the oplog. The {ts: Timestamp(10, 1)} document has {prevOpTime: {ts: Timestamp(5, 1), t: 1}}.
      3. DocumentSourceCursor buffers a batch of documents.
      4. Oplog truncation occurs and now {ts: Timestamp(6, 1)} is the oldest oplog entry.
      5. $graphLookup finds the {ts: Timestamp(10, 1)} document but fails to find a {ts: Timestamp(5, 1)} document.

      The ReshardingOplogFetcher could confirm the prevOpTime value is {prevOpTime: {ts: Timestamp(0, 0), t: -1}} whenever it sees a new transaction. The aggregation pipeline that resharding's oplog fetching is using guarantees all of the oplog entries from the same multi-document transaction are contiguous so it only needs to track this for a single multi-document transaction at a time. We may as well verify the entire prevOpTime chain values for the later entries too.

            Assignee:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Reporter:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: