Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-96561

Make ReshardingOplogFetcher insert oplog entries in batch

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 8.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Cluster Scalability
    • Fully Compatible
    • Cluster Scalability 2024-11-11
    • 200

      Currently, the ReshardingOplogFetcher performs only one insert in each storage transaction (i.e. insert the oplog entries separately). We should perform multiple inserts in each storage transaction (using WriteUnitOfWork::kGroupForTransaction). This optimization is required by SERVER-94800, since it will introduce a doc for tracking the count of oplog entries fetched so that the ReshardingRecipientService can use it to recover the resharding metrics after failover/shutdown instead of using the fast count like we do today. To do that, every time the ReshardingOplogFetcher inserts oplog entries into the oplog buffer collection, it needs to atomically update the count in this doc in the fetcher progress collection. Doing this optimization will enable us to avoid doubling the number of writes. Let N be the number of oplog entries and B > 1 be the batch size, then we would do (N + N/B) writes instead of 2N writes. 

            Assignee:
            cheahuychou.mao@mongodb.com Cheahuychou Mao
            Reporter:
            cheahuychou.mao@mongodb.com Cheahuychou Mao
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: