Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-58472

Implement a balancer merge chunks machinery

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 5.1.0-rc0
    • Affects Version/s: 5.1.0
    • Component/s: None
    • None
    • Fully Compatible
    • Sharding EMEA 2021-08-09, Sharding EMEA 2021-08-23, Sharding EMEA 2021-09-06

      The balancer needs to support new functionality which will be required for the algorithm in SERVER-58471

      After putting the balancer into a "merge chunks" mode via the per-collection command developed as part of SERVER-58270 ,
      the balancer needs to work in a special mode where we attempt to merge and move chunks to shards where they can be merged.

      The end goal is to reach a optimal number of chunks based on the default chunk size and the size of the data.
      After the merge chunks algorithm signals convergence, the balancer needs to move back into the default mode.

       

      • Estimate chunk sizes by using the dataSize estimation API
      • Waiting for chunk size estimation should not completely block balancer, if sizes for chunks are unknown we need to queue the estimation operation and then move on to the next operation
      • Store chunk sizes in config.chunk documents for later processing stages
      • Balancer needs to perform merges operations which the merge algorithm decided on

      See Kal’s script for proof of concept

      Below a schematic of the new balancer round

            Assignee:
            allison.easton@mongodb.com Allison Easton
            Reporter:
            simon.gratzer@mongodb.com Simon Gratzer (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: