Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22616

Make balancer architecture more scalable by allowing parallel chunk migrations

    • Type: Icon: New Feature New Feature
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Sharding
    • None

      1. Moving one chunk at a time is not a feature of an allegedly "linear scalable" product.

      2. Waiting for chunks to be deleted on a target shard, before moving chunks of a totally different collection, is not a scalable design.
      Note: I'm not referring to the wait which is controllable via waitForDelete, I'm referring to not starting at all - i.e. "can't accept new chunks because there are still X deletes from previous migration"

      3. If you detect this behavior, why not move to the next chunk in the meantime? other shards can accept chunks. The current bottleneck for our system at the moment is the range deleter on a single shard. Until it finishes deleting, no balancing occurs in the entire cluster. This is ridicoulus.

            Assignee:
            Unassigned Unassigned
            Reporter:
            yonido Yoni Douek
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: