Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-4313

sharded systems should throttle writes when chunk migrations cannot keep up

    • Type: Icon: Improvement Improvement
    • Resolution: Won't Do
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Sharding
    • Sharding EMEA

      In some cases data may be arriving at a shard faster than it can be migrated elsewhere. The typical situation where this is seen is on a brand new collection which does not have many chunks yet, to which a very large bulk import is sent. In this situation, it is necessary to throttle the injection at the shard; if injest rate is, for a long period of time, higher than migration output rate, the node will "overflow" with either data or workload (often the latter). Presplitting is a good technique to avoid this situation currently.

      Thus this ticket is to throttle writes s.t. migrations do not fall behind.

      For "phase one" inserts are the only thing that needs throttling. We can make a phase 2 ticket after that.

            Assignee:
            backlog-server-sharding-emea [DO NOT USE] Backlog - Sharding EMEA
            Reporter:
            dwight@mongodb.com Dwight Merriman
            Votes:
            3 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: