Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30391

Add QoS features to MongoDB

    • Type: Icon: Improvement Improvement
    • Resolution: Won't Do
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Usability
    • None
    • Service Arch

      I am currently writing an application that generates sustained, heavy writes for about one hour at >200Mbit/s. This application is regularly causing performance issues on the mongodb side.

      It currently runs relatively fine on our atlas production setup using write-concern: majority and a single insertation thread, but it is causing very high loads (>90% disk utilization) on our test setup. This is in would be desired (maxing out the server's resources is good because we get more performance), but we had a db crash in the past (https://jira.mongodb.org/browse/MMSSUPPORT-14543 caused probably by a performance difference between the primary and replication – https://jira.mongodb.org/browse/SERVER-24242) and we are still not sure this could not happen again.

      In addition, using write-concern: majority causes a significant latency wich seems to be slowing down our application by a lot (now taking 3h instead of 1h since I activated this).

      It would be very useful if there was a way to explicitly mark operations as high-impact/best-effort/background jobs. Mongodb should make sure that all operations that are not marked in such a way are given priority before the high-impact tasks. Mongodb should also make sure that high-impact operations can never overtax the server; these operations should either be rejected or blocked until enough resources are available to process them.

      (Blocking would be very useful because then on my side I could write a scheduler that reduces the number of insertation threads or throttles them when many jobs are rejected).

      This way it would be possible to operate high-load jobs at maximum efficiency without stealing resources from routine operations.

            Assignee:
            backlog-server-servicearch [DO NOT USE] Backlog - Service Architecture
            Reporter:
            karo Karolin Varner
            Votes:
            1 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: