Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-53667

High rate of journal flushes on secondary in 4.4

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Execution Team 2021-01-25, Repl 2021-03-08, Repl 2021-03-22, Repl 2021-04-05, Repl 2022-03-21, Repl 2022-04-04, Repl 2022-04-18, Repl 2022-05-02, Repl 2022-05-16, Repl 2022-05-30, Repl 2022-06-13, Repl 2022-06-27

      A simple workload with a single thread doing updates as fast as possible sees the following:

      • 4.2
        • primary
          • updates: 1200/s
          • log flushes: 300/s
        • secondary
          • batches: 280/s
          • log flushes: 300/s
      • 4.4
        • primary
          • updates: 1250/s
          • log flushes: 10/s; much improved in 4.4 due to PM-1274
        • seccondary
          • batches: 1200/s
          • log flushes: 2400/s; much worse in 4.4 than 4.2, and much higher than primary

      This creates a much higher i/o requirement on the secondary in 4.4, and much higher on the secondary than the primary.

      Two possible issues:

      • Do we need to flush more often on the secondary than the primary?
      • Do we need to do 2 flushes per batch?

        1. primary-asis.png
          primary-asis.png
          5 kB
        2. Screen Shot 2021-03-03 at 2.03.12 PM.png
          Screen Shot 2021-03-03 at 2.03.12 PM.png
          36 kB
        3. Screen Shot 2021-03-03 at 2.03.34 PM.png
          Screen Shot 2021-03-03 at 2.03.34 PM.png
          30 kB
        4. Screen Shot 2021-03-03 at 2.35.57 PM.png
          Screen Shot 2021-03-03 at 2.35.57 PM.png
          34 kB
        5. Screen Shot 2021-03-03 at 2.36.01 PM.png
          Screen Shot 2021-03-03 at 2.36.01 PM.png
          33 kB
        6. Screenshot 2021-04-20 at 10.14.07.png
          Screenshot 2021-04-20 at 10.14.07.png
          95 kB
        7. Screenshot 2021-04-20 at 10.14.07-1.png
          Screenshot 2021-04-20 at 10.14.07-1.png
          95 kB
        8. secondary_as_is.png
          secondary_as_is.png
          5 kB

            Assignee:
            matthew.russotto@mongodb.com Matthew Russotto
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            44 Start watching this issue

              Created:
              Updated:
              Resolved: