Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-18314

Stall during fdatasync phase of checkpoints under WiredTiger and EXT4

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.2
    • Component/s: WiredTiger
    • ALL

      • YCSB 30M documents, 10 fields, ~1kB/document, total ~30GB
      • 50/50 read/update workload
      • 40 GB cache, 128 GB memory, 32 CPUs
      • slow SSD disk (~80-100 MB/s)
      • no journal (to simplify the situation)
      • per mongostat, cache is at 100% utilization, 80% dirty pretty much throughout the test.

      During each checkpoint two calls to fdatasync are made. Because this scenario is i/o constrained the fdatasyncs take a substantial amount of time, and during both fdatasync calls throughput falls to exactly 0 for the duration of the fdatasync. This is seen in A-B, C-D, E-F, G-H, I-J, K-L below.

      In many, but not all, such cases WT bumps the "eviction server unable to reach goal" counter.

      Similar test with a larger cache (the default 64GB) does not show this issue.

      Note: this is the same test as reported in SERVER-18315; opening two separate tickets to track what may be separate issues.

        1. try-13.png
          try-13.png
          145 kB
        2. try-14.png
          try-14.png
          105 kB
        3. try-16.png
          try-16.png
          65 kB
        4. try-21.png
          try-21.png
          142 kB
        5. try-23.png
          try-23.png
          114 kB
        6. try-25.png
          try-25.png
          116 kB
        7. try-26.png
          try-26.png
          114 kB
        8. try-29.png
          try-29.png
          117 kB
        9. lz4.png
          lz4.png
          9 kB
        10. snappy.png
          snappy.png
          11 kB
        11. count=1000000.png
          count=1000000.png
          10 kB
        12. try-30.png
          try-30.png
          77 kB
        13. try-33.png
          try-33.png
          62 kB
        14. try-34.png
          try-34.png
          60 kB
        15. try-35.png
          try-35.png
          63 kB
        16. try-36.png
          try-36.png
          63 kB

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            38 Start watching this issue

              Created:
              Updated:
              Resolved: