Explore using zero filled journal files to reduce fsync latency

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Storage Engines
    • StorEng - Refinement Pipeline
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      WT already has a config option (log.zero_fill) but it defaults to off and we don't enable it. The interesting thing is that it lets you make a tradeoff between disk bandwidth and write latency. It means that journal files are written twice, but substantially reduces the latency to fsync the journal. I observed significant improvements to ycsb (which is latency bound) by turning that on.

      I don't know if we want to arbitrarily make that tradeoff or make it a new easy tunable. A cool way to do it (but much more of a project) would be to do it dynamically based on how much we are limited by write throughput (on EBS we have a hard cap and we can check how close we are to hitting it).

            Assignee:
            Keith Smith
            Reporter:
            Mathias Stearn
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated: