Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1157

LSM merges in update-heavy workloads

    • Type: Icon: Task Task
    • Resolution: Done
    • WT2.4.0
    • Affects Version/s: None
    • Component/s: None

      The voxer-130k-short wtperf run dropped between 10-15% in performance between builds [287] (http://mjc.homeunix.org:8180/job/wiredtiger-perf-voxer-130k/287/) and [288] (http://mjc.homeunix.org:8180/job/wiredtiger-perf-voxer-130k/288/) (here's the plot, it's the drop at the [end of July] (http://mjc.homeunix.org:8180/job/wiredtiger-perf-voxer-130k/plot/).

      The change that's responsible is number 4 in build 288 (839742b, "Switch from using the checkpoint lock to serialize bulk-load close and database checkpoints, to using a new WT_DATA_HANDLE lock").

      Here are the runs:

      build 287:
      read: avg: 21239788
      insert: update: avg: 317920
      
      build 287 + change 1
      read: avg: 21346948
      insert: update: avg: 320332
      
      build 287 + change 4 (skipping changes 2 & 3, that was a change and its reversion)
      read: avg: 18351485
      insert: update: avg: 314058
      

      I've been looking at this, and I'm suspicious there's an open/close pattern in LSM that I don't understand, where the close handle spinlock is being held by checkpoint (implying a bulk-loaded file, I think), and is blocking LSM from making forward progress? It could be the reverse, but that should just slow down a checkpoint, it shouldn't affect the read performance of the run.

      @michaelcahill, any thoughts?

            Assignee:
            Unassigned Unassigned
            Reporter:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: