Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12433

Limit background compaction activity based on eviction strategy

    • Type: Icon: Improvement Improvement
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Compaction
    • Storage Engines
    • 8
    • StorEng - Defined Pipeline, 2024-02-20_A_near-death_puffin, 2024-03-05 - Claronald

      Compaction can trigger eviction as it may generate enough dirty content that eviction needs to kick in to evict it.

      In SERVER-80124, we have observed scenarios where compaction, eviction and checkpoint are blocking each other:

      • Compaction generates dirty content from the table it is compacting, enough for eviction to be triggered
      • Checkpoint processes that same table
      • Eviction needs to remove the dirty content but cannot as checkpoint is running on the table, see this check.

      One idea here is to limit the pressure from background compaction on the system by checking the eviction strategy. If the strategy corresponds to WT_CACHE_EVICT_DIRTY_HARD, it would be better to stop background compaction until the cache is in a better state.

        1. ycsb-perf.png
          ycsb-perf.png
          181 kB
        2. Screenshot 2024-02-22 at 9.15.40 am.png
          Screenshot 2024-02-22 at 9.15.40 am.png
          154 kB
        3. Screenshot 2024-02-22 at 9.12.22 am.png
          Screenshot 2024-02-22 at 9.12.22 am.png
          174 kB
        4. background_compact_stress.txt
          0.9 kB

            Assignee:
            sean.watt@mongodb.com Sean Watt
            Reporter:
            etienne.petrel@mongodb.com Etienne Petrel
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: