Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-13693

improving vertical scalability of eviction

    • Storage Engines
    • 5
    • StorEng - 2024-11-26

      sys-perf 3 node replset 2024  ycsb.load_256thread.2024-05 with .8xlarge's is a good reproducer of this issue. You can see from the FTDC data as soon as it hits 20% dirty performance drops significantly (due to now using application threads on both the primary and secondary so write time is slower and wmajority is also slower)

      A couple of things I noticed:

      1. 1 eviction worker scales better than 4 - 16 eviction workers. Is this because it puts 1/3 of LRU in the eviction buffer then each eviction thread takes 1/2 of eviction buffer so with multithreaded setups there is diminishing returns? 1/2 -> 1/4 -> 1/8 etc...
      2. Perhaps it is better locality and 1 eviction worker per collection (btree) would be a more logical way to scale?

       
      Comparison of performance

      1. 90 kops/s with current setup
      2. 300 kops/s 60% dirty_trigger and 1 eviction worker thread (16 eviction worker threads cannot keep dirty level below 60% but 1 can somehow)

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            daniel.hill@mongodb.com Daniel Hill
            Votes:
            0 Vote for this issue
            Watchers:
            16 Start watching this issue

              Created:
              Updated: