sys-perf 3 node replset 2024  ycsb.load_256thread.2024-05 with .8xlarge's is a good reproducer of this issue. You can see from the FTDC data as soon as it hits 20% dirty performance drops significantly (due to now using application threads on both the primary and secondary so write time is slower and wmajority is also slower)

A couple of things I noticed:

1. 1 eviction worker scales better than 4 - 16 eviction workers. Is this because it puts 1/3 of LRU in the eviction buffer then each eviction thread takes 1/2 of eviction buffer so with multithreaded setups there is diminishing returns? 1/2 -> 1/4 -> 1/8 etc...
2. Perhaps it is better locality and 1 eviction worker per collection (btree) would be a more logical way to scale?

 
Comparison of performance

1. 90 kops/s with current setup
2. 300 kops/s 60% dirty_trigger and 1 eviction worker thread (16 eviction worker threads cannot keep dirty level below 60% but 1 can somehow)