Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-3580

workgen: dynamically change workload characteristics

    • Type: Icon: Improvement Improvement
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Tools
    • None

      Discussed with alexander.gorrod the desire for workgen to be able to dynamically modify its workload, perhaps to maintain a certain eviction rate. This could be managed as a periodic callback in Python. As a strawman to facilitate discussion, here's some code illustrating how it could work:

      # Assume that ins_thread, etc. already set up as in runner/multi_btree_heavy_stress.py
      threads = ins_thread * 1 + upd_thread * 1 + read_thread * 2
      workload = Workload(context, threads)
      
      # Define a callback that will be called periodically
      from wiredtiger import stat
      class MyCallback(WorkloadCallback):
          def __init__(self):
              self.mult = 1
              self.dirty = 0
      
          def callback(self, workload, conn):
              session = conn.open_session()
              cur = session.open_cursor('statistics:', None, None)
              dirty = cur[stat.conn.cache_eviction_dirty][2]
              new_dirty = self.dirty - dirty
              self.dirty = dirty
              session.close()
              if new_dirty < 100000:
                  self.mult += 1
              elif self.mult > 2:
                  self.mult -= 1
              threads = ins_thread * self.mult + upd_thread * self.mult + read_thread * 1
              workload.reset_threads(threads)
      
      workload.options.callback = MyCallback()
      workload.options.callback_time = 1  # every second
      workload.options.run_time = 30
      workload.options.report_interval = 1
      workload.run(conn)
      

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            donald.anderson@mongodb.com Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: