Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7343

Write a script that executes many-collection-test.py

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.1, 4.4.8, 5.0.2, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • 5
    • Storage - Ra 2021-06-28, Storage - Ra 2021-07-12

      The million collection test was written years ago and is very outdated on how it gets executed. It merges the latest MongoDB with the latest WiredTiger and compiles them into a binary for the test to execute. This might have suited us when we dropped WiredTiger into MongoDB/master once in several months. Now that we do a drop almost every day, we are practically wasting effort merging the two repositories. Most of the runner script can be done away with and the test will need a complete re-write with the new features in the plan.

      The test runs with much fewer collections than a million. 19701 at the moment, though enough, to begin with, we will like to have the capability to have more as the testing progresses. Ideally, we would have several runs with a varied amount of collections, so we get an idea of how performance (or extent of the stalls) varies with the number of collections. Hence,

      • Have the capability to run the test on a "configured" amount of collections.
      • Have the capability to run multiple tests with a variation on the test parameters.
      • Eventually we will like the test to become a performance and a qualifying test on dhandle growth, so a means to capture performance relative to the test parameters like the number of collections will be useful.

      Edit:

      In WT-7336, a new many-collection test was written in order to be able to create workload where stalls happen while gathering statistics.

      This ticket is dedicated to:

      • the creation of a script that triggers many-collection-test.py. It is possible to start from the script that has been posted by Sulabh in WT-7336.
      • The creation of an evergreen test that triggers the new script.

            Assignee:
            etienne.petrel@mongodb.com Etienne Petrel
            Reporter:
            sulabh.mahajan@mongodb.com Sulabh Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: