Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-6981

Add randomness to Python test suite runs

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.0, 4.9.0, 4.4.4
    • Affects Version/s: None
    • Component/s: None
    • 5
    • Storage - Ra 2021-01-11, Storage - Ra 2021-01-25

      There is some use of random functions in the Python test suite to generate somewhat random situations or input when appropriate. However we (purposely) use the same seed to get predictable and reproducible runs each time. This is usually appropriate for a functional test suite. However, we might expose some errors if we were to change the initial seed.

      Investigate having a separate run (like our long run) that generates a random seed on each run that is reported at the beginning of the run, and a way to rerun with a given seed. I think for our regular pull request runs, it's still appropriate to have "predictable" randomness.

      Note that if you look for uses of `suite_random` in the test suite there are only a half dozen. However, one of those uses is in how long lists of scenarios are pruned. Changing which scenarios are run may give us some more test coverage.

            Assignee:
            jie.chen@mongodb.com Jie Chen
            Reporter:
            donald.anderson@mongodb.com Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: