-
Type: Improvement
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Fully Compatible
-
v4.2, v4.0
-
DAG 2019-09-09
-
0
-
2
The "CleanEveryN" test hook gets run every "N" tests. Due to the way tests are run, this could be run against a different test every execution. This means that when we use test runtime to calculate timeouts, we might not properly account for the "CleanEveryN" runtime and set a timeout too short. This is most problematic on suites made up of lots of short running tests. If we had a maximum number of tests per suite we used when dividing the tests up, this would no longer be a problem.
As a server engineer,
I want there to be a maximum number of tests per generated sub-suite,
So that the "CleanEveryN" hook does not cause timeouts.
AC:
- All suites that run the "CleanEveryN" hook set a maximum number of tests per suite.