-
Type: New Feature
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Fully Compatible
-
TIG 2017-05-08
The buildscripts/combine_reports.py script takes the paths to at least one report.json file as command line arguments and writes a combined report.json file to its stdout. The order of test executions within each report.json file should be preserved and the merged order should follow the order in which the report.json files were specified on the command line.
Since a JS test isn't consider unreliable if one of the dynamic tests (i.e. a resmoke.py hook) fails after the test runs, the combine_reports.py script should exit with a non-zero return code even if a dynamic test has the "silentfail" status.
- (0) All tests have status "pass", or only non-dynamic tests have status "silentfail".
- (32) At least one dynamic test has status "silentfail", but no tests have status "fail" or "timeout".
- (31) At least one test has status "fail" or "timeout".
Note: A test can be considered dynamic if its name contains a ":" character.
- has to be done before
-
SERVER-28784 Update "run tests" function to run unreliable tests separately
- Closed