-
Type:
Task
-
Resolution: Done
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Fully Compatible
-
TIG 2017-05-08, TIG 2017-05-29
-
0
When running a patch build, the "run tests" function in the etc/evergreen.yml project configuration should run resmoke.py once with
--suites=<suite_name> --excludeWithAnyTags=<suite_tags_to_exclude> ...
--tagFile=etc/test_lifecycle.yml
--excludeWithAnyTags=unreliable,
unreliable|${task_name},
unreliable|${task_name}|${build_variant},
unreliable|${task_name}|${build_variant}|${distro_id}
--reportFailureStatus=fail
--reportFile=report_reliable.json
and a second time with
--suites=<suite_name> --excludeWithAnyTags=<suite_tags_to_exclude> ...
--tagFile=etc/test_lifecycle.yml
--includeWithAnyTags=unreliable,
unreliable|${task_name},
unreliable|${task_name}|${build_variant},
unreliable|${task_name}|${build_variant}|${distro_id}
--reportFailureStatus=silentfail
--reportFile=report_unreliable.json
The exit code of the "run tests" function should be based on the exit code of the combine_reports.py script. The combine_reports.py script should also be run as part of the "post" task so that it happens even if the task times out since the SIGUSR1 and Windows event object should cause resmoke.py to write out a report.json file.
Note: The "unreliable*" tags should not be used when running in the Evergreen mainline.
- has to be done after
-
SERVER-28783 Add buildscripts/combine_reports.py script for combining report.json files
-
- Closed
-
- is related to
-
SERVER-29507 Task not marked as failure if resmoke.py fails to start fixtures
-
- Closed
-
- related to
-
-
- Closed
-