-
Type: Improvement
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
DAG 2021-06-28
-
2
If task generation is not constrained to depend on the completion of compile_dist_test, then the task generation tasks can end up queued in front of compile_dist_test. That's essentially a priority inversion, since the tasks generated by the task generation tasks can't actually execute until after archive_dist_test_[debug] runs, and that task depends on compile_dist_test. Worse, other tasks which depend on compile_dist_test, like compile_unittests, are also delayed if task generation preempts compile_dist_test, despite the fact that they could run in parallel with task generation, since the generated tasks don't depend on those artifacts.
Another factor to consider is that if compile_dist_test fails, then all of the task generation work was entirely wasted. While that may not be that common an occurrence in the commit queue or waterfall, it is a very common occurrence in patch builds.
The best way to address this would be to make task generation depend on the start of archive_dist_test[_debug]. That way, task generation would run in parallel with archiving the results of compile_dist_test. If compile_dist_test fails, then archive_dist_test will never start and no task generation will happen. If compile_dist_test passes, then task generation would proceed in parallel with archiving up the results if compile_dist_test, and that task has a very low failure rate, so there is little chance of wasted work. Unfortunately, we can't make task generation wait on the start of a task until we have EVG-14537.
Since we don't currently have the ability to make task generation wait on the start of archive_dist_test[_debug], we should do the next best thing, and make it wait on the completion of compile_dist_test, which is very nearly the same thing.
- is depended on by
-
SERVER-56922 Task generation should depend on the start of archive_dist_test[_debug]
- Closed
- is duplicated by
-
SERVER-57003 Generate all tasks for a build variant in a single task
- Closed
- related to
-
SERVER-57003 Generate all tasks for a build variant in a single task
- Closed