Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-56921

Task generation should depend on the completion of compile_dist_test

    • DAG 2021-06-28
    • 2

      If task generation is not constrained to depend on the completion of compile_dist_test, then the task generation tasks can end up queued in front of compile_dist_test. That's essentially a priority inversion, since the tasks generated by the task generation tasks can't actually execute until after archive_dist_test_[debug] runs, and that task depends on compile_dist_test. Worse, other tasks which depend on compile_dist_test, like compile_unittests, are also delayed if task generation preempts compile_dist_test, despite the fact that they could run in parallel with task generation, since the generated tasks don't depend on those artifacts.

      Another factor to consider is that if compile_dist_test fails, then all of the task generation work was entirely wasted. While that may not be that common an occurrence in the commit queue or waterfall, it is a very common occurrence in patch builds.

      The best way to address this would be to make task generation depend on the start of archive_dist_test[_debug]. That way, task generation would run in parallel with archiving the results of compile_dist_test. If compile_dist_test fails, then archive_dist_test will never start and no task generation will happen. If compile_dist_test passes, then task generation would proceed in parallel with archiving up the results if compile_dist_test, and that task has a very low failure rate, so there is little chance of wasted work. Unfortunately, we can't make task generation wait on the start of a task until we have EVG-14537.

      Since we don't currently have the ability to make task generation wait on the start of archive_dist_test[_debug], we should do the next best thing, and make it wait on the completion of compile_dist_test, which is very nearly the same thing.

            Assignee:
            david.bradford@mongodb.com David Bradford (Inactive)
            Reporter:
            andrew.morrow@mongodb.com Andrew Morrow (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: