Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-88040

Make benchmark tasks run faster in Evergreen

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Build
    • Build OnDeck

      Running a benchmark in Evergreen currently consists of two steps:

      1. Compiling the benchmarks for whatever variant you are running on
      2. Actually running the benchmarks once they're compiled

      This is a very slow process right now due to the fact the compile task is currently generating both debug and non-debug information in one task and creating a single tarball with both sets of information in it. Untarred, that information is 140GB in size (120+ GB of which is just debug information). 

      Over the course of the two steps outlined above, the tasks can spend 90min+ just tarring, uploading, downloading, and then untarring that information. 

      We should split out the debug information from the artifacts we need to just run the benchmarks so that we can spend less time downloading information that isn't needed by the task.

      -----------------------------------------------------------------------------------

      AC:

      • Split up compile_upload_benchmarks in a similar way as to how archive_dist_test and archive_dist_test_debug are
      • Make the benchmark tasks only rely upon the archive_dist_test task equivalent
      • Make the benchmark tasks not download the debug information
      • Reduce the upfront time it takes to run benchmark tasks

            Assignee:
            daniel.moody@mongodb.com Daniel Moody
            Reporter:
            jeff.zambory@mongodb.com Jeffrey Zambory
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: