Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-21073

Add option to resmoke.py to automatically re-run tests failed tests when running with multiple jobs

    • Type: Icon: Improvement Improvement
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.0-rc0
    • Component/s: Testing Infrastructure
    • Decision Automation Group

      As a way to improve the user experience when debugging locally, it would be useful to have an option where, if you were running many tests in parallel, it would repeat the failing ones in a serial execution once it was finished.

      The common use case I can think of, is something like the following. A user runs

      python buildscripts/resmoke.py --suites=<suite> -j8
      

      Halfway through that, it fails on some test. The problem is, it is very difficult to figure out why that test failed, since all the logs are intermingled. The user then runs the following to start debugging the test:

      python buildscripts/resmoke.py --executor=<suite> path/to/failing/test.js
      

      It would be nice if resmoke.py had some sort of option to automatically do that. I would be tempted to have it on by default, since we could easily turn it off everywhere in our evergreen.yml where we wouldn't want it on. The only thing that would be weird is logging to a file. If we did nothing intelligent, the second run would override the first. If the test were flaky, it might not fail the second time, and then the logs of the first run (and all other successful tests) would be gone. It would be better to either append to the file, or to write to a different file, say repeated_tests.log or something like that.

            Assignee:
            backlog-server-dag [DO NOT ASSIGN] Backlog - Decision Automation Group (DAG) (Inactive)
            Reporter:
            charlie.swanson@mongodb.com Charlie Swanson
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: