Add buildscripts/combine_reports.py script for combining report.json files

XMLWordPrintableJSON

    • Type: New Feature
    • Resolution: Done
    • Priority: Major - P3
    • 3.5.7
    • Affects Version/s: None
    • Component/s: Testing Infrastructure
    • None
    • Fully Compatible
    • TIG 2017-05-08
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      The buildscripts/combine_reports.py script takes the paths to at least one report.json file as command line arguments and writes a combined report.json file to its stdout. The order of test executions within each report.json file should be preserved and the merged order should follow the order in which the report.json files were specified on the command line.

      Since a JS test isn't consider unreliable if one of the dynamic tests (i.e. a resmoke.py hook) fails after the test runs, the combine_reports.py script should exit with a non-zero return code even if a dynamic test has the "silentfail" status.

      • (0) All tests have status "pass", or only non-dynamic tests have status "silentfail".
      • (32) At least one dynamic test has status "silentfail", but no tests have status "fail" or "timeout".
      • (31) At least one test has status "fail" or "timeout".

      Note: A test can be considered dynamic if its name contains a ":" character.

            Assignee:
            Jonathan Abrahams (Inactive)
            Reporter:
            Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: