-
Type: New Feature
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Fully Compatible
-
TIG 2018-04-23
-
0
Create a Python script that given (a) the name of an Evergreen task, (b) a list of build variants, and (c) a target execution time, uses Evergreen’s REST API to compute the time taken by recent successful executions of the test and generates resmoke.py YAML suite files running subsets of the tests taking no longer than the target execution time.
- It should be possible to modify buildscripts/test_failures.py to report the duration of successful executions instead of just the number of their occurrences. Doing so would avoid needing to create another wrapper around the /test_history Evergreen endpoint.
- The algorithm for determining how to split up a resmoke.py test suite should prefer to generate more resmoke.py YAML suite files that are well under the target execution time than to generate any resmoke.py YAML suite files that are above the target execution time given the historical average of each test's execution time.
The Python script should create multiple resmoke.py YAML suite files in the buildscripts/resmokeconfig/suites/ directory along the lines of the following, with perhaps some additional information written to stdout describing what build variants the new Evergreen tasks should be run against.
# This file was generated by buildscripts/generate_resmoke_suites.py and manual edits should also be # made to sharding.yml in order to ensure running the full test suite locally matches the behavior # of running parts of the test suite in Evergreen. It was generated against commit ... # with the following options: ... test_kind: js_test selector: # The following tests take approximately # 17 minutes to run on Enterprise RHEL 6.2 # 23 minutes to run on Linux DEBUG # 34 minutes to run on Enterprise Windows 2008R2 roots: - jstests/sharding/aggregation_currentop.js - jstests/sharding/cleanup_orphaned_auth.js - jstests/sharding/crash_mongos_against_upgraded_cluster.js - jstests/sharding/cursor_valid_after_shard_stepdown.js - jstests/sharding/delete_during_migrate.js - jstests/sharding/mapReduce_nonSharded.js - jstests/sharding/migration_id_index.js - jstests/sharding/migration_ignore_interrupts_3.js - jstests/sharding/migration_server_status.js - jstests/sharding/min_optime_recovery_on_successful_move_chunk_commit.js - jstests/sharding/mongos_query_comment.js - jstests/sharding/mongos_shard_failure_tolerance.js - jstests/sharding/movechunk_with_default_paranoia.js - jstests/sharding/movechunk_with_moveParanoia.js - jstests/sharding/mr_shard_version.js - jstests/sharding/read_does_not_create_namespaces.js - jstests/sharding/shard_aware_init.js - jstests/sharding/shard_kill_and_pooling.js - jstests/sharding/tag_auto_split.js - jstests/sharding/tag_range.js executor: config: shell_options: nodb: '' readMode: commands
- is depended on by
-
SERVER-34330 Split up long-running Evergreen tasks into shorter execution units
- Closed