-
Type: New Feature
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
Fully Compatible
-
v3.6, v3.4
-
TIG 2018-02-12
-
15
There are test failures that are difficult to investigate without having access to the generated data files (e.g. backup_restore.js, wt_unclean_shutdown.js). We should consider either (a) adding support for tasks defined in etc/evergreen.yml to upload data files to S3, or (b) defining a new task in etc/evergreen.yml that runs these specific tests.
If we decide to add support for archiving data files in Evergreen without changing the resmoke.py suite definitions (option "a"), then we'll need a mechanism to upload data files immediately after any test failure. This may require the changes from EVG-694 to execute the s3.put command in a resmoke.py after_test hook. (It is unlikely we could get away with moving the data files to a separate directory and uploading all of them at end of the suite without potentially filling up the disks on our EC2 machines.) If we decide to define a new task for archiving data files in Evergreen (option "b"), then we could simply run it with using continue_on_failure=false.
Note: The archiving of the FTDC data is currently flawed, as only the FTDC data from the mongod processes in the last test (not necessarily even a failure) is uploaded to S3.
Technique currently available for patch builds
diff --git a/etc/evergreen.yml b/etc/evergreen.yml index 79bed55..f4b896e 100644 --- a/etc/evergreen.yml +++ b/etc/evergreen.yml @@ -286,7 +286,7 @@ functions: extra_args="$extra_args --continueOnFailure" fi - ${path_prefix} ${asan_symbolizer} ${enable_lsan} ${rlp_environment} ${python|python} buildscripts/resmoke.py ${resmoke_args} $extra_args ${test_flags} --log=buildlogger --reportFile=report.json + ! ${path_prefix} ${asan_symbolizer} ${enable_lsan} ${rlp_environment} ${python|python} buildscripts/resmoke.py ${resmoke_args} $extra_args ${test_flags} --log=buildlogger --reportFile=report.json "run jstestfuzz": - command: shell.exec @@ -1212,8 +1212,24 @@ tasks: - func: "do setup" - func: "run tests" vars: - resmoke_args: --suites=no_passthrough --storageEngine=wiredTiger - run_multiple_jobs: true + resmoke_args: --executor=no_passthrough jstests/noPassthrough/backup_restore.js --storageEngine=wiredTiger --wiredTigerEngineConfigString="verbose=[checkpoint]" --repeat=50 + continue_on_failure: false + - command: archive.targz_pack + params: + target: "dbpath.tgz" + source_dir: "/data/db" + include: + - "**" + - command: s3.put + params: + aws_key: ${aws_key} + aws_secret: ${aws_secret} + local_file: dbpath.tgz + remote_file: mongodb-mongo-master/${build_variant}/${revision}/no_passthrough/${task_id}-${execution}.tgz + bucket: mciuploads + permissions: public-read + content_type: ${content_type|application/x-gzip} + display_name: Files in dbpath - Execution ${execution} - <<: *task_template name: noPassthroughWithMongod
- is depended on by
-
SERVER-32898 Create unit test for resmoke module archival.py
- Closed
-
SERVER-33144 Support archiving data files in Evergreen on test failure - Windows
- Closed
-
SERVER-33193 Enable test failure archive for specific tasks and hooks
- Closed
- related to
-
SERVER-36111 Bug in job.py related to functionality for archiving data files in 3.4
- Closed
-
SERVER-31562 dump replica set oplogs at the end of every failed test
- Closed