-
Type:
Improvement
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Fully Compatible
-
v3.4
-
TIG 2017-08-21
-
0
We occasionally experience network connectivity issues with the Evergreen API server while running tasks in Evergreen. This causes burn_in_tests.py to exit with a non-zero return code because of an uncaught connection error.
[2017/08/02 15:25:46.431] # Capture a list of new and modified tests.
[2017/08/02 15:25:46.431] /opt/mongodbtoolchain/v2/bin/python2 buildscripts/burn_in_tests.py --branch=master --buildVariant=suse12 --testListOutfile=jstests/new_tests.json --noExec $burn_in_args
[2017/08/02 15:25:53.510] Traceback (most recent call last):
[2017/08/02 15:25:53.510] File "buildscripts/burn_in_tests.py", line 416, in <module>
[2017/08/02 15:25:53.510] main()
[2017/08/02 15:25:53.510] File "buildscripts/burn_in_tests.py", line 376, in main
[2017/08/02 15:25:53.510] values.check_evergreen)
[2017/08/02 15:25:53.510] File "buildscripts/burn_in_tests.py", line 158, in find_changed_tests
[2017/08/02 15:25:53.510] last_activated = find_last_activated_task(revs_to_check, buildvariant, branch_name)
[2017/08/02 15:25:53.510] File "buildscripts/burn_in_tests.py", line 124, in find_last_activated_task
[2017/08/02 15:25:53.510] response = requests.get(api_prefix + "projects/" + project + "/revisions/" + githash)
[2017/08/02 15:25:53.510] File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/api.py", line 70, in get
[2017/08/02 15:25:53.511] return request('get', url, params=params, **kwargs)
[2017/08/02 15:25:53.511] File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/api.py", line 56, in request
[2017/08/02 15:25:53.511] return session.request(method=method, url=url, **kwargs)
[2017/08/02 15:25:53.511] File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
[2017/08/02 15:25:53.616] resp = self.send(prep, **send_kwargs)
[2017/08/02 15:25:53.616] File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
[2017/08/02 15:25:53.616] r = adapter.send(request, **kwargs)
[2017/08/02 15:25:53.616] File "/opt/mongodbtoolchain/v2/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
[2017/08/02 15:25:53.756] raise ConnectionError(e, request=request)
[2017/08/02 15:25:53.756] requests.exceptions.ConnectionError: HTTPConnectionPool(host='evergreen-api.mongodb.com', port=8080): Max retries exceeded with url: /rest/v1/projects/mongodb-mongo-master/revisions/fd01541d77ca1455c603c411f395d6ce34fab6f1 (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc1f2eb5c90>: Failed to establish a new connection: [Errno 111] Connection refused',))
We should remove the invocation of burn_in_tests.py the "compile" task and instead change the "burn_in_tests" task to run the git.get_project command to clone the mongodb/mongo repository and apply the user's patch if necessary. If we continue use the --noExec invocation of burn_in_tests.py, then we can make this logic in the "compile" task a command of type="system" so that failing to communicate with the Evergreen API server causes the task to turn purple.
- related to
-
SERVER-30634 Git detects all files as having changed on Windows causing burn_in_tests task to time out in Evergreen
-
- Closed
-