Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-5373

Million collection test timed out due to pymongo connection reset

    • Type: Icon: Bug Bug
    • Resolution: Gone away
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • 3
    • Storage Engines 2020-01-27, Storage Engines 2020-02-10, Storage Engines 2020-02-24, Storage Engines 2020-03-09, Storage Engines 2020-04-06

      The million collection test started to experience pymongo connection reset problem, and timed out after 2 hrs running the Evergreen shell command. The Evergreen task failed 6 out of the past 7 runs. The latest failure run:

      https://evergreen.mongodb.com/task/wiredtiger_large_scale_test_million_collection_test_5624464bba22fd9876fecf412c8eaa2e009eab85_19_12_27_02_39_23

      [2019/12/28 05:19:36.660] Starting a new run
      [2019/12/28 05:19:36.660] Start populating documents into collections ...
      [2019/12/28 05:19:36.660] Finish populating 10000 documents into each of the collections
      [2019/12/28 05:19:36.660] Command for the run:
      [2019/12/28 05:19:36.660]   java -jar ../POCDriver/bin/POCDriver.jar -i 0 -u 100 -k 0 -q 10000 -z 10000 -d 300 -y 4000 --collectionKeyMax 10000 -o ../results/out.csv -t 64 -b 1
      [2019/12/28 05:22:40.284] + tee ../results/results.txt
      [2019/12/28 05:22:40.284] Traceback (most recent call last):
      [2019/12/28 05:22:40.284]   File "largescale/run-test.py", line 218, in <module>
      [2019/12/28 05:22:40.284]     out, avg_l = get_last_ops(client)
      [2019/12/28 05:22:40.284]   File "largescale/run-test.py", line 45, in get_last_ops
      [2019/12/28 05:22:40.284]     res = client.admin.command('serverStatus')
      [2019/12/28 05:22:40.284]   File "/usr/lib64/python2.7/site-packages/pymongo/database.py", line 516, in command
      [2019/12/28 05:22:40.285]     codec_options, **kwargs)
      [2019/12/28 05:22:40.285]   File "/usr/lib64/python2.7/site-packages/pymongo/database.py", line 428, in _command
      [2019/12/28 05:22:40.285]     parse_write_concern_error=parse_write_concern_error)
      [2019/12/28 05:22:40.285]   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 482, in command
      [2019/12/28 05:22:40.286]     self._raise_connection_failure(error)
      [2019/12/28 05:22:40.286]   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 608, in _raise_connection_failure
      [2019/12/28 05:22:40.286]     _raise_connection_failure(self.address, error)
      [2019/12/28 05:22:40.286]   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 269, in _raise_connection_failure
      [2019/12/28 05:22:40.286]     raise AutoReconnect(msg)
      [2019/12/28 05:22:40.286] pymongo.errors.AutoReconnect: localhost:27017: [Errno 104] Connection reset by peer
      [2019/12/28 07:22:40.465] Running task-timeout commands.
      [2019/12/28 07:22:40.465] Command stopped early: context canceled
      [2019/12/28 07:22:40.465] + exit_code=1 

            Assignee:
            donald.anderson@mongodb.com Donald Anderson
            Reporter:
            luke.chen@mongodb.com Luke Chen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: