Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-10569

Fix no space left on device failure for format-abort-recovery-stress-test

    • Type: Icon: Build Failure Build Failure
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • 3
    • StorEng - Defined Pipeline

      format-abort-recovery-stress-test on ubuntu2004-stress-tests

      Host: i-031c3cc2e415c6454
      Project: wiredtiger
      Commit: feb6de96

      Task Logs:

      format-abort-recovery-stress-test task_log

      Logs:

      Segmentation Failure
      ./format.sh: line 668:  7962 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 483:  7995 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 668:  7927 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 411:  7894 Segmentation fault      nohup setsid $cmd > $log 2>&1
      ./format.sh: line 411:  9266 Segmentation fault      nohup setsid $cmd > $log 2>&1
      Link: https://evergreen.mongodb.com/task_log_raw/wiredtiger_ubuntu2004_stress_tests_format_abort_recovery_stress_test_feb6de965c15e2ca722ff70094c2f37f46265ebf_23_02_05_22_11_38/0?type=T#L812
      
      Verbose Error message
          [1675637787:820968][10598:0x7fdfdc7fe700], t, statlog-server: [WT_VERB_DEFAULT][ERROR]: __posix_file_write, 625: /data/mci/53deca030469c434ec62e12e52f7f867/wiredtiger/cmake_build/test/format/RUNDIR.2/./WiredTigerStat.05.22: handle-write: pwrite: failed to write 4631 bytes at offset 10567680: No space left on device
          [1675637787:821012][10598:0x7fdfdc7fe700], t, statlog-server: [WT_VERB_DEFAULT][ERROR]: __statlog_server, 599: statistics log server error: No space left on device
          [1675637787:821018][10598:0x7fdfdc7fe700], t, statlog-server: [WT_VERB_DEFAULT][ERROR]: __statlog_server, 599: the process must exit and restart: WT_PANIC: WiredTiger library panic
          [1675637787:821024][10598:0x7fdfdc7fe700], t, statlog-server: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 28: aborting WiredTiger library
          [1675637798:773924][10639:0x7f4e06b5f800], t, file:WiredTiger.wt, WT_CONNECTION.close: [WT_VERB_DEFAULT][ERROR]: __posix_open_file, 815: /data/mci/53deca030469c434ec62e12e52f7f867/wiredtiger/cmake_build/test/format/RUNDIR.5/WiredTiger.turtle.set: handle-open: open: No space left on device
          [1675637798:773981][10639:0x7f4e06b5f800], t, file:WiredTiger.wt, WT_CONNECTION.close: [WT_VERB_DEFAULT][ERROR]: __wt_block_checkpoint_resolve, 945: WiredTiger.wt: the checkpoint failed, the system must restart: Invalid argument
          [1675637798:773988][10639:0x7f4e06b5f800], t, file:WiredTiger.wt, WT_CONNECTION.close: [WT_VERB_DEFAULT][ERROR]: __wt_block_checkpoint_resolve, 945: the process must exit and restart: WT_PANIC: WiredTiger library panic
          [1675637798:773995][10639:0x7f4e06b5f800], t, file:WiredTiger.wt, WT_CONNECTION.close: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 28: aborting WiredTiger library
      Link: https://evergreen.mongodb.com/task_log_raw/wiredtiger_ubuntu2004_stress_tests_format_abort_recovery_stress_test_feb6de965c15e2ca722ff70094c2f37f46265ebf_23_02_05_22_11_38/0?type=T#L833
      

      logs

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Xgen-BuildBaron-User xgen-buildbaron-user
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: