Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-4172

Add diagnostic hazard pointer checks in more places before freeing refs

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.6.9, 4.0.2, 4.1.2, WT3.2.0
    • Affects Version/s: None
    • Component/s: None
    • Storage Non-NYC 2018-07-16, Storage Engines 2018-07-30

      We have seen an automated test failure where it looks like we are freeing a ref that should not have been freed. We can't see a code path where that could happen, so have added additional diagnostic information to hopefully be able to root cause if it happens again.

      test/format triggered a segfault while doing stress overflow testing, specifically, below line of execution: 

      eval $CMD file_type=row leaf_page_max=9 internal_page_max=9 key_min=256 value_min=256 || cleanup 

      Jenkins run:

      http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress-ppc/18538

      Stack trace:

      Backtrace:
      /lib64/libSegFault.so(+0x1734)[0x3fff883d1734]
      ??:0(??)[0x3fff88400478]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/../src/include/cell.i:769(__cell_data_ref)[0x101593dc]
      ??:0(??)[0x3fff541885e0]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/../src/include/cell.i:737(__wt_cell_unpack)[0x102071d4]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/../src/include/btree.i:1051(__wt_row_leaf_value_cell)[0x10208b3c]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/../src/include/cursor.i:468(__cursor_row_slot_return)[0x1020abd4]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/../src/btree/bt_curprev.c:523(__cursor_row_prev)[0x1020c200]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/../src/btree/bt_curprev.c:627(__wt_btcur_prev)[0x1020c724]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/../src/cursor/cur_file.c:147(__curfile_prev)[0x10188c74]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/test/format/../../../test/format/ops.c:1343 (discriminator 2)(nextprev)[0x1000edb8]
      /home/jenkins/jenkins/workspace/wiredtiger-test-format-stress-ppc/build_posix/test/format/../../../test/format/ops.c:1085(ops)[0x1000df6c]
      /lib64/power8/libpthread.so.0(+0x8944)[0x3fff88328944]
      /lib64/power8/libc.so.6(clone+0x98)[0x3fff881a7640] 

      The configuration:

       ############################################
       # RUN PARAMETERS
       ############################################
       abort=0
       alter=0
       auto_throttle=1
       backups=0
       bitcnt=2
       bloom=1
       bloom_bit_count=4
       bloom_hash_count=21
       bloom_oldest=1
       cache=68
       cache_minimum=20
       checkpoints=on
       checkpoint_log_size=155
       checkpoint_wait=52
       checksum=uncompressed
       chunk_size=1
       compaction=0
       compression=snappy
       data_extend=0
       data_source=table
       delete_pct=13
       dictionary=0
       direct_io=0
       encryption=rotn-7
       evict_max=5
       file_type=row-store
       firstfit=0
       huffman_key=0
       huffman_value=0
       independent_thread_rng=1
       in_memory=0
       insert_pct=34
       internal_key_truncation=0
       internal_page_max=9
       isolation=random
       key_gap=1
       key_max=256
       key_min=256
       leaf_page_max=9
       leak_memory=0
       logging=0
       logging_archive=1
       logging_compression=none
       logging_file_max=44452
       logging_prealloc=1
       long_running_txn=0
       lsm_worker_threads=4
       merge_max=13
       mmap=0
       modify_pct=22
       ops=0
       prefix_compression=1
       prefix_compression_min=8
       prepare=0
       quiet=1
       read_pct=0
       rebalance=1
       repeat_data_pct=44
       reverse=0
       rows=1000000
       runs=1
       salvage=1
       split_pct=75
       statistics=1
       statistics_server=0
       threads=9
       timer=4
       timing_stress_checkpoint=0
       timing_stress_lookaside_sweep=0
       timing_stress_split_1=0
       timing_stress_split_2=0
       timing_stress_split_3=0
       timing_stress_split_4=0
       timing_stress_split_5=0
       timing_stress_split_6=0
       timing_stress_split_7=0
       timing_stress_split_8=0
       timing_stress_split_9=0
       transaction_timestamps=0
       transaction-frequency=19
       truncate=1
       value_max=2497
       value_min=256
       verify=1
       wiredtiger_config=
       write_pct=31
       ############################################
      

        1. stacks.txt
          42 kB
          Keith Bostic

            Assignee:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Reporter:
            luke.chen@mongodb.com Luke Chen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: