There have been a few very long running test/format jobs recently, a common configuration is:
timing_stress_split_9=1
I attached in a debugger and the job is here:
(gdb) where #0 0x00007f50cfc367a3 in select () from /lib64/libc.so.6 #1 0x000000000045b147 in __wt_sleep (seconds=0, micro_seconds=13340) at ../src/os_posix/os_sleep.c:32 #2 0x000000000050deb8 in __wt_timing_stress (session=0x7f50d13281b0, flag=1024) at ../src/include/misc.i:272 #3 0x0000000000510886 in __wt_row_search (session=0x7f50d13281b0, srch_key=0x2248f50, leaf=0x0, cbt=0x2248e50, insert=false, restore=false) at ../src/btree/row_srch.c:435 #4 0x00000000005965c3 in __cursor_row_search (session=0x7f50d13281b0, cbt=0x2248e50, leaf=0x0, insert=false) at ../src/btree/bt_cursor.c:385 #5 0x0000000000596c2e in __wt_btcur_search (cbt=0x2248e50) at ../src/btree/bt_cursor.c:530 #6 0x000000000052f07c in __curfile_search (cursor=0x2248e50) at ../src/cursor/cur_file.c:200 #7 0x000000000040a07c in read_row_worker (cursor=0x2248e50, keyno=613197, key=0x7ffc6730f190, value=0x7ffc6730f160, sn=false) at ../../../test/format/ops.c:1252 #8 0x0000000000409eee in wts_read_scan () at ../../../test/format/ops.c:1202 #9 0x000000000040d89c in main (argc=4, argv=0x7ffc6730f398) at ../../../test/format/t.c:207 (gdb) f 8 #8 0x0000000000409eee in wts_read_scan () at ../../../test/format/ops.c:1202 1202 switch (ret = read_row_worker( (gdb) p keyno $14 = 613197 (gdb) p g.key_cnt $11 = 1000000
After about 21 hours.
The long running job showing characteristics above:
http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress/67017/