I'm seeing an error on session.close:
[1397453897:730451][12959:00b7ffe5ff7f0000], t, file:__wt0000.wt, session.close: the WiredTiger library cannot continue; the process must exit and restart
t: session.close: WT_PANIC: WiredTiger library panic
The application I'm using doesn't do anything too tricky. It starts a number of threads, each thread creates and opens a single table, and execute updates on that table. Once a certain number of updates have been completed, that thread then calls session.close. The application doesn't use explicit transactions, and also doesn't explicitly close the cursor.
The failure reproduces quickly, but not every time I run the test. The call stack and some other information:
(gdb) where #0 __wt_errx (session=session@entry=0x6b08a0, fmt=fmt@entry=0x496b98 "reconciliation illegally skipped an update") at ../src/support/err.c:317 WT-1 0x0000000000444c0d in __rec_write_wrapup (session=session@entry=0x6b08a0, r=r@entry=0x7fffd0003cd0, page=page@entry=0x7fffd0048010) at ../src/btree/rec_write.c:4638 WT-2 0x000000000044ce8f in __wt_rec_write (session=session@entry=0x6b08a0, ref=ref@entry=0x7fffd0004900, salvage=salvage@entry=0x0, flags=flags@entry=4) at ../src/btree/rec_write.c:413 WT-3 0x00000000004380d4 in __evict_file (syncop=4, session=0x6b08a0) at ../src/btree/bt_sync.c:214 WT-4 __wt_bt_cache_op (session=session@entry=0x6b08a0, ckptbase=<optimised out>, op=op@entry=4) at ../src/btree/bt_sync.c:325 WT-5 0x0000000000427c40 in __checkpoint_worker (session=<optimised out>, cfg=cfg@entry=0x0, is_checkpoint=is_checkpoint@entry=0) at ../src/txn/txn_ckpt.c:761 WT-6 0x00000000004282aa in __wt_checkpoint_close ( session=session@entry=0x6b08a0) at ../src/txn/txn_ckpt.c:875 WT-7 0x0000000000454e0c in __wt_conn_btree_sync_and_close ( session=session@entry=0x6b08a0) at ../src/conn/conn_dhandle.c:208 WT-8 0x0000000000455142 in __wt_conn_btree_close ( session=session@entry=0x6b08a0, locked=locked@entry=0) at ../src/conn/conn_dhandle.c:620 WT-9 0x000000000042271c in __wt_session_discard_btree ( session=session@entry=0x6b08a0, dhandle_cache=0x0) at ../src/session/session_dhandle.c:434 WT-10 0x0000000000421220 in __session_close_cache (session=<optimised out>) at ../src/session/session_api.c:39 WT-11 __session_close (wt_session=0x6b08a0, config=<optimised out>) at ../src/session/session_api.c:108 WT-12 0x0000000000402d4a in worker (arg=0x6c8ac0) at ../../../test/checkpoint/workers.c:202 WT-13 0x00007ffff77b8f6e in start_thread (arg=0x7fffe5ffb700) at pthread_create.c:311 WT-14 0x00007ffff74e39cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 (gdb) up 12 WT-12 0x0000000000402d4a in worker (arg=0x6c8ac0) at ../../../test/checkpoint/workers.c:202 202 if ((ret = session->close(session, NULL)) != 0) (gdb) p *cookie $20 = {id = 0, type = COL, uri = "table:__wt0000", '\000' <repeats 113 times>, session = 0x6b08a0, cursor = 0x7fffd0004020} (gdb) down 12 WT-1 0x0000000000444c0d in __rec_write_wrapup (session=session@entry=0x6b08a0, r=r@entry=0x7fffd0003cd0, page=page@entry=0x7fffd0048010) at ../src/btree/rec_write.c:4638 4638 WT_PANIC_RETX(session, (gdb) up WT-2 0x000000000044ce8f in __wt_rec_write (session=session@entry=0x6b08a0, ref=ref@entry=0x7fffd0004900, salvage=salvage@entry=0x0, flags=flags@entry=4) at ../src/btree/rec_write.c:413 413 ret = __rec_write_wrapup(session, r, page); (gdb) up WT-3 0x00000000004380d4 in __evict_file (syncop=4, session=0x6b08a0) at ../src/btree/bt_sync.c:214 214 WT_ERR(__wt_rec_write( (gdb) p *page $22 = {u = {intl = {recno = 1, parent_ref = 0x7fffd0048058, __index = 0x7fffd004c9c0}, row = {d = 0x1, ins = 0x7fffd0048058, upd = 0x7fffd004c9c0, entries = 2333}, col_fix = {recno = 1, bitf = 0x7fffd0048058 "(", entries = 3489974720}, col_var = {recno = 1, d = 0x7fffd0048058, repeats = 0x7fffd004c9c0, nrepeats = 2333, entries = 4693}}, dsk = 0x7fffd0042000, modify = 0x7fffd0004270, read_gen = 0, memory_footprint = 797902, type = 4 '\004', flags_atomic = 18 '\022'} (gdb) p *page->modify $23 = {disk_snap_min = 610900, rec_max_txn = 0, rec_min_skipped_txn = 159807, update_txn = 610869, bytes_dirty = 797902, u1 = {replace = { addr = 0x7fffd07a80c0 "\300\001\206\344\242Q\005i", size = 8 '\b', type = 3 '\003', reuse = 0 '\000'}, m = {multi = 0x7fffd07a80c0, multi_entries = 776}}, u2 = {root_split = 0x0, leaf = {append = 0x0, update = 0x7fffd0058020}}, ovfl_track = 0x0, write_gen = 3228, checkpoint_gen = 0, page_lock = 15 '\017', flags = 4 '\004'}
- is related to
-
WT-844 Flushing cached files on session close not always ideal
- Closed
-
WT-891 wtperf test1 with 8 tables
- Closed
-
WT-1017 voxer 10k - reconciliation illegally skipped an update
- Closed
- related to
-
WT-1 placeholder WT-1
- Closed
-
WT-2 What does metadata look like?
- Closed
-
WT-3 What file formats are required?
- Closed
-
WT-4 Flexible cursor traversals
- Closed
-
WT-5 How does pget work: is it necessary?
- Closed
-
WT-6 Complex schema example
- Closed
-
WT-7 Do we need the handle->err/errx methods?
- Closed
-
WT-9 Does adding schema need to be transactional?
- Closed
-
WT-10 Basic "getting started" tutorial
- Closed
-
WT-11 placeholder #11
- Closed
-
WT-12 Write more examples
- Closed
-
WT-13 Define supported platforms
- Closed
-
WT-14 Windows build
- Closed
-
WT-1018 Close files in a separate thread.
- Closed