Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-330

large numbers of overflow records slow performance

XMLWordPrintableJSON

      Alex writes:

      I've been running test/format today, and have noticed that merges can take a really long time. I stopped one instance in a debugger, and saw the following call stack:

      #0  0x0000000000473026 in __wt_rec_track_ovfl_reuse (session=0x161b5e0,
    page=0x7f72283fe580, data=0x7f722809c250, data_size=1811,
    addrp=0x7f722f4c7b70, addr_sizep=0x7f722f4c7b7c, foundp=0x7f722f4c7b78)
    at ../src/btree/rec_track.c:261
WT-1  0x00000000004424f4 in __rec_cell_build_ovfl (session=0x161b5e0,
    kv=0x7f7220000990, type=128 '\200', rle=0) at ../src/btree/rec_write.c:3849
WT-2  0x0000000000442407 in __rec_cell_build_val (session=0x161b5e0,
    data=0x7f7215c27c00, size=1353, rle=0) at ../src/btree/rec_write.c:3810
WT-3  0x000000000043de6b in __wt_rec_row_bulk_insert (cbulk=0x7f72241b9ce0)
    at ../src/btree/rec_write.c:1442
WT-4  0x0000000000477446 in __wt_bulk_insert (cbulk=0x7f72241b9ce0)
    at ../src/btree/bt_bulk.c:99
WT-5  0x0000000000474fc7 in __curbulk_insert (cursor=0x7f72241b9ce0)
    at ../src/cursor/cur_bulk.c:28
WT-6  0x000000000045425b in __wt_lsm_major_merge (session=0x161b5e0,
    lsm_tree=0x162cdb0) at ../src/lsm/lsm_merge.c:156
WT-7  0x0000000000412a16 in __wt_lsm_worker (arg=0x162cdb0)
    at ../src/lsm/lsm_worker.c:92

      When I looked at track_ovfl_reuse the mod->track_entries value was 295700. From what I can tell, each insert is linearly traversing the list of entries in track_ovfl_reuse only to return not found (and then grow the list).

            Assignee:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Reporter:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: