Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2035

index read leads to read of main table

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • WT2.7.0
    • Affects Version/s: WT2.6.1
    • Component/s: None
    • None

      Hi!

      We have a table with index and sometimes want to read only index.
      I.e. we're only interested in pairs (index key, primary key)

      So we tried to use index cursor projection as described here

      But it seems that WiredTiger nevertheless reads the main table although it doesn't need to (I think).

      The problem is that our main table is quite big (several GB's) and scanning through index lead to it being read non-sequentially which is quite slow.

      Small demo example attached (should be build with -rdynamic)
      Output I get:

      $ ./read_index
      Filling DB...
      total 481716
      -rw-r--r-- 1 sbn eng    757760 Aug  7 15:03 main_index.wti
      -rw-r--r-- 1 sbn eng 491974656 Aug  7 15:03 main.wt
      -rw-r--r-- 1 sbn eng        45 Aug  7 15:03 WiredTiger
      -rw-r--r-- 1 sbn eng       298 Aug  7 15:03 WiredTiger.basecfg
      -rw-r--r-- 1 sbn eng        21 Aug  7 15:03 WiredTiger.lock
      -rw-r--r-- 1 sbn eng       938 Aug  7 15:03 WiredTiger.turtle
      -rw-r--r-- 1 sbn eng     36864 Aug  7 15:03 WiredTiger.wt
      
      Reading index using index: cursor...
      total 0
      lrwx------ 1 sbn eng 64 Aug  7 15:03 0 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 1 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 2 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 3 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.lock
      lrwx------ 1 sbn eng 64 Aug  7 15:03 4 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.wt
      lrwx------ 1 sbn eng 64 Aug  7 15:03 5 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/main_index.wti
      lrwx------ 1 sbn eng 64 Aug  7 15:03 6 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/main.wt
      key_format = i, value_format = i
      pread() fd stats:
      5:      745472
      6:      491962368
      
      Reading index using file: cursor...
      total 0
      lrwx------ 1 sbn eng 64 Aug  7 15:03 0 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 1 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 2 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 3 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.lock
      lrwx------ 1 sbn eng 64 Aug  7 15:03 4 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.wt
      lrwx------ 1 sbn eng 64 Aug  7 15:03 5 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/main_index.wti
      key_format = ii, value_format = 
      pread() fd stats:
      5:      745472
      

      Thanks!

            Assignee:
            donald.anderson@mongodb.com Donald Anderson
            Reporter:
            Dmitri Shubin Dmitri Shubin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: