-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
This references https://jira.mongodb.org/browse/SERVER-17078. Similar to the earlier issue, things get much slower as there are more tables to open. I (again) modified ex_stat to create many tables and then open a stat cursor on each one. On my Mac it takes 17 seconds to open/get a stat/close with a stat cursor on 10000 tables. As the Jira issue point out, it appears that the three loops in _wt_open (twice walking the fh list) and _wt_block_open eat a lot of time.
I added name hashes for both of those structures to avoid the strcmp and that cuts it down to 15.5 seconds. Zoom shows us spending, overall, 12.3 seconds in *wt_curstat_open of which 2.8 seconds (23%) is *wt_block_open (1.5 seconds) and __wt_open (1.3 seconds) and I've confirmed the line numbers are the hash comparison. (Another 2.1 seconds (17%) is spent in config_next.)
Running with a statistics logging on and adding a stat counter in those loops shows that for 17084 curstat open calls, we were in the block name loop, and both open loops 36 million times each or an average of 2200 iterations through each loop for every curstat open call. It looks like both of those structures may need the hash buckets like dhandles.
- is related to
-
SERVER-17078 show databases taking extraordinarily long with wiredTiger
- Closed
-
WT-1734 Statistics cursor reset
- Closed
-
WT-1848 Build indexes for the base configuration strings.
- Closed
-
WT-1802 Make directory sync operation optional
- Closed
-
WT-1865 Performance degrades as more files are open.
- Closed
- related to
-
WT-1635 Add name_hash and hash bucket queues for fh and block. WT-1634
- Closed