-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
After fixing WT-6229, WT still issues hundreds of repeated reads of the same data from the HS file during shutdown. This extra IO is bad for performance.
Although I detected this problem during shutdown, it may occur during regular checkpoint processing, as well.
Using the test case Eric Mikie describes in BF-17346, I enabled verbose read debugging during shutdown by setting the WT_VERB_READ flag at the beginning of wt_txn_global_shutdown(). I sorted and counted the resulting messages to see how many times WT issues reads to the same block. The following reads from WiredTigerHS.wt each occurred 600+ times:
file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=94208, len=12288 file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=28672, len=12288 file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=16384, len=12288 file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=86016, len=8192 file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=69632, len=16384 file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=40960, len=12288 file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=53248, len=16384 file:WiredTigerHS.wt, WT_CURSOR.search_near: [WT_VERB_READ] read: /data/db0/job0/mongorunner /backupRestore/mongod-20020/WiredTigerHS.wt, fd=22, offset=4096, len=12288
This test uses a 1GB WT cache size and the total size of its data files is ~5MB. So it is unlikely that these reads are caused by capacity misses in the cache. Rather, it seems like there are additional cases (similar to WT-6229) where WT evicts HS pages that should remain in the cache.