-
Type:
Improvement
-
Resolution: Won't Do
-
Priority:
Major - P3
-
None
-
Affects Version/s: 4.2.0
-
Component/s: Backup, Block Manager
-
(copied to CRM)
-
8
-
Storage Engines 2019-12-16, Storage Engines 2019-12-30, Storage Engines 2020-01-13
-
v4.2
Consider a case where there are a lot of tables with some random subset only receiving a small amount of data between checkpoints. A checkpoint modifies the file: row (due to a changing LSN and other fields, e.g: [1]), inside the WiredTiger.wt file for each table that received an update. This results in the new image of the block that row belongs to getting written out to disk. Additionally, because backup cursors stop WT from reusing data from interim checkpoints, the file growth here can be surprisingly large compared to the growth of user data that was actually inserted over the same amount of time.
[1]
access_pattern_hint=none,allocation_size=4KB,app_metadata=,assert=(commit_timestamp=none,durable_timestamp=none,read_timestamp=none),block_allocation=best,block_compressor=,cache_resident=false,checkpoint=(WiredTigerCheckpoint.10=(addr="018581e4341ae4908681e4424107b48781e47820e09f808080e26fc0cfc0",order=10,time=1567107756,size=8192,newest_durable_ts=0,oldest_start_ts=0,oldest_start_txn=0,newest_stop_ts=-1,newest_stop_txn=-11,write_gen=30)),checkpoint_lsn=(2,184448),checksum=uncompressed,collator=,columns=,dictionary=0,encryption=(keyid=,name=),format=btree,huffman_key=,huffman_value=,id=2,ignore_in_memory_cache_size=false,internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=4KB,key_format=u,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=0,log=(enabled=true),memory_page_image_max=0,memory_page_max=5MB,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,value_format=u,version=(major=1,minor=1)