Support online compaction.
Because the WiredTiger btree implementation supports variable-size pages (with a minimum page size of 512B), and pages will naturally shrink as key/value pairs are removed, compaction is not expected to have much impact in reducing the amount of fragmentation within individual pages (if a larger minimum page size is specified, for example, to map to optimium underlying disk I/O transfer sizes, then there may be additional fragmentation within individual page that is worth addressing at some point).
However, if a large chunk of the btree is discarded, the file should shrink. This means we should compact the file by moving pages at the end of the file toward the beginning of the file, and truncating the space at the end of the file.
We considered freeing blocks within the file back to the filesystem, but believe support for doing that is filesystem dependent, and do not plan to take that approach.
It is important that compaction not affect the buffer cache.
Additionally, add an API that estimates the current fragmentation of the table (this is either the file size vs. the last checkpoint size, in other words, it's the result of a dump/load, or the file size vs. bytes on the free-list for the last checkpoint).
- is related to
-
WT-365 On-line compaction
- Closed