Bulk-load blocks checkpoint operations (bulk-load has an exclusive lock on a handle, and checkpoint will wait to acquire a lock on the handle). This is a bug, we can't block checkpoints for a multi-hour bulk load.
We should fix this with two changes:
1. Change attempts to open handles that are exclusive (bulk-load, verify, salvage, upgrade) to return EBUSY. This is already the case in the other direction, that is, attempts to bulk-load, verify, salvage, or upgrade an object that is open in another cursor already returns EBUSY. While it is possible an application might want cursor-open to wait on a bulk-load operation, it's just as likely an application doesn't want to wait for a few hours while a bulk-load finishes, and (1) it's simpler for us to return EBUSY consistently, and (2) it's possible for an application to implement "loop until open succeeds" code, but it isn't possible for an application to implement "don't wait if it's going to take a long time code", and if we don't make this change, we're going to end up driving a O_NONBLOCK flag down through the API a couple of years from now.
2. Change checkpoint to simply create empty-file checkpoints for files which return EBUSY on the open. (We can't just skip checkpoints for bulk-loaded files – if the application creates a named checkpoint, a checkpoint for the name has to exist.) This means a checkpoint of a partially-loaded bulk-file will look empty, but I don't see any problem with that.
- is related to
-
WT-332 bulk-load blocks checkpoint
- Closed