This issue is based on the discussion and results from WT-515.

In summary, if the application configures O_DIRECT on Linux, and then uses a system utility to copy files returned by the hot backup cursor (for example, cp or dd), WiredTiger sees problems: our guess is it's because the wrong versions of blocks are being returned from the OS cache to the WiredTiger reads, but that's just a guess.

Michael, here's what I was suggesting last night: add a WT_CURSOR based method that allows reads of the underlying object using WiredTiger's file descriptor, something relatively simple like:

cursor.read(cursor, void *buf, size_t buf_size, size_t *bytesp);

The method takes a buffer and size and returns [something] in the buffer, that if written into a file with same name as the file returned by the backup cursor, can be subsequently loaded back into WiredTiger.

In the case of the standard file manager, we'd copy file blocks into the buffer, in the case of a data-source, it would be something else, probably data-source dependent.

I'd make this the only way to do hot backups: it's not going to be significantly slower than using system utilities, and it means other data-sources are always supported.

Alternatively, we could have hot backup take a destination directory name and we could do all the work inside the engine, but that limits how/where the application can do hot backups, if that matters.

michaelcahill