In the server, the maximum BSON size for user documents is 16mb. Internally, we have a maximum size of 16mb + 16kb for internal documents that need to be slightly larger than the user maximum. An example is the oplog: when users insert a document that's close to 16MB in size, the corresponding oplog entry may exceed the user maximum to account for oplog entry metadata.
In the Cloud backup restore process, a PIT restore manually inserts oplog entries from a source cluster into a target cluster. If the oplog entries generated on the source cluster are sufficiently large, the oplog entries that we try to insert into the target cluster may be larger than 16mb. In that case, since the insert is happening via the regular insert codepath, it will fail validation here. This poses a problem for the restore process, as Cloud cannot get past these large documents. Users will be required to PIT restore to an earlier timestamp, or use a snapshot that already includes the large document, which may be undesirable.
In the server, we already have internal places where we skip document size validation, on account of oplog entries potentially exceeding the size limit. We should expose an undocumented server parameter to allow Cloud to skip document validation, specifically to address this edge case.
- related to
-
SERVER-84734 Oplog entries can come close to 16 MB + 16 KB size
- Open
-
SERVER-96086 Added document key in oplog entries can cause insert to fail due to size limit
- Backlog