Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-84734

Oplog entries can come close to 16 MB + 16 KB size

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 6.0.0, 7.0.0
    • Component/s: None
    • Replication
    • ALL

      The maximum size of a BSON document is 16 MB. Since insert oplog entries must contain the entire BSON document on insert, along with other metadata, we allow for 16 KB of buffer space for internal BSON documents - therefore the internal limit is 16 MB + 16 KB - otherwise no oplog entry would be able to contain an inserted BSON document of size 16 MB.

      However, usually there isn't enough metadata in an oplog entry for it to actually reach the internal limit of 16 MB + 16 KB. Unfortunately, after SERVER-61891, it became possible for oplog entries to come close to the internal limit of 16 MB + 16 KB (when inserts with big _id s are performed). This isn't problematic at the time of writing the oplog, since the oplog entry is still under the limit, but it is at the time of reading the oplog entry.

      When you read an oplog entry, the cursor response, which itself is an internal BSON and is therefore subject to the 16 MB + 16 KB, will contain the oplog entry, along with more metadata. Since the oplog entry itself has come close to the 16 MB + 16 KB limit, the cursor response is unable to hold the oplog entry along with more metadata as this causes the response to cross the 16 MB + 16 KB limit.

      This can cause replication to stop, because the primary will not be able to create a response to send to the secondaries. See links.

            Assignee:
            backlog-server-repl [DO NOT USE] Backlog - Replication Team
            Reporter:
            vishnu.kaushik@mongodb.com Vishnu Kaushik
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: