-
Type: Task
-
Resolution: Unresolved
-
Priority: Minor - P4
-
None
-
Affects Version/s: None
-
Component/s: All Tools
-
None
-
1
-
537
Problem
The go-driver will split up bulk insert commands whose documents exceed 16MB, but all MongoDB tools incorrectly assume that bulk insert commands whose documents total size does not exceed 48MB will be inserted as a single command. This makes MongoDB tools unnecessarily wait to fill up at 48MB buffer before flushing the content to the database.
Relevant thread: https://mongodb.slack.com/archives/C7WJZNUTA/p1682000864772909
Relevant documentation: https://www.mongodb.com/docs/manual/reference/command/hello/#mongodb-data-hello.maxBsonObjectSize
Solution & Acceptance Criteria
Update MongoDB tools to use the correct 16MB (16 * 1024 * 1024) limit when inserting documents in bulk.
Code location: https://github.com/mongodb/mongo-tools/blob/306328af6931516b4344e32c2823f9294472456c/common/db/buffered_bulk.go#L20
Impact
MongoDB tools will no longer unnecessarily wait to fill a 48MB buffer before flushing the content, making the tools use the server resources more effectively.