Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1943

PyMongo does not validate bson document size in OP_MSG bulk writes

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • 5.0
    • Affects Version/s: None
    • Component/s: None
    • None

      On 4.0.11 a bulk insert with one small document followed by a document much larger than 16MiB. ie coll.insert_many([small_doc, very_large_doc]): 

        File "/Users/shane/git/mongo-python-driver/test/test_encryption.py", line 448, in test_large_op_msg_bulk
    self.db.coll.bulk_write([InsertOne(doc1), InsertOne(doc2)])
  File "/Users/shane/git/mongo-python-driver/pymongo/collection.py", line 504, in bulk_write
    bulk_api_result = blk.execute(write_concern, session)
  File "/Users/shane/git/mongo-python-driver/pymongo/bulk.py", line 511, in execute
    return self.execute_command(generator, write_concern, session)
  File "/Users/shane/git/mongo-python-driver/pymongo/bulk.py", line 346, in execute_command
    self.is_retryable, retryable_bulk, s, self)
  File "/Users/shane/git/mongo-python-driver/pymongo/mongo_client.py", line 1366, in _retry_with_session
    return func(session, sock_info, retryable)
  File "/Users/shane/git/mongo-python-driver/pymongo/bulk.py", line 341, in retryable_bulk
    retryable, full_result)
  File "/Users/shane/git/mongo-python-driver/pymongo/bulk.py", line 295, in _execute_command
    result, to_send = bwc.execute(ops, client)
  File "/Users/shane/git/mongo-python-driver/pymongo/message.py", line 896, in execute
    result = self.write_command(request_id, msg, to_send)
  File "/Users/shane/git/mongo-python-driver/pymongo/message.py", line 977, in write_command
    reply = self.sock_info.write_command(request_id, msg)
  File "/Users/shane/git/mongo-python-driver/pymongo/pool.py", line 690, in write_command
    helpers._check_command_response(result)
  File "/Users/shane/git/mongo-python-driver/pymongo/helpers.py", line 159, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: assertion src/mongo/util/hex.cpp:98

      The server log shows this:

      2019-08-06T17:09:42.571-0700 D NETWORK  [conn41] invalid message: BSONObjectTooLarge BSONObjectTooLarge: BSONObj size: 33552491 (0x1FFF86B) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: "encryption_exceeds_16mib_2" --
2019-08-06T17:09:42.571-0700 D COMMAND  [conn41] assertion while parsing command: UnknownError: assertion src/mongo/util/hex.cpp:98

      PyMongo should validate the sizes of documents it encodes for OP_MSG.

            Assignee:
            Unassigned Unassigned
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: