Very large Bulk batched write operations inside a transaction fail with transaction conflicts.
Error from the reproduction script:
Creating operations Starting transaction with 200 operations Start session 25e4510e-da67-47c6-9634-4d28242ca3ab Synthetic op to start transaction Attempting bulk write of 200 operations Inside transaction 1 Synthetic op to start transaction Attempting bulk write of 200 operations Inside transaction 2 Synthetic op to start transaction ... Attempting bulk write of 200 operations Inside transaction 164 Traceback (most recent call last): File "bulk_transactions.py", line 84, in <module> session.with_transaction( File "/usr/local/lib/python3.9/site-packages/pymongo/client_session.py", line 620, in with_transaction ret = callback(self) File "bulk_transactions.py", line 72, in perform_operation res = coll.bulk_write(ops, ordered=False, session=s) File "/usr/local/lib/python3.9/site-packages/pymongo/collection.py", line 508, in bulk_write bulk_api_result = blk.execute(write_concern, session) File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 529, in execute return self.execute_command(generator, write_concern, session) File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 360, in execute_command client._retry_with_session( File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1438, in _retry_with_session return self._retry_internal(retryable, func, session, bulk) File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1470, in _retry_internal return func(session, sock_info, retryable) File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 354, in retryable_bulk self._execute_command( File "/usr/local/lib/python3.9/site-packages/pymongo/bulk.py", line 310, in _execute_command result, to_send = bwc.execute(ops, client) File "/usr/local/lib/python3.9/site-packages/pymongo/message.py", line 950, in execute result = self.write_command(request_id, msg, to_send) File "/usr/local/lib/python3.9/site-packages/pymongo/message.py", line 1042, in write_command reply = self.sock_info.write_command(request_id, msg) File "/usr/local/lib/python3.9/site-packages/pymongo/pool.py", line 798, in write_command helpers._check_command_response(result, self.max_wire_version) File "/usr/local/lib/python3.9/site-packages/pymongo/helpers.py", line 167, in _check_command_response raise OperationFailure(errmsg, code, response, max_wire_version) pymongo.errors.OperationFailure: Encountered error from localhost:27018 during a transaction :: caused by :: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction., full error: {'ok': 0.0, 'errmsg': 'Encountered error from localhost:27018 during a transaction :: caused by :: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.', 'code': 112, 'codeName': 'WriteConflict', 'operationTime': Timestamp(1632204487, 4), '$clusterTime': {'clusterTime': Timestamp(1632204487, 4), 'signature': {'hash': b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'keyId': 0}}, 'errorLabels': ['TransientTransactionError']}
Python version: 3.9.7
pymongo version: 3.12.0
mongod version: 4.4.6-ent (sharded)
- related to
-
PYTHON-2915 Session persistence failure under transaction callback with large Bulk ops
- Closed