-
Type: Bug
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: 4.1.11
-
Component/s: None
-
None
-
ALL
-
-
Storage NYC 2019-05-20, Sharding 2019-05-06
-
45
Various driver transaction spec tests are failing on latest sharded clusters. The one commonality between the failures is that they all use read concern "snapshot" and they all fail with SnapshotTooOld on the first operation in the transaction. Was there a recent change in how "snapshot" works?
Although this issue fails consistently in Evergreen, creating a standalone repro has proven difficult. When starting a fresh sharded cluster the attached repro (PYTHON-1796.py) succeeds. However if I run the entire driver test suite against the cluster first, then the repro starts failing consistently with SnapshotTooOld. Here's an example evergreen failure from today:
https://evergreen.mongodb.com/task/mongo_python_driver_tests_python_version_supports_openssl_110_test_ssl__auth_ssl~auth_ssl_python_version_requires_openssl_102_plus~3.7_test_latest_sharded_cluster_ea8941ef5d3f60a227cb89021ef7d65d7b06c6e1_19_04_16_21_00_56
And here's the repro failing:
$ python PYTHON-1796.py First transaction command failed: {'ok': 0.0, 'errmsg': 'Transaction 060cae38-7be3-4240-babe-65739200884c:1 was aborted on statement 0 due to: a non-retryable snapshot error :: caused by :: Encountered error from localhost:27019 during a transaction :: caused by :: Read timestamp Timestamp(1555453221, 10) is older than the oldest available timestamp.', 'code': 239, 'codeName': 'SnapshotTooOld', 'operationTime': Timestamp(1555453221, 11), '$clusterTime': {'clusterTime': Timestamp(1555453221, 11), 'signature': {'hash': b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'keyId': 0}}, 'errorLabels': ['TransientTransactionError']} Traceback (most recent call last): File "PYTHON-1796.py", line 13, in <module> coll.insert_one({'_id': i}, session=s) File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 694, in insert_one session=session), File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 608, in _insert bypass_doc_val, session) File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 596, in _insert_one acknowledged, _insert_command, session) File "/home/ubuntu/mongo-python-driver/pymongo/mongo_client.py", line 1342, in _retryable_write return self._retry_with_session(retryable, func, s, None) File "/home/ubuntu/mongo-python-driver/pymongo/mongo_client.py", line 1295, in _retry_with_session return func(session, sock_info, retryable) File "/home/ubuntu/mongo-python-driver/pymongo/collection.py", line 591, in _insert_command retryable_write=retryable_write) File "/home/ubuntu/mongo-python-driver/pymongo/pool.py", line 579, in command unacknowledged=unacknowledged) File "/home/ubuntu/mongo-python-driver/pymongo/network.py", line 150, in command parse_write_concern_error=parse_write_concern_error) File "/home/ubuntu/mongo-python-driver/pymongo/helpers.py", line 155, in _check_command_response raise OperationFailure(msg % errmsg, code, response) pymongo.errors.OperationFailure: Transaction 060cae38-7be3-4240-babe-65739200884c:1 was aborted on statement 0 due to: a non-retryable snapshot error :: caused by :: Encountered error from localhost:27019 during a transaction :: caused by :: Read timestamp Timestamp(1555453221, 10) is older than the oldest available timestamp.
I can reproduce this on the latest version:
db version v4.1.10-94-ga654dcf
git version: a654dcf592ea7ed65426a0de96b4079ff4fc6716
- causes
-
PYTHON-1796 Transaction tests that use readConcern "snapshot" are failing with snapshot errors
- Closed
- depends on
-
SERVER-36956 Replace the statistic that dynamically resizes the snapshot history window
- Closed
- is depended on by
-
CSHARP-2629 Remove workaround for SERVER-40685
- Closed
- is related to
-
PYTHON-1901 Avoid using readConcern snapshot in sharded transaction tests
- Closed
-
CSHARP-2601 Fix consistently failing convenient api for transactions test on latest
- Closed
-
SERVER-41472 Create a workload to demonstrate the MongoDB usage difficulties of the WT WT_STAT_CONN_CACHE_LOOKASIDE_SCORE stat
- Closed
-
RUBY-1942 Avoid using readConcern snapshot in sharded transaction tests
- Closed
- related to
-
CDRIVER-3301 Unskip txn tests with readConcern=snapshot against mongos
- Closed