We have a MongoDB cluster with 2 mongod(replicaset).

After high load testing we found out that the queries start to fail and we get the following error:

2019-12-25 11:04:19,664 - ERROR:MainThread [celery.app.trace-256]: Task http_service.async_tasks.faceID_clear_up.face_process_util_by_step_async[619d7335-baa2-4927-8f34-32d45cfc313c] raised unexpected: OperationError('Update failed (cannot add session into the cache)',)
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/mongoengine/queryset/base.py", line 532, in update
result = update_func(query, update, upsert=upsert)
File "/opt/conda/lib/python3.6/site-packages/pymongo/collection.py", line 1076, in update_many
session=session),
File "/opt/conda/lib/python3.6/site-packages/pymongo/collection.py", line 856, in _update_retryable
_update, session)
File "/opt/conda/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1491, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/opt/conda/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session
return func(session, sock_info, retryable)
File "/opt/conda/lib/python3.6/site-packages/pymongo/collection.py", line 852, in _update
retryable_write=retryable_write)
File "/opt/conda/lib/python3.6/site-packages/pymongo/collection.py", line 822, in _update
retryable_write=retryable_write).copy()
File "/opt/conda/lib/python3.6/site-packages/pymongo/pool.py", line 613, in command
user_fields=user_fields)
File "/opt/conda/lib/python3.6/site-packages/pymongo/network.py", line 167, in command
parse_write_concern_error=parse_write_concern_error)
File "/opt/conda/lib/python3.6/site-packages/pymongo/helpers.py", line 159, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: cannot add session into the cache

There is a similar bug at: https://jira.mongodb.org/browse/SERVER-39044

Followed its' advise, We upgrade MongoDB from 4.0.4 to 4.0.13, however, it does no disapear.

After many try, I found a clue: When I upgrade pymongo from 3.7 to 3.10, this bug appear. And when downgrading from 3.10 to 3.7, it disappear.

Obviously, the root cause may be in pymongo 3.10.

Thanks a lot!