Investigation

Scoped connection errors exist in a custom logical session refresh jsCore passthrough suite. This suite sets the logical session refresh time to 100ms.

At the end of each job, secondaries cannot create a connection to the primary because "only one usage of each socket address (protocol/network address/port) is normally permitted."

Throughout the job's lifetime, we see instances of "scoped connection" ... "not being returned to the pool" from all nodes.

This is the ordering of events I was able to put together from the highlighted lines on the Lobster log for one of the jobs.

1. The LogicalSessionCache Refresh thread on the primary starts a replica set monitor on itself (is this normal?)
2. The primary can’t find the primary (itself) in context of the replica set
3. The primary eventually finds the primary in context of the replica set
3. The primary and both secondaries creates scoped connections to the primary that are being terminated without being in a fail state (…"scoped connection"… "not being returned to the pool")
4. Client requests restart of the primary (I'm not sure if this actually happens, but why else would the CONTROL startup lines be repeated so late in the test?)
5. The LogicalSessionCache Refresh and Reap threads on secondaries now get connpool errors attempting to connect to the primary

Problem

We are letting ScopedDBConnections go out of scope before calling done() – the call to done() places the connection back on the pool.

Proposed fix

Call done() on the connection before the connection goes out of scope.