-
Type: Epic
-
Resolution: Won't Do
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Done
-
Implement a spec-compliant connection pool
Summary
libmongoc does not have a notion of a per-server connection pool, but rather a client pool. There may be several benefits to reworking a mongoc_client_t to utilize a connection pool, as described in the Connection Monitoring and Pooling spec.
- wrapping drivers (C++ in particular) may benefit from a more idiomatic client model.
- fair waiting for connections (libmongoc's client pool does not have fair waiting)
- less total sockets created (different threads can utilize a shared pool of sockets, but clients from libmongoc's client pool own their sockets, which necessitates one socket per server per thread)
- implementation of remaining CMAP monitoring events (e.g. PoolClearedEvent, which could be used by wrapping drivers to detect if a fork occurs)
This would be a large change, likely require refactoring large parts of the driver. And it would increase the maintenance burden. Scoping should involve investigating whether this really is a worthwhile change.
- Should we test performance of libmongoc's client pool against a prototype mongoc_client_t with connection pool? This would further validate that such a large change is warranted. Since clients from libmongoc's client pool own their sockets, they do not need to lock a shared structure to obtain a socket. Additionally, clients can be popped/pushed back quickly, for the duration of a single operation (which I believe is what Swift does).
- Do we only make a mongoc_client_t thread safe? Or a mongoc_database_t/mongoc_collection_t as well.
- Do we want to support a connection pool for single-threaded mode? CMAP does not require multiple threads. And there may be legitimate uses for a connection pool in a single-threaded environment. For example, it is currently not possible to have more than one socket open against a server on a single threaded mongoc_client_t. That makes it impossible to have more than one exhaust cursor open on a single server. If so, let's also amend the CMAP spec to note that CMAP events are excluded from connections used only for monitoring, but connections shared between monitoring and application use are not excluded.
Improvements to test infrastructure and coverage (CDRIVER-2733) should likely be a prerequisite, and would provide more confidence in such a large change.
Motivation
Cast of Characters
- Product Owner: Rachelle Palmer
- Project Lead: Kevin Albertson
- Program Manager: Alexander Golin
Documentation
Scope Document
[Technical Design Document|]
- is depended on by
-
CDRIVER-3214 Ignore ConnectionReadyEvent in CMAP pool creation test
- Closed
-
CDRIVER-3816 Avoiding connection storms
- Development Complete
-
CXX-2133 Avoiding connection storms
- Development Complete
-
CDRIVER-2871 Implement Connection Monitoring and Pooling spec
- Closed
-
CDRIVER-4501 Add log messages to CMAP spec
- Closed
-
CDRIVER-3868 Record command monitoring and CMAP events and save statistics
- Closed
-
CXX-2166 Record command monitoring and CMAP events and save statistics
- Closed
-
CDRIVER-3791 Ensure ConnectionCreatedEvents are emitted before ConnectionReadyEvents
- Closed
-
CDRIVER-4513 CMAP connection establishment failure tests
- Closed
- related to
-
CDRIVER-5714 Skip flaky /Stepdown/getmore test cases
- Closed