This might be a Mongo C driver issue, not a database issue, but since the only variable changed between my tests is the server configuration, I created this under SERVER.

I am running Mongo DB v4.2.1 on Windows and the Mongo C driver v1.16.1-1.el8 on CentOS 8. This is a development machine, so the Mongo DB server is configured to run as a replica set with a single server. CentOS is running in HyperV, but looking at the CPU usage in VTune profiling results below, this is not caused by overloading CPUs.

What I noticed is that when I run an application test against this server configured as a standalone server, the test runs about 3 times faster than when it is configured as a replica set.

I tried to reproduce the same problem with a small Mongo Shell script and while it doesn't show the same difference, the test against a replica set still runs about 1.3-1.5 times slower against a replica set (~32s vs. ~48s). Sometimes the difference is as little as 1.2 times, but never as close as on Windows. The same test running Mongo Shell on Windows differs only about 1.1 times and is more consistent in terms of variance.

The two VTune screenshots below are bottom-up stacks showing the same application test doing about 80K look-ups by _id and single-document inserts in a database that contains about 2M documents. Same database in both tests. The only change is adding and removing replSetName in the configuration file and restarting the database server.

This VTune screenshot is against a standalone server. The bottom two green lines show CPU usage for application 2 threads interacting with Mongo C. You can see how each is using about 50-60% of CPU and the test runs for 90 seconds. The wait time in the poll function is 61.8 seconds for both threads combined.

This VTune screenshot shows the exact same application test running against the exact same Mongo DB server, but configured as a replica set. Notice on the CPU usage scale that the duration of the test is now 260 seconds and CPU usage for each application thread is barely 25%. The wait time now is 418 seconds for both threads combined.

The call stack is exactly the same for both screenshots.

Looking at Mongo C driver source, I see that _mongoc_socket_wait uses a fixed timeout, which is consistent with VTune profiling results, but I'm not sure if it's the server that changes send/receive timing or it's the client that polls differently. I didn't have a chance to profile the source.

Let me know if you want me to try anything in my configuration.