Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-26740

Total connections is not stable and spiky when user authentication is enabled

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.10
    • Component/s: Networking, Security
    • None
    • ALL

      MongoDB cluster is consist of 28 shards and 15 mongos.
      There's about 4K/sec queries per shard and small DMLs and there's about 600~1000 connections(from mongos) per shard.

      MongoDB connections graph is spiky only when user authentication is enabled Not with disabled authentication. And java client driver return errors(like below) or slow-responses when user authentication is enabled. (I attached connections and query/sec graphs).

      [ERROR] [c.k.s.m.g.r.RequestExecuteCallable] execute (71): Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 400 has been exceeded.
      com.mongodb.MongoWaitQueueFullException: Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 400 has been exceeded.

      Not exactly 5 minutes, but connection spike(and client queue error) is happened about 5 minutes intervals.
      I am not sure "ConnectionPool::kDefaultHostTimeout" cause this issue yet, But I think 5 minutes is too short to shutdown connection pool. it seems so aggressive. Is there any reason to choose 5 minutes to shutdown pool ?

      https://github.com/mongodb/mongo/blob/master/src/mongo/executor/connection_pool.cpp#L177

        1. Query.png
          Query.png
          45 kB
        2. diagnostic.data.tar.gz
          10.85 MB
        3. Connections.png
          Connections.png
          38 kB

            Assignee:
            spencer.jackson@mongodb.com Spencer Jackson
            Reporter:
            sunguck.lee@gmail.com 아나 하리
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: