Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-29120

mongos connections increase indefinitely on chunk md refresh

    • Type: Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.12
    • Component/s: Networking, Sharding
    • None
    • ALL

      We have a pretty intensive process that stresses our mongod servers. When this process is running, we encounter the following issue: suddenly, there's a sudden increase in # of connections mongos opens, and then it starts taking more and more cpu until it becomes totally stuck. Please see the 2 graphs attached.

      When this happens, mongos logs shows A LOT of open/close connection messages (to the mongod servers). This trigger for this process seems to be a chunk move. When the move is complete, the ChunkManager refreshes - and then the connections starts opening and closing endlessly.

      Note that our mongod+mongos run on super strong machines, c4.4xlarge.

      Is this known / any workaround?

      SERVER-22611 (opened by myself) seems to be related, but fixed only in 3.4, we are using 3.2.12.

        1. dan1.png
          121 kB
          Yoni Douek
        2. dan2.png
          97 kB
          Yoni Douek

            Assignee:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Reporter:
            yonido Yoni Douek
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: