Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-4567

mongos process doesn't update on DNS change

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 1.8.4
    • Component/s: Sharding
    • ALL

      After changing the DNS entry and IP for a shard (by removing it from /etc/hosts), the mongos process continued to look for the old IP. DNS should be re-resolved at "trying reconnect" step.

      Partial logs attached, restarting the mongos process fixed the problem.

      Tue Dec 27 19:35:54 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:35:54 [Balancer] reconnect mongoshard05:27017 failed couldn't connect to server mongoshard05:27017
      Tue Dec 27 19:35:54 [Balancer] ReplicaSetMonitor::_checkConnection: caught exception mongoshard05:27017 socket exception
      Tue Dec 27 19:35:54 [Balancer] trying reconnect to mongoshard06:27017
      Tue Dec 27 19:35:57 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:35:59 [Balancer] reconnect mongoshard06:27017 failed couldn't connect to server mongoshard06:27017
      Tue Dec 27 19:35:59 [Balancer] ReplicaSetMonitor::_checkConnection: caught exception mongoshard06:27017 socket exception
      Tue Dec 27 19:35:59 [ReplicaSetMonitorWatcher] trying reconnect to mongoshard04:27017
      Tue Dec 27 19:36:00 [Balancer] ~ScopedDbConnection: _conn != null
      Tue Dec 27 19:36:00 [Balancer] caught exception while doing balance: socket exception
      Tue Dec 27 19:36:04 [ReplicaSetMonitorWatcher] reconnect mongoshard04:27017 failed couldn't connect to server mongoshard04:27017
      Tue Dec 27 19:36:04 [ReplicaSetMonitorWatcher] trying reconnect to mongoshard05:27017
      Tue Dec 27 19:36:06 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:09 [ReplicaSetMonitorWatcher] reconnect mongoshard05:27017 failed couldn't connect to server mongoshard05:27017
      Tue Dec 27 19:36:09 [ReplicaSetMonitorWatcher] trying reconnect to mongoshard06:27017
      Tue Dec 27 19:36:11 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:12 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:14 [ReplicaSetMonitorWatcher] reconnect mongoshard06:27017 failed couldn't connect to server mongoshard06:27017
      Tue Dec 27 19:36:15 [ReplicaSetMonitorWatcher] trying reconnect to mongoshard04:27017
      Tue Dec 27 19:36:15 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:16 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:16 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:19 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:20 [ReplicaSetMonitorWatcher] reconnect mongoshard04:27017 failed couldn't connect to server mongoshard04:27017
      Tue Dec 27 19:36:20 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: caught exception mongoshard04:27017 socket exception
      Tue Dec 27 19:36:20 [ReplicaSetMonitorWatcher] trying reconnect to mongoshard05:27017
      Tue Dec 27 19:36:22 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:23 [WriteBackListener] WriteBackListener exception : socket exception
      Tue Dec 27 19:36:25 [ReplicaSetMonitorWatcher] reconnect mongoshard05:27017 failed couldn't connect to server mongoshard05:27017
      Tue Dec 27 19:36:25 [ReplicaSetMonitorWatcher] ReplicaSetMonitor::_checkConnection: caught exception mongoshard05:27017 socket exception
      Tue Dec 27 19:36:25 [ReplicaSetMonitorWatcher] trying reconnect to mongoshard06:27017
      

            Assignee:
            richard.kreuter Richard Kreuter (Inactive)
            Reporter:
            siteops@reputation.com Site Operations
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: