We have a sharded cluster, with mongos on each of the app servers. If I stop a secondary on the first shard, then from the replica set point of view, everything seems ok (primary still up). However, our application servers are then unable to query the local mongos (the queries just time out).

Looking at the mongos logs, there are errors like this:

W NETWORK [Balancer] Failed to connect to <IP of replica set SECONDARY>, reason: errno:111 Connection refused
W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to <IP of replica set SECONDARY>, reason: errno:111 Connection refused