2016-03-13T08:25:03.148+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:25:03.148+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:25:03.148+0000-56e523df18fa188537623c5a", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857503148), what: "balancer.round", ns: "", details: { executionTimeMillis: 10589, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:25:03.152+0000 I NETWORK [conn4076236] end connection 10.0.216.90:52771 (1506 connections now open) 2016-03-13T08:25:03.153+0000 I NETWORK [conn4076235] end connection 10.0.216.90:52770 (1505 connections now open) 2016-03-13T08:25:03.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52784 #4076250 (1506 connections now open) 2016-03-13T08:25:03.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52785 #4076251 (1507 connections now open) 2016-03-13T08:25:03.879+0000 I NETWORK [conn4076237] end connection 10.0.20.53:46042 (1506 connections now open) 2016-03-13T08:25:03.879+0000 I NETWORK [conn4076238] end connection 10.0.20.53:46041 (1506 connections now open) 2016-03-13T08:25:03.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46056 #4076252 (1506 connections now open) 2016-03-13T08:25:03.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46057 #4076253 (1507 connections now open) 2016-03-13T08:25:06.437+0000 I NETWORK [conn4076240] end connection 10.0.179.209:35937 (1506 connections now open) 2016-03-13T08:25:06.437+0000 I NETWORK [conn4076239] end connection 10.0.179.209:35936 (1506 connections now open) 2016-03-13T08:25:06.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35950 #4076254 (1506 connections now open) 2016-03-13T08:25:06.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35949 #4076255 (1507 connections now open) 2016-03-13T08:25:06.866+0000 I NETWORK [conn4076241] end connection 10.0.26.69:37781 (1506 connections now open) 2016-03-13T08:25:06.866+0000 I NETWORK [conn4076242] end connection 10.0.26.69:37782 (1506 connections now open) 2016-03-13T08:25:06.867+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37807 #4076256 (1506 connections now open) 2016-03-13T08:25:06.867+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37806 #4076257 (1507 connections now open) 2016-03-13T08:25:07.006+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:25:06.880+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:25:09.927+0000 I NETWORK [conn4076247] end connection 10.0.27.237:59434 (1506 connections now open) 2016-03-13T08:25:09.927+0000 I NETWORK [conn4076246] end connection 10.0.27.237:59433 (1506 connections now open) 2016-03-13T08:25:09.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59454 #4076258 (1506 connections now open) 2016-03-13T08:25:09.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59453 #4076259 (1507 connections now open) 2016-03-13T08:25:11.929+0000 I NETWORK [conn4076248] end connection 10.0.7.113:54747 (1506 connections now open) 2016-03-13T08:25:11.929+0000 I NETWORK [conn4076249] end connection 10.0.7.113:54748 (1506 connections now open) 2016-03-13T08:25:11.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54761 #4076260 (1506 connections now open) 2016-03-13T08:25:11.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54762 #4076261 (1507 connections now open) 2016-03-13T08:25:12.434+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:12.435+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:12.435+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:25:12.435+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:25:13.165+0000 I NETWORK [conn4076250] end connection 10.0.216.90:52784 (1506 connections now open) 2016-03-13T08:25:13.165+0000 I NETWORK [conn4076251] end connection 10.0.216.90:52785 (1505 connections now open) 2016-03-13T08:25:13.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52795 #4076262 (1506 connections now open) 2016-03-13T08:25:13.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52796 #4076263 (1507 connections now open) 2016-03-13T08:25:13.194+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:13.195+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:13.195+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:13.195+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:25:13.696+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:13.697+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:13.697+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:13.697+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:25:13.892+0000 I NETWORK [conn4076252] end connection 10.0.20.53:46056 (1506 connections now open) 2016-03-13T08:25:13.892+0000 I NETWORK [conn4076253] end connection 10.0.20.53:46057 (1506 connections now open) 2016-03-13T08:25:13.893+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46076 #4076264 (1506 connections now open) 2016-03-13T08:25:13.893+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46077 #4076265 (1507 connections now open) 2016-03-13T08:25:14.198+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:14.198+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:14.198+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:14.198+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:25:14.699+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:14.700+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:14.700+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:14.700+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:25:15.201+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:15.201+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:15.201+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:15.201+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:25:15.702+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:15.703+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:15.703+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:15.703+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:25:15.703+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:25:15.703+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:25:15.703+0000-56e523eb18fa188537623c5b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857515703), what: "balancer.round", ns: "", details: { executionTimeMillis: 2528, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:25:16.118+0000 I NETWORK [conn4068831] end connection 10.0.20.53:38271 (1506 connections now open) 2016-03-13T08:25:16.118+0000 I NETWORK [conn4074228] end connection 10.0.216.90:50576 (1506 connections now open) 2016-03-13T08:25:16.118+0000 I NETWORK [conn4067965] end connection 10.0.27.237:50829 (1504 connections now open) 2016-03-13T08:25:16.118+0000 I NETWORK [conn4066179] end connection 10.0.7.113:43986 (1503 connections now open) 2016-03-13T08:25:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52801 #4076266 (1504 connections now open) 2016-03-13T08:25:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46080 #4076267 (1505 connections now open) 2016-03-13T08:25:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59462 #4076268 (1506 connections now open) 2016-03-13T08:25:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54769 #4076269 (1507 connections now open) 2016-03-13T08:25:16.450+0000 I NETWORK [conn4076254] end connection 10.0.179.209:35950 (1506 connections now open) 2016-03-13T08:25:16.450+0000 I NETWORK [conn4076255] end connection 10.0.179.209:35949 (1506 connections now open) 2016-03-13T08:25:16.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35964 #4076270 (1506 connections now open) 2016-03-13T08:25:16.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35965 #4076271 (1507 connections now open) 2016-03-13T08:25:16.879+0000 I NETWORK [conn4076257] end connection 10.0.26.69:37806 (1506 connections now open) 2016-03-13T08:25:16.879+0000 I NETWORK [conn4076256] end connection 10.0.26.69:37807 (1506 connections now open) 2016-03-13T08:25:16.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37818 #4076272 (1506 connections now open) 2016-03-13T08:25:16.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37819 #4076273 (1507 connections now open) 2016-03-13T08:25:19.940+0000 I NETWORK [conn4076258] end connection 10.0.27.237:59454 (1506 connections now open) 2016-03-13T08:25:19.940+0000 I NETWORK [conn4076259] end connection 10.0.27.237:59453 (1506 connections now open) 2016-03-13T08:25:19.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59465 #4076274 (1506 connections now open) 2016-03-13T08:25:19.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59464 #4076275 (1507 connections now open) 2016-03-13T08:25:21.942+0000 I NETWORK [conn4076260] end connection 10.0.7.113:54761 (1506 connections now open) 2016-03-13T08:25:21.942+0000 I NETWORK [conn4076261] end connection 10.0.7.113:54762 (1506 connections now open) 2016-03-13T08:25:21.943+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54783 #4076276 (1506 connections now open) 2016-03-13T08:25:21.943+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54784 #4076277 (1507 connections now open) 2016-03-13T08:25:22.437+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:25:23.178+0000 I NETWORK [conn4076263] end connection 10.0.216.90:52796 (1506 connections now open) 2016-03-13T08:25:23.178+0000 I NETWORK [conn4076262] end connection 10.0.216.90:52795 (1506 connections now open) 2016-03-13T08:25:23.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52815 #4076278 (1506 connections now open) 2016-03-13T08:25:23.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52816 #4076279 (1507 connections now open) 2016-03-13T08:25:23.906+0000 I NETWORK [conn4076264] end connection 10.0.20.53:46076 (1506 connections now open) 2016-03-13T08:25:23.906+0000 I NETWORK [conn4076265] end connection 10.0.20.53:46077 (1506 connections now open) 2016-03-13T08:25:23.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46088 #4076280 (1506 connections now open) 2016-03-13T08:25:23.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46087 #4076281 (1507 connections now open) 2016-03-13T08:25:25.764+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:25:25.776+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:25.776+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:25.776+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:25.776+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:25:26.277+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:26.278+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:26.278+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:26.278+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:25:26.463+0000 I NETWORK [conn4076270] end connection 10.0.179.209:35964 (1506 connections now open) 2016-03-13T08:25:26.463+0000 I NETWORK [conn4076271] end connection 10.0.179.209:35965 (1506 connections now open) 2016-03-13T08:25:26.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35984 #4076282 (1506 connections now open) 2016-03-13T08:25:26.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35985 #4076283 (1507 connections now open) 2016-03-13T08:25:26.779+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:26.780+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:26.780+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:26.780+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:25:26.892+0000 I NETWORK [conn4076273] end connection 10.0.26.69:37819 (1506 connections now open) 2016-03-13T08:25:26.892+0000 I NETWORK [conn4076272] end connection 10.0.26.69:37818 (1505 connections now open) 2016-03-13T08:25:26.893+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37831 #4076284 (1506 connections now open) 2016-03-13T08:25:26.893+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37832 #4076285 (1507 connections now open) 2016-03-13T08:25:27.280+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:27.281+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:27.281+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:27.281+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:25:27.782+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:27.783+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:27.783+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:27.783+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:25:28.113+0000 I NETWORK [conn4067900] end connection 10.0.179.209:55736 (1506 connections now open) 2016-03-13T08:25:28.113+0000 I NETWORK [conn4064789] end connection 10.0.26.69:53693 (1505 connections now open) 2016-03-13T08:25:28.113+0000 I NETWORK [conn4064261] end connection 10.0.7.113:41922 (1504 connections now open) 2016-03-13T08:25:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35987 #4076286 (1505 connections now open) 2016-03-13T08:25:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37835 #4076287 (1506 connections now open) 2016-03-13T08:25:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54791 #4076288 (1507 connections now open) 2016-03-13T08:25:28.283+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:28.284+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:28.284+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:28.284+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:25:28.785+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:28.786+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:28.786+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:28.786+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:25:29.287+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:29.287+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:29.287+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:29.287+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:25:29.788+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:29.789+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:29.789+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:29.789+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:25:29.953+0000 I NETWORK [conn4076275] end connection 10.0.27.237:59464 (1506 connections now open) 2016-03-13T08:25:29.953+0000 I NETWORK [conn4076274] end connection 10.0.27.237:59465 (1506 connections now open) 2016-03-13T08:25:29.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59480 #4076289 (1506 connections now open) 2016-03-13T08:25:29.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59479 #4076290 (1507 connections now open) 2016-03-13T08:25:30.290+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:30.291+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:30.291+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:30.291+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:25:30.792+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:30.792+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:30.792+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:30.792+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:25:31.293+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:31.294+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:31.294+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:31.294+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:25:31.795+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:31.796+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:31.796+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:31.796+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:25:31.956+0000 I NETWORK [conn4076277] end connection 10.0.7.113:54784 (1506 connections now open) 2016-03-13T08:25:31.956+0000 I NETWORK [conn4076276] end connection 10.0.7.113:54783 (1506 connections now open) 2016-03-13T08:25:31.956+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54794 #4076291 (1506 connections now open) 2016-03-13T08:25:31.956+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54793 #4076292 (1507 connections now open) 2016-03-13T08:25:32.296+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:32.297+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:32.297+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:32.297+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:25:32.440+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:32.440+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:32.440+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:25:32.440+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:25:32.798+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:32.799+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:32.799+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:32.799+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:25:33.191+0000 I NETWORK [conn4076278] end connection 10.0.216.90:52815 (1506 connections now open) 2016-03-13T08:25:33.191+0000 I NETWORK [conn4076279] end connection 10.0.216.90:52816 (1506 connections now open) 2016-03-13T08:25:33.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52829 #4076293 (1506 connections now open) 2016-03-13T08:25:33.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52828 #4076294 (1507 connections now open) 2016-03-13T08:25:33.300+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:33.300+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:33.300+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:33.300+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:25:33.801+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:33.802+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:33.802+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:33.802+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:25:33.919+0000 I NETWORK [conn4076281] end connection 10.0.20.53:46087 (1506 connections now open) 2016-03-13T08:25:33.919+0000 I NETWORK [conn4076280] end connection 10.0.20.53:46088 (1506 connections now open) 2016-03-13T08:25:33.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46105 #4076295 (1506 connections now open) 2016-03-13T08:25:33.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46104 #4076296 (1507 connections now open) 2016-03-13T08:25:34.303+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:34.303+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:34.303+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:34.303+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:25:34.804+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:34.805+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:34.805+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:34.805+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:25:35.306+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:35.306+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:35.306+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:35.306+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:25:35.807+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:35.808+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:35.808+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:35.808+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:25:36.309+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:36.310+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:36.310+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:36.310+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:25:36.310+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:25:36.310+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:25:36.310+0000-56e5240018fa188537623c5c", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857536310), what: "balancer.round", ns: "", details: { executionTimeMillis: 10579, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:25:36.476+0000 I NETWORK [conn4076282] end connection 10.0.179.209:35984 (1506 connections now open) 2016-03-13T08:25:36.476+0000 I NETWORK [conn4076283] end connection 10.0.179.209:35985 (1506 connections now open) 2016-03-13T08:25:36.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35993 #4076297 (1506 connections now open) 2016-03-13T08:25:36.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:35994 #4076298 (1507 connections now open) 2016-03-13T08:25:36.905+0000 I NETWORK [conn4076285] end connection 10.0.26.69:37832 (1506 connections now open) 2016-03-13T08:25:36.905+0000 I NETWORK [conn4076284] end connection 10.0.26.69:37831 (1505 connections now open) 2016-03-13T08:25:36.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37853 #4076299 (1506 connections now open) 2016-03-13T08:25:36.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37852 #4076300 (1507 connections now open) 2016-03-13T08:25:37.342+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:25:37.006+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:25:39.966+0000 I NETWORK [conn4076289] end connection 10.0.27.237:59480 (1506 connections now open) 2016-03-13T08:25:39.966+0000 I NETWORK [conn4076290] end connection 10.0.27.237:59479 (1506 connections now open) 2016-03-13T08:25:39.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59497 #4076301 (1506 connections now open) 2016-03-13T08:25:39.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59498 #4076302 (1507 connections now open) 2016-03-13T08:25:41.969+0000 I NETWORK [conn4076292] end connection 10.0.7.113:54793 (1506 connections now open) 2016-03-13T08:25:41.969+0000 I NETWORK [conn4076291] end connection 10.0.7.113:54794 (1506 connections now open) 2016-03-13T08:25:41.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54805 #4076303 (1506 connections now open) 2016-03-13T08:25:41.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54806 #4076304 (1507 connections now open) 2016-03-13T08:25:42.443+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:42.444+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:42.444+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:25:42.444+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:25:43.204+0000 I NETWORK [conn4076294] end connection 10.0.216.90:52828 (1506 connections now open) 2016-03-13T08:25:43.204+0000 I NETWORK [conn4076293] end connection 10.0.216.90:52829 (1506 connections now open) 2016-03-13T08:25:43.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52840 #4076305 (1506 connections now open) 2016-03-13T08:25:43.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52841 #4076306 (1507 connections now open) 2016-03-13T08:25:43.933+0000 I NETWORK [conn4076295] end connection 10.0.20.53:46105 (1506 connections now open) 2016-03-13T08:25:43.933+0000 I NETWORK [conn4076296] end connection 10.0.20.53:46104 (1506 connections now open) 2016-03-13T08:25:43.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46122 #4076307 (1506 connections now open) 2016-03-13T08:25:43.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46121 #4076308 (1507 connections now open) 2016-03-13T08:25:46.118+0000 I NETWORK [conn4069579] end connection 10.0.179.209:57433 (1506 connections now open) 2016-03-13T08:25:46.118+0000 I NETWORK [conn4067148] end connection 10.0.216.90:43011 (1505 connections now open) 2016-03-13T08:25:46.118+0000 I NETWORK [conn4065752] end connection 10.0.20.53:35096 (1504 connections now open) 2016-03-13T08:25:46.118+0000 I NETWORK [conn4068008] end connection 10.0.27.237:50874 (1503 connections now open) 2016-03-13T08:25:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52847 #4076309 (1504 connections now open) 2016-03-13T08:25:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46126 #4076310 (1505 connections now open) 2016-03-13T08:25:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59504 #4076311 (1506 connections now open) 2016-03-13T08:25:46.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36005 #4076312 (1507 connections now open) 2016-03-13T08:25:46.376+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:46.376+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:46.376+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:46.376+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:25:46.489+0000 I NETWORK [conn4076297] end connection 10.0.179.209:35993 (1506 connections now open) 2016-03-13T08:25:46.489+0000 I NETWORK [conn4076298] end connection 10.0.179.209:35994 (1506 connections now open) 2016-03-13T08:25:46.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36007 #4076313 (1506 connections now open) 2016-03-13T08:25:46.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36006 #4076314 (1507 connections now open) 2016-03-13T08:25:46.877+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:46.878+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:46.878+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:46.878+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:25:46.927+0000 I NETWORK [conn4076300] end connection 10.0.26.69:37852 (1506 connections now open) 2016-03-13T08:25:46.928+0000 I NETWORK [conn4076299] end connection 10.0.26.69:37853 (1505 connections now open) 2016-03-13T08:25:46.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37862 #4076315 (1506 connections now open) 2016-03-13T08:25:46.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37863 #4076316 (1507 connections now open) 2016-03-13T08:25:47.379+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:47.379+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:47.379+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:47.379+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:25:47.880+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:47.881+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:47.881+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:47.881+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:25:48.382+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:48.383+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:48.383+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:48.383+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:25:48.883+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:48.884+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:48.884+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:48.884+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:25:48.884+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:25:48.884+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:25:48.884+0000-56e5240c18fa188537623c5d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857548884), what: "balancer.round", ns: "", details: { executionTimeMillis: 2543, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:25:49.987+0000 I NETWORK [conn4076302] end connection 10.0.27.237:59498 (1506 connections now open) 2016-03-13T08:25:49.987+0000 I NETWORK [conn4076301] end connection 10.0.27.237:59497 (1506 connections now open) 2016-03-13T08:25:49.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59506 #4076317 (1506 connections now open) 2016-03-13T08:25:49.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59507 #4076318 (1507 connections now open) 2016-03-13T08:25:51.983+0000 I NETWORK [conn4076303] end connection 10.0.7.113:54805 (1506 connections now open) 2016-03-13T08:25:51.983+0000 I NETWORK [conn4076304] end connection 10.0.7.113:54806 (1506 connections now open) 2016-03-13T08:25:51.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54825 #4076319 (1506 connections now open) 2016-03-13T08:25:51.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54824 #4076320 (1507 connections now open) 2016-03-13T08:25:52.446+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:25:53.226+0000 I NETWORK [conn4076305] end connection 10.0.216.90:52840 (1506 connections now open) 2016-03-13T08:25:53.226+0000 I NETWORK [conn4076306] end connection 10.0.216.90:52841 (1506 connections now open) 2016-03-13T08:25:53.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52861 #4076321 (1506 connections now open) 2016-03-13T08:25:53.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52862 #4076322 (1507 connections now open) 2016-03-13T08:25:53.954+0000 I NETWORK [conn4076307] end connection 10.0.20.53:46122 (1506 connections now open) 2016-03-13T08:25:53.954+0000 I NETWORK [conn4076308] end connection 10.0.20.53:46121 (1506 connections now open) 2016-03-13T08:25:53.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46133 #4076323 (1506 connections now open) 2016-03-13T08:25:53.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46134 #4076324 (1507 connections now open) 2016-03-13T08:25:56.502+0000 I NETWORK [conn4076314] end connection 10.0.179.209:36006 (1506 connections now open) 2016-03-13T08:25:56.502+0000 I NETWORK [conn4076313] end connection 10.0.179.209:36007 (1506 connections now open) 2016-03-13T08:25:56.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36025 #4076325 (1506 connections now open) 2016-03-13T08:25:56.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36026 #4076326 (1507 connections now open) 2016-03-13T08:25:56.941+0000 I NETWORK [conn4076315] end connection 10.0.26.69:37862 (1506 connections now open) 2016-03-13T08:25:56.941+0000 I NETWORK [conn4076316] end connection 10.0.26.69:37863 (1506 connections now open) 2016-03-13T08:25:56.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37875 #4076327 (1506 connections now open) 2016-03-13T08:25:56.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37874 #4076328 (1507 connections now open) 2016-03-13T08:25:58.113+0000 I NETWORK [conn4069041] end connection 10.0.179.209:56887 (1506 connections now open) 2016-03-13T08:25:58.113+0000 I NETWORK [conn4072619] end connection 10.0.216.90:48861 (1505 connections now open) 2016-03-13T08:25:58.113+0000 I NETWORK [conn4064834] end connection 10.0.26.69:53742 (1504 connections now open) 2016-03-13T08:25:58.113+0000 I NETWORK [conn4067691] end connection 10.0.27.237:50539 (1503 connections now open) 2016-03-13T08:25:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36028 #4076329 (1504 connections now open) 2016-03-13T08:25:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52868 #4076330 (1505 connections now open) 2016-03-13T08:25:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37879 #4076331 (1506 connections now open) 2016-03-13T08:25:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59518 #4076332 (1507 connections now open) 2016-03-13T08:25:58.933+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:25:58.944+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:58.944+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:58.944+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:58.945+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:25:59.445+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:59.446+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:59.446+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:59.446+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:25:59.947+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:25:59.948+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:25:59.948+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:25:59.948+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:26:00.000+0000 I NETWORK [conn4076317] end connection 10.0.27.237:59506 (1506 connections now open) 2016-03-13T08:26:00.000+0000 I NETWORK [conn4076318] end connection 10.0.27.237:59507 (1506 connections now open) 2016-03-13T08:26:00.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59520 #4076333 (1506 connections now open) 2016-03-13T08:26:00.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59519 #4076334 (1507 connections now open) 2016-03-13T08:26:00.449+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:00.449+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:00.449+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:00.449+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:26:00.950+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:00.951+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:00.951+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:00.951+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:26:01.452+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:01.453+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:01.453+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:01.453+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:26:01.954+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:01.954+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:01.954+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:01.954+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:26:02.005+0000 I NETWORK [conn4076319] end connection 10.0.7.113:54825 (1506 connections now open) 2016-03-13T08:26:02.005+0000 I NETWORK [conn4076320] end connection 10.0.7.113:54824 (1506 connections now open) 2016-03-13T08:26:02.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54836 #4076335 (1506 connections now open) 2016-03-13T08:26:02.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54837 #4076336 (1507 connections now open) 2016-03-13T08:26:02.449+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:02.450+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:02.450+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:26:02.450+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:26:02.455+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:02.456+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:02.456+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:02.456+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:26:02.957+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:02.957+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:02.957+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:02.957+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:26:03.239+0000 I NETWORK [conn4076322] end connection 10.0.216.90:52862 (1506 connections now open) 2016-03-13T08:26:03.239+0000 I NETWORK [conn4076321] end connection 10.0.216.90:52861 (1506 connections now open) 2016-03-13T08:26:03.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52873 #4076337 (1506 connections now open) 2016-03-13T08:26:03.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52874 #4076338 (1507 connections now open) 2016-03-13T08:26:03.458+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:03.459+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:03.459+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:03.459+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:26:03.960+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:03.960+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:03.960+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:03.960+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:26:03.968+0000 I NETWORK [conn4076324] end connection 10.0.20.53:46134 (1506 connections now open) 2016-03-13T08:26:03.968+0000 I NETWORK [conn4076323] end connection 10.0.20.53:46133 (1506 connections now open) 2016-03-13T08:26:03.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46146 #4076339 (1506 connections now open) 2016-03-13T08:26:03.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46147 #4076340 (1507 connections now open) 2016-03-13T08:26:04.461+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:04.462+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:04.462+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:04.462+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:26:04.963+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:04.964+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:04.964+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:04.964+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:26:05.464+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:05.465+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:05.465+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:05.465+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:26:05.966+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:05.967+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:05.967+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:05.967+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:26:06.468+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:06.468+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:06.468+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:06.468+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:26:06.515+0000 I NETWORK [conn4076326] end connection 10.0.179.209:36026 (1506 connections now open) 2016-03-13T08:26:06.515+0000 I NETWORK [conn4076325] end connection 10.0.179.209:36025 (1505 connections now open) 2016-03-13T08:26:06.515+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36037 #4076341 (1506 connections now open) 2016-03-13T08:26:06.515+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36038 #4076342 (1507 connections now open) 2016-03-13T08:26:06.955+0000 I NETWORK [conn4076327] end connection 10.0.26.69:37875 (1506 connections now open) 2016-03-13T08:26:06.955+0000 I NETWORK [conn4076328] end connection 10.0.26.69:37874 (1506 connections now open) 2016-03-13T08:26:06.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37899 #4076343 (1506 connections now open) 2016-03-13T08:26:06.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37898 #4076344 (1507 connections now open) 2016-03-13T08:26:06.969+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:06.970+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:06.970+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:06.970+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:26:07.471+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:07.471+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:07.471+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:07.471+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:26:07.483+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:26:07.342+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:26:07.972+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:07.973+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:07.973+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:07.973+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:26:08.474+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:08.475+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:08.475+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:08.475+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:26:08.975+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:08.976+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:08.976+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:08.976+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:26:09.477+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:09.478+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:09.478+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:09.478+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:26:09.478+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:26:09.478+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:26:09.478+0000-56e5242118fa188537623c5e", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857569478), what: "balancer.round", ns: "", details: { executionTimeMillis: 10565, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:26:10.012+0000 I NETWORK [conn4076333] end connection 10.0.27.237:59520 (1506 connections now open) 2016-03-13T08:26:10.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59538 #4076345 (1507 connections now open) 2016-03-13T08:26:10.013+0000 I NETWORK [conn4076334] end connection 10.0.27.237:59519 (1506 connections now open) 2016-03-13T08:26:10.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59539 #4076346 (1507 connections now open) 2016-03-13T08:26:12.018+0000 I NETWORK [conn4076335] end connection 10.0.7.113:54836 (1506 connections now open) 2016-03-13T08:26:12.018+0000 I NETWORK [conn4076336] end connection 10.0.7.113:54837 (1505 connections now open) 2016-03-13T08:26:12.018+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54852 #4076347 (1506 connections now open) 2016-03-13T08:26:12.019+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54851 #4076348 (1507 connections now open) 2016-03-13T08:26:12.453+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:12.453+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:12.454+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:26:12.454+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:26:13.261+0000 I NETWORK [conn4076338] end connection 10.0.216.90:52874 (1506 connections now open) 2016-03-13T08:26:13.261+0000 I NETWORK [conn4076337] end connection 10.0.216.90:52873 (1506 connections now open) 2016-03-13T08:26:13.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52885 #4076349 (1506 connections now open) 2016-03-13T08:26:13.262+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52884 #4076350 (1507 connections now open) 2016-03-13T08:26:13.990+0000 I NETWORK [conn4076339] end connection 10.0.20.53:46146 (1506 connections now open) 2016-03-13T08:26:13.990+0000 I NETWORK [conn4076340] end connection 10.0.20.53:46147 (1506 connections now open) 2016-03-13T08:26:13.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46164 #4076351 (1507 connections now open) 2016-03-13T08:26:13.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46165 #4076352 (1507 connections now open) 2016-03-13T08:26:16.118+0000 I NETWORK [conn4068958] end connection 10.0.20.53:38391 (1506 connections now open) 2016-03-13T08:26:16.118+0000 I NETWORK [conn4070037] end connection 10.0.27.237:53005 (1505 connections now open) 2016-03-13T08:26:16.118+0000 I NETWORK [conn4068775] end connection 10.0.26.69:57977 (1505 connections now open) 2016-03-13T08:26:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46169 #4076353 (1505 connections now open) 2016-03-13T08:26:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37909 #4076354 (1506 connections now open) 2016-03-13T08:26:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59547 #4076355 (1507 connections now open) 2016-03-13T08:26:16.528+0000 I NETWORK [conn4076341] end connection 10.0.179.209:36037 (1506 connections now open) 2016-03-13T08:26:16.528+0000 I NETWORK [conn4076342] end connection 10.0.179.209:36038 (1506 connections now open) 2016-03-13T08:26:16.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36050 #4076356 (1506 connections now open) 2016-03-13T08:26:16.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36049 #4076357 (1507 connections now open) 2016-03-13T08:26:16.976+0000 I NETWORK [conn4076343] end connection 10.0.26.69:37899 (1506 connections now open) 2016-03-13T08:26:16.976+0000 I NETWORK [conn4076344] end connection 10.0.26.69:37898 (1506 connections now open) 2016-03-13T08:26:16.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37912 #4076358 (1506 connections now open) 2016-03-13T08:26:16.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37911 #4076359 (1507 connections now open) 2016-03-13T08:26:19.524+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:19.524+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:19.525+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:19.525+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:26:20.025+0000 I NETWORK [conn4076345] end connection 10.0.27.237:59538 (1506 connections now open) 2016-03-13T08:26:20.025+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:20.025+0000 I NETWORK [conn4076346] end connection 10.0.27.237:59539 (1505 connections now open) 2016-03-13T08:26:20.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59549 #4076360 (1506 connections now open) 2016-03-13T08:26:20.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59550 #4076361 (1507 connections now open) 2016-03-13T08:26:20.026+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:20.026+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:20.026+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:26:20.527+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:20.528+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:20.528+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:20.528+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:26:21.029+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:21.029+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:21.029+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:21.029+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:26:21.530+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:21.531+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:21.531+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:21.531+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:26:22.031+0000 I NETWORK [conn4076347] end connection 10.0.7.113:54852 (1506 connections now open) 2016-03-13T08:26:22.031+0000 I NETWORK [conn4076348] end connection 10.0.7.113:54851 (1506 connections now open) 2016-03-13T08:26:22.031+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54873 #4076362 (1506 connections now open) 2016-03-13T08:26:22.031+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54872 #4076363 (1507 connections now open) 2016-03-13T08:26:22.032+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:22.033+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:22.033+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:22.033+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:26:22.033+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:26:22.033+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:26:22.033+0000-56e5242e18fa188537623c5f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857582033), what: "balancer.round", ns: "", details: { executionTimeMillis: 2534, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:26:22.456+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:26:23.274+0000 I NETWORK [conn4076350] end connection 10.0.216.90:52884 (1506 connections now open) 2016-03-13T08:26:23.274+0000 I NETWORK [conn4076349] end connection 10.0.216.90:52885 (1506 connections now open) 2016-03-13T08:26:23.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52907 #4076364 (1506 connections now open) 2016-03-13T08:26:23.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52906 #4076365 (1507 connections now open) 2016-03-13T08:26:24.003+0000 I NETWORK [conn4076352] end connection 10.0.20.53:46165 (1506 connections now open) 2016-03-13T08:26:24.004+0000 I NETWORK [conn4076351] end connection 10.0.20.53:46164 (1505 connections now open) 2016-03-13T08:26:24.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46175 #4076366 (1506 connections now open) 2016-03-13T08:26:24.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46176 #4076367 (1507 connections now open) 2016-03-13T08:26:26.541+0000 I NETWORK [conn4076356] end connection 10.0.179.209:36050 (1506 connections now open) 2016-03-13T08:26:26.541+0000 I NETWORK [conn4076357] end connection 10.0.179.209:36049 (1506 connections now open) 2016-03-13T08:26:26.542+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36071 #4076368 (1506 connections now open) 2016-03-13T08:26:26.542+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36072 #4076369 (1507 connections now open) 2016-03-13T08:26:26.990+0000 I NETWORK [conn4076358] end connection 10.0.26.69:37912 (1506 connections now open) 2016-03-13T08:26:26.990+0000 I NETWORK [conn4076359] end connection 10.0.26.69:37911 (1506 connections now open) 2016-03-13T08:26:26.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37923 #4076370 (1506 connections now open) 2016-03-13T08:26:26.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37924 #4076371 (1507 connections now open) 2016-03-13T08:26:28.114+0000 I NETWORK [conn4065110] end connection 10.0.20.53:34458 (1506 connections now open) 2016-03-13T08:26:28.114+0000 I NETWORK [conn4073891] end connection 10.0.179.209:33511 (1506 connections now open) 2016-03-13T08:26:28.114+0000 I NETWORK [conn4066452] end connection 10.0.216.90:42268 (1504 connections now open) 2016-03-13T08:26:28.114+0000 I NETWORK [conn4063560] end connection 10.0.26.69:52400 (1503 connections now open) 2016-03-13T08:26:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46183 #4076372 (1504 connections now open) 2016-03-13T08:26:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36074 #4076373 (1505 connections now open) 2016-03-13T08:26:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52913 #4076374 (1506 connections now open) 2016-03-13T08:26:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37928 #4076375 (1507 connections now open) 2016-03-13T08:26:30.038+0000 I NETWORK [conn4076361] end connection 10.0.27.237:59550 (1506 connections now open) 2016-03-13T08:26:30.038+0000 I NETWORK [conn4076360] end connection 10.0.27.237:59549 (1506 connections now open) 2016-03-13T08:26:30.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59564 #4076376 (1506 connections now open) 2016-03-13T08:26:30.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59563 #4076377 (1507 connections now open) 2016-03-13T08:26:32.044+0000 I NETWORK [conn4076363] end connection 10.0.7.113:54872 (1506 connections now open) 2016-03-13T08:26:32.044+0000 I NETWORK [conn4076362] end connection 10.0.7.113:54873 (1506 connections now open) 2016-03-13T08:26:32.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54884 #4076378 (1507 connections now open) 2016-03-13T08:26:32.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54883 #4076379 (1507 connections now open) 2016-03-13T08:26:32.089+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:26:32.091+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:32.101+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:32.101+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:32.101+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:26:32.459+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:32.459+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:32.459+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:26:32.459+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:26:32.601+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:32.602+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:32.602+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:32.602+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:26:33.103+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:33.104+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:33.104+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:33.104+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:26:33.287+0000 I NETWORK [conn4076365] end connection 10.0.216.90:52906 (1506 connections now open) 2016-03-13T08:26:33.287+0000 I NETWORK [conn4076364] end connection 10.0.216.90:52907 (1506 connections now open) 2016-03-13T08:26:33.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52919 #4076380 (1506 connections now open) 2016-03-13T08:26:33.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52920 #4076381 (1507 connections now open) 2016-03-13T08:26:33.605+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:33.605+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:33.605+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:33.605+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:26:34.017+0000 I NETWORK [conn4076367] end connection 10.0.20.53:46176 (1506 connections now open) 2016-03-13T08:26:34.017+0000 I NETWORK [conn4076366] end connection 10.0.20.53:46175 (1506 connections now open) 2016-03-13T08:26:34.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46191 #4076382 (1506 connections now open) 2016-03-13T08:26:34.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46192 #4076383 (1507 connections now open) 2016-03-13T08:26:34.106+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:34.107+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:34.107+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:34.107+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:26:34.608+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:34.608+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:34.608+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:34.608+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:26:35.109+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:35.110+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:35.110+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:35.110+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:26:35.611+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:35.612+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:35.612+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:35.612+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:26:36.113+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:36.113+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:36.113+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:36.113+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:26:36.554+0000 I NETWORK [conn4076368] end connection 10.0.179.209:36071 (1506 connections now open) 2016-03-13T08:26:36.554+0000 I NETWORK [conn4076369] end connection 10.0.179.209:36072 (1506 connections now open) 2016-03-13T08:26:36.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36081 #4076384 (1506 connections now open) 2016-03-13T08:26:36.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36080 #4076385 (1507 connections now open) 2016-03-13T08:26:36.614+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:36.615+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:36.615+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:36.615+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:26:37.004+0000 I NETWORK [conn4076371] end connection 10.0.26.69:37924 (1506 connections now open) 2016-03-13T08:26:37.004+0000 I NETWORK [conn4076370] end connection 10.0.26.69:37923 (1506 connections now open) 2016-03-13T08:26:37.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37944 #4076386 (1506 connections now open) 2016-03-13T08:26:37.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37945 #4076387 (1507 connections now open) 2016-03-13T08:26:37.116+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:37.116+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:37.116+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:37.116+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:26:37.617+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:37.618+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:37.618+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:37.618+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:26:37.732+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:26:37.483+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:26:38.119+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:38.119+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:38.119+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:38.119+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:26:38.620+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:38.621+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:38.621+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:38.621+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:26:39.122+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:39.123+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:39.123+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:39.123+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:26:39.624+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:39.624+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:39.624+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:39.624+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:26:40.052+0000 I NETWORK [conn4076376] end connection 10.0.27.237:59564 (1506 connections now open) 2016-03-13T08:26:40.052+0000 I NETWORK [conn4076377] end connection 10.0.27.237:59563 (1506 connections now open) 2016-03-13T08:26:40.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59580 #4076388 (1506 connections now open) 2016-03-13T08:26:40.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59579 #4076389 (1507 connections now open) 2016-03-13T08:26:40.125+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:40.126+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:40.126+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:40.126+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:26:40.627+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:40.628+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:40.628+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:40.628+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:26:41.128+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:41.129+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:41.129+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:41.129+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:26:41.630+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:41.631+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:41.631+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:41.631+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:26:42.057+0000 I NETWORK [conn4076378] end connection 10.0.7.113:54884 (1506 connections now open) 2016-03-13T08:26:42.057+0000 I NETWORK [conn4076379] end connection 10.0.7.113:54883 (1506 connections now open) 2016-03-13T08:26:42.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54894 #4076390 (1506 connections now open) 2016-03-13T08:26:42.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54895 #4076391 (1507 connections now open) 2016-03-13T08:26:42.132+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:42.132+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:42.132+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:42.132+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:26:42.462+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:42.463+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:42.463+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:26:42.463+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:26:42.633+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:42.634+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:42.634+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:42.634+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:26:42.634+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:26:42.634+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:26:42.634+0000-56e5244218fa188537623c60", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857602634), what: "balancer.round", ns: "", details: { executionTimeMillis: 10571, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:26:43.300+0000 I NETWORK [conn4076381] end connection 10.0.216.90:52920 (1506 connections now open) 2016-03-13T08:26:43.300+0000 I NETWORK [conn4076380] end connection 10.0.216.90:52919 (1506 connections now open) 2016-03-13T08:26:43.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52931 #4076392 (1506 connections now open) 2016-03-13T08:26:43.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52930 #4076393 (1507 connections now open) 2016-03-13T08:26:44.030+0000 I NETWORK [conn4076382] end connection 10.0.20.53:46191 (1506 connections now open) 2016-03-13T08:26:44.030+0000 I NETWORK [conn4076383] end connection 10.0.20.53:46192 (1506 connections now open) 2016-03-13T08:26:44.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46209 #4076394 (1506 connections now open) 2016-03-13T08:26:44.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46210 #4076395 (1507 connections now open) 2016-03-13T08:26:46.119+0000 I NETWORK [conn4066246] end connection 10.0.179.209:53979 (1506 connections now open) 2016-03-13T08:26:46.119+0000 I NETWORK [conn4067525] end connection 10.0.216.90:43418 (1506 connections now open) 2016-03-13T08:26:46.119+0000 I NETWORK [conn4072199] end connection 10.0.20.53:41770 (1506 connections now open) 2016-03-13T08:26:46.119+0000 I NETWORK [conn4070692] end connection 10.0.7.113:48804 (1503 connections now open) 2016-03-13T08:26:46.119+0000 I NETWORK [conn4068308] end connection 10.0.27.237:51173 (1503 connections now open) 2016-03-13T08:26:46.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46214 #4076396 (1503 connections now open) 2016-03-13T08:26:46.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54901 #4076397 (1504 connections now open) 2016-03-13T08:26:46.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59586 #4076398 (1505 connections now open) 2016-03-13T08:26:46.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52938 #4076399 (1506 connections now open) 2016-03-13T08:26:46.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36091 #4076400 (1507 connections now open) 2016-03-13T08:26:46.568+0000 I NETWORK [conn4076384] end connection 10.0.179.209:36081 (1506 connections now open) 2016-03-13T08:26:46.568+0000 I NETWORK [conn4076385] end connection 10.0.179.209:36080 (1506 connections now open) 2016-03-13T08:26:46.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36092 #4076401 (1506 connections now open) 2016-03-13T08:26:46.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36093 #4076402 (1507 connections now open) 2016-03-13T08:26:47.024+0000 I NETWORK [conn4076386] end connection 10.0.26.69:37944 (1506 connections now open) 2016-03-13T08:26:47.024+0000 I NETWORK [conn4076387] end connection 10.0.26.69:37945 (1506 connections now open) 2016-03-13T08:26:47.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37955 #4076403 (1506 connections now open) 2016-03-13T08:26:47.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37956 #4076404 (1507 connections now open) 2016-03-13T08:26:50.066+0000 I NETWORK [conn4076388] end connection 10.0.27.237:59580 (1506 connections now open) 2016-03-13T08:26:50.066+0000 I NETWORK [conn4076389] end connection 10.0.27.237:59579 (1506 connections now open) 2016-03-13T08:26:50.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59588 #4076405 (1506 connections now open) 2016-03-13T08:26:50.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59589 #4076406 (1507 connections now open) 2016-03-13T08:26:52.070+0000 I NETWORK [conn4076390] end connection 10.0.7.113:54894 (1506 connections now open) 2016-03-13T08:26:52.070+0000 I NETWORK [conn4076391] end connection 10.0.7.113:54895 (1506 connections now open) 2016-03-13T08:26:52.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54914 #4076407 (1506 connections now open) 2016-03-13T08:26:52.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54915 #4076408 (1507 connections now open) 2016-03-13T08:26:52.466+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:52.466+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:52.466+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:26:52.466+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:26:52.694+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:52.695+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:52.695+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:52.695+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:26:53.196+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:53.196+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:53.196+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:53.196+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:26:53.314+0000 I NETWORK [conn4076392] end connection 10.0.216.90:52931 (1506 connections now open) 2016-03-13T08:26:53.314+0000 I NETWORK [conn4076393] end connection 10.0.216.90:52930 (1506 connections now open) 2016-03-13T08:26:53.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52952 #4076409 (1506 connections now open) 2016-03-13T08:26:53.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52953 #4076410 (1507 connections now open) 2016-03-13T08:26:53.697+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:53.698+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:53.698+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:53.698+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:26:54.052+0000 I NETWORK [conn4076394] end connection 10.0.20.53:46209 (1506 connections now open) 2016-03-13T08:26:54.052+0000 I NETWORK [conn4076395] end connection 10.0.20.53:46210 (1505 connections now open) 2016-03-13T08:26:54.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46221 #4076411 (1506 connections now open) 2016-03-13T08:26:54.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46220 #4076412 (1507 connections now open) 2016-03-13T08:26:54.199+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:54.200+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:54.200+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:54.200+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:26:54.701+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:26:54.701+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:26:54.701+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:26:54.701+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:26:54.701+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:26:54.701+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:26:54.701+0000-56e5244e18fa188537623c61", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857614701), what: "balancer.round", ns: "", details: { executionTimeMillis: 2040, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:26:56.581+0000 I NETWORK [conn4076401] end connection 10.0.179.209:36092 (1506 connections now open) 2016-03-13T08:26:56.581+0000 I NETWORK [conn4076402] end connection 10.0.179.209:36093 (1506 connections now open) 2016-03-13T08:26:56.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36111 #4076413 (1506 connections now open) 2016-03-13T08:26:56.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36110 #4076414 (1507 connections now open) 2016-03-13T08:26:57.037+0000 I NETWORK [conn4076404] end connection 10.0.26.69:37956 (1506 connections now open) 2016-03-13T08:26:57.037+0000 I NETWORK [conn4076403] end connection 10.0.26.69:37955 (1506 connections now open) 2016-03-13T08:26:57.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37966 #4076415 (1506 connections now open) 2016-03-13T08:26:57.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37967 #4076416 (1507 connections now open) 2016-03-13T08:26:58.114+0000 I NETWORK [conn4069256] end connection 10.0.179.209:57100 (1506 connections now open) 2016-03-13T08:26:58.114+0000 I NETWORK [conn4065781] end connection 10.0.216.90:41539 (1506 connections now open) 2016-03-13T08:26:58.114+0000 I NETWORK [conn4063600] end connection 10.0.26.69:52442 (1506 connections now open) 2016-03-13T08:26:58.114+0000 I NETWORK [conn4071142] end connection 10.0.7.113:49280 (1503 connections now open) 2016-03-13T08:26:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52959 #4076417 (1504 connections now open) 2016-03-13T08:26:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37970 #4076418 (1505 connections now open) 2016-03-13T08:26:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36114 #4076419 (1506 connections now open) 2016-03-13T08:26:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54921 #4076420 (1507 connections now open) 2016-03-13T08:27:00.080+0000 I NETWORK [conn4076406] end connection 10.0.27.237:59589 (1506 connections now open) 2016-03-13T08:27:00.080+0000 I NETWORK [conn4076405] end connection 10.0.27.237:59588 (1506 connections now open) 2016-03-13T08:27:00.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59602 #4076421 (1506 connections now open) 2016-03-13T08:27:00.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59603 #4076422 (1507 connections now open) 2016-03-13T08:27:02.092+0000 I NETWORK [conn4076407] end connection 10.0.7.113:54914 (1506 connections now open) 2016-03-13T08:27:02.092+0000 I NETWORK [conn4076408] end connection 10.0.7.113:54915 (1506 connections now open) 2016-03-13T08:27:02.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54925 #4076423 (1506 connections now open) 2016-03-13T08:27:02.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54926 #4076424 (1507 connections now open) 2016-03-13T08:27:02.469+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:27:03.328+0000 I NETWORK [conn4076409] end connection 10.0.216.90:52952 (1506 connections now open) 2016-03-13T08:27:03.328+0000 I NETWORK [conn4076410] end connection 10.0.216.90:52953 (1506 connections now open) 2016-03-13T08:27:03.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52964 #4076425 (1506 connections now open) 2016-03-13T08:27:03.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52965 #4076426 (1507 connections now open) 2016-03-13T08:27:04.065+0000 I NETWORK [conn4076412] end connection 10.0.20.53:46220 (1506 connections now open) 2016-03-13T08:27:04.065+0000 I NETWORK [conn4076411] end connection 10.0.20.53:46221 (1506 connections now open) 2016-03-13T08:27:04.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46235 #4076427 (1506 connections now open) 2016-03-13T08:27:04.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46236 #4076428 (1507 connections now open) 2016-03-13T08:27:04.775+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:27:04.788+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:04.789+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:04.789+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:04.789+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:27:05.290+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:05.290+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:05.290+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:05.290+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:27:05.791+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:05.792+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:05.792+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:05.792+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:27:06.293+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:06.293+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:06.293+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:06.293+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:27:06.594+0000 I NETWORK [conn4076414] end connection 10.0.179.209:36110 (1506 connections now open) 2016-03-13T08:27:06.594+0000 I NETWORK [conn4076413] end connection 10.0.179.209:36111 (1506 connections now open) 2016-03-13T08:27:06.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36122 #4076429 (1506 connections now open) 2016-03-13T08:27:06.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36123 #4076430 (1507 connections now open) 2016-03-13T08:27:06.794+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:06.795+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:06.795+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:06.795+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:27:07.051+0000 I NETWORK [conn4076415] end connection 10.0.26.69:37966 (1506 connections now open) 2016-03-13T08:27:07.051+0000 I NETWORK [conn4076416] end connection 10.0.26.69:37967 (1505 connections now open) 2016-03-13T08:27:07.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37989 #4076431 (1506 connections now open) 2016-03-13T08:27:07.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:37990 #4076432 (1507 connections now open) 2016-03-13T08:27:07.296+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:07.297+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:07.297+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:07.297+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:27:07.798+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:07.798+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:07.798+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:07.798+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:27:07.977+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:27:07.732+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:27:08.299+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:08.300+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:08.300+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:08.300+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:27:08.801+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:08.801+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:08.801+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:08.801+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:27:09.302+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:09.303+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:09.303+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:09.303+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:27:09.804+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:09.805+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:09.805+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:09.805+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:27:10.093+0000 I NETWORK [conn4076422] end connection 10.0.27.237:59603 (1506 connections now open) 2016-03-13T08:27:10.093+0000 I NETWORK [conn4076421] end connection 10.0.27.237:59602 (1506 connections now open) 2016-03-13T08:27:10.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59620 #4076433 (1506 connections now open) 2016-03-13T08:27:10.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59619 #4076434 (1507 connections now open) 2016-03-13T08:27:10.305+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:10.306+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:10.306+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:10.306+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:27:10.807+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:10.808+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:10.808+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:10.808+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:27:11.309+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:11.309+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:11.309+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:11.309+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:27:11.810+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:11.811+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:11.811+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:11.811+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:27:12.106+0000 I NETWORK [conn4076424] end connection 10.0.7.113:54926 (1506 connections now open) 2016-03-13T08:27:12.106+0000 I NETWORK [conn4076423] end connection 10.0.7.113:54925 (1506 connections now open) 2016-03-13T08:27:12.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54942 #4076435 (1506 connections now open) 2016-03-13T08:27:12.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54941 #4076436 (1507 connections now open) 2016-03-13T08:27:12.312+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:12.312+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:12.314+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:12.314+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:27:12.472+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:12.472+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:12.472+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:27:12.472+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:27:12.815+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:12.815+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:12.815+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:12.815+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:27:13.316+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:13.317+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:13.317+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:13.317+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:27:13.341+0000 I NETWORK [conn4076426] end connection 10.0.216.90:52965 (1506 connections now open) 2016-03-13T08:27:13.341+0000 I NETWORK [conn4076425] end connection 10.0.216.90:52964 (1506 connections now open) 2016-03-13T08:27:13.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52974 #4076437 (1506 connections now open) 2016-03-13T08:27:13.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52975 #4076438 (1507 connections now open) 2016-03-13T08:27:13.818+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:13.818+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:13.818+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:13.818+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:27:14.079+0000 I NETWORK [conn4076428] end connection 10.0.20.53:46236 (1506 connections now open) 2016-03-13T08:27:14.079+0000 I NETWORK [conn4076427] end connection 10.0.20.53:46235 (1506 connections now open) 2016-03-13T08:27:14.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46255 #4076439 (1506 connections now open) 2016-03-13T08:27:14.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46254 #4076440 (1507 connections now open) 2016-03-13T08:27:14.319+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:14.320+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:14.320+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:14.320+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:27:14.821+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:14.821+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:14.821+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:14.821+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:27:15.322+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:15.323+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:15.323+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:15.323+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:27:15.323+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:27:15.323+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:27:15.323+0000-56e5246318fa188537623c62", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857635323), what: "balancer.round", ns: "", details: { executionTimeMillis: 10586, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:27:16.117+0000 I NETWORK [conn4070945] end connection 10.0.216.90:47062 (1506 connections now open) 2016-03-13T08:27:16.118+0000 I NETWORK [conn4070966] end connection 10.0.20.53:40493 (1506 connections now open) 2016-03-13T08:27:16.118+0000 I NETWORK [conn4065154] end connection 10.0.26.69:54089 (1505 connections now open) 2016-03-13T08:27:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52982 #4076441 (1505 connections now open) 2016-03-13T08:27:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46259 #4076442 (1506 connections now open) 2016-03-13T08:27:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38001 #4076443 (1507 connections now open) 2016-03-13T08:27:16.608+0000 I NETWORK [conn4076430] end connection 10.0.179.209:36123 (1506 connections now open) 2016-03-13T08:27:16.608+0000 I NETWORK [conn4076429] end connection 10.0.179.209:36122 (1506 connections now open) 2016-03-13T08:27:16.608+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36133 #4076444 (1506 connections now open) 2016-03-13T08:27:16.608+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36134 #4076445 (1507 connections now open) 2016-03-13T08:27:17.064+0000 I NETWORK [conn4076431] end connection 10.0.26.69:37989 (1506 connections now open) 2016-03-13T08:27:17.064+0000 I NETWORK [conn4076432] end connection 10.0.26.69:37990 (1506 connections now open) 2016-03-13T08:27:17.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38003 #4076446 (1506 connections now open) 2016-03-13T08:27:17.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38002 #4076447 (1507 connections now open) 2016-03-13T08:27:20.106+0000 I NETWORK [conn4076433] end connection 10.0.27.237:59620 (1506 connections now open) 2016-03-13T08:27:20.106+0000 I NETWORK [conn4076434] end connection 10.0.27.237:59619 (1506 connections now open) 2016-03-13T08:27:20.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59632 #4076448 (1506 connections now open) 2016-03-13T08:27:20.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59633 #4076449 (1507 connections now open) 2016-03-13T08:27:22.119+0000 I NETWORK [conn4076436] end connection 10.0.7.113:54941 (1506 connections now open) 2016-03-13T08:27:22.119+0000 I NETWORK [conn4076435] end connection 10.0.7.113:54942 (1506 connections now open) 2016-03-13T08:27:22.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54962 #4076450 (1506 connections now open) 2016-03-13T08:27:22.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54963 #4076451 (1507 connections now open) 2016-03-13T08:27:22.475+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:22.476+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:22.476+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:27:22.476+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:27:23.354+0000 I NETWORK [conn4076438] end connection 10.0.216.90:52975 (1506 connections now open) 2016-03-13T08:27:23.354+0000 I NETWORK [conn4076437] end connection 10.0.216.90:52974 (1506 connections now open) 2016-03-13T08:27:23.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52996 #4076452 (1506 connections now open) 2016-03-13T08:27:23.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:52997 #4076453 (1507 connections now open) 2016-03-13T08:27:24.093+0000 I NETWORK [conn4076440] end connection 10.0.20.53:46254 (1506 connections now open) 2016-03-13T08:27:24.093+0000 I NETWORK [conn4076439] end connection 10.0.20.53:46255 (1506 connections now open) 2016-03-13T08:27:24.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46266 #4076454 (1506 connections now open) 2016-03-13T08:27:24.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46265 #4076455 (1507 connections now open) 2016-03-13T08:27:25.386+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:25.387+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:25.387+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:25.387+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:27:25.888+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:25.889+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:25.889+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:25.889+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:27:26.390+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:26.390+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:26.390+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:26.390+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:27:26.621+0000 I NETWORK [conn4076445] end connection 10.0.179.209:36134 (1506 connections now open) 2016-03-13T08:27:26.621+0000 I NETWORK [conn4076444] end connection 10.0.179.209:36133 (1506 connections now open) 2016-03-13T08:27:26.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36156 #4076456 (1506 connections now open) 2016-03-13T08:27:26.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36155 #4076457 (1507 connections now open) 2016-03-13T08:27:26.891+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:26.892+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:26.892+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:26.892+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:27:27.077+0000 I NETWORK [conn4076446] end connection 10.0.26.69:38003 (1506 connections now open) 2016-03-13T08:27:27.077+0000 I NETWORK [conn4076447] end connection 10.0.26.69:38002 (1506 connections now open) 2016-03-13T08:27:27.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38015 #4076458 (1506 connections now open) 2016-03-13T08:27:27.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38016 #4076459 (1507 connections now open) 2016-03-13T08:27:27.393+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:27.393+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:27.393+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:27.393+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:27:27.894+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:27.895+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:27.895+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:27.895+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:27:27.895+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:27:27.895+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:27:27.895+0000-56e5246f18fa188537623c63", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857647895), what: "balancer.round", ns: "", details: { executionTimeMillis: 2546, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:27:28.114+0000 I NETWORK [conn4072131] end connection 10.0.20.53:41709 (1506 connections now open) 2016-03-13T08:27:28.114+0000 I NETWORK [conn4067731] end connection 10.0.27.237:50581 (1506 connections now open) 2016-03-13T08:27:28.114+0000 I NETWORK [conn4067754] end connection 10.0.216.90:43653 (1506 connections now open) 2016-03-13T08:27:28.114+0000 I NETWORK [conn4067342] end connection 10.0.7.113:45224 (1503 connections now open) 2016-03-13T08:27:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46273 #4076460 (1504 connections now open) 2016-03-13T08:27:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53003 #4076461 (1505 connections now open) 2016-03-13T08:27:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59644 #4076462 (1506 connections now open) 2016-03-13T08:27:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54970 #4076463 (1507 connections now open) 2016-03-13T08:27:30.119+0000 I NETWORK [conn4076449] end connection 10.0.27.237:59633 (1506 connections now open) 2016-03-13T08:27:30.119+0000 I NETWORK [conn4076448] end connection 10.0.27.237:59632 (1506 connections now open) 2016-03-13T08:27:30.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59646 #4076464 (1506 connections now open) 2016-03-13T08:27:30.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59647 #4076465 (1507 connections now open) 2016-03-13T08:27:32.132+0000 I NETWORK [conn4076451] end connection 10.0.7.113:54963 (1506 connections now open) 2016-03-13T08:27:32.132+0000 I NETWORK [conn4076450] end connection 10.0.7.113:54962 (1506 connections now open) 2016-03-13T08:27:32.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54973 #4076466 (1506 connections now open) 2016-03-13T08:27:32.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54972 #4076467 (1507 connections now open) 2016-03-13T08:27:32.478+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:27:33.367+0000 I NETWORK [conn4076452] end connection 10.0.216.90:52996 (1506 connections now open) 2016-03-13T08:27:33.367+0000 I NETWORK [conn4076453] end connection 10.0.216.90:52997 (1506 connections now open) 2016-03-13T08:27:33.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53009 #4076468 (1506 connections now open) 2016-03-13T08:27:33.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53010 #4076469 (1507 connections now open) 2016-03-13T08:27:34.106+0000 I NETWORK [conn4076455] end connection 10.0.20.53:46265 (1506 connections now open) 2016-03-13T08:27:34.106+0000 I NETWORK [conn4076454] end connection 10.0.20.53:46266 (1506 connections now open) 2016-03-13T08:27:34.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46282 #4076470 (1506 connections now open) 2016-03-13T08:27:34.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46283 #4076471 (1507 connections now open) 2016-03-13T08:27:36.634+0000 I NETWORK [conn4076457] end connection 10.0.179.209:36155 (1506 connections now open) 2016-03-13T08:27:36.634+0000 I NETWORK [conn4076456] end connection 10.0.179.209:36156 (1506 connections now open) 2016-03-13T08:27:36.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36165 #4076472 (1506 connections now open) 2016-03-13T08:27:36.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36164 #4076473 (1507 connections now open) 2016-03-13T08:27:37.090+0000 I NETWORK [conn4076458] end connection 10.0.26.69:38015 (1506 connections now open) 2016-03-13T08:27:37.090+0000 I NETWORK [conn4076459] end connection 10.0.26.69:38016 (1506 connections now open) 2016-03-13T08:27:37.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38036 #4076474 (1506 connections now open) 2016-03-13T08:27:37.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38037 #4076475 (1507 connections now open) 2016-03-13T08:27:37.952+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:27:37.965+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:37.966+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:37.966+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:37.966+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:27:38.145+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:27:37.977+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:27:38.467+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:38.467+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:38.467+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:38.467+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:27:38.968+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:38.969+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:38.969+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:38.969+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:27:39.470+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:39.471+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:39.471+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:39.471+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:27:39.972+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:39.972+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:39.972+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:39.972+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:27:40.132+0000 I NETWORK [conn4076464] end connection 10.0.27.237:59646 (1506 connections now open) 2016-03-13T08:27:40.132+0000 I NETWORK [conn4076465] end connection 10.0.27.237:59647 (1506 connections now open) 2016-03-13T08:27:40.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59663 #4076476 (1506 connections now open) 2016-03-13T08:27:40.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59664 #4076477 (1507 connections now open) 2016-03-13T08:27:40.473+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:40.474+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:40.474+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:40.474+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:27:40.975+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:40.975+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:40.975+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:40.975+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:27:41.476+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:41.477+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:41.477+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:41.477+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:27:41.978+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:41.978+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:41.978+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:41.979+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:27:42.145+0000 I NETWORK [conn4076467] end connection 10.0.7.113:54972 (1506 connections now open) 2016-03-13T08:27:42.145+0000 I NETWORK [conn4076466] end connection 10.0.7.113:54973 (1506 connections now open) 2016-03-13T08:27:42.145+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54983 #4076478 (1506 connections now open) 2016-03-13T08:27:42.145+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:54984 #4076479 (1507 connections now open) 2016-03-13T08:27:42.479+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:42.480+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:42.480+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:42.480+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:27:42.481+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:42.481+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:42.481+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:27:42.481+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:27:42.981+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:42.981+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:42.982+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:42.982+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:27:43.380+0000 I NETWORK [conn4076469] end connection 10.0.216.90:53010 (1506 connections now open) 2016-03-13T08:27:43.380+0000 I NETWORK [conn4076468] end connection 10.0.216.90:53009 (1505 connections now open) 2016-03-13T08:27:43.380+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53021 #4076480 (1506 connections now open) 2016-03-13T08:27:43.381+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53022 #4076481 (1507 connections now open) 2016-03-13T08:27:43.482+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:43.483+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:43.483+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:43.483+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:27:43.984+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:43.985+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:43.985+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:43.985+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:27:44.120+0000 I NETWORK [conn4076470] end connection 10.0.20.53:46282 (1506 connections now open) 2016-03-13T08:27:44.120+0000 I NETWORK [conn4076471] end connection 10.0.20.53:46283 (1506 connections now open) 2016-03-13T08:27:44.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46301 #4076482 (1506 connections now open) 2016-03-13T08:27:44.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46300 #4076483 (1507 connections now open) 2016-03-13T08:27:44.486+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:44.486+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:44.486+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:44.486+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:27:44.987+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:44.988+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:44.988+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:44.988+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:27:45.489+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:45.489+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:45.489+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:45.490+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:27:45.990+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:45.991+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:45.991+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:45.991+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:27:46.117+0000 I NETWORK [conn4068096] end connection 10.0.27.237:50962 (1506 connections now open) 2016-03-13T08:27:46.118+0000 I NETWORK [conn4064960] end connection 10.0.216.90:40636 (1505 connections now open) 2016-03-13T08:27:46.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59670 #4076484 (1506 connections now open) 2016-03-13T08:27:46.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53027 #4076485 (1507 connections now open) 2016-03-13T08:27:46.492+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:46.493+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:46.493+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:46.493+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:27:46.646+0000 I NETWORK [conn4076472] end connection 10.0.179.209:36165 (1506 connections now open) 2016-03-13T08:27:46.647+0000 I NETWORK [conn4076473] end connection 10.0.179.209:36164 (1505 connections now open) 2016-03-13T08:27:46.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36178 #4076486 (1506 connections now open) 2016-03-13T08:27:46.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36177 #4076487 (1507 connections now open) 2016-03-13T08:27:46.994+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:46.994+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:46.994+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:46.994+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:27:47.111+0000 I NETWORK [conn4076474] end connection 10.0.26.69:38036 (1506 connections now open) 2016-03-13T08:27:47.111+0000 I NETWORK [conn4076475] end connection 10.0.26.69:38037 (1505 connections now open) 2016-03-13T08:27:47.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38047 #4076488 (1506 connections now open) 2016-03-13T08:27:47.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38048 #4076489 (1507 connections now open) 2016-03-13T08:27:47.495+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:47.496+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:47.496+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:47.496+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:27:47.997+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:47.997+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:47.997+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:47.997+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:27:48.498+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:48.499+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:48.499+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:48.499+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:27:48.499+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:27:48.499+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:27:48.499+0000-56e5248418fa188537623c64", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857668499), what: "balancer.round", ns: "", details: { executionTimeMillis: 10582, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:27:50.145+0000 I NETWORK [conn4076477] end connection 10.0.27.237:59664 (1506 connections now open) 2016-03-13T08:27:50.145+0000 I NETWORK [conn4076476] end connection 10.0.27.237:59663 (1506 connections now open) 2016-03-13T08:27:50.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59672 #4076490 (1506 connections now open) 2016-03-13T08:27:50.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59673 #4076491 (1507 connections now open) 2016-03-13T08:27:52.158+0000 I NETWORK [conn4076478] end connection 10.0.7.113:54983 (1506 connections now open) 2016-03-13T08:27:52.158+0000 I NETWORK [conn4076479] end connection 10.0.7.113:54984 (1506 connections now open) 2016-03-13T08:27:52.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55005 #4076492 (1506 connections now open) 2016-03-13T08:27:52.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55006 #4076493 (1507 connections now open) 2016-03-13T08:27:52.484+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:52.484+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:52.485+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:27:52.485+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:27:53.393+0000 I NETWORK [conn4076480] end connection 10.0.216.90:53021 (1506 connections now open) 2016-03-13T08:27:53.393+0000 I NETWORK [conn4076481] end connection 10.0.216.90:53022 (1505 connections now open) 2016-03-13T08:27:53.393+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53041 #4076494 (1506 connections now open) 2016-03-13T08:27:53.393+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53042 #4076495 (1507 connections now open) 2016-03-13T08:27:54.143+0000 I NETWORK [conn4076483] end connection 10.0.20.53:46300 (1506 connections now open) 2016-03-13T08:27:54.143+0000 I NETWORK [conn4076482] end connection 10.0.20.53:46301 (1506 connections now open) 2016-03-13T08:27:54.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46310 #4076496 (1506 connections now open) 2016-03-13T08:27:54.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46311 #4076497 (1507 connections now open) 2016-03-13T08:27:56.667+0000 I NETWORK [conn4076486] end connection 10.0.179.209:36178 (1506 connections now open) 2016-03-13T08:27:56.667+0000 I NETWORK [conn4076487] end connection 10.0.179.209:36177 (1505 connections now open) 2016-03-13T08:27:56.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36195 #4076498 (1506 connections now open) 2016-03-13T08:27:56.668+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36196 #4076499 (1507 connections now open) 2016-03-13T08:27:57.124+0000 I NETWORK [conn4076489] end connection 10.0.26.69:38048 (1506 connections now open) 2016-03-13T08:27:57.124+0000 I NETWORK [conn4076488] end connection 10.0.26.69:38047 (1506 connections now open) 2016-03-13T08:27:57.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38059 #4076500 (1506 connections now open) 2016-03-13T08:27:57.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38060 #4076501 (1507 connections now open) 2016-03-13T08:27:58.113+0000 I NETWORK [conn4067672] end connection 10.0.20.53:37046 (1506 connections now open) 2016-03-13T08:27:58.113+0000 I NETWORK [conn4065822] end connection 10.0.216.90:41585 (1506 connections now open) 2016-03-13T08:27:58.113+0000 I NETWORK [conn4067106] end connection 10.0.26.69:56206 (1506 connections now open) 2016-03-13T08:27:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53048 #4076502 (1505 connections now open) 2016-03-13T08:27:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46318 #4076503 (1506 connections now open) 2016-03-13T08:27:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38063 #4076504 (1507 connections now open) 2016-03-13T08:27:58.114+0000 I NETWORK [conn4063814] end connection 10.0.27.237:46686 (1506 connections now open) 2016-03-13T08:27:58.114+0000 I NETWORK [conn4068647] end connection 10.0.7.113:46591 (1506 connections now open) 2016-03-13T08:27:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59685 #4076505 (1506 connections now open) 2016-03-13T08:27:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55014 #4076506 (1507 connections now open) 2016-03-13T08:27:58.544+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:58.545+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:58.545+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:58.545+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:27:59.046+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:59.046+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:59.046+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:59.047+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:27:59.547+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:27:59.548+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:27:59.548+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:27:59.548+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:28:00.049+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:00.050+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:00.050+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:00.050+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:28:00.158+0000 I NETWORK [conn4076491] end connection 10.0.27.237:59673 (1506 connections now open) 2016-03-13T08:28:00.158+0000 I NETWORK [conn4076490] end connection 10.0.27.237:59672 (1506 connections now open) 2016-03-13T08:28:00.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59686 #4076507 (1506 connections now open) 2016-03-13T08:28:00.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59687 #4076508 (1507 connections now open) 2016-03-13T08:28:00.551+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:00.551+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:00.551+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:00.551+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:28:01.052+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:01.053+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:01.053+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:01.053+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:28:01.053+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:28:01.053+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:28:01.053+0000-56e5249118fa188537623c65", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857681053), what: "balancer.round", ns: "", details: { executionTimeMillis: 2536, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:28:02.179+0000 I NETWORK [conn4076492] end connection 10.0.7.113:55005 (1506 connections now open) 2016-03-13T08:28:02.179+0000 I NETWORK [conn4076493] end connection 10.0.7.113:55006 (1506 connections now open) 2016-03-13T08:28:02.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55018 #4076509 (1506 connections now open) 2016-03-13T08:28:02.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55019 #4076510 (1507 connections now open) 2016-03-13T08:28:02.487+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:28:03.406+0000 I NETWORK [conn4076494] end connection 10.0.216.90:53041 (1506 connections now open) 2016-03-13T08:28:03.406+0000 I NETWORK [conn4076495] end connection 10.0.216.90:53042 (1506 connections now open) 2016-03-13T08:28:03.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53054 #4076511 (1506 connections now open) 2016-03-13T08:28:03.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53053 #4076512 (1507 connections now open) 2016-03-13T08:28:04.156+0000 I NETWORK [conn4076497] end connection 10.0.20.53:46311 (1506 connections now open) 2016-03-13T08:28:04.156+0000 I NETWORK [conn4076496] end connection 10.0.20.53:46310 (1506 connections now open) 2016-03-13T08:28:04.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46326 #4076513 (1506 connections now open) 2016-03-13T08:28:04.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46325 #4076514 (1507 connections now open) 2016-03-13T08:28:06.679+0000 I NETWORK [conn4076498] end connection 10.0.179.209:36195 (1506 connections now open) 2016-03-13T08:28:06.680+0000 I NETWORK [conn4076499] end connection 10.0.179.209:36196 (1505 connections now open) 2016-03-13T08:28:06.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36205 #4076515 (1506 connections now open) 2016-03-13T08:28:06.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36206 #4076516 (1507 connections now open) 2016-03-13T08:28:07.137+0000 I NETWORK [conn4076500] end connection 10.0.26.69:38059 (1506 connections now open) 2016-03-13T08:28:07.137+0000 I NETWORK [conn4076501] end connection 10.0.26.69:38060 (1506 connections now open) 2016-03-13T08:28:07.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38083 #4076517 (1506 connections now open) 2016-03-13T08:28:07.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38082 #4076518 (1507 connections now open) 2016-03-13T08:28:08.415+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:28:08.145+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:28:10.171+0000 I NETWORK [conn4076507] end connection 10.0.27.237:59686 (1506 connections now open) 2016-03-13T08:28:10.171+0000 I NETWORK [conn4076508] end connection 10.0.27.237:59687 (1505 connections now open) 2016-03-13T08:28:10.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59705 #4076519 (1506 connections now open) 2016-03-13T08:28:10.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59706 #4076520 (1507 connections now open) 2016-03-13T08:28:11.123+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:28:11.127+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:11.136+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:11.136+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:11.136+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:28:11.637+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:11.638+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:11.638+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:11.638+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:28:12.139+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:12.139+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:12.139+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:12.139+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:28:12.193+0000 I NETWORK [conn4076509] end connection 10.0.7.113:55018 (1506 connections now open) 2016-03-13T08:28:12.193+0000 I NETWORK [conn4076510] end connection 10.0.7.113:55019 (1506 connections now open) 2016-03-13T08:28:12.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55032 #4076521 (1506 connections now open) 2016-03-13T08:28:12.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55031 #4076522 (1507 connections now open) 2016-03-13T08:28:12.489+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:12.490+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:12.490+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:28:12.490+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:28:12.640+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:12.641+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:12.641+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:12.641+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:28:13.142+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:13.142+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:13.142+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:13.142+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:28:13.419+0000 I NETWORK [conn4076511] end connection 10.0.216.90:53054 (1506 connections now open) 2016-03-13T08:28:13.419+0000 I NETWORK [conn4076512] end connection 10.0.216.90:53053 (1506 connections now open) 2016-03-13T08:28:13.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53065 #4076523 (1506 connections now open) 2016-03-13T08:28:13.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53064 #4076524 (1507 connections now open) 2016-03-13T08:28:13.643+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:13.644+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:13.644+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:13.644+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:28:14.145+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:14.145+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:14.146+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:14.146+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:28:14.170+0000 I NETWORK [conn4076513] end connection 10.0.20.53:46326 (1506 connections now open) 2016-03-13T08:28:14.170+0000 I NETWORK [conn4076514] end connection 10.0.20.53:46325 (1506 connections now open) 2016-03-13T08:28:14.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46344 #4076525 (1506 connections now open) 2016-03-13T08:28:14.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46343 #4076526 (1507 connections now open) 2016-03-13T08:28:14.646+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:14.647+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:14.647+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:14.647+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:28:15.148+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:15.149+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:15.149+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:15.149+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:28:15.650+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:15.650+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:15.650+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:15.650+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:28:16.117+0000 I NETWORK [conn4072030] end connection 10.0.179.209:59873 (1506 connections now open) 2016-03-13T08:28:16.118+0000 I NETWORK [conn4072534] end connection 10.0.26.69:33802 (1505 connections now open) 2016-03-13T08:28:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36217 #4076527 (1506 connections now open) 2016-03-13T08:28:16.118+0000 I NETWORK [conn4068562] end connection 10.0.27.237:51434 (1504 connections now open) 2016-03-13T08:28:16.118+0000 I NETWORK [conn4068707] end connection 10.0.7.113:46658 (1504 connections now open) 2016-03-13T08:28:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38094 #4076528 (1505 connections now open) 2016-03-13T08:28:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59716 #4076529 (1506 connections now open) 2016-03-13T08:28:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55040 #4076530 (1507 connections now open) 2016-03-13T08:28:16.151+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:16.152+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:16.152+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:16.152+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:28:16.653+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:16.653+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:16.653+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:16.653+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:28:16.692+0000 I NETWORK [conn4076515] end connection 10.0.179.209:36205 (1506 connections now open) 2016-03-13T08:28:16.693+0000 I NETWORK [conn4076516] end connection 10.0.179.209:36206 (1506 connections now open) 2016-03-13T08:28:16.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36218 #4076531 (1506 connections now open) 2016-03-13T08:28:16.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36219 #4076532 (1507 connections now open) 2016-03-13T08:28:17.150+0000 I NETWORK [conn4076517] end connection 10.0.26.69:38083 (1506 connections now open) 2016-03-13T08:28:17.150+0000 I NETWORK [conn4076518] end connection 10.0.26.69:38082 (1506 connections now open) 2016-03-13T08:28:17.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38096 #4076533 (1506 connections now open) 2016-03-13T08:28:17.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38095 #4076534 (1507 connections now open) 2016-03-13T08:28:17.154+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:17.155+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:17.155+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:17.155+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:28:17.656+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:17.656+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:17.656+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:17.656+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:28:18.158+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:18.158+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:18.158+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:18.158+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:28:18.659+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:18.660+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:18.660+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:18.660+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:28:19.161+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:19.161+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:19.161+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:19.161+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:28:19.662+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:19.663+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:19.663+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:19.663+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:28:20.164+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:20.164+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:20.164+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:20.164+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:28:20.185+0000 I NETWORK [conn4076519] end connection 10.0.27.237:59705 (1506 connections now open) 2016-03-13T08:28:20.185+0000 I NETWORK [conn4076520] end connection 10.0.27.237:59706 (1506 connections now open) 2016-03-13T08:28:20.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59717 #4076535 (1506 connections now open) 2016-03-13T08:28:20.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59718 #4076536 (1507 connections now open) 2016-03-13T08:28:20.665+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:20.666+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:20.666+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:20.666+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:28:21.167+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:21.168+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:21.168+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:21.168+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:28:21.668+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:21.669+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:21.669+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:21.669+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:28:21.669+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:28:21.669+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:28:21.669+0000-56e524a518fa188537623c66", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857701669), what: "balancer.round", ns: "", details: { executionTimeMillis: 10582, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:28:22.206+0000 I NETWORK [conn4076521] end connection 10.0.7.113:55032 (1506 connections now open) 2016-03-13T08:28:22.206+0000 I NETWORK [conn4076522] end connection 10.0.7.113:55031 (1506 connections now open) 2016-03-13T08:28:22.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55053 #4076537 (1506 connections now open) 2016-03-13T08:28:22.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55054 #4076538 (1507 connections now open) 2016-03-13T08:28:22.493+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:22.494+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:22.494+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:28:22.494+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:28:23.433+0000 I NETWORK [conn4076523] end connection 10.0.216.90:53065 (1506 connections now open) 2016-03-13T08:28:23.433+0000 I NETWORK [conn4076524] end connection 10.0.216.90:53064 (1506 connections now open) 2016-03-13T08:28:23.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53085 #4076539 (1506 connections now open) 2016-03-13T08:28:23.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53086 #4076540 (1507 connections now open) 2016-03-13T08:28:24.183+0000 I NETWORK [conn4076525] end connection 10.0.20.53:46344 (1506 connections now open) 2016-03-13T08:28:24.183+0000 I NETWORK [conn4076526] end connection 10.0.20.53:46343 (1506 connections now open) 2016-03-13T08:28:24.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46353 #4076541 (1506 connections now open) 2016-03-13T08:28:24.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46354 #4076542 (1507 connections now open) 2016-03-13T08:28:26.705+0000 I NETWORK [conn4076532] end connection 10.0.179.209:36219 (1506 connections now open) 2016-03-13T08:28:26.705+0000 I NETWORK [conn4076531] end connection 10.0.179.209:36218 (1506 connections now open) 2016-03-13T08:28:26.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36239 #4076543 (1506 connections now open) 2016-03-13T08:28:26.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36238 #4076544 (1507 connections now open) 2016-03-13T08:28:27.163+0000 I NETWORK [conn4076533] end connection 10.0.26.69:38096 (1506 connections now open) 2016-03-13T08:28:27.163+0000 I NETWORK [conn4076534] end connection 10.0.26.69:38095 (1506 connections now open) 2016-03-13T08:28:27.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38107 #4076545 (1506 connections now open) 2016-03-13T08:28:27.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38108 #4076546 (1507 connections now open) 2016-03-13T08:28:28.114+0000 I NETWORK [conn4064685] end connection 10.0.20.53:34040 (1506 connections now open) 2016-03-13T08:28:28.114+0000 I NETWORK [conn4069429] end connection 10.0.179.209:57269 (1506 connections now open) 2016-03-13T08:28:28.114+0000 I NETWORK [conn4074170] end connection 10.0.216.90:50520 (1506 connections now open) 2016-03-13T08:28:28.114+0000 I NETWORK [conn4067632] end connection 10.0.7.113:45520 (1503 connections now open) 2016-03-13T08:28:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46360 #4076547 (1504 connections now open) 2016-03-13T08:28:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36242 #4076548 (1505 connections now open) 2016-03-13T08:28:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53092 #4076549 (1506 connections now open) 2016-03-13T08:28:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55061 #4076550 (1507 connections now open) 2016-03-13T08:28:30.198+0000 I NETWORK [conn4076535] end connection 10.0.27.237:59717 (1506 connections now open) 2016-03-13T08:28:30.198+0000 I NETWORK [conn4076536] end connection 10.0.27.237:59718 (1506 connections now open) 2016-03-13T08:28:30.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59733 #4076551 (1506 connections now open) 2016-03-13T08:28:30.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59734 #4076552 (1507 connections now open) 2016-03-13T08:28:31.726+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:31.726+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:31.726+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:31.726+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:28:32.219+0000 I NETWORK [conn4076537] end connection 10.0.7.113:55053 (1506 connections now open) 2016-03-13T08:28:32.219+0000 I NETWORK [conn4076538] end connection 10.0.7.113:55054 (1506 connections now open) 2016-03-13T08:28:32.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55063 #4076553 (1506 connections now open) 2016-03-13T08:28:32.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55064 #4076554 (1507 connections now open) 2016-03-13T08:28:32.227+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:32.228+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:32.228+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:32.228+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:28:32.497+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:32.497+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:32.498+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:28:32.498+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:28:32.729+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:32.730+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:32.730+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:32.730+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:28:33.231+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:33.231+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:33.231+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:33.231+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:28:33.446+0000 I NETWORK [conn4076539] end connection 10.0.216.90:53085 (1506 connections now open) 2016-03-13T08:28:33.446+0000 I NETWORK [conn4076540] end connection 10.0.216.90:53086 (1506 connections now open) 2016-03-13T08:28:33.447+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53098 #4076555 (1506 connections now open) 2016-03-13T08:28:33.447+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53099 #4076556 (1507 connections now open) 2016-03-13T08:28:33.732+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:33.733+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:33.733+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:33.733+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:28:33.733+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:28:33.733+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:28:33.733+0000-56e524b118fa188537623c67", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857713733), what: "balancer.round", ns: "", details: { executionTimeMillis: 2047, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:28:34.196+0000 I NETWORK [conn4076542] end connection 10.0.20.53:46354 (1506 connections now open) 2016-03-13T08:28:34.196+0000 I NETWORK [conn4076541] end connection 10.0.20.53:46353 (1506 connections now open) 2016-03-13T08:28:34.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46369 #4076557 (1506 connections now open) 2016-03-13T08:28:34.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46370 #4076558 (1507 connections now open) 2016-03-13T08:28:36.719+0000 I NETWORK [conn4076544] end connection 10.0.179.209:36238 (1506 connections now open) 2016-03-13T08:28:36.719+0000 I NETWORK [conn4076543] end connection 10.0.179.209:36239 (1506 connections now open) 2016-03-13T08:28:36.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36247 #4076559 (1506 connections now open) 2016-03-13T08:28:36.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36248 #4076560 (1507 connections now open) 2016-03-13T08:28:37.177+0000 I NETWORK [conn4076545] end connection 10.0.26.69:38107 (1506 connections now open) 2016-03-13T08:28:37.177+0000 I NETWORK [conn4076546] end connection 10.0.26.69:38108 (1506 connections now open) 2016-03-13T08:28:37.186+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38129 #4076561 (1506 connections now open) 2016-03-13T08:28:37.186+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38128 #4076562 (1507 connections now open) 2016-03-13T08:28:38.578+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:28:38.415+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:28:40.212+0000 I NETWORK [conn4076551] end connection 10.0.27.237:59733 (1506 connections now open) 2016-03-13T08:28:40.212+0000 I NETWORK [conn4076552] end connection 10.0.27.237:59734 (1506 connections now open) 2016-03-13T08:28:40.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59750 #4076563 (1506 connections now open) 2016-03-13T08:28:40.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59749 #4076564 (1507 connections now open) 2016-03-13T08:28:42.232+0000 I NETWORK [conn4076554] end connection 10.0.7.113:55064 (1506 connections now open) 2016-03-13T08:28:42.232+0000 I NETWORK [conn4076553] end connection 10.0.7.113:55063 (1506 connections now open) 2016-03-13T08:28:42.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55073 #4076565 (1507 connections now open) 2016-03-13T08:28:42.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55074 #4076566 (1507 connections now open) 2016-03-13T08:28:42.500+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:28:43.459+0000 I NETWORK [conn4076556] end connection 10.0.216.90:53099 (1506 connections now open) 2016-03-13T08:28:43.459+0000 I NETWORK [conn4076555] end connection 10.0.216.90:53098 (1506 connections now open) 2016-03-13T08:28:43.460+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53111 #4076567 (1506 connections now open) 2016-03-13T08:28:43.460+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53110 #4076568 (1507 connections now open) 2016-03-13T08:28:43.811+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:28:43.823+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:43.824+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:43.824+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:43.824+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:28:44.209+0000 I NETWORK [conn4076557] end connection 10.0.20.53:46369 (1506 connections now open) 2016-03-13T08:28:44.209+0000 I NETWORK [conn4076558] end connection 10.0.20.53:46370 (1505 connections now open) 2016-03-13T08:28:44.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46387 #4076569 (1506 connections now open) 2016-03-13T08:28:44.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46388 #4076570 (1507 connections now open) 2016-03-13T08:28:44.325+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:44.326+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:44.326+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:44.326+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:28:44.827+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:44.827+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:44.827+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:44.827+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:28:45.328+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:45.328+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:45.329+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:45.329+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:28:45.830+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:45.830+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:45.830+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:45.830+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:28:46.118+0000 I NETWORK [conn4068348] end connection 10.0.26.69:57524 (1506 connections now open) 2016-03-13T08:28:46.118+0000 I NETWORK [conn4066846] end connection 10.0.7.113:44716 (1505 connections now open) 2016-03-13T08:28:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38138 #4076571 (1506 connections now open) 2016-03-13T08:28:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55081 #4076572 (1507 connections now open) 2016-03-13T08:28:46.331+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:46.332+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:46.332+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:46.332+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:28:46.732+0000 I NETWORK [conn4076560] end connection 10.0.179.209:36248 (1506 connections now open) 2016-03-13T08:28:46.732+0000 I NETWORK [conn4076559] end connection 10.0.179.209:36247 (1506 connections now open) 2016-03-13T08:28:46.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36260 #4076573 (1506 connections now open) 2016-03-13T08:28:46.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36259 #4076574 (1507 connections now open) 2016-03-13T08:28:46.832+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:46.833+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:46.833+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:46.833+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:28:47.199+0000 I NETWORK [conn4076561] end connection 10.0.26.69:38129 (1506 connections now open) 2016-03-13T08:28:47.199+0000 I NETWORK [conn4076562] end connection 10.0.26.69:38128 (1506 connections now open) 2016-03-13T08:28:47.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38140 #4076575 (1506 connections now open) 2016-03-13T08:28:47.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38139 #4076576 (1507 connections now open) 2016-03-13T08:28:47.334+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:47.335+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:47.335+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:47.335+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:28:47.836+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:47.836+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:47.836+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:47.836+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:28:48.337+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:48.338+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:48.338+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:48.338+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:28:48.839+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:48.839+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:48.840+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:48.840+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:28:49.340+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:49.341+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:49.341+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:49.341+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:28:49.842+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:49.843+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:49.843+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:49.843+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:28:50.225+0000 I NETWORK [conn4076564] end connection 10.0.27.237:59749 (1506 connections now open) 2016-03-13T08:28:50.225+0000 I NETWORK [conn4076563] end connection 10.0.27.237:59750 (1506 connections now open) 2016-03-13T08:28:50.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59761 #4076577 (1506 connections now open) 2016-03-13T08:28:50.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59760 #4076578 (1507 connections now open) 2016-03-13T08:28:50.343+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:50.344+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:50.344+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:50.344+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:28:50.845+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:50.846+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:50.846+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:50.846+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:28:51.347+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:51.347+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:51.347+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:51.347+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:28:51.848+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:51.849+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:51.849+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:51.849+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:28:52.245+0000 I NETWORK [conn4076565] end connection 10.0.7.113:55073 (1506 connections now open) 2016-03-13T08:28:52.245+0000 I NETWORK [conn4076566] end connection 10.0.7.113:55074 (1506 connections now open) 2016-03-13T08:28:52.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55095 #4076579 (1506 connections now open) 2016-03-13T08:28:52.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55094 #4076580 (1507 connections now open) 2016-03-13T08:28:52.350+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:52.350+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:52.351+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:52.351+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:28:52.502+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:52.503+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:52.503+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:28:52.503+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:28:52.851+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:52.852+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:52.852+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:52.852+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:28:53.353+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:53.354+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:53.354+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:53.354+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:28:53.473+0000 I NETWORK [conn4076568] end connection 10.0.216.90:53110 (1506 connections now open) 2016-03-13T08:28:53.473+0000 I NETWORK [conn4076567] end connection 10.0.216.90:53111 (1506 connections now open) 2016-03-13T08:28:53.474+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53130 #4076581 (1506 connections now open) 2016-03-13T08:28:53.474+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53131 #4076582 (1507 connections now open) 2016-03-13T08:28:53.855+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:53.855+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:53.855+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:53.855+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:28:54.243+0000 I NETWORK [conn4076569] end connection 10.0.20.53:46387 (1506 connections now open) 2016-03-13T08:28:54.243+0000 I NETWORK [conn4076570] end connection 10.0.20.53:46388 (1506 connections now open) 2016-03-13T08:28:54.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46398 #4076583 (1506 connections now open) 2016-03-13T08:28:54.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46399 #4076584 (1507 connections now open) 2016-03-13T08:28:54.356+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:28:54.357+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:28:54.357+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:28:54.357+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:28:54.357+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:28:54.357+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:28:54.357+0000-56e524c618fa188537623c68", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857734357), what: "balancer.round", ns: "", details: { executionTimeMillis: 10581, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:28:56.755+0000 I NETWORK [conn4076573] end connection 10.0.179.209:36260 (1506 connections now open) 2016-03-13T08:28:56.755+0000 I NETWORK [conn4076574] end connection 10.0.179.209:36259 (1506 connections now open) 2016-03-13T08:28:56.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36278 #4076585 (1506 connections now open) 2016-03-13T08:28:56.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36277 #4076586 (1507 connections now open) 2016-03-13T08:28:57.212+0000 I NETWORK [conn4076576] end connection 10.0.26.69:38139 (1506 connections now open) 2016-03-13T08:28:57.212+0000 I NETWORK [conn4076575] end connection 10.0.26.69:38140 (1506 connections now open) 2016-03-13T08:28:57.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38153 #4076587 (1506 connections now open) 2016-03-13T08:28:57.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38152 #4076588 (1507 connections now open) 2016-03-13T08:28:58.113+0000 I NETWORK [conn4065795] end connection 10.0.179.209:53509 (1506 connections now open) 2016-03-13T08:28:58.113+0000 I NETWORK [conn4063667] end connection 10.0.20.53:33039 (1506 connections now open) 2016-03-13T08:28:58.113+0000 I NETWORK [conn4065952] end connection 10.0.216.90:41721 (1506 connections now open) 2016-03-13T08:28:58.114+0000 I NETWORK [conn4067839] end connection 10.0.7.113:45746 (1505 connections now open) 2016-03-13T08:28:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53139 #4076589 (1507 connections now open) 2016-03-13T08:28:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36282 #4076590 (1505 connections now open) 2016-03-13T08:28:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46405 #4076591 (1506 connections now open) 2016-03-13T08:28:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55103 #4076592 (1507 connections now open) 2016-03-13T08:29:00.248+0000 I NETWORK [conn4076577] end connection 10.0.27.237:59761 (1506 connections now open) 2016-03-13T08:29:00.248+0000 I NETWORK [conn4076578] end connection 10.0.27.237:59760 (1506 connections now open) 2016-03-13T08:29:00.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59776 #4076593 (1506 connections now open) 2016-03-13T08:29:00.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59775 #4076594 (1507 connections now open) 2016-03-13T08:29:02.267+0000 I NETWORK [conn4076579] end connection 10.0.7.113:55095 (1506 connections now open) 2016-03-13T08:29:02.267+0000 I NETWORK [conn4076580] end connection 10.0.7.113:55094 (1506 connections now open) 2016-03-13T08:29:02.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55107 #4076595 (1506 connections now open) 2016-03-13T08:29:02.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55108 #4076596 (1507 connections now open) 2016-03-13T08:29:02.506+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:02.507+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:02.507+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:29:02.507+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:29:03.487+0000 I NETWORK [conn4076582] end connection 10.0.216.90:53131 (1506 connections now open) 2016-03-13T08:29:03.487+0000 I NETWORK [conn4076581] end connection 10.0.216.90:53130 (1506 connections now open) 2016-03-13T08:29:03.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53145 #4076597 (1506 connections now open) 2016-03-13T08:29:03.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53144 #4076598 (1507 connections now open) 2016-03-13T08:29:04.256+0000 I NETWORK [conn4076583] end connection 10.0.20.53:46398 (1506 connections now open) 2016-03-13T08:29:04.256+0000 I NETWORK [conn4076584] end connection 10.0.20.53:46399 (1506 connections now open) 2016-03-13T08:29:04.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46414 #4076599 (1506 connections now open) 2016-03-13T08:29:04.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46413 #4076600 (1507 connections now open) 2016-03-13T08:29:04.517+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:04.518+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:04.518+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:04.518+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:29:05.019+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:05.020+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:05.020+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:05.020+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:29:05.521+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:05.521+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:05.521+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:05.521+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:29:06.022+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:06.023+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:06.023+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:06.023+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:29:06.524+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:06.524+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:06.524+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:06.524+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:29:06.768+0000 I NETWORK [conn4076585] end connection 10.0.179.209:36278 (1506 connections now open) 2016-03-13T08:29:06.768+0000 I NETWORK [conn4076586] end connection 10.0.179.209:36277 (1506 connections now open) 2016-03-13T08:29:06.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36289 #4076601 (1506 connections now open) 2016-03-13T08:29:06.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36290 #4076602 (1507 connections now open) 2016-03-13T08:29:07.025+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:07.026+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:07.026+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:07.026+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:29:07.026+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:29:07.026+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:29:07.026+0000-56e524d318fa188537623c69", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857747026), what: "balancer.round", ns: "", details: { executionTimeMillis: 2554, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:29:07.226+0000 I NETWORK [conn4076588] end connection 10.0.26.69:38152 (1506 connections now open) 2016-03-13T08:29:07.226+0000 I NETWORK [conn4076587] end connection 10.0.26.69:38153 (1506 connections now open) 2016-03-13T08:29:07.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38175 #4076603 (1506 connections now open) 2016-03-13T08:29:07.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38174 #4076604 (1507 connections now open) 2016-03-13T08:29:08.729+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:29:08.579+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:29:10.261+0000 I NETWORK [conn4076594] end connection 10.0.27.237:59775 (1506 connections now open) 2016-03-13T08:29:10.261+0000 I NETWORK [conn4076593] end connection 10.0.27.237:59776 (1506 connections now open) 2016-03-13T08:29:10.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59793 #4076605 (1506 connections now open) 2016-03-13T08:29:10.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59794 #4076606 (1507 connections now open) 2016-03-13T08:29:12.280+0000 I NETWORK [conn4076595] end connection 10.0.7.113:55107 (1506 connections now open) 2016-03-13T08:29:12.280+0000 I NETWORK [conn4076596] end connection 10.0.7.113:55108 (1506 connections now open) 2016-03-13T08:29:12.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55121 #4076607 (1506 connections now open) 2016-03-13T08:29:12.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55122 #4076608 (1507 connections now open) 2016-03-13T08:29:12.509+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:29:13.499+0000 I NETWORK [conn4076598] end connection 10.0.216.90:53144 (1506 connections now open) 2016-03-13T08:29:13.500+0000 I NETWORK [conn4076597] end connection 10.0.216.90:53145 (1505 connections now open) 2016-03-13T08:29:13.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53156 #4076609 (1506 connections now open) 2016-03-13T08:29:13.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53157 #4076610 (1507 connections now open) 2016-03-13T08:29:14.270+0000 I NETWORK [conn4076599] end connection 10.0.20.53:46414 (1506 connections now open) 2016-03-13T08:29:14.270+0000 I NETWORK [conn4076600] end connection 10.0.20.53:46413 (1506 connections now open) 2016-03-13T08:29:14.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46432 #4076611 (1506 connections now open) 2016-03-13T08:29:14.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46433 #4076612 (1507 connections now open) 2016-03-13T08:29:16.117+0000 I NETWORK [conn4071096] end connection 10.0.179.209:58941 (1506 connections now open) 2016-03-13T08:29:16.118+0000 I NETWORK [conn4064727] end connection 10.0.26.69:53634 (1505 connections now open) 2016-03-13T08:29:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36300 #4076613 (1506 connections now open) 2016-03-13T08:29:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38185 #4076614 (1507 connections now open) 2016-03-13T08:29:16.782+0000 I NETWORK [conn4076601] end connection 10.0.179.209:36289 (1506 connections now open) 2016-03-13T08:29:16.782+0000 I NETWORK [conn4076602] end connection 10.0.179.209:36290 (1506 connections now open) 2016-03-13T08:29:16.782+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36301 #4076615 (1506 connections now open) 2016-03-13T08:29:16.782+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36302 #4076616 (1507 connections now open) 2016-03-13T08:29:17.094+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:29:17.096+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:17.106+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:17.106+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:17.106+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:29:17.240+0000 I NETWORK [conn4076604] end connection 10.0.26.69:38174 (1506 connections now open) 2016-03-13T08:29:17.240+0000 I NETWORK [conn4076603] end connection 10.0.26.69:38175 (1506 connections now open) 2016-03-13T08:29:17.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38186 #4076617 (1506 connections now open) 2016-03-13T08:29:17.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38187 #4076618 (1507 connections now open) 2016-03-13T08:29:17.606+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:17.607+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:17.607+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:17.607+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:29:18.108+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:18.109+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:18.109+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:18.109+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:29:18.610+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:18.610+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:18.610+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:18.610+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:29:19.111+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:19.112+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:19.112+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:19.112+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:29:19.613+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:19.613+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:19.613+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:19.613+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:29:20.114+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:20.115+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:20.115+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:20.115+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:29:20.274+0000 I NETWORK [conn4076606] end connection 10.0.27.237:59794 (1506 connections now open) 2016-03-13T08:29:20.274+0000 I NETWORK [conn4076605] end connection 10.0.27.237:59793 (1506 connections now open) 2016-03-13T08:29:20.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59806 #4076619 (1506 connections now open) 2016-03-13T08:29:20.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59807 #4076620 (1507 connections now open) 2016-03-13T08:29:20.616+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:20.617+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:20.617+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:20.617+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:29:21.118+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:21.118+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:21.118+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:21.118+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:29:21.619+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:21.620+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:21.620+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:21.620+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:29:22.121+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:22.121+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:22.121+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:22.121+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:29:22.294+0000 I NETWORK [conn4076608] end connection 10.0.7.113:55122 (1506 connections now open) 2016-03-13T08:29:22.294+0000 I NETWORK [conn4076607] end connection 10.0.7.113:55121 (1506 connections now open) 2016-03-13T08:29:22.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55142 #4076621 (1506 connections now open) 2016-03-13T08:29:22.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55143 #4076622 (1507 connections now open) 2016-03-13T08:29:22.512+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:22.513+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:22.513+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:29:22.513+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:29:22.622+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:22.623+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:22.623+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:22.623+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:29:23.124+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:23.124+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:23.124+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:23.124+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:29:23.512+0000 I NETWORK [conn4076610] end connection 10.0.216.90:53157 (1506 connections now open) 2016-03-13T08:29:23.512+0000 I NETWORK [conn4076609] end connection 10.0.216.90:53156 (1506 connections now open) 2016-03-13T08:29:23.513+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53176 #4076623 (1506 connections now open) 2016-03-13T08:29:23.513+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53175 #4076624 (1507 connections now open) 2016-03-13T08:29:23.625+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:23.626+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:23.626+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:23.626+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:29:24.127+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:24.128+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:24.128+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:24.128+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:29:24.284+0000 I NETWORK [conn4076611] end connection 10.0.20.53:46432 (1506 connections now open) 2016-03-13T08:29:24.284+0000 I NETWORK [conn4076612] end connection 10.0.20.53:46433 (1506 connections now open) 2016-03-13T08:29:24.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46442 #4076625 (1506 connections now open) 2016-03-13T08:29:24.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46441 #4076626 (1507 connections now open) 2016-03-13T08:29:24.629+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:24.629+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:24.629+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:24.629+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:29:25.130+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:25.131+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:25.131+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:25.131+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:29:25.631+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:25.632+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:25.632+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:25.632+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:29:26.133+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:26.134+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:26.134+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:26.134+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:29:26.634+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:26.635+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:26.635+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:26.635+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:29:26.794+0000 I NETWORK [conn4076616] end connection 10.0.179.209:36302 (1506 connections now open) 2016-03-13T08:29:26.795+0000 I NETWORK [conn4076615] end connection 10.0.179.209:36301 (1505 connections now open) 2016-03-13T08:29:26.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36322 #4076627 (1507 connections now open) 2016-03-13T08:29:26.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36323 #4076628 (1507 connections now open) 2016-03-13T08:29:27.136+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:27.137+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:27.137+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:27.137+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:29:27.254+0000 I NETWORK [conn4076617] end connection 10.0.26.69:38186 (1506 connections now open) 2016-03-13T08:29:27.254+0000 I NETWORK [conn4076618] end connection 10.0.26.69:38187 (1506 connections now open) 2016-03-13T08:29:27.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38199 #4076629 (1506 connections now open) 2016-03-13T08:29:27.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38198 #4076630 (1507 connections now open) 2016-03-13T08:29:27.638+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:27.638+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:27.638+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:27.638+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:29:27.638+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:29:27.638+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:29:27.638+0000-56e524e718fa188537623c6a", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857767638), what: "balancer.round", ns: "", details: { executionTimeMillis: 10580, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:29:28.113+0000 I NETWORK [conn4064475] end connection 10.0.179.209:52083 (1506 connections now open) 2016-03-13T08:29:28.113+0000 I NETWORK [conn4071246] end connection 10.0.20.53:40772 (1506 connections now open) 2016-03-13T08:29:28.114+0000 I NETWORK [conn4067569] end connection 10.0.27.237:50406 (1504 connections now open) 2016-03-13T08:29:28.113+0000 I NETWORK [conn4070437] end connection 10.0.26.69:59773 (1505 connections now open) 2016-03-13T08:29:28.114+0000 I NETWORK [conn4068645] end connection 10.0.216.90:44599 (1502 connections now open) 2016-03-13T08:29:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36326 #4076631 (1503 connections now open) 2016-03-13T08:29:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53183 #4076632 (1504 connections now open) 2016-03-13T08:29:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46449 #4076633 (1505 connections now open) 2016-03-13T08:29:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38202 #4076634 (1506 connections now open) 2016-03-13T08:29:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59818 #4076635 (1507 connections now open) 2016-03-13T08:29:30.288+0000 I NETWORK [conn4076619] end connection 10.0.27.237:59806 (1506 connections now open) 2016-03-13T08:29:30.288+0000 I NETWORK [conn4076620] end connection 10.0.27.237:59807 (1506 connections now open) 2016-03-13T08:29:30.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59820 #4076636 (1506 connections now open) 2016-03-13T08:29:30.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59821 #4076637 (1507 connections now open) 2016-03-13T08:29:32.307+0000 I NETWORK [conn4076622] end connection 10.0.7.113:55143 (1506 connections now open) 2016-03-13T08:29:32.307+0000 I NETWORK [conn4076621] end connection 10.0.7.113:55142 (1506 connections now open) 2016-03-13T08:29:32.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55152 #4076638 (1506 connections now open) 2016-03-13T08:29:32.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55153 #4076639 (1507 connections now open) 2016-03-13T08:29:32.515+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:32.516+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:32.516+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:29:32.516+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:29:33.525+0000 I NETWORK [conn4076624] end connection 10.0.216.90:53175 (1506 connections now open) 2016-03-13T08:29:33.525+0000 I NETWORK [conn4076623] end connection 10.0.216.90:53176 (1506 connections now open) 2016-03-13T08:29:33.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53188 #4076640 (1506 connections now open) 2016-03-13T08:29:33.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53189 #4076641 (1507 connections now open) 2016-03-13T08:29:34.298+0000 I NETWORK [conn4076626] end connection 10.0.20.53:46441 (1506 connections now open) 2016-03-13T08:29:34.298+0000 I NETWORK [conn4076625] end connection 10.0.20.53:46442 (1506 connections now open) 2016-03-13T08:29:34.299+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46459 #4076642 (1506 connections now open) 2016-03-13T08:29:34.299+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46458 #4076643 (1507 connections now open) 2016-03-13T08:29:36.807+0000 I NETWORK [conn4076627] end connection 10.0.179.209:36322 (1506 connections now open) 2016-03-13T08:29:36.807+0000 I NETWORK [conn4076628] end connection 10.0.179.209:36323 (1505 connections now open) 2016-03-13T08:29:36.808+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36331 #4076644 (1506 connections now open) 2016-03-13T08:29:36.808+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36332 #4076645 (1507 connections now open) 2016-03-13T08:29:37.267+0000 I NETWORK [conn4076630] end connection 10.0.26.69:38198 (1506 connections now open) 2016-03-13T08:29:37.267+0000 I NETWORK [conn4076629] end connection 10.0.26.69:38199 (1505 connections now open) 2016-03-13T08:29:37.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38209 #4076646 (1506 connections now open) 2016-03-13T08:29:37.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38210 #4076647 (1507 connections now open) 2016-03-13T08:29:37.709+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:37.710+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:37.710+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:37.710+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:29:38.211+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:38.212+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:38.212+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:38.212+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:29:38.713+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:38.713+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:38.713+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:38.713+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:29:38.928+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:29:38.729+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:29:39.214+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:39.215+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:39.215+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:39.215+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:29:39.716+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:39.717+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:39.717+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:39.717+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:29:40.217+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:40.218+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:40.218+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:40.218+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:29:40.218+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:29:40.218+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:29:40.218+0000-56e524f418fa188537623c6b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857780218), what: "balancer.round", ns: "", details: { executionTimeMillis: 2544, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:29:40.301+0000 I NETWORK [conn4076636] end connection 10.0.27.237:59820 (1506 connections now open) 2016-03-13T08:29:40.301+0000 I NETWORK [conn4076637] end connection 10.0.27.237:59821 (1506 connections now open) 2016-03-13T08:29:40.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59836 #4076648 (1506 connections now open) 2016-03-13T08:29:40.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59837 #4076649 (1507 connections now open) 2016-03-13T08:29:42.320+0000 I NETWORK [conn4076639] end connection 10.0.7.113:55153 (1506 connections now open) 2016-03-13T08:29:42.320+0000 I NETWORK [conn4076638] end connection 10.0.7.113:55152 (1506 connections now open) 2016-03-13T08:29:42.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55164 #4076650 (1506 connections now open) 2016-03-13T08:29:42.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55163 #4076651 (1507 connections now open) 2016-03-13T08:29:42.518+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:29:43.539+0000 I NETWORK [conn4076641] end connection 10.0.216.90:53189 (1506 connections now open) 2016-03-13T08:29:43.539+0000 I NETWORK [conn4076640] end connection 10.0.216.90:53188 (1506 connections now open) 2016-03-13T08:29:43.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53201 #4076652 (1506 connections now open) 2016-03-13T08:29:43.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53202 #4076653 (1507 connections now open) 2016-03-13T08:29:44.311+0000 I NETWORK [conn4076642] end connection 10.0.20.53:46459 (1506 connections now open) 2016-03-13T08:29:44.311+0000 I NETWORK [conn4076643] end connection 10.0.20.53:46458 (1506 connections now open) 2016-03-13T08:29:44.312+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46477 #4076654 (1506 connections now open) 2016-03-13T08:29:44.312+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46478 #4076655 (1507 connections now open) 2016-03-13T08:29:46.117+0000 I NETWORK [conn4064619] end connection 10.0.179.209:52229 (1506 connections now open) 2016-03-13T08:29:46.118+0000 I NETWORK [conn4069195] end connection 10.0.26.69:58433 (1505 connections now open) 2016-03-13T08:29:46.118+0000 I NETWORK [conn4070858] end connection 10.0.7.113:48986 (1504 connections now open) 2016-03-13T08:29:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38228 #4076656 (1505 connections now open) 2016-03-13T08:29:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55170 #4076657 (1506 connections now open) 2016-03-13T08:29:46.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36342 #4076658 (1507 connections now open) 2016-03-13T08:29:46.821+0000 I NETWORK [conn4076644] end connection 10.0.179.209:36331 (1506 connections now open) 2016-03-13T08:29:46.821+0000 I NETWORK [conn4076645] end connection 10.0.179.209:36332 (1506 connections now open) 2016-03-13T08:29:46.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36343 #4076659 (1506 connections now open) 2016-03-13T08:29:46.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36344 #4076660 (1507 connections now open) 2016-03-13T08:29:47.288+0000 I NETWORK [conn4076647] end connection 10.0.26.69:38210 (1506 connections now open) 2016-03-13T08:29:47.288+0000 I NETWORK [conn4076646] end connection 10.0.26.69:38209 (1506 connections now open) 2016-03-13T08:29:47.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38230 #4076661 (1506 connections now open) 2016-03-13T08:29:47.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38229 #4076662 (1507 connections now open) 2016-03-13T08:29:50.277+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:29:50.279+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:50.314+0000 I NETWORK [conn4076649] end connection 10.0.27.237:59837 (1506 connections now open) 2016-03-13T08:29:50.314+0000 I NETWORK [conn4076648] end connection 10.0.27.237:59836 (1506 connections now open) 2016-03-13T08:29:50.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59846 #4076663 (1506 connections now open) 2016-03-13T08:29:50.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59845 #4076664 (1507 connections now open) 2016-03-13T08:29:50.342+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:50.342+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:50.342+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:29:50.843+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:50.844+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:50.844+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:50.844+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:29:51.345+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:51.345+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:51.345+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:51.345+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:29:51.846+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:51.847+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:51.847+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:51.847+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:29:52.334+0000 I NETWORK [conn4076650] end connection 10.0.7.113:55164 (1506 connections now open) 2016-03-13T08:29:52.334+0000 I NETWORK [conn4076651] end connection 10.0.7.113:55163 (1506 connections now open) 2016-03-13T08:29:52.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55185 #4076665 (1506 connections now open) 2016-03-13T08:29:52.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55184 #4076666 (1507 connections now open) 2016-03-13T08:29:52.347+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:52.348+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:52.348+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:52.348+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:29:52.521+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:52.522+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:52.522+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:29:52.522+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:29:52.849+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:52.850+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:52.850+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:52.850+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:29:53.350+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:53.351+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:53.351+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:53.351+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:29:53.552+0000 I NETWORK [conn4076653] end connection 10.0.216.90:53202 (1506 connections now open) 2016-03-13T08:29:53.552+0000 I NETWORK [conn4076652] end connection 10.0.216.90:53201 (1506 connections now open) 2016-03-13T08:29:53.561+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53220 #4076667 (1506 connections now open) 2016-03-13T08:29:53.562+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53221 #4076668 (1507 connections now open) 2016-03-13T08:29:53.852+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:53.853+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:53.853+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:53.853+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:29:54.325+0000 I NETWORK [conn4076654] end connection 10.0.20.53:46477 (1506 connections now open) 2016-03-13T08:29:54.325+0000 I NETWORK [conn4076655] end connection 10.0.20.53:46478 (1506 connections now open) 2016-03-13T08:29:54.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46488 #4076669 (1506 connections now open) 2016-03-13T08:29:54.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46487 #4076670 (1507 connections now open) 2016-03-13T08:29:54.353+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:54.354+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:54.354+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:54.354+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:29:54.855+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:54.856+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:54.856+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:54.856+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:29:55.357+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:55.357+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:55.357+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:55.357+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:29:55.858+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:55.859+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:55.859+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:55.859+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:29:56.360+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:56.360+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:56.360+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:56.360+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:29:56.834+0000 I NETWORK [conn4076660] end connection 10.0.179.209:36344 (1506 connections now open) 2016-03-13T08:29:56.834+0000 I NETWORK [conn4076659] end connection 10.0.179.209:36343 (1506 connections now open) 2016-03-13T08:29:56.843+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36360 #4076671 (1506 connections now open) 2016-03-13T08:29:56.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36361 #4076672 (1507 connections now open) 2016-03-13T08:29:56.861+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:56.862+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:56.862+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:56.862+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:29:57.301+0000 I NETWORK [conn4076662] end connection 10.0.26.69:38229 (1506 connections now open) 2016-03-13T08:29:57.301+0000 I NETWORK [conn4076661] end connection 10.0.26.69:38230 (1506 connections now open) 2016-03-13T08:29:57.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38238 #4076673 (1506 connections now open) 2016-03-13T08:29:57.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38237 #4076674 (1507 connections now open) 2016-03-13T08:29:57.363+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:57.363+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:57.363+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:57.363+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:29:57.864+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:57.865+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:57.865+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:57.865+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:29:58.120+0000 I NETWORK [conn4063752] end connection 10.0.20.53:33122 (1506 connections now open) 2016-03-13T08:29:58.121+0000 I NETWORK [conn4074395] end connection 10.0.26.69:35777 (1506 connections now open) 2016-03-13T08:29:58.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38245 #4076675 (1506 connections now open) 2016-03-13T08:29:58.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46495 #4076676 (1507 connections now open) 2016-03-13T08:29:58.366+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:58.366+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:58.366+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:58.366+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:29:58.867+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:58.868+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:58.868+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:58.868+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:29:59.369+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:59.370+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:59.370+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:59.370+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:29:59.871+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:29:59.871+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:29:59.871+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:29:59.871+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:30:00.336+0000 I NETWORK [conn4076664] end connection 10.0.27.237:59845 (1506 connections now open) 2016-03-13T08:30:00.336+0000 I NETWORK [conn4076663] end connection 10.0.27.237:59846 (1506 connections now open) 2016-03-13T08:30:00.337+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59858 #4076677 (1506 connections now open) 2016-03-13T08:30:00.337+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59859 #4076678 (1507 connections now open) 2016-03-13T08:30:00.372+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:00.373+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:00.373+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:00.373+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:30:00.874+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:00.875+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:00.875+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:00.875+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:30:00.875+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:30:00.875+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:30:00.875+0000-56e5250818fa188537623c6c", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857800875), what: "balancer.round", ns: "", details: { executionTimeMillis: 10630, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:30:02.356+0000 I NETWORK [conn4076666] end connection 10.0.7.113:55184 (1506 connections now open) 2016-03-13T08:30:02.356+0000 I NETWORK [conn4076665] end connection 10.0.7.113:55185 (1506 connections now open) 2016-03-13T08:30:02.357+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55197 #4076679 (1506 connections now open) 2016-03-13T08:30:02.357+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55198 #4076680 (1507 connections now open) 2016-03-13T08:30:02.524+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:02.525+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:02.525+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:30:02.525+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:30:03.574+0000 I NETWORK [conn4076668] end connection 10.0.216.90:53221 (1506 connections now open) 2016-03-13T08:30:03.574+0000 I NETWORK [conn4076667] end connection 10.0.216.90:53220 (1506 connections now open) 2016-03-13T08:30:03.575+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53235 #4076681 (1506 connections now open) 2016-03-13T08:30:03.575+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53234 #4076682 (1507 connections now open) 2016-03-13T08:30:04.346+0000 I NETWORK [conn4076669] end connection 10.0.20.53:46488 (1506 connections now open) 2016-03-13T08:30:04.346+0000 I NETWORK [conn4076670] end connection 10.0.20.53:46487 (1506 connections now open) 2016-03-13T08:30:04.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46504 #4076683 (1506 connections now open) 2016-03-13T08:30:04.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46505 #4076684 (1507 connections now open) 2016-03-13T08:30:06.855+0000 I NETWORK [conn4076671] end connection 10.0.179.209:36360 (1506 connections now open) 2016-03-13T08:30:06.855+0000 I NETWORK [conn4076672] end connection 10.0.179.209:36361 (1506 connections now open) 2016-03-13T08:30:06.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36372 #4076685 (1506 connections now open) 2016-03-13T08:30:06.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36373 #4076686 (1507 connections now open) 2016-03-13T08:30:07.314+0000 I NETWORK [conn4076673] end connection 10.0.26.69:38238 (1506 connections now open) 2016-03-13T08:30:07.314+0000 I NETWORK [conn4076674] end connection 10.0.26.69:38237 (1506 connections now open) 2016-03-13T08:30:07.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38257 #4076687 (1506 connections now open) 2016-03-13T08:30:07.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38256 #4076688 (1507 connections now open) 2016-03-13T08:30:09.183+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:30:08.928+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:30:10.349+0000 I NETWORK [conn4076678] end connection 10.0.27.237:59859 (1506 connections now open) 2016-03-13T08:30:10.349+0000 I NETWORK [conn4076677] end connection 10.0.27.237:59858 (1506 connections now open) 2016-03-13T08:30:10.350+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59878 #4076689 (1506 connections now open) 2016-03-13T08:30:10.350+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59879 #4076690 (1507 connections now open) 2016-03-13T08:30:10.949+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:10.950+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:10.950+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:10.950+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:30:11.451+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:11.451+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:11.451+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:11.451+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:30:11.952+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:11.953+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:11.953+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:11.953+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:30:12.369+0000 I NETWORK [conn4076679] end connection 10.0.7.113:55197 (1506 connections now open) 2016-03-13T08:30:12.369+0000 I NETWORK [conn4076680] end connection 10.0.7.113:55198 (1506 connections now open) 2016-03-13T08:30:12.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55214 #4076691 (1506 connections now open) 2016-03-13T08:30:12.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55213 #4076692 (1507 connections now open) 2016-03-13T08:30:12.454+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:12.454+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:12.454+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:12.454+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:30:12.528+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:12.529+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:12.529+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:30:12.529+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:30:12.955+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:12.956+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:12.956+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:12.956+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:30:12.956+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:30:12.956+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:30:12.956+0000-56e5251418fa188537623c6d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857812956), what: "balancer.round", ns: "", details: { executionTimeMillis: 2045, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:30:13.587+0000 I NETWORK [conn4076681] end connection 10.0.216.90:53235 (1506 connections now open) 2016-03-13T08:30:13.587+0000 I NETWORK [conn4076682] end connection 10.0.216.90:53234 (1506 connections now open) 2016-03-13T08:30:13.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53247 #4076693 (1506 connections now open) 2016-03-13T08:30:13.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53248 #4076694 (1507 connections now open) 2016-03-13T08:30:14.360+0000 I NETWORK [conn4076684] end connection 10.0.20.53:46505 (1506 connections now open) 2016-03-13T08:30:14.360+0000 I NETWORK [conn4076683] end connection 10.0.20.53:46504 (1506 connections now open) 2016-03-13T08:30:14.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46522 #4076695 (1506 connections now open) 2016-03-13T08:30:14.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46523 #4076696 (1507 connections now open) 2016-03-13T08:30:16.119+0000 I NETWORK [conn4069153] end connection 10.0.26.69:58384 (1506 connections now open) 2016-03-13T08:30:16.119+0000 I NETWORK [conn4068661] end connection 10.0.216.90:44613 (1506 connections now open) 2016-03-13T08:30:16.119+0000 I NETWORK [conn4062894] end connection 10.0.7.113:40442 (1506 connections now open) 2016-03-13T08:30:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53252 #4076697 (1505 connections now open) 2016-03-13T08:30:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38278 #4076698 (1506 connections now open) 2016-03-13T08:30:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55221 #4076699 (1507 connections now open) 2016-03-13T08:30:16.868+0000 I NETWORK [conn4076685] end connection 10.0.179.209:36372 (1506 connections now open) 2016-03-13T08:30:16.868+0000 I NETWORK [conn4076686] end connection 10.0.179.209:36373 (1506 connections now open) 2016-03-13T08:30:16.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36386 #4076700 (1506 connections now open) 2016-03-13T08:30:16.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36387 #4076701 (1507 connections now open) 2016-03-13T08:30:17.327+0000 I NETWORK [conn4076687] end connection 10.0.26.69:38257 (1506 connections now open) 2016-03-13T08:30:17.327+0000 I NETWORK [conn4076688] end connection 10.0.26.69:38256 (1506 connections now open) 2016-03-13T08:30:17.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38280 #4076702 (1506 connections now open) 2016-03-13T08:30:17.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38279 #4076703 (1507 connections now open) 2016-03-13T08:30:20.362+0000 I NETWORK [conn4076690] end connection 10.0.27.237:59879 (1506 connections now open) 2016-03-13T08:30:20.362+0000 I NETWORK [conn4076689] end connection 10.0.27.237:59878 (1505 connections now open) 2016-03-13T08:30:20.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59891 #4076704 (1506 connections now open) 2016-03-13T08:30:20.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59892 #4076705 (1507 connections now open) 2016-03-13T08:30:22.383+0000 I NETWORK [conn4076691] end connection 10.0.7.113:55214 (1506 connections now open) 2016-03-13T08:30:22.383+0000 I NETWORK [conn4076692] end connection 10.0.7.113:55213 (1506 connections now open) 2016-03-13T08:30:22.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55236 #4076706 (1507 connections now open) 2016-03-13T08:30:22.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55235 #4076707 (1507 connections now open) 2016-03-13T08:30:22.531+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:30:23.022+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:30:23.026+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:23.048+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:23.048+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:23.048+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:30:23.549+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:23.549+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:23.549+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:23.549+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:30:23.600+0000 I NETWORK [conn4076693] end connection 10.0.216.90:53247 (1506 connections now open) 2016-03-13T08:30:23.600+0000 I NETWORK [conn4076694] end connection 10.0.216.90:53248 (1505 connections now open) 2016-03-13T08:30:23.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53269 #4076708 (1506 connections now open) 2016-03-13T08:30:23.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53270 #4076709 (1507 connections now open) 2016-03-13T08:30:24.050+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:24.051+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:24.051+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:24.051+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:30:24.373+0000 I NETWORK [conn4076696] end connection 10.0.20.53:46523 (1506 connections now open) 2016-03-13T08:30:24.373+0000 I NETWORK [conn4076695] end connection 10.0.20.53:46522 (1506 connections now open) 2016-03-13T08:30:24.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46534 #4076710 (1506 connections now open) 2016-03-13T08:30:24.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46533 #4076711 (1507 connections now open) 2016-03-13T08:30:24.552+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:24.552+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:24.552+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:24.552+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:30:25.053+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:25.054+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:25.054+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:25.054+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:30:25.555+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:25.555+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:25.556+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:25.556+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:30:26.056+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:26.057+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:26.057+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:26.057+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:30:26.558+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:26.559+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:26.559+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:26.559+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:30:26.881+0000 I NETWORK [conn4076700] end connection 10.0.179.209:36386 (1506 connections now open) 2016-03-13T08:30:26.881+0000 I NETWORK [conn4076701] end connection 10.0.179.209:36387 (1506 connections now open) 2016-03-13T08:30:26.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36406 #4076712 (1506 connections now open) 2016-03-13T08:30:26.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36407 #4076713 (1507 connections now open) 2016-03-13T08:30:27.060+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:27.060+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:27.060+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:27.060+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:30:27.340+0000 I NETWORK [conn4076702] end connection 10.0.26.69:38280 (1506 connections now open) 2016-03-13T08:30:27.340+0000 I NETWORK [conn4076703] end connection 10.0.26.69:38279 (1506 connections now open) 2016-03-13T08:30:27.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38289 #4076714 (1506 connections now open) 2016-03-13T08:30:27.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38288 #4076715 (1507 connections now open) 2016-03-13T08:30:27.561+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:27.562+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:27.562+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:27.562+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:30:28.063+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:28.064+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:28.064+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:28.064+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:30:28.113+0000 I NETWORK [conn4071328] end connection 10.0.179.209:59179 (1506 connections now open) 2016-03-13T08:30:28.113+0000 I NETWORK [conn4072967] end connection 10.0.20.53:42578 (1506 connections now open) 2016-03-13T08:30:28.113+0000 I NETWORK [conn4068409] end connection 10.0.26.69:57589 (1504 connections now open) 2016-03-13T08:30:28.113+0000 I NETWORK [conn4065543] end connection 10.0.27.237:48391 (1504 connections now open) 2016-03-13T08:30:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36410 #4076716 (1504 connections now open) 2016-03-13T08:30:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46540 #4076717 (1505 connections now open) 2016-03-13T08:30:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38296 #4076718 (1506 connections now open) 2016-03-13T08:30:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59905 #4076719 (1507 connections now open) 2016-03-13T08:30:28.565+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:28.565+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:28.565+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:28.565+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:30:29.066+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:29.067+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:29.067+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:29.067+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:30:29.568+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:29.568+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:29.568+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:29.568+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:30:30.069+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:30.070+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:30.070+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:30.070+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:30:30.375+0000 I NETWORK [conn4076704] end connection 10.0.27.237:59891 (1506 connections now open) 2016-03-13T08:30:30.375+0000 I NETWORK [conn4076705] end connection 10.0.27.237:59892 (1506 connections now open) 2016-03-13T08:30:30.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59906 #4076720 (1506 connections now open) 2016-03-13T08:30:30.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59907 #4076721 (1507 connections now open) 2016-03-13T08:30:30.571+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:30.571+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:30.572+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:30.572+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:30:31.072+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:31.073+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:31.073+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:31.073+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:30:31.574+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:31.575+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:31.575+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:31.575+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:30:32.075+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:32.076+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:32.076+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:32.076+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:30:32.396+0000 I NETWORK [conn4076707] end connection 10.0.7.113:55235 (1506 connections now open) 2016-03-13T08:30:32.396+0000 I NETWORK [conn4076706] end connection 10.0.7.113:55236 (1506 connections now open) 2016-03-13T08:30:32.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55245 #4076722 (1506 connections now open) 2016-03-13T08:30:32.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55246 #4076723 (1507 connections now open) 2016-03-13T08:30:32.554+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:32.555+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:32.555+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:30:32.555+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:30:32.577+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:32.578+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:32.578+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:32.578+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:30:33.079+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:33.079+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:33.080+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:33.080+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:30:33.580+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:33.581+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:33.581+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:33.581+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:30:33.581+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:30:33.581+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:30:33.581+0000-56e5252918fa188537623c6e", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857833581), what: "balancer.round", ns: "", details: { executionTimeMillis: 10594, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:30:33.613+0000 I NETWORK [conn4076709] end connection 10.0.216.90:53270 (1506 connections now open) 2016-03-13T08:30:33.613+0000 I NETWORK [conn4076708] end connection 10.0.216.90:53269 (1506 connections now open) 2016-03-13T08:30:33.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53282 #4076724 (1506 connections now open) 2016-03-13T08:30:33.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53283 #4076725 (1507 connections now open) 2016-03-13T08:30:34.387+0000 I NETWORK [conn4076711] end connection 10.0.20.53:46533 (1506 connections now open) 2016-03-13T08:30:34.387+0000 I NETWORK [conn4076710] end connection 10.0.20.53:46534 (1506 connections now open) 2016-03-13T08:30:34.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46549 #4076726 (1506 connections now open) 2016-03-13T08:30:34.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46550 #4076727 (1507 connections now open) 2016-03-13T08:30:36.895+0000 I NETWORK [conn4076713] end connection 10.0.179.209:36407 (1506 connections now open) 2016-03-13T08:30:36.895+0000 I NETWORK [conn4076712] end connection 10.0.179.209:36406 (1506 connections now open) 2016-03-13T08:30:36.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36415 #4076728 (1506 connections now open) 2016-03-13T08:30:36.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36416 #4076729 (1507 connections now open) 2016-03-13T08:30:37.353+0000 I NETWORK [conn4076715] end connection 10.0.26.69:38288 (1506 connections now open) 2016-03-13T08:30:37.353+0000 I NETWORK [conn4076714] end connection 10.0.26.69:38289 (1506 connections now open) 2016-03-13T08:30:37.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38303 #4076730 (1506 connections now open) 2016-03-13T08:30:37.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38304 #4076731 (1507 connections now open) 2016-03-13T08:30:39.466+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:30:39.183+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:30:40.388+0000 I NETWORK [conn4076721] end connection 10.0.27.237:59907 (1506 connections now open) 2016-03-13T08:30:40.388+0000 I NETWORK [conn4076720] end connection 10.0.27.237:59906 (1506 connections now open) 2016-03-13T08:30:40.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59923 #4076732 (1506 connections now open) 2016-03-13T08:30:40.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59922 #4076733 (1507 connections now open) 2016-03-13T08:30:42.409+0000 I NETWORK [conn4076722] end connection 10.0.7.113:55245 (1506 connections now open) 2016-03-13T08:30:42.409+0000 I NETWORK [conn4076723] end connection 10.0.7.113:55246 (1506 connections now open) 2016-03-13T08:30:42.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55256 #4076734 (1506 connections now open) 2016-03-13T08:30:42.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55257 #4076735 (1507 connections now open) 2016-03-13T08:30:42.558+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:42.559+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:42.559+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:30:42.559+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:30:43.626+0000 I NETWORK [conn4076725] end connection 10.0.216.90:53283 (1506 connections now open) 2016-03-13T08:30:43.626+0000 I NETWORK [conn4076724] end connection 10.0.216.90:53282 (1505 connections now open) 2016-03-13T08:30:43.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53296 #4076736 (1506 connections now open) 2016-03-13T08:30:43.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53295 #4076737 (1507 connections now open) 2016-03-13T08:30:43.640+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:43.640+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:43.640+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:43.640+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:30:44.141+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:44.142+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:44.142+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:44.142+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:30:44.400+0000 I NETWORK [conn4076727] end connection 10.0.20.53:46550 (1506 connections now open) 2016-03-13T08:30:44.401+0000 I NETWORK [conn4076726] end connection 10.0.20.53:46549 (1505 connections now open) 2016-03-13T08:30:44.401+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46558 #4076738 (1506 connections now open) 2016-03-13T08:30:44.401+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46559 #4076739 (1507 connections now open) 2016-03-13T08:30:44.643+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:44.644+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:44.644+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:44.644+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:30:45.145+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:45.145+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:45.145+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:45.145+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:30:45.646+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:45.647+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:45.647+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:45.647+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:30:46.118+0000 I NETWORK [conn4065026] end connection 10.0.26.69:53955 (1506 connections now open) 2016-03-13T08:30:46.118+0000 I NETWORK [conn4065994] end connection 10.0.27.237:48842 (1505 connections now open) 2016-03-13T08:30:46.118+0000 I NETWORK [conn4065251] end connection 10.0.7.113:42983 (1504 connections now open) 2016-03-13T08:30:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59929 #4076740 (1505 connections now open) 2016-03-13T08:30:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38322 #4076741 (1506 connections now open) 2016-03-13T08:30:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55264 #4076742 (1507 connections now open) 2016-03-13T08:30:46.148+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:46.148+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:46.148+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:46.148+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:30:46.149+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:30:46.149+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:30:46.149+0000-56e5253618fa188537623c6f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857846149), what: "balancer.round", ns: "", details: { executionTimeMillis: 2532, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:30:46.908+0000 I NETWORK [conn4076729] end connection 10.0.179.209:36416 (1506 connections now open) 2016-03-13T08:30:46.908+0000 I NETWORK [conn4076728] end connection 10.0.179.209:36415 (1506 connections now open) 2016-03-13T08:30:46.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36428 #4076743 (1506 connections now open) 2016-03-13T08:30:46.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36427 #4076744 (1507 connections now open) 2016-03-13T08:30:47.375+0000 I NETWORK [conn4076731] end connection 10.0.26.69:38304 (1506 connections now open) 2016-03-13T08:30:47.375+0000 I NETWORK [conn4076730] end connection 10.0.26.69:38303 (1506 connections now open) 2016-03-13T08:30:47.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38323 #4076745 (1506 connections now open) 2016-03-13T08:30:47.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38324 #4076746 (1507 connections now open) 2016-03-13T08:30:50.401+0000 I NETWORK [conn4076732] end connection 10.0.27.237:59923 (1506 connections now open) 2016-03-13T08:30:50.401+0000 I NETWORK [conn4076733] end connection 10.0.27.237:59922 (1506 connections now open) 2016-03-13T08:30:50.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59930 #4076747 (1506 connections now open) 2016-03-13T08:30:50.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59931 #4076748 (1507 connections now open) 2016-03-13T08:30:52.422+0000 I NETWORK [conn4076734] end connection 10.0.7.113:55256 (1506 connections now open) 2016-03-13T08:30:52.422+0000 I NETWORK [conn4076735] end connection 10.0.7.113:55257 (1506 connections now open) 2016-03-13T08:30:52.432+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55278 #4076749 (1506 connections now open) 2016-03-13T08:30:52.432+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55277 #4076750 (1507 connections now open) 2016-03-13T08:30:52.561+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:30:53.638+0000 I NETWORK [conn4076737] end connection 10.0.216.90:53295 (1506 connections now open) 2016-03-13T08:30:53.639+0000 I NETWORK [conn4076736] end connection 10.0.216.90:53296 (1506 connections now open) 2016-03-13T08:30:53.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53315 #4076751 (1506 connections now open) 2016-03-13T08:30:53.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53314 #4076752 (1507 connections now open) 2016-03-13T08:30:54.414+0000 I NETWORK [conn4076738] end connection 10.0.20.53:46558 (1506 connections now open) 2016-03-13T08:30:54.414+0000 I NETWORK [conn4076739] end connection 10.0.20.53:46559 (1506 connections now open) 2016-03-13T08:30:54.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46578 #4076753 (1506 connections now open) 2016-03-13T08:30:54.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46579 #4076754 (1507 connections now open) 2016-03-13T08:30:56.197+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:30:56.201+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:56.209+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:56.209+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:56.209+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:30:56.710+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:56.711+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:56.711+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:56.711+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:30:56.928+0000 I NETWORK [conn4076743] end connection 10.0.179.209:36428 (1506 connections now open) 2016-03-13T08:30:56.929+0000 I NETWORK [conn4076744] end connection 10.0.179.209:36427 (1505 connections now open) 2016-03-13T08:30:56.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36445 #4076755 (1506 connections now open) 2016-03-13T08:30:56.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36446 #4076756 (1507 connections now open) 2016-03-13T08:30:57.212+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:57.212+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:57.212+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:57.212+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:30:57.388+0000 I NETWORK [conn4076745] end connection 10.0.26.69:38323 (1506 connections now open) 2016-03-13T08:30:57.388+0000 I NETWORK [conn4076746] end connection 10.0.26.69:38324 (1506 connections now open) 2016-03-13T08:30:57.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38332 #4076757 (1506 connections now open) 2016-03-13T08:30:57.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38333 #4076758 (1507 connections now open) 2016-03-13T08:30:57.713+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:57.714+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:57.714+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:57.714+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:30:58.113+0000 I NETWORK [conn4069001] end connection 10.0.179.209:56841 (1506 connections now open) 2016-03-13T08:30:58.113+0000 I NETWORK [conn4064432] end connection 10.0.20.53:33784 (1505 connections now open) 2016-03-13T08:30:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36449 #4076759 (1506 connections now open) 2016-03-13T08:30:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46586 #4076760 (1507 connections now open) 2016-03-13T08:30:58.215+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:58.216+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:58.216+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:58.216+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:30:58.717+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:58.717+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:58.717+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:58.717+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:30:59.218+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:59.219+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:59.219+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:59.219+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:30:59.720+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:30:59.720+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:30:59.720+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:30:59.720+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:31:00.221+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:00.222+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:00.222+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:00.222+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:31:00.414+0000 I NETWORK [conn4076747] end connection 10.0.27.237:59930 (1506 connections now open) 2016-03-13T08:31:00.414+0000 I NETWORK [conn4076748] end connection 10.0.27.237:59931 (1506 connections now open) 2016-03-13T08:31:00.415+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59946 #4076761 (1506 connections now open) 2016-03-13T08:31:00.415+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59945 #4076762 (1507 connections now open) 2016-03-13T08:31:00.723+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:00.723+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:00.724+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:00.724+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:31:01.224+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:01.225+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:01.225+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:01.225+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:31:01.726+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:01.727+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:01.727+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:01.727+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:31:02.228+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:02.228+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:02.228+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:02.228+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:31:02.445+0000 I NETWORK [conn4076749] end connection 10.0.7.113:55278 (1506 connections now open) 2016-03-13T08:31:02.445+0000 I NETWORK [conn4076750] end connection 10.0.7.113:55277 (1506 connections now open) 2016-03-13T08:31:02.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55291 #4076763 (1506 connections now open) 2016-03-13T08:31:02.446+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55292 #4076764 (1507 connections now open) 2016-03-13T08:31:02.563+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:02.564+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:02.564+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:31:02.564+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:31:02.729+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:02.730+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:02.730+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:02.730+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:31:03.231+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:03.231+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:03.231+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:03.231+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:31:03.660+0000 I NETWORK [conn4076752] end connection 10.0.216.90:53314 (1506 connections now open) 2016-03-13T08:31:03.661+0000 I NETWORK [conn4076751] end connection 10.0.216.90:53315 (1505 connections now open) 2016-03-13T08:31:03.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53327 #4076765 (1506 connections now open) 2016-03-13T08:31:03.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53328 #4076766 (1507 connections now open) 2016-03-13T08:31:03.732+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:03.733+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:03.733+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:03.733+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:31:04.234+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:04.234+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:04.234+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:04.234+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:31:04.436+0000 I NETWORK [conn4076753] end connection 10.0.20.53:46578 (1506 connections now open) 2016-03-13T08:31:04.436+0000 I NETWORK [conn4076754] end connection 10.0.20.53:46579 (1506 connections now open) 2016-03-13T08:31:04.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46589 #4076767 (1506 connections now open) 2016-03-13T08:31:04.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46590 #4076768 (1507 connections now open) 2016-03-13T08:31:04.735+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:04.736+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:04.736+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:04.736+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:31:05.237+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:05.238+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:05.238+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:05.238+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:31:05.739+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:05.739+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:05.739+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:05.739+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:31:06.240+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:06.241+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:06.241+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:06.241+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:31:06.742+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:06.743+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:06.743+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:06.743+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:31:06.743+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:31:06.743+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:31:06.743+0000-56e5254a18fa188537623c70", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857866743), what: "balancer.round", ns: "", details: { executionTimeMillis: 10575, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:31:06.941+0000 I NETWORK [conn4076755] end connection 10.0.179.209:36445 (1506 connections now open) 2016-03-13T08:31:06.941+0000 I NETWORK [conn4076756] end connection 10.0.179.209:36446 (1505 connections now open) 2016-03-13T08:31:06.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36456 #4076769 (1506 connections now open) 2016-03-13T08:31:06.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36457 #4076770 (1507 connections now open) 2016-03-13T08:31:07.402+0000 I NETWORK [conn4076758] end connection 10.0.26.69:38333 (1506 connections now open) 2016-03-13T08:31:07.402+0000 I NETWORK [conn4076757] end connection 10.0.26.69:38332 (1506 connections now open) 2016-03-13T08:31:07.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38348 #4076771 (1506 connections now open) 2016-03-13T08:31:07.403+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38349 #4076772 (1507 connections now open) 2016-03-13T08:31:09.753+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:31:09.467+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:31:10.427+0000 I NETWORK [conn4076761] end connection 10.0.27.237:59946 (1506 connections now open) 2016-03-13T08:31:10.427+0000 I NETWORK [conn4076762] end connection 10.0.27.237:59945 (1506 connections now open) 2016-03-13T08:31:10.428+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59964 #4076773 (1506 connections now open) 2016-03-13T08:31:10.428+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59963 #4076774 (1507 connections now open) 2016-03-13T08:31:12.458+0000 I NETWORK [conn4076763] end connection 10.0.7.113:55291 (1506 connections now open) 2016-03-13T08:31:12.459+0000 I NETWORK [conn4076764] end connection 10.0.7.113:55292 (1505 connections now open) 2016-03-13T08:31:12.459+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55305 #4076775 (1506 connections now open) 2016-03-13T08:31:12.459+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55306 #4076776 (1507 connections now open) 2016-03-13T08:31:12.567+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:12.568+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:12.568+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:31:12.568+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:31:13.674+0000 I NETWORK [conn4076766] end connection 10.0.216.90:53328 (1506 connections now open) 2016-03-13T08:31:13.674+0000 I NETWORK [conn4076765] end connection 10.0.216.90:53327 (1506 connections now open) 2016-03-13T08:31:13.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53340 #4076777 (1506 connections now open) 2016-03-13T08:31:13.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53341 #4076778 (1507 connections now open) 2016-03-13T08:31:14.449+0000 I NETWORK [conn4076767] end connection 10.0.20.53:46589 (1506 connections now open) 2016-03-13T08:31:14.449+0000 I NETWORK [conn4076768] end connection 10.0.20.53:46590 (1506 connections now open) 2016-03-13T08:31:14.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46602 #4076779 (1506 connections now open) 2016-03-13T08:31:14.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46603 #4076780 (1507 connections now open) 2016-03-13T08:31:16.118+0000 I NETWORK [conn4071182] end connection 10.0.179.209:59027 (1506 connections now open) 2016-03-13T08:31:16.118+0000 I NETWORK [conn4067775] end connection 10.0.216.90:43668 (1505 connections now open) 2016-03-13T08:31:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36467 #4076781 (1506 connections now open) 2016-03-13T08:31:16.118+0000 I NETWORK [conn4068434] end connection 10.0.27.237:51307 (1504 connections now open) 2016-03-13T08:31:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53345 #4076782 (1506 connections now open) 2016-03-13T08:31:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59973 #4076783 (1507 connections now open) 2016-03-13T08:31:16.786+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:16.786+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:16.786+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:16.786+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:31:16.954+0000 I NETWORK [conn4076770] end connection 10.0.179.209:36457 (1506 connections now open) 2016-03-13T08:31:16.954+0000 I NETWORK [conn4076769] end connection 10.0.179.209:36456 (1506 connections now open) 2016-03-13T08:31:16.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36469 #4076784 (1506 connections now open) 2016-03-13T08:31:16.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36468 #4076785 (1507 connections now open) 2016-03-13T08:31:17.287+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:17.288+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:17.288+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:17.288+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:31:17.415+0000 I NETWORK [conn4076772] end connection 10.0.26.69:38349 (1506 connections now open) 2016-03-13T08:31:17.415+0000 I NETWORK [conn4076771] end connection 10.0.26.69:38348 (1506 connections now open) 2016-03-13T08:31:17.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38370 #4076786 (1506 connections now open) 2016-03-13T08:31:17.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38371 #4076787 (1507 connections now open) 2016-03-13T08:31:17.789+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:17.789+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:17.789+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:17.789+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:31:18.290+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:18.291+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:18.291+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:18.291+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:31:18.792+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:18.793+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:18.793+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:18.793+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:31:19.294+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:19.294+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:19.294+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:19.294+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:31:19.294+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:31:19.294+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:31:19.294+0000-56e5255718fa188537623c71", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857879294), what: "balancer.round", ns: "", details: { executionTimeMillis: 2517, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:31:20.441+0000 I NETWORK [conn4076773] end connection 10.0.27.237:59964 (1506 connections now open) 2016-03-13T08:31:20.441+0000 I NETWORK [conn4076774] end connection 10.0.27.237:59963 (1506 connections now open) 2016-03-13T08:31:20.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59976 #4076788 (1506 connections now open) 2016-03-13T08:31:20.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59975 #4076789 (1507 connections now open) 2016-03-13T08:31:22.471+0000 I NETWORK [conn4076775] end connection 10.0.7.113:55305 (1506 connections now open) 2016-03-13T08:31:22.471+0000 I NETWORK [conn4076776] end connection 10.0.7.113:55306 (1505 connections now open) 2016-03-13T08:31:22.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55325 #4076790 (1506 connections now open) 2016-03-13T08:31:22.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55326 #4076791 (1507 connections now open) 2016-03-13T08:31:22.570+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:31:23.687+0000 I NETWORK [conn4076777] end connection 10.0.216.90:53340 (1506 connections now open) 2016-03-13T08:31:23.687+0000 I NETWORK [conn4076778] end connection 10.0.216.90:53341 (1506 connections now open) 2016-03-13T08:31:23.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53361 #4076792 (1506 connections now open) 2016-03-13T08:31:23.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53362 #4076793 (1507 connections now open) 2016-03-13T08:31:24.463+0000 I NETWORK [conn4076780] end connection 10.0.20.53:46603 (1506 connections now open) 2016-03-13T08:31:24.463+0000 I NETWORK [conn4076779] end connection 10.0.20.53:46602 (1506 connections now open) 2016-03-13T08:31:24.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46624 #4076794 (1506 connections now open) 2016-03-13T08:31:24.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46623 #4076795 (1507 connections now open) 2016-03-13T08:31:26.967+0000 I NETWORK [conn4076785] end connection 10.0.179.209:36468 (1506 connections now open) 2016-03-13T08:31:26.967+0000 I NETWORK [conn4076784] end connection 10.0.179.209:36469 (1506 connections now open) 2016-03-13T08:31:26.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36490 #4076796 (1506 connections now open) 2016-03-13T08:31:26.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36489 #4076797 (1507 connections now open) 2016-03-13T08:31:27.429+0000 I NETWORK [conn4076787] end connection 10.0.26.69:38371 (1506 connections now open) 2016-03-13T08:31:27.429+0000 I NETWORK [conn4076786] end connection 10.0.26.69:38370 (1506 connections now open) 2016-03-13T08:31:27.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38377 #4076798 (1506 connections now open) 2016-03-13T08:31:27.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38378 #4076799 (1507 connections now open) 2016-03-13T08:31:28.113+0000 I NETWORK [conn4066076] end connection 10.0.216.90:41858 (1506 connections now open) 2016-03-13T08:31:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53369 #4076800 (1507 connections now open) 2016-03-13T08:31:28.114+0000 I NETWORK [conn4068240] end connection 10.0.27.237:51103 (1506 connections now open) 2016-03-13T08:31:28.114+0000 I NETWORK [conn4067924] end connection 10.0.7.113:45833 (1506 connections now open) 2016-03-13T08:31:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59987 #4076801 (1506 connections now open) 2016-03-13T08:31:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55332 #4076802 (1507 connections now open) 2016-03-13T08:31:29.351+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:31:29.354+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:29.362+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:29.362+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:29.362+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:31:29.863+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:29.864+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:29.864+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:29.864+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:31:30.368+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:30.368+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:30.368+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:30.368+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:31:30.455+0000 I NETWORK [conn4076789] end connection 10.0.27.237:59975 (1506 connections now open) 2016-03-13T08:31:30.455+0000 I NETWORK [conn4076788] end connection 10.0.27.237:59976 (1506 connections now open) 2016-03-13T08:31:30.455+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59990 #4076803 (1506 connections now open) 2016-03-13T08:31:30.455+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:59989 #4076804 (1507 connections now open) 2016-03-13T08:31:30.869+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:30.870+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:30.870+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:30.870+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:31:31.371+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:31.372+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:31.372+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:31.372+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:31:31.872+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:31.873+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:31.873+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:31.873+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:31:32.374+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:32.375+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:32.375+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:32.375+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:31:32.484+0000 I NETWORK [conn4076790] end connection 10.0.7.113:55325 (1506 connections now open) 2016-03-13T08:31:32.484+0000 I NETWORK [conn4076791] end connection 10.0.7.113:55326 (1506 connections now open) 2016-03-13T08:31:32.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55335 #4076805 (1506 connections now open) 2016-03-13T08:31:32.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55334 #4076806 (1507 connections now open) 2016-03-13T08:31:32.573+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:32.573+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:32.573+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:31:32.573+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:31:32.876+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:32.876+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:32.876+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:32.876+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:31:33.377+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:33.378+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:33.378+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:33.378+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:31:33.700+0000 I NETWORK [conn4076792] end connection 10.0.216.90:53361 (1506 connections now open) 2016-03-13T08:31:33.700+0000 I NETWORK [conn4076793] end connection 10.0.216.90:53362 (1506 connections now open) 2016-03-13T08:31:33.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53376 #4076807 (1506 connections now open) 2016-03-13T08:31:33.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53375 #4076808 (1507 connections now open) 2016-03-13T08:31:33.879+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:33.880+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:33.880+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:33.880+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:31:34.380+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:34.381+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:34.381+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:34.381+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:31:34.477+0000 I NETWORK [conn4076794] end connection 10.0.20.53:46624 (1506 connections now open) 2016-03-13T08:31:34.477+0000 I NETWORK [conn4076795] end connection 10.0.20.53:46623 (1506 connections now open) 2016-03-13T08:31:34.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46635 #4076809 (1506 connections now open) 2016-03-13T08:31:34.478+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46636 #4076810 (1507 connections now open) 2016-03-13T08:31:34.882+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:34.883+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:34.883+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:34.883+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:31:35.384+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:35.384+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:35.384+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:35.384+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:31:35.885+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:35.886+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:35.886+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:35.886+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:31:36.387+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:36.387+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:36.387+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:36.387+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:31:36.888+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:36.889+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:36.889+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:36.889+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:31:36.980+0000 I NETWORK [conn4076796] end connection 10.0.179.209:36490 (1506 connections now open) 2016-03-13T08:31:36.980+0000 I NETWORK [conn4076797] end connection 10.0.179.209:36489 (1506 connections now open) 2016-03-13T08:31:36.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36499 #4076811 (1506 connections now open) 2016-03-13T08:31:36.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36500 #4076812 (1507 connections now open) 2016-03-13T08:31:37.390+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:37.391+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:37.391+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:37.391+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:31:37.442+0000 I NETWORK [conn4076798] end connection 10.0.26.69:38377 (1506 connections now open) 2016-03-13T08:31:37.442+0000 I NETWORK [conn4076799] end connection 10.0.26.69:38378 (1506 connections now open) 2016-03-13T08:31:37.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38394 #4076813 (1506 connections now open) 2016-03-13T08:31:37.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38395 #4076814 (1507 connections now open) 2016-03-13T08:31:37.891+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:37.892+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:37.892+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:37.892+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:31:38.393+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:38.394+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:38.394+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:38.394+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:31:38.895+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:38.895+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:38.895+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:38.895+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:31:39.396+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:39.401+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:39.401+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:39.401+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:31:39.902+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:39.902+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:39.902+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:39.902+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:31:39.902+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:31:39.902+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:31:39.902+0000-56e5256b18fa188537623c72", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857899902), what: "balancer.round", ns: "", details: { executionTimeMillis: 10588, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:31:39.935+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:31:39.753+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:31:40.468+0000 I NETWORK [conn4076804] end connection 10.0.27.237:59989 (1506 connections now open) 2016-03-13T08:31:40.468+0000 I NETWORK [conn4076803] end connection 10.0.27.237:59990 (1506 connections now open) 2016-03-13T08:31:40.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60008 #4076815 (1506 connections now open) 2016-03-13T08:31:40.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60007 #4076816 (1507 connections now open) 2016-03-13T08:31:42.497+0000 I NETWORK [conn4076805] end connection 10.0.7.113:55335 (1506 connections now open) 2016-03-13T08:31:42.497+0000 I NETWORK [conn4076806] end connection 10.0.7.113:55334 (1506 connections now open) 2016-03-13T08:31:42.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55346 #4076817 (1506 connections now open) 2016-03-13T08:31:42.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55347 #4076818 (1507 connections now open) 2016-03-13T08:31:42.576+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:42.577+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:42.577+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:31:42.577+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:31:43.713+0000 I NETWORK [conn4076808] end connection 10.0.216.90:53375 (1506 connections now open) 2016-03-13T08:31:43.713+0000 I NETWORK [conn4076807] end connection 10.0.216.90:53376 (1506 connections now open) 2016-03-13T08:31:43.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53389 #4076819 (1506 connections now open) 2016-03-13T08:31:43.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53388 #4076820 (1507 connections now open) 2016-03-13T08:31:44.490+0000 I NETWORK [conn4076809] end connection 10.0.20.53:46635 (1506 connections now open) 2016-03-13T08:31:44.490+0000 I NETWORK [conn4076810] end connection 10.0.20.53:46636 (1506 connections now open) 2016-03-13T08:31:44.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46647 #4076821 (1507 connections now open) 2016-03-13T08:31:44.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46648 #4076822 (1507 connections now open) 2016-03-13T08:31:46.118+0000 I NETWORK [conn4069042] end connection 10.0.20.53:38479 (1506 connections now open) 2016-03-13T08:31:46.118+0000 I NETWORK [conn4069324] end connection 10.0.26.69:58568 (1505 connections now open) 2016-03-13T08:31:46.118+0000 I NETWORK [conn4067258] end connection 10.0.27.237:50107 (1504 connections now open) 2016-03-13T08:31:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46660 #4076823 (1505 connections now open) 2016-03-13T08:31:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38411 #4076824 (1506 connections now open) 2016-03-13T08:31:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60015 #4076825 (1507 connections now open) 2016-03-13T08:31:46.993+0000 I NETWORK [conn4076812] end connection 10.0.179.209:36500 (1506 connections now open) 2016-03-13T08:31:46.993+0000 I NETWORK [conn4076811] end connection 10.0.179.209:36499 (1505 connections now open) 2016-03-13T08:31:47.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36511 #4076826 (1506 connections now open) 2016-03-13T08:31:47.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36512 #4076827 (1507 connections now open) 2016-03-13T08:31:47.463+0000 I NETWORK [conn4076814] end connection 10.0.26.69:38395 (1506 connections now open) 2016-03-13T08:31:47.463+0000 I NETWORK [conn4076813] end connection 10.0.26.69:38394 (1506 connections now open) 2016-03-13T08:31:47.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38413 #4076828 (1506 connections now open) 2016-03-13T08:31:47.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38414 #4076829 (1507 connections now open) 2016-03-13T08:31:49.973+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:49.973+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:49.973+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:49.973+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:31:50.474+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:50.475+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:50.475+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:50.475+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:31:50.490+0000 I NETWORK [conn4076816] end connection 10.0.27.237:60007 (1506 connections now open) 2016-03-13T08:31:50.490+0000 I NETWORK [conn4076815] end connection 10.0.27.237:60008 (1506 connections now open) 2016-03-13T08:31:50.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60017 #4076830 (1506 connections now open) 2016-03-13T08:31:50.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60016 #4076831 (1507 connections now open) 2016-03-13T08:31:50.976+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:50.977+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:50.977+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:50.977+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:31:51.478+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:51.478+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:51.478+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:51.478+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:31:51.979+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:51.980+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:51.980+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:51.980+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:31:52.481+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:31:52.481+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:31:52.481+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:31:52.481+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:31:52.482+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:31:52.482+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:31:52.482+0000-56e5257818fa188537623c73", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857912482), what: "balancer.round", ns: "", details: { executionTimeMillis: 2545, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:31:52.510+0000 I NETWORK [conn4076817] end connection 10.0.7.113:55346 (1506 connections now open) 2016-03-13T08:31:52.510+0000 I NETWORK [conn4076818] end connection 10.0.7.113:55347 (1506 connections now open) 2016-03-13T08:31:52.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55368 #4076832 (1506 connections now open) 2016-03-13T08:31:52.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55367 #4076833 (1507 connections now open) 2016-03-13T08:31:52.579+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:31:53.726+0000 I NETWORK [conn4076820] end connection 10.0.216.90:53388 (1506 connections now open) 2016-03-13T08:31:53.726+0000 I NETWORK [conn4076819] end connection 10.0.216.90:53389 (1505 connections now open) 2016-03-13T08:31:53.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53408 #4076834 (1506 connections now open) 2016-03-13T08:31:53.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53409 #4076835 (1507 connections now open) 2016-03-13T08:31:54.504+0000 I NETWORK [conn4076822] end connection 10.0.20.53:46648 (1506 connections now open) 2016-03-13T08:31:54.504+0000 I NETWORK [conn4076821] end connection 10.0.20.53:46647 (1506 connections now open) 2016-03-13T08:31:54.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46666 #4076836 (1506 connections now open) 2016-03-13T08:31:54.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46667 #4076837 (1507 connections now open) 2016-03-13T08:31:57.014+0000 I NETWORK [conn4076826] end connection 10.0.179.209:36511 (1506 connections now open) 2016-03-13T08:31:57.014+0000 I NETWORK [conn4076827] end connection 10.0.179.209:36512 (1506 connections now open) 2016-03-13T08:31:57.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36529 #4076838 (1506 connections now open) 2016-03-13T08:31:57.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36530 #4076839 (1507 connections now open) 2016-03-13T08:31:57.476+0000 I NETWORK [conn4076828] end connection 10.0.26.69:38413 (1506 connections now open) 2016-03-13T08:31:57.476+0000 I NETWORK [conn4076829] end connection 10.0.26.69:38414 (1506 connections now open) 2016-03-13T08:31:57.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38422 #4076840 (1506 connections now open) 2016-03-13T08:31:57.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38421 #4076841 (1507 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [conn4066676] end connection 10.0.179.209:54447 (1506 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [conn4066159] end connection 10.0.216.90:41951 (1506 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [conn4066933] end connection 10.0.26.69:56022 (1506 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [conn4070309] end connection 10.0.27.237:53301 (1503 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53416 #4076842 (1504 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36532 #4076843 (1505 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38426 #4076844 (1506 connections now open) 2016-03-13T08:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60029 #4076845 (1507 connections now open) 2016-03-13T08:32:00.503+0000 I NETWORK [conn4076831] end connection 10.0.27.237:60016 (1506 connections now open) 2016-03-13T08:32:00.503+0000 I NETWORK [conn4076830] end connection 10.0.27.237:60017 (1505 connections now open) 2016-03-13T08:32:00.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60031 #4076846 (1506 connections now open) 2016-03-13T08:32:00.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60032 #4076847 (1507 connections now open) 2016-03-13T08:32:02.532+0000 I NETWORK [conn4076833] end connection 10.0.7.113:55367 (1506 connections now open) 2016-03-13T08:32:02.532+0000 I NETWORK [conn4076832] end connection 10.0.7.113:55368 (1505 connections now open) 2016-03-13T08:32:02.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55379 #4076848 (1506 connections now open) 2016-03-13T08:32:02.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55380 #4076849 (1507 connections now open) 2016-03-13T08:32:02.561+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:32:02.563+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:02.573+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:02.573+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:02.573+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:32:02.582+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:02.582+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:02.582+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:32:02.582+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:32:03.073+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:03.074+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:03.074+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:03.074+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:32:03.575+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:03.576+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:03.576+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:03.576+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:32:03.747+0000 I NETWORK [conn4076835] end connection 10.0.216.90:53409 (1506 connections now open) 2016-03-13T08:32:03.747+0000 I NETWORK [conn4076834] end connection 10.0.216.90:53408 (1506 connections now open) 2016-03-13T08:32:03.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53423 #4076850 (1506 connections now open) 2016-03-13T08:32:03.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53422 #4076851 (1507 connections now open) 2016-03-13T08:32:04.076+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:04.077+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:04.077+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:04.077+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:32:04.518+0000 I NETWORK [conn4076836] end connection 10.0.20.53:46666 (1506 connections now open) 2016-03-13T08:32:04.518+0000 I NETWORK [conn4076837] end connection 10.0.20.53:46667 (1506 connections now open) 2016-03-13T08:32:04.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46676 #4076852 (1506 connections now open) 2016-03-13T08:32:04.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46677 #4076853 (1507 connections now open) 2016-03-13T08:32:04.578+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:04.579+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:04.579+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:04.579+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:32:05.080+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:05.080+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:05.080+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:05.080+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:32:05.581+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:05.582+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:05.582+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:05.582+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:32:06.083+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:06.083+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:06.083+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:06.083+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:32:06.584+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:06.585+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:06.585+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:06.585+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:32:07.027+0000 I NETWORK [conn4076839] end connection 10.0.179.209:36530 (1506 connections now open) 2016-03-13T08:32:07.027+0000 I NETWORK [conn4076838] end connection 10.0.179.209:36529 (1506 connections now open) 2016-03-13T08:32:07.028+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36540 #4076854 (1506 connections now open) 2016-03-13T08:32:07.028+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36541 #4076855 (1507 connections now open) 2016-03-13T08:32:07.086+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:07.087+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:07.087+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:07.087+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:32:07.490+0000 I NETWORK [conn4076841] end connection 10.0.26.69:38421 (1506 connections now open) 2016-03-13T08:32:07.490+0000 I NETWORK [conn4076840] end connection 10.0.26.69:38422 (1506 connections now open) 2016-03-13T08:32:07.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38439 #4076856 (1506 connections now open) 2016-03-13T08:32:07.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38440 #4076857 (1507 connections now open) 2016-03-13T08:32:07.587+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:07.588+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:07.588+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:07.588+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:32:08.089+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:08.090+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:08.090+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:08.090+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:32:08.591+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:08.592+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:08.592+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:08.592+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:32:09.092+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:09.093+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:09.093+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:09.093+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:32:09.594+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:09.595+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:09.595+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:09.595+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:32:10.096+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:10.096+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:10.097+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:10.097+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:32:10.130+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:32:09.935+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:32:10.516+0000 I NETWORK [conn4076846] end connection 10.0.27.237:60031 (1506 connections now open) 2016-03-13T08:32:10.516+0000 I NETWORK [conn4076847] end connection 10.0.27.237:60032 (1505 connections now open) 2016-03-13T08:32:10.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60049 #4076858 (1506 connections now open) 2016-03-13T08:32:10.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60050 #4076859 (1507 connections now open) 2016-03-13T08:32:10.597+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:10.598+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:10.598+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:10.598+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:32:11.099+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:11.100+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:11.100+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:11.100+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:32:11.601+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:11.601+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:11.601+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:11.601+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:32:12.102+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:12.103+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:12.103+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:12.103+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:32:12.545+0000 I NETWORK [conn4076848] end connection 10.0.7.113:55379 (1506 connections now open) 2016-03-13T08:32:12.545+0000 I NETWORK [conn4076849] end connection 10.0.7.113:55380 (1505 connections now open) 2016-03-13T08:32:12.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55394 #4076860 (1506 connections now open) 2016-03-13T08:32:12.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55393 #4076861 (1507 connections now open) 2016-03-13T08:32:12.585+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:12.586+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:12.586+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:32:12.586+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:32:12.604+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:12.604+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:12.605+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:12.605+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:32:13.106+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:13.106+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:13.106+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:13.106+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:32:13.106+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:32:13.106+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:32:13.106+0000-56e5258d18fa188537623c74", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857933106), what: "balancer.round", ns: "", details: { executionTimeMillis: 10589, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:32:13.760+0000 I NETWORK [conn4076850] end connection 10.0.216.90:53423 (1506 connections now open) 2016-03-13T08:32:13.760+0000 I NETWORK [conn4076851] end connection 10.0.216.90:53422 (1506 connections now open) 2016-03-13T08:32:13.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53431 #4076862 (1506 connections now open) 2016-03-13T08:32:13.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53432 #4076863 (1507 connections now open) 2016-03-13T08:32:14.528+0000 I NETWORK [conn4068023] end connection 10.0.53.235:35994 (1506 connections now open) 2016-03-13T08:32:14.532+0000 I NETWORK [conn4076853] end connection 10.0.20.53:46677 (1505 connections now open) 2016-03-13T08:32:14.532+0000 I NETWORK [conn4076852] end connection 10.0.20.53:46676 (1505 connections now open) 2016-03-13T08:32:14.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46688 #4076864 (1505 connections now open) 2016-03-13T08:32:14.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46687 #4076865 (1506 connections now open) 2016-03-13T08:32:14.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.53.235:40900 #4076866 (1507 connections now open) 2016-03-13T08:32:16.117+0000 I NETWORK [conn4066955] end connection 10.0.179.209:54734 (1506 connections now open) 2016-03-13T08:32:16.117+0000 I NETWORK [conn4069198] end connection 10.0.20.53:38638 (1506 connections now open) 2016-03-13T08:32:16.117+0000 I NETWORK [conn4073976] end connection 10.0.216.90:50310 (1506 connections now open) 2016-03-13T08:32:16.117+0000 I NETWORK [conn4068263] end connection 10.0.26.69:57430 (1503 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [conn4066956] end connection 10.0.27.237:49811 (1502 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [conn4066134] end connection 10.0.7.113:43937 (1502 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46701 #4076867 (1502 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53439 #4076868 (1503 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36551 #4076869 (1504 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38460 #4076870 (1505 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60059 #4076871 (1506 connections now open) 2016-03-13T08:32:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55400 #4076872 (1507 connections now open) 2016-03-13T08:32:17.041+0000 I NETWORK [conn4076854] end connection 10.0.179.209:36540 (1506 connections now open) 2016-03-13T08:32:17.041+0000 I NETWORK [conn4076855] end connection 10.0.179.209:36541 (1506 connections now open) 2016-03-13T08:32:17.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36554 #4076873 (1506 connections now open) 2016-03-13T08:32:17.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36553 #4076874 (1507 connections now open) 2016-03-13T08:32:17.503+0000 I NETWORK [conn4076857] end connection 10.0.26.69:38440 (1506 connections now open) 2016-03-13T08:32:17.503+0000 I NETWORK [conn4076856] end connection 10.0.26.69:38439 (1506 connections now open) 2016-03-13T08:32:17.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38462 #4076875 (1506 connections now open) 2016-03-13T08:32:17.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38463 #4076876 (1507 connections now open) 2016-03-13T08:32:20.529+0000 I NETWORK [conn4076859] end connection 10.0.27.237:60050 (1506 connections now open) 2016-03-13T08:32:20.529+0000 I NETWORK [conn4076858] end connection 10.0.27.237:60049 (1506 connections now open) 2016-03-13T08:32:20.530+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60061 #4076877 (1506 connections now open) 2016-03-13T08:32:20.530+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60060 #4076878 (1507 connections now open) 2016-03-13T08:32:22.558+0000 I NETWORK [conn4076861] end connection 10.0.7.113:55393 (1506 connections now open) 2016-03-13T08:32:22.558+0000 I NETWORK [conn4076860] end connection 10.0.7.113:55394 (1506 connections now open) 2016-03-13T08:32:22.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55413 #4076879 (1506 connections now open) 2016-03-13T08:32:22.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55414 #4076880 (1507 connections now open) 2016-03-13T08:32:22.589+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:22.590+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:22.590+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:32:22.590+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:32:23.163+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:23.164+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:23.164+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:23.164+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:32:23.665+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:23.666+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:23.666+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:23.666+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:32:23.773+0000 I NETWORK [conn4076863] end connection 10.0.216.90:53432 (1506 connections now open) 2016-03-13T08:32:23.773+0000 I NETWORK [conn4076862] end connection 10.0.216.90:53431 (1506 connections now open) 2016-03-13T08:32:23.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53445 #4076881 (1506 connections now open) 2016-03-13T08:32:23.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53446 #4076882 (1507 connections now open) 2016-03-13T08:32:24.166+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:24.167+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:24.167+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:24.167+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:32:24.545+0000 I NETWORK [conn4076864] end connection 10.0.20.53:46688 (1506 connections now open) 2016-03-13T08:32:24.546+0000 I NETWORK [conn4076865] end connection 10.0.20.53:46687 (1505 connections now open) 2016-03-13T08:32:24.546+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46707 #4076883 (1506 connections now open) 2016-03-13T08:32:24.546+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46708 #4076884 (1507 connections now open) 2016-03-13T08:32:24.668+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:24.669+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:24.669+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:24.669+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:32:25.170+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:25.170+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:25.170+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:25.170+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:32:25.170+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:32:25.170+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:32:25.170+0000-56e5259918fa188537623c75", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857945170), what: "balancer.round", ns: "", details: { executionTimeMillis: 2036, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:32:27.054+0000 I NETWORK [conn4076873] end connection 10.0.179.209:36554 (1506 connections now open) 2016-03-13T08:32:27.054+0000 I NETWORK [conn4076874] end connection 10.0.179.209:36553 (1506 connections now open) 2016-03-13T08:32:27.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36574 #4076885 (1506 connections now open) 2016-03-13T08:32:27.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36575 #4076886 (1507 connections now open) 2016-03-13T08:32:27.516+0000 I NETWORK [conn4076876] end connection 10.0.26.69:38463 (1506 connections now open) 2016-03-13T08:32:27.516+0000 I NETWORK [conn4076875] end connection 10.0.26.69:38462 (1506 connections now open) 2016-03-13T08:32:27.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38469 #4076887 (1506 connections now open) 2016-03-13T08:32:27.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38470 #4076888 (1507 connections now open) 2016-03-13T08:32:28.113+0000 I NETWORK [conn4065909] end connection 10.0.216.90:41675 (1506 connections now open) 2016-03-13T08:32:28.113+0000 I NETWORK [conn4069259] end connection 10.0.26.69:58497 (1506 connections now open) 2016-03-13T08:32:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53461 #4076889 (1506 connections now open) 2016-03-13T08:32:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38473 #4076890 (1507 connections now open) 2016-03-13T08:32:30.542+0000 I NETWORK [conn4076877] end connection 10.0.27.237:60061 (1506 connections now open) 2016-03-13T08:32:30.542+0000 I NETWORK [conn4076878] end connection 10.0.27.237:60060 (1506 connections now open) 2016-03-13T08:32:30.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60076 #4076891 (1506 connections now open) 2016-03-13T08:32:30.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60075 #4076892 (1507 connections now open) 2016-03-13T08:32:32.571+0000 I NETWORK [conn4076879] end connection 10.0.7.113:55413 (1506 connections now open) 2016-03-13T08:32:32.571+0000 I NETWORK [conn4076880] end connection 10.0.7.113:55414 (1506 connections now open) 2016-03-13T08:32:32.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55424 #4076893 (1506 connections now open) 2016-03-13T08:32:32.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55423 #4076894 (1507 connections now open) 2016-03-13T08:32:32.592+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:32:33.786+0000 I NETWORK [conn4076881] end connection 10.0.216.90:53445 (1506 connections now open) 2016-03-13T08:32:33.786+0000 I NETWORK [conn4076882] end connection 10.0.216.90:53446 (1506 connections now open) 2016-03-13T08:32:33.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53467 #4076895 (1506 connections now open) 2016-03-13T08:32:33.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53468 #4076896 (1507 connections now open) 2016-03-13T08:32:34.559+0000 I NETWORK [conn4076883] end connection 10.0.20.53:46707 (1506 connections now open) 2016-03-13T08:32:34.559+0000 I NETWORK [conn4076884] end connection 10.0.20.53:46708 (1506 connections now open) 2016-03-13T08:32:34.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46719 #4076897 (1506 connections now open) 2016-03-13T08:32:34.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46720 #4076898 (1507 connections now open) 2016-03-13T08:32:35.211+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:32:35.213+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:35.223+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:35.223+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:35.223+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:32:35.724+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:35.724+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:35.724+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:35.724+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:32:36.225+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:36.226+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:36.226+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:36.226+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:32:36.727+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:36.727+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:36.727+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:36.727+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:32:37.067+0000 I NETWORK [conn4076885] end connection 10.0.179.209:36574 (1506 connections now open) 2016-03-13T08:32:37.067+0000 I NETWORK [conn4076886] end connection 10.0.179.209:36575 (1506 connections now open) 2016-03-13T08:32:37.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36584 #4076899 (1506 connections now open) 2016-03-13T08:32:37.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36583 #4076900 (1507 connections now open) 2016-03-13T08:32:37.228+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:37.229+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:37.229+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:37.229+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:32:37.529+0000 I NETWORK [conn4076888] end connection 10.0.26.69:38470 (1506 connections now open) 2016-03-13T08:32:37.529+0000 I NETWORK [conn4076887] end connection 10.0.26.69:38469 (1506 connections now open) 2016-03-13T08:32:37.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38483 #4076901 (1506 connections now open) 2016-03-13T08:32:37.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38484 #4076902 (1507 connections now open) 2016-03-13T08:32:37.730+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:37.730+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:37.731+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:37.731+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:32:38.231+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:38.232+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:38.232+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:38.232+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:32:38.733+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:38.734+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:38.734+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:38.734+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:32:39.235+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:39.235+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:39.235+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:39.235+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:32:39.736+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:39.737+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:39.737+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:39.737+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:32:40.238+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:40.238+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:40.238+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:40.238+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:32:40.426+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:32:40.130+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:32:40.556+0000 I NETWORK [conn4076892] end connection 10.0.27.237:60075 (1506 connections now open) 2016-03-13T08:32:40.556+0000 I NETWORK [conn4076891] end connection 10.0.27.237:60076 (1506 connections now open) 2016-03-13T08:32:40.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60092 #4076903 (1506 connections now open) 2016-03-13T08:32:40.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60093 #4076904 (1507 connections now open) 2016-03-13T08:32:40.739+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:40.740+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:40.740+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:40.740+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:32:41.241+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:41.241+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:41.241+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:41.241+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:32:41.742+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:41.743+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:41.743+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:41.743+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:32:42.244+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:42.244+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:42.245+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:42.245+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:32:42.585+0000 I NETWORK [conn4076893] end connection 10.0.7.113:55424 (1506 connections now open) 2016-03-13T08:32:42.585+0000 I NETWORK [conn4076894] end connection 10.0.7.113:55423 (1506 connections now open) 2016-03-13T08:32:42.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55434 #4076905 (1506 connections now open) 2016-03-13T08:32:42.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55433 #4076906 (1507 connections now open) 2016-03-13T08:32:42.595+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:42.595+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:42.595+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:32:42.595+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:32:42.745+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:42.746+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:42.746+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:42.746+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:32:43.247+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:43.248+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:43.248+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:43.248+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:32:43.749+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:43.750+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:43.750+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:43.750+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:32:43.799+0000 I NETWORK [conn4076896] end connection 10.0.216.90:53468 (1506 connections now open) 2016-03-13T08:32:43.799+0000 I NETWORK [conn4076895] end connection 10.0.216.90:53467 (1506 connections now open) 2016-03-13T08:32:43.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53477 #4076907 (1506 connections now open) 2016-03-13T08:32:43.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53476 #4076908 (1507 connections now open) 2016-03-13T08:32:44.250+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:44.251+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:44.251+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:44.251+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:32:44.572+0000 I NETWORK [conn4076897] end connection 10.0.20.53:46719 (1506 connections now open) 2016-03-13T08:32:44.572+0000 I NETWORK [conn4076898] end connection 10.0.20.53:46720 (1506 connections now open) 2016-03-13T08:32:44.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46731 #4076909 (1506 connections now open) 2016-03-13T08:32:44.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46730 #4076910 (1507 connections now open) 2016-03-13T08:32:44.752+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:44.753+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:44.753+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:44.753+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:32:45.254+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:45.254+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:45.254+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:45.254+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:32:45.755+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:45.756+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:45.756+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:45.756+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:32:45.756+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:32:45.756+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:32:45.756+0000-56e525ad18fa188537623c76", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857965756), what: "balancer.round", ns: "", details: { executionTimeMillis: 10567, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:32:46.117+0000 I NETWORK [conn4072492] end connection 10.0.20.53:42063 (1506 connections now open) 2016-03-13T08:32:46.117+0000 I NETWORK [conn4067817] end connection 10.0.216.90:43710 (1506 connections now open) 2016-03-13T08:32:46.117+0000 I NETWORK [conn4069414] end connection 10.0.26.69:58658 (1504 connections now open) 2016-03-13T08:32:46.117+0000 I NETWORK [conn4062772] end connection 10.0.179.209:50235 (1503 connections now open) 2016-03-13T08:32:46.117+0000 I NETWORK [conn4067692] end connection 10.0.7.113:45581 (1502 connections now open) 2016-03-13T08:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53484 #4076911 (1503 connections now open) 2016-03-13T08:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46745 #4076912 (1504 connections now open) 2016-03-13T08:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38501 #4076913 (1505 connections now open) 2016-03-13T08:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36595 #4076914 (1506 connections now open) 2016-03-13T08:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55441 #4076915 (1507 connections now open) 2016-03-13T08:32:47.080+0000 I NETWORK [conn4076900] end connection 10.0.179.209:36583 (1506 connections now open) 2016-03-13T08:32:47.080+0000 I NETWORK [conn4076899] end connection 10.0.179.209:36584 (1506 connections now open) 2016-03-13T08:32:47.089+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36597 #4076916 (1506 connections now open) 2016-03-13T08:32:47.089+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36598 #4076917 (1507 connections now open) 2016-03-13T08:32:47.551+0000 I NETWORK [conn4076901] end connection 10.0.26.69:38483 (1506 connections now open) 2016-03-13T08:32:47.551+0000 I NETWORK [conn4076902] end connection 10.0.26.69:38484 (1506 connections now open) 2016-03-13T08:32:47.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38504 #4076918 (1506 connections now open) 2016-03-13T08:32:47.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38503 #4076919 (1507 connections now open) 2016-03-13T08:32:50.578+0000 I NETWORK [conn4076904] end connection 10.0.27.237:60093 (1506 connections now open) 2016-03-13T08:32:50.578+0000 I NETWORK [conn4076903] end connection 10.0.27.237:60092 (1506 connections now open) 2016-03-13T08:32:50.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60101 #4076920 (1506 connections now open) 2016-03-13T08:32:50.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60102 #4076921 (1507 connections now open) 2016-03-13T08:32:52.598+0000 I NETWORK [conn4076906] end connection 10.0.7.113:55433 (1506 connections now open) 2016-03-13T08:32:52.598+0000 I NETWORK [conn4076905] end connection 10.0.7.113:55434 (1506 connections now open) 2016-03-13T08:32:52.598+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:52.599+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:52.599+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:32:52.599+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:32:52.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55454 #4076922 (1506 connections now open) 2016-03-13T08:32:52.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55455 #4076923 (1507 connections now open) 2016-03-13T08:32:53.812+0000 I NETWORK [conn4076907] end connection 10.0.216.90:53477 (1506 connections now open) 2016-03-13T08:32:53.812+0000 I NETWORK [conn4076908] end connection 10.0.216.90:53476 (1505 connections now open) 2016-03-13T08:32:53.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53492 #4076924 (1506 connections now open) 2016-03-13T08:32:53.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53491 #4076925 (1507 connections now open) 2016-03-13T08:32:54.584+0000 I NETWORK [conn4076910] end connection 10.0.20.53:46730 (1506 connections now open) 2016-03-13T08:32:54.584+0000 I NETWORK [conn4076909] end connection 10.0.20.53:46731 (1506 connections now open) 2016-03-13T08:32:54.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46752 #4076926 (1506 connections now open) 2016-03-13T08:32:54.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46751 #4076927 (1507 connections now open) 2016-03-13T08:32:55.828+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:55.829+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:55.829+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:55.829+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:32:56.330+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:56.331+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:56.331+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:56.331+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:32:56.832+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:56.832+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:56.832+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:56.832+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:32:57.101+0000 I NETWORK [conn4076917] end connection 10.0.179.209:36598 (1506 connections now open) 2016-03-13T08:32:57.101+0000 I NETWORK [conn4076916] end connection 10.0.179.209:36597 (1506 connections now open) 2016-03-13T08:32:57.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36616 #4076928 (1506 connections now open) 2016-03-13T08:32:57.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36615 #4076929 (1507 connections now open) 2016-03-13T08:32:57.333+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:57.334+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:57.334+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:57.334+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:32:57.564+0000 I NETWORK [conn4076918] end connection 10.0.26.69:38504 (1506 connections now open) 2016-03-13T08:32:57.564+0000 I NETWORK [conn4076919] end connection 10.0.26.69:38503 (1506 connections now open) 2016-03-13T08:32:57.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38511 #4076930 (1506 connections now open) 2016-03-13T08:32:57.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38510 #4076931 (1507 connections now open) 2016-03-13T08:32:57.834+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:57.835+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:57.835+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:57.835+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:32:58.113+0000 I NETWORK [conn4069721] end connection 10.0.179.209:57571 (1506 connections now open) 2016-03-13T08:32:58.113+0000 I NETWORK [conn4066697] end connection 10.0.216.90:42541 (1505 connections now open) 2016-03-13T08:32:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36618 #4076932 (1506 connections now open) 2016-03-13T08:32:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53509 #4076933 (1507 connections now open) 2016-03-13T08:32:58.336+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:32:58.337+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:32:58.337+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:32:58.337+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:32:58.337+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:32:58.337+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:32:58.337+0000-56e525ba18fa188537623c77", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857978337), what: "balancer.round", ns: "", details: { executionTimeMillis: 2545, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:33:00.591+0000 I NETWORK [conn4076921] end connection 10.0.27.237:60102 (1506 connections now open) 2016-03-13T08:33:00.591+0000 I NETWORK [conn4076920] end connection 10.0.27.237:60101 (1506 connections now open) 2016-03-13T08:33:00.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60115 #4076934 (1506 connections now open) 2016-03-13T08:33:00.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60116 #4076935 (1507 connections now open) 2016-03-13T08:33:02.601+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:33:02.620+0000 I NETWORK [conn4076922] end connection 10.0.7.113:55454 (1506 connections now open) 2016-03-13T08:33:02.620+0000 I NETWORK [conn4076923] end connection 10.0.7.113:55455 (1506 connections now open) 2016-03-13T08:33:02.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55468 #4076936 (1506 connections now open) 2016-03-13T08:33:02.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55467 #4076937 (1507 connections now open) 2016-03-13T08:33:03.834+0000 I NETWORK [conn4076925] end connection 10.0.216.90:53491 (1506 connections now open) 2016-03-13T08:33:03.834+0000 I NETWORK [conn4076924] end connection 10.0.216.90:53492 (1506 connections now open) 2016-03-13T08:33:03.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53514 #4076938 (1506 connections now open) 2016-03-13T08:33:03.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53515 #4076939 (1507 connections now open) 2016-03-13T08:33:04.606+0000 I NETWORK [conn4076927] end connection 10.0.20.53:46751 (1506 connections now open) 2016-03-13T08:33:04.606+0000 I NETWORK [conn4076926] end connection 10.0.20.53:46752 (1506 connections now open) 2016-03-13T08:33:04.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46761 #4076940 (1506 connections now open) 2016-03-13T08:33:04.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46762 #4076941 (1507 connections now open) 2016-03-13T08:33:07.114+0000 I NETWORK [conn4076928] end connection 10.0.179.209:36616 (1506 connections now open) 2016-03-13T08:33:07.114+0000 I NETWORK [conn4076929] end connection 10.0.179.209:36615 (1506 connections now open) 2016-03-13T08:33:07.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36627 #4076942 (1506 connections now open) 2016-03-13T08:33:07.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36626 #4076943 (1507 connections now open) 2016-03-13T08:33:07.578+0000 I NETWORK [conn4076931] end connection 10.0.26.69:38510 (1506 connections now open) 2016-03-13T08:33:07.578+0000 I NETWORK [conn4076930] end connection 10.0.26.69:38511 (1506 connections now open) 2016-03-13T08:33:07.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38527 #4076944 (1506 connections now open) 2016-03-13T08:33:07.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38526 #4076945 (1507 connections now open) 2016-03-13T08:33:08.390+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:33:08.393+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:08.401+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:08.401+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:08.401+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:33:08.902+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:08.903+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:08.903+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:08.903+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:33:09.403+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:09.404+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:09.404+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:09.404+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:33:09.905+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:09.906+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:09.906+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:09.906+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:33:10.406+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:10.407+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:10.407+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:10.407+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:33:10.587+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:33:10.426+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:33:10.603+0000 I NETWORK [conn4076935] end connection 10.0.27.237:60116 (1506 connections now open) 2016-03-13T08:33:10.603+0000 I NETWORK [conn4076934] end connection 10.0.27.237:60115 (1506 connections now open) 2016-03-13T08:33:10.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60135 #4076946 (1506 connections now open) 2016-03-13T08:33:10.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60134 #4076947 (1507 connections now open) 2016-03-13T08:33:10.908+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:10.909+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:10.909+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:10.909+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:33:11.409+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:11.410+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:11.410+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:11.410+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:33:11.911+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:11.912+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:11.912+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:11.912+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:33:12.413+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:12.413+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:12.414+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:12.414+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:33:12.604+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:12.605+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:12.605+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:33:12.605+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:33:12.633+0000 I NETWORK [conn4076937] end connection 10.0.7.113:55467 (1506 connections now open) 2016-03-13T08:33:12.633+0000 I NETWORK [conn4076936] end connection 10.0.7.113:55468 (1506 connections now open) 2016-03-13T08:33:12.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55482 #4076948 (1506 connections now open) 2016-03-13T08:33:12.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55483 #4076949 (1507 connections now open) 2016-03-13T08:33:12.914+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:12.915+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:12.915+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:12.915+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:33:13.416+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:13.417+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:13.417+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:13.417+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:33:13.847+0000 I NETWORK [conn4076938] end connection 10.0.216.90:53514 (1506 connections now open) 2016-03-13T08:33:13.847+0000 I NETWORK [conn4076939] end connection 10.0.216.90:53515 (1506 connections now open) 2016-03-13T08:33:13.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53522 #4076950 (1506 connections now open) 2016-03-13T08:33:13.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53521 #4076951 (1507 connections now open) 2016-03-13T08:33:13.917+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:13.918+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:13.918+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:13.918+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:33:14.419+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:14.420+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:14.420+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:14.420+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:33:14.620+0000 I NETWORK [conn4076940] end connection 10.0.20.53:46761 (1506 connections now open) 2016-03-13T08:33:14.620+0000 I NETWORK [conn4076941] end connection 10.0.20.53:46762 (1506 connections now open) 2016-03-13T08:33:14.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46773 #4076952 (1506 connections now open) 2016-03-13T08:33:14.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46774 #4076953 (1507 connections now open) 2016-03-13T08:33:14.921+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:14.921+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:14.921+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:14.921+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:33:15.422+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:15.423+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:15.423+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:15.423+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:33:15.924+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:15.924+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:15.924+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:15.924+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:33:16.117+0000 I NETWORK [conn4063936] end connection 10.0.179.209:51499 (1506 connections now open) 2016-03-13T08:33:16.117+0000 I NETWORK [conn4068197] end connection 10.0.20.53:37601 (1505 connections now open) 2016-03-13T08:33:16.117+0000 I NETWORK [conn4065044] end connection 10.0.216.90:40724 (1504 connections now open) 2016-03-13T08:33:16.118+0000 I NETWORK [conn4066741] end connection 10.0.27.237:49606 (1503 connections now open) 2016-03-13T08:33:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36637 #4076954 (1504 connections now open) 2016-03-13T08:33:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46787 #4076955 (1505 connections now open) 2016-03-13T08:33:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53530 #4076956 (1506 connections now open) 2016-03-13T08:33:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60144 #4076957 (1507 connections now open) 2016-03-13T08:33:16.425+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:16.426+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:16.426+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:16.426+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:33:16.927+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:16.927+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:16.927+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:16.927+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:33:17.127+0000 I NETWORK [conn4076943] end connection 10.0.179.209:36626 (1506 connections now open) 2016-03-13T08:33:17.127+0000 I NETWORK [conn4076942] end connection 10.0.179.209:36627 (1506 connections now open) 2016-03-13T08:33:17.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36640 #4076958 (1506 connections now open) 2016-03-13T08:33:17.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36639 #4076959 (1507 connections now open) 2016-03-13T08:33:17.428+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:17.429+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:17.429+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:17.429+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:33:17.591+0000 I NETWORK [conn4076944] end connection 10.0.26.69:38527 (1506 connections now open) 2016-03-13T08:33:17.591+0000 I NETWORK [conn4076945] end connection 10.0.26.69:38526 (1506 connections now open) 2016-03-13T08:33:17.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38551 #4076960 (1506 connections now open) 2016-03-13T08:33:17.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38552 #4076961 (1507 connections now open) 2016-03-13T08:33:17.930+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:17.930+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:17.930+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:17.930+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:33:18.431+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:18.432+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:18.432+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:18.432+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:33:18.933+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:18.933+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:18.934+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:18.934+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:33:18.934+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:33:18.934+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:33:18.934+0000-56e525ce18fa188537623c78", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457857998934), what: "balancer.round", ns: "", details: { executionTimeMillis: 10574, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:33:20.616+0000 I NETWORK [conn4076946] end connection 10.0.27.237:60135 (1506 connections now open) 2016-03-13T08:33:20.616+0000 I NETWORK [conn4076947] end connection 10.0.27.237:60134 (1506 connections now open) 2016-03-13T08:33:20.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60146 #4076962 (1506 connections now open) 2016-03-13T08:33:20.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60145 #4076963 (1507 connections now open) 2016-03-13T08:33:22.608+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:22.609+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:22.609+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:33:22.609+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:33:22.646+0000 I NETWORK [conn4076948] end connection 10.0.7.113:55482 (1506 connections now open) 2016-03-13T08:33:22.646+0000 I NETWORK [conn4076949] end connection 10.0.7.113:55483 (1506 connections now open) 2016-03-13T08:33:22.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55504 #4076964 (1506 connections now open) 2016-03-13T08:33:22.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55503 #4076965 (1507 connections now open) 2016-03-13T08:33:23.860+0000 I NETWORK [conn4076950] end connection 10.0.216.90:53522 (1506 connections now open) 2016-03-13T08:33:23.860+0000 I NETWORK [conn4076951] end connection 10.0.216.90:53521 (1506 connections now open) 2016-03-13T08:33:23.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53536 #4076966 (1506 connections now open) 2016-03-13T08:33:23.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53537 #4076967 (1507 connections now open) 2016-03-13T08:33:24.633+0000 I NETWORK [conn4076952] end connection 10.0.20.53:46773 (1506 connections now open) 2016-03-13T08:33:24.633+0000 I NETWORK [conn4076953] end connection 10.0.20.53:46774 (1506 connections now open) 2016-03-13T08:33:24.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46793 #4076968 (1506 connections now open) 2016-03-13T08:33:24.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46794 #4076969 (1507 connections now open) 2016-03-13T08:33:27.139+0000 I NETWORK [conn4076959] end connection 10.0.179.209:36639 (1506 connections now open) 2016-03-13T08:33:27.139+0000 I NETWORK [conn4076958] end connection 10.0.179.209:36640 (1506 connections now open) 2016-03-13T08:33:27.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36661 #4076970 (1506 connections now open) 2016-03-13T08:33:27.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36660 #4076971 (1507 connections now open) 2016-03-13T08:33:27.604+0000 I NETWORK [conn4076960] end connection 10.0.26.69:38551 (1506 connections now open) 2016-03-13T08:33:27.604+0000 I NETWORK [conn4076961] end connection 10.0.26.69:38552 (1505 connections now open) 2016-03-13T08:33:27.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38558 #4076972 (1506 connections now open) 2016-03-13T08:33:27.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38559 #4076973 (1507 connections now open) 2016-03-13T08:33:28.113+0000 I NETWORK [conn4069111] end connection 10.0.216.90:45099 (1506 connections now open) 2016-03-13T08:33:28.113+0000 I NETWORK [conn4068988] end connection 10.0.7.113:46955 (1505 connections now open) 2016-03-13T08:33:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53553 #4076974 (1507 connections now open) 2016-03-13T08:33:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55512 #4076975 (1507 connections now open) 2016-03-13T08:33:29.000+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:29.001+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:29.001+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:29.001+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:33:29.502+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:29.502+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:29.502+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:29.502+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:33:30.003+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:30.004+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:30.004+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:30.004+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:33:30.505+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:30.505+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:30.505+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:30.505+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:33:30.629+0000 I NETWORK [conn4076963] end connection 10.0.27.237:60145 (1506 connections now open) 2016-03-13T08:33:30.629+0000 I NETWORK [conn4076962] end connection 10.0.27.237:60146 (1506 connections now open) 2016-03-13T08:33:30.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60160 #4076976 (1506 connections now open) 2016-03-13T08:33:30.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60159 #4076977 (1507 connections now open) 2016-03-13T08:33:31.006+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:31.007+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:31.007+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:31.007+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:33:31.508+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:31.509+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:31.509+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:31.509+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:33:31.509+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:33:31.509+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:33:31.509+0000-56e525db18fa188537623c79", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858011509), what: "balancer.round", ns: "", details: { executionTimeMillis: 2546, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:33:32.611+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:33:32.659+0000 I NETWORK [conn4076965] end connection 10.0.7.113:55503 (1506 connections now open) 2016-03-13T08:33:32.659+0000 I NETWORK [conn4076964] end connection 10.0.7.113:55504 (1506 connections now open) 2016-03-13T08:33:32.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55514 #4076978 (1506 connections now open) 2016-03-13T08:33:32.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55515 #4076979 (1507 connections now open) 2016-03-13T08:33:33.873+0000 I NETWORK [conn4076967] end connection 10.0.216.90:53537 (1506 connections now open) 2016-03-13T08:33:33.873+0000 I NETWORK [conn4076966] end connection 10.0.216.90:53536 (1506 connections now open) 2016-03-13T08:33:33.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53559 #4076980 (1506 connections now open) 2016-03-13T08:33:33.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53560 #4076981 (1507 connections now open) 2016-03-13T08:33:34.647+0000 I NETWORK [conn4076968] end connection 10.0.20.53:46793 (1506 connections now open) 2016-03-13T08:33:34.647+0000 I NETWORK [conn4076969] end connection 10.0.20.53:46794 (1506 connections now open) 2016-03-13T08:33:34.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46805 #4076982 (1506 connections now open) 2016-03-13T08:33:34.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46806 #4076983 (1507 connections now open) 2016-03-13T08:33:37.153+0000 I NETWORK [conn4076971] end connection 10.0.179.209:36660 (1506 connections now open) 2016-03-13T08:33:37.153+0000 I NETWORK [conn4076970] end connection 10.0.179.209:36661 (1506 connections now open) 2016-03-13T08:33:37.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36670 #4076984 (1506 connections now open) 2016-03-13T08:33:37.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36671 #4076985 (1507 connections now open) 2016-03-13T08:33:37.617+0000 I NETWORK [conn4076973] end connection 10.0.26.69:38559 (1506 connections now open) 2016-03-13T08:33:37.617+0000 I NETWORK [conn4076972] end connection 10.0.26.69:38558 (1505 connections now open) 2016-03-13T08:33:37.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38572 #4076986 (1506 connections now open) 2016-03-13T08:33:37.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38573 #4076987 (1507 connections now open) 2016-03-13T08:33:40.641+0000 I NETWORK [conn4076976] end connection 10.0.27.237:60160 (1506 connections now open) 2016-03-13T08:33:40.641+0000 I NETWORK [conn4076977] end connection 10.0.27.237:60159 (1506 connections now open) 2016-03-13T08:33:40.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60177 #4076988 (1506 connections now open) 2016-03-13T08:33:40.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60178 #4076989 (1507 connections now open) 2016-03-13T08:33:40.743+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:33:40.587+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:33:41.564+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:33:41.566+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:41.574+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:41.574+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:41.574+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:33:42.075+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:42.076+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:42.076+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:42.076+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:33:42.577+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:42.577+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:42.577+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:42.578+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:33:42.614+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:42.615+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:42.615+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:33:42.615+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:33:42.672+0000 I NETWORK [conn4076979] end connection 10.0.7.113:55515 (1506 connections now open) 2016-03-13T08:33:42.672+0000 I NETWORK [conn4076978] end connection 10.0.7.113:55514 (1506 connections now open) 2016-03-13T08:33:42.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55526 #4076990 (1506 connections now open) 2016-03-13T08:33:42.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55527 #4076991 (1507 connections now open) 2016-03-13T08:33:43.078+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:43.079+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:43.079+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:43.079+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:33:43.580+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:43.581+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:43.581+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:43.581+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:33:43.885+0000 I NETWORK [conn4076980] end connection 10.0.216.90:53559 (1506 connections now open) 2016-03-13T08:33:43.885+0000 I NETWORK [conn4076981] end connection 10.0.216.90:53560 (1506 connections now open) 2016-03-13T08:33:43.886+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53567 #4076992 (1506 connections now open) 2016-03-13T08:33:43.886+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53568 #4076993 (1507 connections now open) 2016-03-13T08:33:44.082+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:44.082+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:44.082+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:44.082+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:33:44.583+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:44.584+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:44.584+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:44.584+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:33:44.661+0000 I NETWORK [conn4076982] end connection 10.0.20.53:46805 (1506 connections now open) 2016-03-13T08:33:44.661+0000 I NETWORK [conn4076983] end connection 10.0.20.53:46806 (1506 connections now open) 2016-03-13T08:33:44.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46817 #4076994 (1506 connections now open) 2016-03-13T08:33:44.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46818 #4076995 (1507 connections now open) 2016-03-13T08:33:45.085+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:45.085+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:45.085+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:45.085+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:33:45.586+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:45.587+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:45.587+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:45.587+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:33:46.088+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:46.089+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:46.089+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:46.089+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:33:46.117+0000 I NETWORK [conn4069239] end connection 10.0.179.209:57090 (1506 connections now open) 2016-03-13T08:33:46.117+0000 I NETWORK [conn4074629] end connection 10.0.7.113:53010 (1505 connections now open) 2016-03-13T08:33:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36680 #4076996 (1506 connections now open) 2016-03-13T08:33:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55533 #4076997 (1507 connections now open) 2016-03-13T08:33:46.590+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:46.590+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:46.590+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:46.590+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:33:47.091+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:47.092+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:47.092+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:47.092+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:33:47.165+0000 I NETWORK [conn4076985] end connection 10.0.179.209:36671 (1506 connections now open) 2016-03-13T08:33:47.166+0000 I NETWORK [conn4076984] end connection 10.0.179.209:36670 (1505 connections now open) 2016-03-13T08:33:47.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36682 #4076998 (1506 connections now open) 2016-03-13T08:33:47.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36683 #4076999 (1507 connections now open) 2016-03-13T08:33:47.593+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:47.593+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:47.593+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:47.593+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:33:47.639+0000 I NETWORK [conn4076986] end connection 10.0.26.69:38572 (1506 connections now open) 2016-03-13T08:33:47.639+0000 I NETWORK [conn4076987] end connection 10.0.26.69:38573 (1506 connections now open) 2016-03-13T08:33:47.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38593 #4077000 (1506 connections now open) 2016-03-13T08:33:47.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38594 #4077001 (1507 connections now open) 2016-03-13T08:33:48.094+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:48.095+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:48.095+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:48.095+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:33:48.596+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:48.597+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:48.597+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:48.597+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:33:49.098+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:49.098+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:49.099+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:49.099+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:33:49.599+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:49.600+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:49.600+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:49.600+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:33:50.101+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:50.102+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:50.102+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:50.102+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:33:50.603+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:50.603+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:50.603+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:50.603+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:33:50.663+0000 I NETWORK [conn4076988] end connection 10.0.27.237:60177 (1506 connections now open) 2016-03-13T08:33:50.663+0000 I NETWORK [conn4076989] end connection 10.0.27.237:60178 (1506 connections now open) 2016-03-13T08:33:50.663+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60187 #4077002 (1506 connections now open) 2016-03-13T08:33:50.663+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60186 #4077003 (1507 connections now open) 2016-03-13T08:33:51.104+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:51.105+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:51.105+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:51.105+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:33:51.606+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:51.606+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:51.607+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:51.607+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:33:52.107+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:52.108+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:52.108+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:33:52.108+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:33:52.108+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:33:52.108+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:33:52.108+0000-56e525f018fa188537623c7a", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858032108), what: "balancer.round", ns: "", details: { executionTimeMillis: 10577, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:33:52.617+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:33:52.618+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:33:52.618+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:33:52.618+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:33:52.685+0000 I NETWORK [conn4076990] end connection 10.0.7.113:55526 (1506 connections now open) 2016-03-13T08:33:52.685+0000 I NETWORK [conn4076991] end connection 10.0.7.113:55527 (1506 connections now open) 2016-03-13T08:33:52.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55547 #4077004 (1506 connections now open) 2016-03-13T08:33:52.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55546 #4077005 (1507 connections now open) 2016-03-13T08:33:53.898+0000 I NETWORK [conn4076992] end connection 10.0.216.90:53567 (1506 connections now open) 2016-03-13T08:33:53.898+0000 I NETWORK [conn4076993] end connection 10.0.216.90:53568 (1506 connections now open) 2016-03-13T08:33:53.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53582 #4077006 (1506 connections now open) 2016-03-13T08:33:53.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53583 #4077007 (1507 connections now open) 2016-03-13T08:33:54.674+0000 I NETWORK [conn4076995] end connection 10.0.20.53:46818 (1506 connections now open) 2016-03-13T08:33:54.674+0000 I NETWORK [conn4076994] end connection 10.0.20.53:46817 (1506 connections now open) 2016-03-13T08:33:54.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46838 #4077008 (1506 connections now open) 2016-03-13T08:33:54.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46837 #4077009 (1507 connections now open) 2016-03-13T08:33:57.188+0000 I NETWORK [conn4076998] end connection 10.0.179.209:36682 (1506 connections now open) 2016-03-13T08:33:57.188+0000 I NETWORK [conn4076999] end connection 10.0.179.209:36683 (1506 connections now open) 2016-03-13T08:33:57.188+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36701 #4077010 (1506 connections now open) 2016-03-13T08:33:57.188+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36700 #4077011 (1507 connections now open) 2016-03-13T08:33:57.653+0000 I NETWORK [conn4077000] end connection 10.0.26.69:38593 (1506 connections now open) 2016-03-13T08:33:57.653+0000 I NETWORK [conn4077001] end connection 10.0.26.69:38594 (1506 connections now open) 2016-03-13T08:33:57.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38600 #4077012 (1506 connections now open) 2016-03-13T08:33:57.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38601 #4077013 (1507 connections now open) 2016-03-13T08:33:58.112+0000 I NETWORK [conn4069068] end connection 10.0.179.209:56918 (1506 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [conn4068305] end connection 10.0.20.53:37711 (1505 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [conn4074758] end connection 10.0.216.90:51156 (1504 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [conn4070395] end connection 10.0.26.69:59727 (1504 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [conn4076505] end connection 10.0.27.237:59685 (1502 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36703 #4077014 (1503 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46844 #4077015 (1504 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53600 #4077016 (1505 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38604 #4077017 (1506 connections now open) 2016-03-13T08:33:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60198 #4077018 (1507 connections now open) 2016-03-13T08:34:00.676+0000 I NETWORK [conn4077002] end connection 10.0.27.237:60187 (1506 connections now open) 2016-03-13T08:34:00.676+0000 I NETWORK [conn4077003] end connection 10.0.27.237:60186 (1506 connections now open) 2016-03-13T08:34:00.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60201 #4077019 (1506 connections now open) 2016-03-13T08:34:00.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60200 #4077020 (1507 connections now open) 2016-03-13T08:34:02.166+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:02.166+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:02.166+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:02.166+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:34:02.621+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:02.622+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:02.622+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:34:02.622+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:34:02.667+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:02.668+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:02.668+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:02.668+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:34:02.707+0000 I NETWORK [conn4077004] end connection 10.0.7.113:55547 (1506 connections now open) 2016-03-13T08:34:02.707+0000 I NETWORK [conn4077005] end connection 10.0.7.113:55546 (1506 connections now open) 2016-03-13T08:34:02.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55558 #4077021 (1506 connections now open) 2016-03-13T08:34:02.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55559 #4077022 (1507 connections now open) 2016-03-13T08:34:03.169+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:03.169+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:03.169+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:03.169+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:34:03.670+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:03.671+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:03.671+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:03.671+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:34:03.920+0000 I NETWORK [conn4077007] end connection 10.0.216.90:53583 (1506 connections now open) 2016-03-13T08:34:03.920+0000 I NETWORK [conn4077006] end connection 10.0.216.90:53582 (1506 connections now open) 2016-03-13T08:34:03.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53605 #4077023 (1506 connections now open) 2016-03-13T08:34:03.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53606 #4077024 (1507 connections now open) 2016-03-13T08:34:04.172+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:04.173+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:04.173+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:04.173+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:34:04.173+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:34:04.173+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:34:04.173+0000-56e525fc18fa188537623c7b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858044173), what: "balancer.round", ns: "", details: { executionTimeMillis: 2028, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:34:04.697+0000 I NETWORK [conn4077008] end connection 10.0.20.53:46838 (1506 connections now open) 2016-03-13T08:34:04.697+0000 I NETWORK [conn4077009] end connection 10.0.20.53:46837 (1506 connections now open) 2016-03-13T08:34:04.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46847 #4077025 (1506 connections now open) 2016-03-13T08:34:04.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46848 #4077026 (1507 connections now open) 2016-03-13T08:34:07.201+0000 I NETWORK [conn4077011] end connection 10.0.179.209:36700 (1506 connections now open) 2016-03-13T08:34:07.201+0000 I NETWORK [conn4077010] end connection 10.0.179.209:36701 (1506 connections now open) 2016-03-13T08:34:07.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36711 #4077027 (1506 connections now open) 2016-03-13T08:34:07.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36712 #4077028 (1507 connections now open) 2016-03-13T08:34:07.666+0000 I NETWORK [conn4077013] end connection 10.0.26.69:38601 (1506 connections now open) 2016-03-13T08:34:07.666+0000 I NETWORK [conn4077012] end connection 10.0.26.69:38600 (1506 connections now open) 2016-03-13T08:34:07.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38617 #4077029 (1506 connections now open) 2016-03-13T08:34:07.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38618 #4077030 (1507 connections now open) 2016-03-13T08:34:10.688+0000 I NETWORK [conn4077019] end connection 10.0.27.237:60201 (1506 connections now open) 2016-03-13T08:34:10.688+0000 I NETWORK [conn4077020] end connection 10.0.27.237:60200 (1506 connections now open) 2016-03-13T08:34:10.689+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60219 #4077031 (1506 connections now open) 2016-03-13T08:34:10.689+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60220 #4077032 (1507 connections now open) 2016-03-13T08:34:10.903+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:34:10.743+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:34:12.624+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:34:12.720+0000 I NETWORK [conn4077022] end connection 10.0.7.113:55559 (1506 connections now open) 2016-03-13T08:34:12.720+0000 I NETWORK [conn4077021] end connection 10.0.7.113:55558 (1506 connections now open) 2016-03-13T08:34:12.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55572 #4077033 (1507 connections now open) 2016-03-13T08:34:12.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55573 #4077034 (1507 connections now open) 2016-03-13T08:34:13.934+0000 I NETWORK [conn4077024] end connection 10.0.216.90:53606 (1506 connections now open) 2016-03-13T08:34:13.934+0000 I NETWORK [conn4077023] end connection 10.0.216.90:53605 (1506 connections now open) 2016-03-13T08:34:13.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53612 #4077035 (1506 connections now open) 2016-03-13T08:34:13.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53613 #4077036 (1507 connections now open) 2016-03-13T08:34:14.223+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:34:14.226+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:14.235+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:14.235+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:14.235+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:34:14.711+0000 I NETWORK [conn4077026] end connection 10.0.20.53:46848 (1506 connections now open) 2016-03-13T08:34:14.711+0000 I NETWORK [conn4077025] end connection 10.0.20.53:46847 (1506 connections now open) 2016-03-13T08:34:14.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46859 #4077037 (1506 connections now open) 2016-03-13T08:34:14.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46860 #4077038 (1507 connections now open) 2016-03-13T08:34:14.736+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:14.737+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:14.737+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:14.737+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:34:15.238+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:15.238+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:15.238+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:15.238+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:34:15.739+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:15.740+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:15.740+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:15.740+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:34:16.117+0000 I NETWORK [conn4067630] end connection 10.0.179.209:55448 (1506 connections now open) 2016-03-13T08:34:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36722 #4077039 (1507 connections now open) 2016-03-13T08:34:16.241+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:16.241+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:16.241+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:16.241+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:34:16.742+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:16.743+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:16.743+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:16.743+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:34:17.214+0000 I NETWORK [conn4077027] end connection 10.0.179.209:36711 (1506 connections now open) 2016-03-13T08:34:17.214+0000 I NETWORK [conn4077028] end connection 10.0.179.209:36712 (1506 connections now open) 2016-03-13T08:34:17.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36723 #4077040 (1506 connections now open) 2016-03-13T08:34:17.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36724 #4077041 (1507 connections now open) 2016-03-13T08:34:17.244+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:17.245+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:17.245+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:17.245+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:34:17.680+0000 I NETWORK [conn4077029] end connection 10.0.26.69:38617 (1506 connections now open) 2016-03-13T08:34:17.680+0000 I NETWORK [conn4077030] end connection 10.0.26.69:38618 (1506 connections now open) 2016-03-13T08:34:17.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38641 #4077042 (1506 connections now open) 2016-03-13T08:34:17.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38642 #4077043 (1507 connections now open) 2016-03-13T08:34:17.746+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:17.746+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:17.746+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:17.746+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:34:18.247+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:18.248+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:18.248+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:18.248+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:34:18.749+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:18.750+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:18.750+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:18.750+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:34:19.251+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:19.251+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:19.251+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:19.251+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:34:19.752+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:19.753+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:19.753+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:19.753+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:34:20.254+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:20.254+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:20.254+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:20.254+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:34:20.701+0000 I NETWORK [conn4077032] end connection 10.0.27.237:60220 (1506 connections now open) 2016-03-13T08:34:20.701+0000 I NETWORK [conn4077031] end connection 10.0.27.237:60219 (1506 connections now open) 2016-03-13T08:34:20.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60231 #4077044 (1506 connections now open) 2016-03-13T08:34:20.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60232 #4077045 (1507 connections now open) 2016-03-13T08:34:20.755+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:20.756+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:20.756+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:20.756+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:34:21.257+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:21.257+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:21.257+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:21.257+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:34:21.758+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:21.759+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:21.759+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:21.759+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:34:22.260+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:22.261+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:22.261+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:22.261+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:34:22.627+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:22.627+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:22.627+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:34:22.627+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:34:22.734+0000 I NETWORK [conn4077033] end connection 10.0.7.113:55572 (1506 connections now open) 2016-03-13T08:34:22.734+0000 I NETWORK [conn4077034] end connection 10.0.7.113:55573 (1506 connections now open) 2016-03-13T08:34:22.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55592 #4077046 (1506 connections now open) 2016-03-13T08:34:22.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55593 #4077047 (1507 connections now open) 2016-03-13T08:34:22.762+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:22.762+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:22.763+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:22.763+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:34:23.264+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:23.265+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:23.265+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:23.265+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:34:23.766+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:23.766+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:23.766+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:23.766+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:34:23.947+0000 I NETWORK [conn4077036] end connection 10.0.216.90:53613 (1506 connections now open) 2016-03-13T08:34:23.947+0000 I NETWORK [conn4077035] end connection 10.0.216.90:53612 (1506 connections now open) 2016-03-13T08:34:23.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53629 #4077048 (1506 connections now open) 2016-03-13T08:34:23.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53628 #4077049 (1507 connections now open) 2016-03-13T08:34:24.267+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:24.268+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:24.268+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:24.268+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:34:24.724+0000 I NETWORK [conn4077037] end connection 10.0.20.53:46859 (1506 connections now open) 2016-03-13T08:34:24.724+0000 I NETWORK [conn4077038] end connection 10.0.20.53:46860 (1506 connections now open) 2016-03-13T08:34:24.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46881 #4077050 (1506 connections now open) 2016-03-13T08:34:24.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46880 #4077051 (1507 connections now open) 2016-03-13T08:34:24.769+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:24.770+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:24.770+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:24.770+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:34:24.770+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:34:24.770+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:34:24.770+0000-56e5261018fa188537623c7c", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858064770), what: "balancer.round", ns: "", details: { executionTimeMillis: 10565, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:34:27.227+0000 I NETWORK [conn4077040] end connection 10.0.179.209:36723 (1506 connections now open) 2016-03-13T08:34:27.227+0000 I NETWORK [conn4077041] end connection 10.0.179.209:36724 (1506 connections now open) 2016-03-13T08:34:27.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36745 #4077052 (1506 connections now open) 2016-03-13T08:34:27.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36746 #4077053 (1507 connections now open) 2016-03-13T08:34:27.693+0000 I NETWORK [conn4077042] end connection 10.0.26.69:38641 (1506 connections now open) 2016-03-13T08:34:27.693+0000 I NETWORK [conn4077043] end connection 10.0.26.69:38642 (1506 connections now open) 2016-03-13T08:34:27.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38649 #4077054 (1506 connections now open) 2016-03-13T08:34:27.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38650 #4077055 (1507 connections now open) 2016-03-13T08:34:28.114+0000 I NETWORK [conn4065709] end connection 10.0.179.209:53419 (1506 connections now open) 2016-03-13T08:34:28.114+0000 I NETWORK [conn4070773] end connection 10.0.26.69:60140 (1505 connections now open) 2016-03-13T08:34:28.114+0000 I NETWORK [conn4071228] end connection 10.0.216.90:47355 (1505 connections now open) 2016-03-13T08:34:28.114+0000 I NETWORK [conn4068139] end connection 10.0.7.113:46053 (1505 connections now open) 2016-03-13T08:34:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36748 #4077056 (1507 connections now open) 2016-03-13T08:34:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38652 #4077057 (1505 connections now open) 2016-03-13T08:34:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53645 #4077058 (1506 connections now open) 2016-03-13T08:34:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55600 #4077059 (1507 connections now open) 2016-03-13T08:34:30.715+0000 I NETWORK [conn4077045] end connection 10.0.27.237:60232 (1506 connections now open) 2016-03-13T08:34:30.715+0000 I NETWORK [conn4077044] end connection 10.0.27.237:60231 (1505 connections now open) 2016-03-13T08:34:30.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60246 #4077060 (1506 connections now open) 2016-03-13T08:34:30.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60247 #4077061 (1507 connections now open) 2016-03-13T08:34:32.631+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:32.631+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:32.631+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:34:32.631+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:34:32.747+0000 I NETWORK [conn4077046] end connection 10.0.7.113:55592 (1506 connections now open) 2016-03-13T08:34:32.747+0000 I NETWORK [conn4077047] end connection 10.0.7.113:55593 (1506 connections now open) 2016-03-13T08:34:32.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55603 #4077062 (1506 connections now open) 2016-03-13T08:34:32.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55602 #4077063 (1507 connections now open) 2016-03-13T08:34:33.961+0000 I NETWORK [conn4077049] end connection 10.0.216.90:53628 (1506 connections now open) 2016-03-13T08:34:33.961+0000 I NETWORK [conn4077048] end connection 10.0.216.90:53629 (1506 connections now open) 2016-03-13T08:34:33.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53651 #4077064 (1506 connections now open) 2016-03-13T08:34:33.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53652 #4077065 (1507 connections now open) 2016-03-13T08:34:34.737+0000 I NETWORK [conn4077051] end connection 10.0.20.53:46880 (1506 connections now open) 2016-03-13T08:34:34.737+0000 I NETWORK [conn4077050] end connection 10.0.20.53:46881 (1506 connections now open) 2016-03-13T08:34:34.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46894 #4077066 (1506 connections now open) 2016-03-13T08:34:34.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46893 #4077067 (1507 connections now open) 2016-03-13T08:34:34.842+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:34.843+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:34.843+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:34.843+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:34:35.344+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:35.344+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:35.344+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:35.344+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:34:35.845+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:35.846+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:35.846+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:35.846+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:34:36.347+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:36.347+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:36.347+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:36.347+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:34:36.848+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:36.849+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:36.849+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:36.849+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:34:37.240+0000 I NETWORK [conn4077053] end connection 10.0.179.209:36746 (1506 connections now open) 2016-03-13T08:34:37.240+0000 I NETWORK [conn4077052] end connection 10.0.179.209:36745 (1506 connections now open) 2016-03-13T08:34:37.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36755 #4077068 (1506 connections now open) 2016-03-13T08:34:37.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36754 #4077069 (1507 connections now open) 2016-03-13T08:34:37.350+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:37.350+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:37.350+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:37.350+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:34:37.350+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:34:37.350+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:34:37.350+0000-56e5261d18fa188537623c7d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858077350), what: "balancer.round", ns: "", details: { executionTimeMillis: 2545, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:34:37.706+0000 I NETWORK [conn4077054] end connection 10.0.26.69:38649 (1506 connections now open) 2016-03-13T08:34:37.706+0000 I NETWORK [conn4077055] end connection 10.0.26.69:38650 (1506 connections now open) 2016-03-13T08:34:37.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38664 #4077070 (1506 connections now open) 2016-03-13T08:34:37.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38663 #4077071 (1507 connections now open) 2016-03-13T08:34:40.728+0000 I NETWORK [conn4077061] end connection 10.0.27.237:60247 (1506 connections now open) 2016-03-13T08:34:40.728+0000 I NETWORK [conn4077060] end connection 10.0.27.237:60246 (1505 connections now open) 2016-03-13T08:34:40.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60264 #4077072 (1506 connections now open) 2016-03-13T08:34:40.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60265 #4077073 (1507 connections now open) 2016-03-13T08:34:41.181+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:34:40.904+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:34:42.633+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:34:42.761+0000 I NETWORK [conn4077063] end connection 10.0.7.113:55602 (1506 connections now open) 2016-03-13T08:34:42.761+0000 I NETWORK [conn4077062] end connection 10.0.7.113:55603 (1506 connections now open) 2016-03-13T08:34:42.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55613 #4077074 (1506 connections now open) 2016-03-13T08:34:42.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55614 #4077075 (1507 connections now open) 2016-03-13T08:34:43.973+0000 I NETWORK [conn4077065] end connection 10.0.216.90:53652 (1506 connections now open) 2016-03-13T08:34:43.973+0000 I NETWORK [conn4077064] end connection 10.0.216.90:53651 (1506 connections now open) 2016-03-13T08:34:43.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53660 #4077076 (1506 connections now open) 2016-03-13T08:34:43.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53659 #4077077 (1507 connections now open) 2016-03-13T08:34:44.750+0000 I NETWORK [conn4077067] end connection 10.0.20.53:46893 (1506 connections now open) 2016-03-13T08:34:44.750+0000 I NETWORK [conn4077066] end connection 10.0.20.53:46894 (1506 connections now open) 2016-03-13T08:34:44.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46906 #4077078 (1506 connections now open) 2016-03-13T08:34:44.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46907 #4077079 (1507 connections now open) 2016-03-13T08:34:46.117+0000 I NETWORK [conn4067041] end connection 10.0.179.209:54824 (1506 connections now open) 2016-03-13T08:34:46.117+0000 I NETWORK [conn4070436] end connection 10.0.216.90:46512 (1506 connections now open) 2016-03-13T08:34:46.117+0000 I NETWORK [conn4073708] end connection 10.0.26.69:35036 (1504 connections now open) 2016-03-13T08:34:46.118+0000 I NETWORK [conn4068900] end connection 10.0.27.237:51793 (1504 connections now open) 2016-03-13T08:34:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36765 #4077080 (1506 connections now open) 2016-03-13T08:34:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38683 #4077081 (1505 connections now open) 2016-03-13T08:34:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53668 #4077082 (1506 connections now open) 2016-03-13T08:34:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60272 #4077083 (1507 connections now open) 2016-03-13T08:34:47.253+0000 I NETWORK [conn4077069] end connection 10.0.179.209:36754 (1506 connections now open) 2016-03-13T08:34:47.253+0000 I NETWORK [conn4077068] end connection 10.0.179.209:36755 (1506 connections now open) 2016-03-13T08:34:47.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36767 #4077084 (1506 connections now open) 2016-03-13T08:34:47.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36766 #4077085 (1507 connections now open) 2016-03-13T08:34:47.424+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:34:47.427+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:47.436+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:47.436+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:47.436+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:34:47.728+0000 I NETWORK [conn4077070] end connection 10.0.26.69:38664 (1506 connections now open) 2016-03-13T08:34:47.728+0000 I NETWORK [conn4077071] end connection 10.0.26.69:38663 (1506 connections now open) 2016-03-13T08:34:47.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38685 #4077086 (1506 connections now open) 2016-03-13T08:34:47.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38686 #4077087 (1507 connections now open) 2016-03-13T08:34:47.937+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:47.938+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:47.938+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:47.938+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:34:48.439+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:48.439+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:48.439+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:48.439+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:34:48.940+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:48.941+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:48.941+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:48.941+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:34:49.442+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:49.443+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:49.443+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:49.443+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:34:49.944+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:49.944+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:49.944+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:49.944+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:34:50.445+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:50.446+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:50.446+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:50.446+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:34:50.750+0000 I NETWORK [conn4077073] end connection 10.0.27.237:60265 (1506 connections now open) 2016-03-13T08:34:50.750+0000 I NETWORK [conn4077072] end connection 10.0.27.237:60264 (1506 connections now open) 2016-03-13T08:34:50.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60273 #4077088 (1506 connections now open) 2016-03-13T08:34:50.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60274 #4077089 (1507 connections now open) 2016-03-13T08:34:50.947+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:50.947+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:50.947+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:50.947+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:34:51.448+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:51.449+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:51.449+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:51.449+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:34:51.950+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:51.951+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:51.951+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:51.951+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:34:52.452+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:52.452+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:52.452+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:52.452+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:34:52.636+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:52.637+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:52.637+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:34:52.637+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:34:52.774+0000 I NETWORK [conn4077075] end connection 10.0.7.113:55614 (1506 connections now open) 2016-03-13T08:34:52.774+0000 I NETWORK [conn4077074] end connection 10.0.7.113:55613 (1506 connections now open) 2016-03-13T08:34:52.782+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55634 #4077090 (1506 connections now open) 2016-03-13T08:34:52.782+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55635 #4077091 (1507 connections now open) 2016-03-13T08:34:52.953+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:52.954+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:52.954+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:52.954+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:34:53.455+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:53.455+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:53.455+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:53.455+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:34:53.956+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:53.957+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:53.957+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:53.957+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:34:53.987+0000 I NETWORK [conn4077077] end connection 10.0.216.90:53659 (1506 connections now open) 2016-03-13T08:34:53.987+0000 I NETWORK [conn4077076] end connection 10.0.216.90:53660 (1506 connections now open) 2016-03-13T08:34:53.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53674 #4077092 (1506 connections now open) 2016-03-13T08:34:53.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53675 #4077093 (1507 connections now open) 2016-03-13T08:34:54.458+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:54.458+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:54.458+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:54.458+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:34:54.764+0000 I NETWORK [conn4077078] end connection 10.0.20.53:46906 (1506 connections now open) 2016-03-13T08:34:54.764+0000 I NETWORK [conn4077079] end connection 10.0.20.53:46907 (1506 connections now open) 2016-03-13T08:34:54.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46926 #4077094 (1506 connections now open) 2016-03-13T08:34:54.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46927 #4077095 (1507 connections now open) 2016-03-13T08:34:54.959+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:54.960+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:54.960+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:54.960+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:34:55.461+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:55.462+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:55.462+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:55.462+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:34:55.963+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:55.963+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:55.963+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:55.963+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:34:56.464+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:56.465+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:56.465+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:56.465+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:34:56.966+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:56.967+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:56.967+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:56.967+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:34:57.275+0000 I NETWORK [conn4077084] end connection 10.0.179.209:36767 (1506 connections now open) 2016-03-13T08:34:57.275+0000 I NETWORK [conn4077085] end connection 10.0.179.209:36766 (1506 connections now open) 2016-03-13T08:34:57.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36786 #4077096 (1506 connections now open) 2016-03-13T08:34:57.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36785 #4077097 (1507 connections now open) 2016-03-13T08:34:57.468+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:57.468+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:57.468+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:57.468+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:34:57.741+0000 I NETWORK [conn4077087] end connection 10.0.26.69:38686 (1506 connections now open) 2016-03-13T08:34:57.741+0000 I NETWORK [conn4077086] end connection 10.0.26.69:38685 (1506 connections now open) 2016-03-13T08:34:57.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38693 #4077098 (1506 connections now open) 2016-03-13T08:34:57.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38694 #4077099 (1507 connections now open) 2016-03-13T08:34:57.969+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:34:57.970+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:34:57.970+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:34:57.970+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:34:57.970+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:34:57.970+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:34:57.970+0000-56e5263118fa188537623c7e", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858097970), what: "balancer.round", ns: "", details: { executionTimeMillis: 10585, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:34:58.113+0000 I NETWORK [conn4064812] end connection 10.0.20.53:34165 (1506 connections now open) 2016-03-13T08:34:58.114+0000 I NETWORK [conn4069322] end connection 10.0.216.90:45322 (1505 connections now open) 2016-03-13T08:34:58.114+0000 I NETWORK [conn4075956] end connection 10.0.26.69:37470 (1504 connections now open) 2016-03-13T08:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46931 #4077100 (1505 connections now open) 2016-03-13T08:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53692 #4077101 (1506 connections now open) 2016-03-13T08:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38697 #4077102 (1507 connections now open) 2016-03-13T08:34:58.115+0000 I NETWORK [conn4067063] end connection 10.0.27.237:49904 (1506 connections now open) 2016-03-13T08:34:58.115+0000 I NETWORK [conn4072074] end connection 10.0.7.113:50270 (1505 connections now open) 2016-03-13T08:34:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60285 #4077103 (1506 connections now open) 2016-03-13T08:34:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55642 #4077104 (1507 connections now open) 2016-03-13T08:35:00.763+0000 I NETWORK [conn4077089] end connection 10.0.27.237:60274 (1506 connections now open) 2016-03-13T08:35:00.763+0000 I NETWORK [conn4077088] end connection 10.0.27.237:60273 (1506 connections now open) 2016-03-13T08:35:00.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60287 #4077105 (1506 connections now open) 2016-03-13T08:35:00.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60288 #4077106 (1507 connections now open) 2016-03-13T08:35:02.640+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:02.641+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:02.641+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:35:02.641+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:35:02.795+0000 I NETWORK [conn4077090] end connection 10.0.7.113:55634 (1506 connections now open) 2016-03-13T08:35:02.795+0000 I NETWORK [conn4077091] end connection 10.0.7.113:55635 (1506 connections now open) 2016-03-13T08:35:02.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55647 #4077107 (1506 connections now open) 2016-03-13T08:35:02.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55648 #4077108 (1507 connections now open) 2016-03-13T08:35:04.009+0000 I NETWORK [conn4077093] end connection 10.0.216.90:53675 (1506 connections now open) 2016-03-13T08:35:04.009+0000 I NETWORK [conn4077092] end connection 10.0.216.90:53674 (1506 connections now open) 2016-03-13T08:35:04.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53698 #4077109 (1506 connections now open) 2016-03-13T08:35:04.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53699 #4077110 (1507 connections now open) 2016-03-13T08:35:04.784+0000 I NETWORK [conn4077095] end connection 10.0.20.53:46927 (1506 connections now open) 2016-03-13T08:35:04.784+0000 I NETWORK [conn4077094] end connection 10.0.20.53:46926 (1506 connections now open) 2016-03-13T08:35:04.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46938 #4077111 (1506 connections now open) 2016-03-13T08:35:04.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46939 #4077112 (1507 connections now open) 2016-03-13T08:35:07.287+0000 I NETWORK [conn4077097] end connection 10.0.179.209:36785 (1506 connections now open) 2016-03-13T08:35:07.287+0000 I NETWORK [conn4077096] end connection 10.0.179.209:36786 (1506 connections now open) 2016-03-13T08:35:07.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36799 #4077113 (1506 connections now open) 2016-03-13T08:35:07.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36798 #4077114 (1507 connections now open) 2016-03-13T08:35:07.754+0000 I NETWORK [conn4077098] end connection 10.0.26.69:38693 (1506 connections now open) 2016-03-13T08:35:07.754+0000 I NETWORK [conn4077099] end connection 10.0.26.69:38694 (1506 connections now open) 2016-03-13T08:35:07.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38712 #4077115 (1506 connections now open) 2016-03-13T08:35:07.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38711 #4077116 (1507 connections now open) 2016-03-13T08:35:08.033+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:08.033+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:08.033+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:08.033+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:35:08.534+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:08.535+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:08.535+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:08.535+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:35:09.036+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:09.036+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:09.036+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:09.036+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:35:09.537+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:09.538+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:09.538+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:09.538+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:35:10.039+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:10.039+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:10.039+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:10.039+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:35:10.540+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:10.541+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:10.541+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:10.541+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:35:10.541+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:35:10.541+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:35:10.541+0000-56e5263e18fa188537623c7f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858110541), what: "balancer.round", ns: "", details: { executionTimeMillis: 2541, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:35:10.776+0000 I NETWORK [conn4077106] end connection 10.0.27.237:60288 (1506 connections now open) 2016-03-13T08:35:10.776+0000 I NETWORK [conn4077105] end connection 10.0.27.237:60287 (1506 connections now open) 2016-03-13T08:35:10.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60307 #4077117 (1506 connections now open) 2016-03-13T08:35:10.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60308 #4077118 (1507 connections now open) 2016-03-13T08:35:11.431+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:35:11.181+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:35:12.643+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:35:12.809+0000 I NETWORK [conn4077107] end connection 10.0.7.113:55647 (1506 connections now open) 2016-03-13T08:35:12.809+0000 I NETWORK [conn4077108] end connection 10.0.7.113:55648 (1506 connections now open) 2016-03-13T08:35:12.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55661 #4077119 (1506 connections now open) 2016-03-13T08:35:12.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55662 #4077120 (1507 connections now open) 2016-03-13T08:35:14.022+0000 I NETWORK [conn4077109] end connection 10.0.216.90:53698 (1506 connections now open) 2016-03-13T08:35:14.022+0000 I NETWORK [conn4077110] end connection 10.0.216.90:53699 (1506 connections now open) 2016-03-13T08:35:14.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53706 #4077121 (1506 connections now open) 2016-03-13T08:35:14.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53707 #4077122 (1507 connections now open) 2016-03-13T08:35:14.798+0000 I NETWORK [conn4077111] end connection 10.0.20.53:46938 (1506 connections now open) 2016-03-13T08:35:14.798+0000 I NETWORK [conn4077112] end connection 10.0.20.53:46939 (1506 connections now open) 2016-03-13T08:35:14.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46950 #4077123 (1506 connections now open) 2016-03-13T08:35:14.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46951 #4077124 (1507 connections now open) 2016-03-13T08:35:16.118+0000 I NETWORK [conn4065125] end connection 10.0.20.53:34471 (1506 connections now open) 2016-03-13T08:35:16.118+0000 I NETWORK [conn4069582] end connection 10.0.26.69:58839 (1506 connections now open) 2016-03-13T08:35:16.118+0000 I NETWORK [conn4073836] end connection 10.0.27.237:57017 (1506 connections now open) 2016-03-13T08:35:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46954 #4077125 (1505 connections now open) 2016-03-13T08:35:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38733 #4077126 (1506 connections now open) 2016-03-13T08:35:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60317 #4077127 (1507 connections now open) 2016-03-13T08:35:17.300+0000 I NETWORK [conn4077113] end connection 10.0.179.209:36799 (1506 connections now open) 2016-03-13T08:35:17.301+0000 I NETWORK [conn4077114] end connection 10.0.179.209:36798 (1505 connections now open) 2016-03-13T08:35:17.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36814 #4077128 (1506 connections now open) 2016-03-13T08:35:17.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36815 #4077129 (1507 connections now open) 2016-03-13T08:35:17.767+0000 I NETWORK [conn4077116] end connection 10.0.26.69:38711 (1506 connections now open) 2016-03-13T08:35:17.767+0000 I NETWORK [conn4077115] end connection 10.0.26.69:38712 (1506 connections now open) 2016-03-13T08:35:17.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38734 #4077130 (1507 connections now open) 2016-03-13T08:35:17.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38735 #4077131 (1507 connections now open) 2016-03-13T08:35:20.585+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:35:20.587+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:20.595+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:20.595+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:20.595+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:35:20.789+0000 I NETWORK [conn4077117] end connection 10.0.27.237:60307 (1506 connections now open) 2016-03-13T08:35:20.789+0000 I NETWORK [conn4077118] end connection 10.0.27.237:60308 (1506 connections now open) 2016-03-13T08:35:20.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60318 #4077132 (1506 connections now open) 2016-03-13T08:35:20.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60319 #4077133 (1507 connections now open) 2016-03-13T08:35:21.096+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:21.097+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:21.097+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:21.097+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:35:21.597+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:21.598+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:21.598+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:21.598+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:35:22.099+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:22.100+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:22.100+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:22.100+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:35:22.601+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:22.601+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:22.601+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:22.601+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:35:22.646+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:22.647+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:22.647+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:35:22.647+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:35:22.822+0000 I NETWORK [conn4077120] end connection 10.0.7.113:55662 (1506 connections now open) 2016-03-13T08:35:22.822+0000 I NETWORK [conn4077119] end connection 10.0.7.113:55661 (1505 connections now open) 2016-03-13T08:35:22.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55683 #4077134 (1507 connections now open) 2016-03-13T08:35:22.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55684 #4077135 (1507 connections now open) 2016-03-13T08:35:23.102+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:23.103+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:23.103+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:23.103+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:35:23.604+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:23.605+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:23.605+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:23.605+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:35:24.035+0000 I NETWORK [conn4077121] end connection 10.0.216.90:53706 (1506 connections now open) 2016-03-13T08:35:24.035+0000 I NETWORK [conn4077122] end connection 10.0.216.90:53707 (1506 connections now open) 2016-03-13T08:35:24.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53719 #4077136 (1506 connections now open) 2016-03-13T08:35:24.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53720 #4077137 (1507 connections now open) 2016-03-13T08:35:24.105+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:24.106+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:24.106+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:24.106+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:35:24.607+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:24.608+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:24.608+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:24.608+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:35:24.812+0000 I NETWORK [conn4077124] end connection 10.0.20.53:46951 (1506 connections now open) 2016-03-13T08:35:24.812+0000 I NETWORK [conn4077123] end connection 10.0.20.53:46950 (1506 connections now open) 2016-03-13T08:35:24.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46970 #4077138 (1506 connections now open) 2016-03-13T08:35:24.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46971 #4077139 (1507 connections now open) 2016-03-13T08:35:25.109+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:25.109+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:25.109+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:25.109+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:35:25.610+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:25.611+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:25.611+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:25.611+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:35:26.112+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:26.112+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:26.112+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:26.112+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:35:26.613+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:26.614+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:26.614+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:26.614+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:35:27.115+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:27.115+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:27.115+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:27.115+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:35:27.313+0000 I NETWORK [conn4077129] end connection 10.0.179.209:36815 (1506 connections now open) 2016-03-13T08:35:27.313+0000 I NETWORK [conn4077128] end connection 10.0.179.209:36814 (1505 connections now open) 2016-03-13T08:35:27.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36833 #4077140 (1506 connections now open) 2016-03-13T08:35:27.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36834 #4077141 (1507 connections now open) 2016-03-13T08:35:27.616+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:27.617+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:27.617+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:27.617+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:35:27.781+0000 I NETWORK [conn4077131] end connection 10.0.26.69:38735 (1506 connections now open) 2016-03-13T08:35:27.781+0000 I NETWORK [conn4077130] end connection 10.0.26.69:38734 (1506 connections now open) 2016-03-13T08:35:27.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38742 #4077142 (1506 connections now open) 2016-03-13T08:35:27.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38743 #4077143 (1507 connections now open) 2016-03-13T08:35:28.113+0000 I NETWORK [conn4064898] end connection 10.0.20.53:34247 (1506 connections now open) 2016-03-13T08:35:28.113+0000 I NETWORK [conn4069494] end connection 10.0.216.90:45504 (1505 connections now open) 2016-03-13T08:35:28.113+0000 I NETWORK [conn4069340] end connection 10.0.26.69:58589 (1504 connections now open) 2016-03-13T08:35:28.113+0000 I NETWORK [conn4068833] end connection 10.0.27.237:51721 (1503 connections now open) 2016-03-13T08:35:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46974 #4077144 (1504 connections now open) 2016-03-13T08:35:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53736 #4077145 (1505 connections now open) 2016-03-13T08:35:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38746 #4077146 (1506 connections now open) 2016-03-13T08:35:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60331 #4077147 (1507 connections now open) 2016-03-13T08:35:28.118+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:28.118+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:28.119+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:28.119+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:35:28.619+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:28.620+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:28.620+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:28.620+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:35:29.121+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:29.122+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:29.122+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:29.122+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:35:29.623+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:29.623+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:29.623+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:29.623+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:35:30.124+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:30.125+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:30.125+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:30.125+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:35:30.626+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:30.627+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:30.627+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:30.627+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:35:30.802+0000 I NETWORK [conn4077132] end connection 10.0.27.237:60318 (1506 connections now open) 2016-03-13T08:35:30.802+0000 I NETWORK [conn4077133] end connection 10.0.27.237:60319 (1506 connections now open) 2016-03-13T08:35:30.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60334 #4077148 (1506 connections now open) 2016-03-13T08:35:30.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60333 #4077149 (1507 connections now open) 2016-03-13T08:35:31.128+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:31.128+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:31.128+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:31.128+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:35:31.128+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:35:31.128+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:35:31.128+0000-56e5265318fa188537623c80", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858131128), what: "balancer.round", ns: "", details: { executionTimeMillis: 10573, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:35:32.650+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:32.650+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:32.650+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:35:32.650+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:35:32.835+0000 I NETWORK [conn4077135] end connection 10.0.7.113:55684 (1506 connections now open) 2016-03-13T08:35:32.835+0000 I NETWORK [conn4077134] end connection 10.0.7.113:55683 (1506 connections now open) 2016-03-13T08:35:32.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55694 #4077150 (1506 connections now open) 2016-03-13T08:35:32.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55693 #4077151 (1507 connections now open) 2016-03-13T08:35:34.048+0000 I NETWORK [conn4077137] end connection 10.0.216.90:53720 (1506 connections now open) 2016-03-13T08:35:34.048+0000 I NETWORK [conn4077136] end connection 10.0.216.90:53719 (1506 connections now open) 2016-03-13T08:35:34.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53743 #4077152 (1506 connections now open) 2016-03-13T08:35:34.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53742 #4077153 (1507 connections now open) 2016-03-13T08:35:34.825+0000 I NETWORK [conn4077139] end connection 10.0.20.53:46971 (1506 connections now open) 2016-03-13T08:35:34.825+0000 I NETWORK [conn4077138] end connection 10.0.20.53:46970 (1506 connections now open) 2016-03-13T08:35:34.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46983 #4077154 (1506 connections now open) 2016-03-13T08:35:34.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46982 #4077155 (1507 connections now open) 2016-03-13T08:35:37.326+0000 I NETWORK [conn4077140] end connection 10.0.179.209:36833 (1506 connections now open) 2016-03-13T08:35:37.326+0000 I NETWORK [conn4077141] end connection 10.0.179.209:36834 (1505 connections now open) 2016-03-13T08:35:37.326+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36842 #4077156 (1506 connections now open) 2016-03-13T08:35:37.326+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36843 #4077157 (1507 connections now open) 2016-03-13T08:35:37.794+0000 I NETWORK [conn4077143] end connection 10.0.26.69:38743 (1506 connections now open) 2016-03-13T08:35:37.794+0000 I NETWORK [conn4077142] end connection 10.0.26.69:38742 (1506 connections now open) 2016-03-13T08:35:37.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38758 #4077158 (1506 connections now open) 2016-03-13T08:35:37.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38757 #4077159 (1507 connections now open) 2016-03-13T08:35:40.815+0000 I NETWORK [conn4077149] end connection 10.0.27.237:60333 (1506 connections now open) 2016-03-13T08:35:40.815+0000 I NETWORK [conn4077148] end connection 10.0.27.237:60334 (1505 connections now open) 2016-03-13T08:35:40.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60351 #4077160 (1506 connections now open) 2016-03-13T08:35:40.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60352 #4077161 (1507 connections now open) 2016-03-13T08:35:41.201+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:41.201+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:41.201+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:41.201+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:35:41.702+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:41.703+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:41.703+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:41.703+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:35:41.709+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:35:41.431+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:35:42.204+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:42.204+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:42.204+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:42.204+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:35:42.653+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:42.654+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:42.654+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:35:42.654+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:35:42.705+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:42.706+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:42.706+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:42.706+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:35:42.848+0000 I NETWORK [conn4077151] end connection 10.0.7.113:55693 (1506 connections now open) 2016-03-13T08:35:42.848+0000 I NETWORK [conn4077150] end connection 10.0.7.113:55694 (1506 connections now open) 2016-03-13T08:35:42.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55704 #4077162 (1506 connections now open) 2016-03-13T08:35:42.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55705 #4077163 (1507 connections now open) 2016-03-13T08:35:43.207+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:43.207+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:43.208+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:43.208+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:35:43.208+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:35:43.208+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:35:43.208+0000-56e5265f18fa188537623c81", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858143208), what: "balancer.round", ns: "", details: { executionTimeMillis: 2044, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:35:44.061+0000 I NETWORK [conn4077153] end connection 10.0.216.90:53742 (1506 connections now open) 2016-03-13T08:35:44.061+0000 I NETWORK [conn4077152] end connection 10.0.216.90:53743 (1506 connections now open) 2016-03-13T08:35:44.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53751 #4077164 (1506 connections now open) 2016-03-13T08:35:44.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53750 #4077165 (1507 connections now open) 2016-03-13T08:35:44.839+0000 I NETWORK [conn4077154] end connection 10.0.20.53:46983 (1506 connections now open) 2016-03-13T08:35:44.839+0000 I NETWORK [conn4077155] end connection 10.0.20.53:46982 (1506 connections now open) 2016-03-13T08:35:44.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46993 #4077166 (1506 connections now open) 2016-03-13T08:35:44.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:46994 #4077167 (1507 connections now open) 2016-03-13T08:35:46.119+0000 I NETWORK [conn4066913] end connection 10.0.179.209:54687 (1506 connections now open) 2016-03-13T08:35:46.119+0000 I NETWORK [conn4065363] end connection 10.0.26.69:54313 (1506 connections now open) 2016-03-13T08:35:46.119+0000 I NETWORK [conn4068496] end connection 10.0.7.113:46422 (1505 connections now open) 2016-03-13T08:35:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38777 #4077168 (1505 connections now open) 2016-03-13T08:35:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36853 #4077169 (1506 connections now open) 2016-03-13T08:35:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55711 #4077170 (1507 connections now open) 2016-03-13T08:35:47.339+0000 I NETWORK [conn4077156] end connection 10.0.179.209:36842 (1506 connections now open) 2016-03-13T08:35:47.339+0000 I NETWORK [conn4077157] end connection 10.0.179.209:36843 (1506 connections now open) 2016-03-13T08:35:47.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36855 #4077171 (1506 connections now open) 2016-03-13T08:35:47.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36854 #4077172 (1507 connections now open) 2016-03-13T08:35:47.816+0000 I NETWORK [conn4077158] end connection 10.0.26.69:38758 (1506 connections now open) 2016-03-13T08:35:47.816+0000 I NETWORK [conn4077159] end connection 10.0.26.69:38757 (1506 connections now open) 2016-03-13T08:35:47.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38778 #4077173 (1506 connections now open) 2016-03-13T08:35:47.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38779 #4077174 (1507 connections now open) 2016-03-13T08:35:50.829+0000 I NETWORK [conn4077161] end connection 10.0.27.237:60352 (1506 connections now open) 2016-03-13T08:35:50.829+0000 I NETWORK [conn4077160] end connection 10.0.27.237:60351 (1506 connections now open) 2016-03-13T08:35:50.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60361 #4077175 (1506 connections now open) 2016-03-13T08:35:50.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60360 #4077176 (1507 connections now open) 2016-03-13T08:35:52.656+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:35:52.861+0000 I NETWORK [conn4077163] end connection 10.0.7.113:55705 (1506 connections now open) 2016-03-13T08:35:52.862+0000 I NETWORK [conn4077162] end connection 10.0.7.113:55704 (1505 connections now open) 2016-03-13T08:35:52.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55722 #4077177 (1506 connections now open) 2016-03-13T08:35:52.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55723 #4077178 (1507 connections now open) 2016-03-13T08:35:53.264+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:35:53.266+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:53.275+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:53.275+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:53.275+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:35:53.776+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:53.777+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:53.777+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:53.777+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:35:54.074+0000 I NETWORK [conn4077165] end connection 10.0.216.90:53750 (1506 connections now open) 2016-03-13T08:35:54.074+0000 I NETWORK [conn4077164] end connection 10.0.216.90:53751 (1506 connections now open) 2016-03-13T08:35:54.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53765 #4077179 (1506 connections now open) 2016-03-13T08:35:54.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53764 #4077180 (1507 connections now open) 2016-03-13T08:35:54.278+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:54.278+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:54.278+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:54.278+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:35:54.779+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:54.780+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:54.780+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:54.780+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:35:54.853+0000 I NETWORK [conn4077167] end connection 10.0.20.53:46994 (1506 connections now open) 2016-03-13T08:35:54.853+0000 I NETWORK [conn4077166] end connection 10.0.20.53:46993 (1506 connections now open) 2016-03-13T08:35:54.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47013 #4077181 (1506 connections now open) 2016-03-13T08:35:54.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47014 #4077182 (1507 connections now open) 2016-03-13T08:35:55.281+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:55.281+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:55.281+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:55.281+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:35:55.782+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:55.783+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:55.783+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:55.783+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:35:56.284+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:56.285+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:56.285+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:56.285+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:35:56.786+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:56.786+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:56.786+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:56.786+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:35:57.287+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:57.288+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:57.288+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:57.288+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:35:57.360+0000 I NETWORK [conn4077172] end connection 10.0.179.209:36854 (1506 connections now open) 2016-03-13T08:35:57.360+0000 I NETWORK [conn4077171] end connection 10.0.179.209:36855 (1506 connections now open) 2016-03-13T08:35:57.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36872 #4077183 (1506 connections now open) 2016-03-13T08:35:57.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36873 #4077184 (1507 connections now open) 2016-03-13T08:35:57.789+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:57.790+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:57.790+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:57.790+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:35:57.830+0000 I NETWORK [conn4077174] end connection 10.0.26.69:38779 (1506 connections now open) 2016-03-13T08:35:57.830+0000 I NETWORK [conn4077173] end connection 10.0.26.69:38778 (1506 connections now open) 2016-03-13T08:35:57.831+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38787 #4077185 (1506 connections now open) 2016-03-13T08:35:57.831+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38788 #4077186 (1507 connections now open) 2016-03-13T08:35:58.113+0000 I NETWORK [conn4067524] end connection 10.0.179.209:55336 (1506 connections now open) 2016-03-13T08:35:58.113+0000 I NETWORK [conn4070519] end connection 10.0.20.53:40030 (1505 connections now open) 2016-03-13T08:35:58.113+0000 I NETWORK [conn4075070] end connection 10.0.26.69:36510 (1504 connections now open) 2016-03-13T08:35:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36875 #4077187 (1505 connections now open) 2016-03-13T08:35:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47018 #4077188 (1506 connections now open) 2016-03-13T08:35:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38790 #4077189 (1507 connections now open) 2016-03-13T08:35:58.291+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:58.291+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:58.291+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:58.291+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:35:58.792+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:58.793+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:58.793+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:58.793+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:35:59.294+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:59.294+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:59.294+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:59.294+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:35:59.795+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:35:59.796+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:35:59.796+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:35:59.796+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:36:00.297+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:00.297+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:00.298+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:00.298+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:36:00.798+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:00.799+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:00.799+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:00.799+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:36:00.851+0000 I NETWORK [conn4077176] end connection 10.0.27.237:60360 (1506 connections now open) 2016-03-13T08:36:00.851+0000 I NETWORK [conn4077175] end connection 10.0.27.237:60361 (1506 connections now open) 2016-03-13T08:36:00.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60375 #4077190 (1506 connections now open) 2016-03-13T08:36:00.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60376 #4077191 (1507 connections now open) 2016-03-13T08:36:01.300+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:01.301+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:01.301+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:01.301+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:36:01.802+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:01.802+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:01.802+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:01.802+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:36:02.303+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:02.304+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:02.304+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:02.304+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:36:02.659+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:02.659+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:02.659+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:36:02.659+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:36:02.805+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:02.806+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:02.806+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:02.806+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:36:02.884+0000 I NETWORK [conn4077178] end connection 10.0.7.113:55723 (1506 connections now open) 2016-03-13T08:36:02.884+0000 I NETWORK [conn4077177] end connection 10.0.7.113:55722 (1506 connections now open) 2016-03-13T08:36:02.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55736 #4077192 (1506 connections now open) 2016-03-13T08:36:02.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55737 #4077193 (1507 connections now open) 2016-03-13T08:36:03.307+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:03.307+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:03.307+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:03.307+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:36:03.808+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:03.809+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:03.809+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:03.809+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:36:03.809+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:36:03.809+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:36:03.809+0000-56e5267318fa188537623c82", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858163809), what: "balancer.round", ns: "", details: { executionTimeMillis: 10580, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:36:04.096+0000 I NETWORK [conn4077179] end connection 10.0.216.90:53765 (1506 connections now open) 2016-03-13T08:36:04.096+0000 I NETWORK [conn4077180] end connection 10.0.216.90:53764 (1506 connections now open) 2016-03-13T08:36:04.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53788 #4077194 (1506 connections now open) 2016-03-13T08:36:04.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53789 #4077195 (1507 connections now open) 2016-03-13T08:36:04.874+0000 I NETWORK [conn4077182] end connection 10.0.20.53:47014 (1506 connections now open) 2016-03-13T08:36:04.874+0000 I NETWORK [conn4077181] end connection 10.0.20.53:47013 (1506 connections now open) 2016-03-13T08:36:04.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47025 #4077196 (1506 connections now open) 2016-03-13T08:36:04.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47024 #4077197 (1507 connections now open) 2016-03-13T08:36:07.373+0000 I NETWORK [conn4077184] end connection 10.0.179.209:36873 (1506 connections now open) 2016-03-13T08:36:07.373+0000 I NETWORK [conn4077183] end connection 10.0.179.209:36872 (1506 connections now open) 2016-03-13T08:36:07.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36883 #4077198 (1506 connections now open) 2016-03-13T08:36:07.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36884 #4077199 (1507 connections now open) 2016-03-13T08:36:07.844+0000 I NETWORK [conn4077186] end connection 10.0.26.69:38788 (1506 connections now open) 2016-03-13T08:36:07.844+0000 I NETWORK [conn4077185] end connection 10.0.26.69:38787 (1506 connections now open) 2016-03-13T08:36:07.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38805 #4077200 (1507 connections now open) 2016-03-13T08:36:07.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38804 #4077201 (1507 connections now open) 2016-03-13T08:36:10.864+0000 I NETWORK [conn4077191] end connection 10.0.27.237:60376 (1506 connections now open) 2016-03-13T08:36:10.864+0000 I NETWORK [conn4077190] end connection 10.0.27.237:60375 (1506 connections now open) 2016-03-13T08:36:10.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60396 #4077202 (1506 connections now open) 2016-03-13T08:36:10.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60395 #4077203 (1507 connections now open) 2016-03-13T08:36:12.039+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:36:11.709+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:36:12.662+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:12.663+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:12.663+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:36:12.663+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:36:12.897+0000 I NETWORK [conn4077192] end connection 10.0.7.113:55736 (1506 connections now open) 2016-03-13T08:36:12.897+0000 I NETWORK [conn4077193] end connection 10.0.7.113:55737 (1506 connections now open) 2016-03-13T08:36:12.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55747 #4077204 (1507 connections now open) 2016-03-13T08:36:12.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55748 #4077205 (1507 connections now open) 2016-03-13T08:36:13.854+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:13.855+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:13.855+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:13.855+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:36:14.109+0000 I NETWORK [conn4077194] end connection 10.0.216.90:53788 (1506 connections now open) 2016-03-13T08:36:14.109+0000 I NETWORK [conn4077195] end connection 10.0.216.90:53789 (1506 connections now open) 2016-03-13T08:36:14.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53796 #4077206 (1506 connections now open) 2016-03-13T08:36:14.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53797 #4077207 (1507 connections now open) 2016-03-13T08:36:14.356+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:14.357+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:14.357+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:14.357+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:36:14.857+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:14.858+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:14.858+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:14.858+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:36:14.888+0000 I NETWORK [conn4077196] end connection 10.0.20.53:47025 (1506 connections now open) 2016-03-13T08:36:14.888+0000 I NETWORK [conn4077197] end connection 10.0.20.53:47024 (1506 connections now open) 2016-03-13T08:36:14.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47036 #4077208 (1506 connections now open) 2016-03-13T08:36:14.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47037 #4077209 (1507 connections now open) 2016-03-13T08:36:15.359+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:15.360+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:15.360+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:15.360+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:36:15.861+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:15.861+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:15.861+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:15.861+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:36:16.118+0000 I NETWORK [conn4065086] end connection 10.0.20.53:34428 (1506 connections now open) 2016-03-13T08:36:16.118+0000 I NETWORK [conn4068069] end connection 10.0.216.90:43983 (1506 connections now open) 2016-03-13T08:36:16.118+0000 I NETWORK [conn4069664] end connection 10.0.26.69:58928 (1504 connections now open) 2016-03-13T08:36:16.118+0000 I NETWORK [conn4066158] end connection 10.0.27.237:49015 (1503 connections now open) 2016-03-13T08:36:16.118+0000 I NETWORK [conn4070647] end connection 10.0.7.113:48760 (1502 connections now open) 2016-03-13T08:36:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47039 #4077210 (1503 connections now open) 2016-03-13T08:36:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53804 #4077211 (1504 connections now open) 2016-03-13T08:36:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38826 #4077212 (1505 connections now open) 2016-03-13T08:36:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60406 #4077213 (1506 connections now open) 2016-03-13T08:36:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55758 #4077214 (1507 connections now open) 2016-03-13T08:36:16.362+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:16.363+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:16.363+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:16.363+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:36:16.363+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:36:16.363+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:36:16.363+0000-56e5268018fa188537623c83", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858176363), what: "balancer.round", ns: "", details: { executionTimeMillis: 2518, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:36:17.387+0000 I NETWORK [conn4077198] end connection 10.0.179.209:36883 (1506 connections now open) 2016-03-13T08:36:17.387+0000 I NETWORK [conn4077199] end connection 10.0.179.209:36884 (1506 connections now open) 2016-03-13T08:36:17.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36898 #4077215 (1506 connections now open) 2016-03-13T08:36:17.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36899 #4077216 (1507 connections now open) 2016-03-13T08:36:17.857+0000 I NETWORK [conn4077200] end connection 10.0.26.69:38805 (1506 connections now open) 2016-03-13T08:36:17.857+0000 I NETWORK [conn4077201] end connection 10.0.26.69:38804 (1506 connections now open) 2016-03-13T08:36:17.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38827 #4077217 (1506 connections now open) 2016-03-13T08:36:17.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38828 #4077218 (1507 connections now open) 2016-03-13T08:36:20.877+0000 I NETWORK [conn4077203] end connection 10.0.27.237:60395 (1506 connections now open) 2016-03-13T08:36:20.877+0000 I NETWORK [conn4077202] end connection 10.0.27.237:60396 (1506 connections now open) 2016-03-13T08:36:20.878+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60408 #4077219 (1506 connections now open) 2016-03-13T08:36:20.878+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60407 #4077220 (1507 connections now open) 2016-03-13T08:36:22.665+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:36:22.910+0000 I NETWORK [conn4077205] end connection 10.0.7.113:55748 (1506 connections now open) 2016-03-13T08:36:22.910+0000 I NETWORK [conn4077204] end connection 10.0.7.113:55747 (1506 connections now open) 2016-03-13T08:36:22.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55762 #4077221 (1506 connections now open) 2016-03-13T08:36:22.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55763 #4077222 (1507 connections now open) 2016-03-13T08:36:24.122+0000 I NETWORK [conn4077206] end connection 10.0.216.90:53796 (1506 connections now open) 2016-03-13T08:36:24.122+0000 I NETWORK [conn4077207] end connection 10.0.216.90:53797 (1506 connections now open) 2016-03-13T08:36:24.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53811 #4077223 (1506 connections now open) 2016-03-13T08:36:24.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53810 #4077224 (1507 connections now open) 2016-03-13T08:36:24.902+0000 I NETWORK [conn4077209] end connection 10.0.20.53:47037 (1506 connections now open) 2016-03-13T08:36:24.902+0000 I NETWORK [conn4077208] end connection 10.0.20.53:47036 (1506 connections now open) 2016-03-13T08:36:24.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47056 #4077225 (1506 connections now open) 2016-03-13T08:36:24.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47057 #4077226 (1507 connections now open) 2016-03-13T08:36:26.413+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:36:26.416+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:26.425+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:26.425+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:26.425+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:36:26.926+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:26.927+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:26.927+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:26.927+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:36:27.399+0000 I NETWORK [conn4077216] end connection 10.0.179.209:36899 (1506 connections now open) 2016-03-13T08:36:27.399+0000 I NETWORK [conn4077215] end connection 10.0.179.209:36898 (1506 connections now open) 2016-03-13T08:36:27.400+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36916 #4077227 (1506 connections now open) 2016-03-13T08:36:27.400+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36917 #4077228 (1507 connections now open) 2016-03-13T08:36:27.428+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:27.428+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:27.428+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:27.428+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:36:27.870+0000 I NETWORK [conn4077217] end connection 10.0.26.69:38827 (1506 connections now open) 2016-03-13T08:36:27.870+0000 I NETWORK [conn4077218] end connection 10.0.26.69:38828 (1506 connections now open) 2016-03-13T08:36:27.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38835 #4077229 (1506 connections now open) 2016-03-13T08:36:27.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38836 #4077230 (1507 connections now open) 2016-03-13T08:36:27.929+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:27.930+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:27.930+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:27.930+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:36:28.112+0000 I NETWORK [conn4066429] end connection 10.0.179.209:54184 (1506 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [conn4065611] end connection 10.0.20.53:34962 (1505 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [conn4066593] end connection 10.0.26.69:55654 (1505 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [conn4074516] end connection 10.0.27.237:57718 (1503 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36920 #4077231 (1504 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [conn4070166] end connection 10.0.7.113:48240 (1502 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38839 #4077232 (1504 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47060 #4077233 (1505 connections now open) 2016-03-13T08:36:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60418 #4077234 (1506 connections now open) 2016-03-13T08:36:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55779 #4077235 (1507 connections now open) 2016-03-13T08:36:28.431+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:28.431+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:28.432+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:28.432+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:36:28.932+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:28.933+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:28.933+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:28.933+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:36:29.434+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:29.435+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:29.435+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:29.435+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:36:29.936+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:29.936+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:29.936+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:29.936+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:36:30.437+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:30.438+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:30.438+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:30.438+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:36:30.890+0000 I NETWORK [conn4077220] end connection 10.0.27.237:60407 (1506 connections now open) 2016-03-13T08:36:30.890+0000 I NETWORK [conn4077219] end connection 10.0.27.237:60408 (1506 connections now open) 2016-03-13T08:36:30.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60420 #4077236 (1506 connections now open) 2016-03-13T08:36:30.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60421 #4077237 (1507 connections now open) 2016-03-13T08:36:30.939+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:30.939+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:30.939+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:30.939+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:36:31.440+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:31.441+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:31.442+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:31.442+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:36:31.943+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:31.943+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:31.944+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:31.944+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:36:32.444+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:32.445+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:32.445+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:32.445+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:36:32.668+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:32.669+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:32.669+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:36:32.669+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:36:32.924+0000 I NETWORK [conn4077222] end connection 10.0.7.113:55763 (1506 connections now open) 2016-03-13T08:36:32.924+0000 I NETWORK [conn4077221] end connection 10.0.7.113:55762 (1506 connections now open) 2016-03-13T08:36:32.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55782 #4077238 (1506 connections now open) 2016-03-13T08:36:32.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55783 #4077239 (1507 connections now open) 2016-03-13T08:36:32.946+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:32.947+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:32.947+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:32.947+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:36:33.447+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:33.448+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:33.448+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:33.448+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:36:33.949+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:33.950+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:33.950+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:33.950+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:36:34.136+0000 I NETWORK [conn4077223] end connection 10.0.216.90:53811 (1506 connections now open) 2016-03-13T08:36:34.136+0000 I NETWORK [conn4077224] end connection 10.0.216.90:53810 (1506 connections now open) 2016-03-13T08:36:34.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53836 #4077240 (1506 connections now open) 2016-03-13T08:36:34.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53835 #4077241 (1507 connections now open) 2016-03-13T08:36:34.450+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:34.451+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:34.451+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:34.451+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:36:34.915+0000 I NETWORK [conn4077225] end connection 10.0.20.53:47056 (1506 connections now open) 2016-03-13T08:36:34.915+0000 I NETWORK [conn4077226] end connection 10.0.20.53:47057 (1506 connections now open) 2016-03-13T08:36:34.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47070 #4077242 (1506 connections now open) 2016-03-13T08:36:34.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47069 #4077243 (1507 connections now open) 2016-03-13T08:36:34.952+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:34.953+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:34.953+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:34.953+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:36:35.453+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:35.454+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:35.454+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:35.454+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:36:35.955+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:35.956+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:35.956+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:35.956+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:36:36.457+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:36.457+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:36.457+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:36.457+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:36:36.958+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:36.959+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:36.959+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:36.959+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:36:36.959+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:36:36.959+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:36:36.959+0000-56e5269418fa188537623c84", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858196959), what: "balancer.round", ns: "", details: { executionTimeMillis: 10581, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:36:37.413+0000 I NETWORK [conn4077228] end connection 10.0.179.209:36917 (1506 connections now open) 2016-03-13T08:36:37.413+0000 I NETWORK [conn4077227] end connection 10.0.179.209:36916 (1506 connections now open) 2016-03-13T08:36:37.414+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36926 #4077244 (1506 connections now open) 2016-03-13T08:36:37.414+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36927 #4077245 (1507 connections now open) 2016-03-13T08:36:37.884+0000 I NETWORK [conn4077229] end connection 10.0.26.69:38835 (1506 connections now open) 2016-03-13T08:36:37.884+0000 I NETWORK [conn4077230] end connection 10.0.26.69:38836 (1506 connections now open) 2016-03-13T08:36:37.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38851 #4077246 (1506 connections now open) 2016-03-13T08:36:37.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38850 #4077247 (1507 connections now open) 2016-03-13T08:36:40.903+0000 I NETWORK [conn4077236] end connection 10.0.27.237:60420 (1506 connections now open) 2016-03-13T08:36:40.903+0000 I NETWORK [conn4077237] end connection 10.0.27.237:60421 (1505 connections now open) 2016-03-13T08:36:40.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60436 #4077248 (1506 connections now open) 2016-03-13T08:36:40.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60437 #4077249 (1507 connections now open) 2016-03-13T08:36:42.275+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:36:42.040+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:36:42.672+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:42.672+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:42.672+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:36:42.672+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:36:42.938+0000 I NETWORK [conn4077238] end connection 10.0.7.113:55782 (1506 connections now open) 2016-03-13T08:36:42.938+0000 I NETWORK [conn4077239] end connection 10.0.7.113:55783 (1506 connections now open) 2016-03-13T08:36:42.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55790 #4077250 (1506 connections now open) 2016-03-13T08:36:42.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55791 #4077251 (1507 connections now open) 2016-03-13T08:36:44.148+0000 I NETWORK [conn4077240] end connection 10.0.216.90:53836 (1506 connections now open) 2016-03-13T08:36:44.148+0000 I NETWORK [conn4077241] end connection 10.0.216.90:53835 (1506 connections now open) 2016-03-13T08:36:44.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53842 #4077252 (1506 connections now open) 2016-03-13T08:36:44.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53843 #4077253 (1507 connections now open) 2016-03-13T08:36:44.928+0000 I NETWORK [conn4077243] end connection 10.0.20.53:47069 (1506 connections now open) 2016-03-13T08:36:44.928+0000 I NETWORK [conn4077242] end connection 10.0.20.53:47070 (1506 connections now open) 2016-03-13T08:36:44.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47081 #4077254 (1506 connections now open) 2016-03-13T08:36:44.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47082 #4077255 (1507 connections now open) 2016-03-13T08:36:46.117+0000 I NETWORK [conn4069366] end connection 10.0.20.53:38814 (1506 connections now open) 2016-03-13T08:36:46.118+0000 I NETWORK [conn4068156] end connection 10.0.216.90:44074 (1505 connections now open) 2016-03-13T08:36:46.118+0000 I NETWORK [conn4066037] end connection 10.0.26.69:55047 (1504 connections now open) 2016-03-13T08:36:46.118+0000 I NETWORK [conn4066553] end connection 10.0.7.113:44393 (1503 connections now open) 2016-03-13T08:36:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47084 #4077256 (1504 connections now open) 2016-03-13T08:36:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53850 #4077257 (1505 connections now open) 2016-03-13T08:36:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38870 #4077258 (1506 connections now open) 2016-03-13T08:36:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55800 #4077259 (1507 connections now open) 2016-03-13T08:36:47.018+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:47.018+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:47.019+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:47.019+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:36:47.426+0000 I NETWORK [conn4077244] end connection 10.0.179.209:36926 (1506 connections now open) 2016-03-13T08:36:47.426+0000 I NETWORK [conn4077245] end connection 10.0.179.209:36927 (1506 connections now open) 2016-03-13T08:36:47.428+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36940 #4077260 (1506 connections now open) 2016-03-13T08:36:47.428+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36939 #4077261 (1507 connections now open) 2016-03-13T08:36:47.519+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:47.520+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:47.520+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:47.520+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:36:47.905+0000 I NETWORK [conn4077247] end connection 10.0.26.69:38850 (1506 connections now open) 2016-03-13T08:36:47.905+0000 I NETWORK [conn4077246] end connection 10.0.26.69:38851 (1506 connections now open) 2016-03-13T08:36:47.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38871 #4077262 (1506 connections now open) 2016-03-13T08:36:47.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38872 #4077263 (1507 connections now open) 2016-03-13T08:36:48.021+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:48.022+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:48.022+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:48.022+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:36:48.523+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:48.523+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:48.523+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:48.523+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:36:49.024+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:49.025+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:49.025+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:49.025+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:36:49.526+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:49.526+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:49.526+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:49.526+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:36:49.526+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:36:49.526+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:36:49.526+0000-56e526a118fa188537623c85", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858209526), what: "balancer.round", ns: "", details: { executionTimeMillis: 2541, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:36:50.915+0000 I NETWORK [conn4077248] end connection 10.0.27.237:60436 (1506 connections now open) 2016-03-13T08:36:50.916+0000 I NETWORK [conn4077249] end connection 10.0.27.237:60437 (1506 connections now open) 2016-03-13T08:36:50.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60447 #4077264 (1506 connections now open) 2016-03-13T08:36:50.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60446 #4077265 (1507 connections now open) 2016-03-13T08:36:52.674+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:36:52.952+0000 I NETWORK [conn4077250] end connection 10.0.7.113:55790 (1506 connections now open) 2016-03-13T08:36:52.952+0000 I NETWORK [conn4077251] end connection 10.0.7.113:55791 (1506 connections now open) 2016-03-13T08:36:52.962+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55804 #4077266 (1506 connections now open) 2016-03-13T08:36:52.962+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55805 #4077267 (1507 connections now open) 2016-03-13T08:36:54.161+0000 I NETWORK [conn4077253] end connection 10.0.216.90:53843 (1506 connections now open) 2016-03-13T08:36:54.161+0000 I NETWORK [conn4077252] end connection 10.0.216.90:53842 (1506 connections now open) 2016-03-13T08:36:54.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53857 #4077268 (1506 connections now open) 2016-03-13T08:36:54.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53856 #4077269 (1507 connections now open) 2016-03-13T08:36:54.941+0000 I NETWORK [conn4077254] end connection 10.0.20.53:47081 (1506 connections now open) 2016-03-13T08:36:54.941+0000 I NETWORK [conn4077255] end connection 10.0.20.53:47082 (1506 connections now open) 2016-03-13T08:36:54.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47102 #4077270 (1506 connections now open) 2016-03-13T08:36:54.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47101 #4077271 (1507 connections now open) 2016-03-13T08:36:57.440+0000 I NETWORK [conn4077261] end connection 10.0.179.209:36939 (1506 connections now open) 2016-03-13T08:36:57.440+0000 I NETWORK [conn4077260] end connection 10.0.179.209:36940 (1505 connections now open) 2016-03-13T08:36:57.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36958 #4077272 (1506 connections now open) 2016-03-13T08:36:57.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36957 #4077273 (1507 connections now open) 2016-03-13T08:36:57.919+0000 I NETWORK [conn4077262] end connection 10.0.26.69:38871 (1506 connections now open) 2016-03-13T08:36:57.919+0000 I NETWORK [conn4077263] end connection 10.0.26.69:38872 (1506 connections now open) 2016-03-13T08:36:57.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38879 #4077274 (1506 connections now open) 2016-03-13T08:36:57.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38880 #4077275 (1507 connections now open) 2016-03-13T08:36:58.114+0000 I NETWORK [conn4074819] end connection 10.0.27.237:58008 (1506 connections now open) 2016-03-13T08:36:58.114+0000 I NETWORK [conn4067175] end connection 10.0.7.113:45050 (1506 connections now open) 2016-03-13T08:36:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60458 #4077276 (1507 connections now open) 2016-03-13T08:36:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55821 #4077277 (1507 connections now open) 2016-03-13T08:36:59.572+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:36:59.574+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:36:59.584+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:36:59.584+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:36:59.584+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:37:00.085+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:00.086+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:00.086+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:00.086+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:37:00.587+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:00.588+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:00.588+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:00.588+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:37:00.937+0000 I NETWORK [conn4077265] end connection 10.0.27.237:60446 (1506 connections now open) 2016-03-13T08:37:00.937+0000 I NETWORK [conn4077264] end connection 10.0.27.237:60447 (1506 connections now open) 2016-03-13T08:37:00.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60461 #4077278 (1506 connections now open) 2016-03-13T08:37:00.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60460 #4077279 (1507 connections now open) 2016-03-13T08:37:01.088+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:01.089+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:01.089+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:01.089+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:37:01.590+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:01.591+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:01.591+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:01.591+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:37:02.092+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:02.092+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:02.092+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:02.092+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:37:02.593+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:02.594+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:02.594+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:02.594+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:37:02.677+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:02.678+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:02.678+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:37:02.678+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:37:02.975+0000 I NETWORK [conn4077267] end connection 10.0.7.113:55805 (1506 connections now open) 2016-03-13T08:37:02.975+0000 I NETWORK [conn4077266] end connection 10.0.7.113:55804 (1506 connections now open) 2016-03-13T08:37:02.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55828 #4077280 (1506 connections now open) 2016-03-13T08:37:02.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55827 #4077281 (1507 connections now open) 2016-03-13T08:37:03.095+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:03.095+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:03.095+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:03.095+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:37:03.596+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:03.597+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:03.597+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:03.597+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:37:04.098+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:04.099+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:04.099+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:04.099+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:37:04.182+0000 I NETWORK [conn4077268] end connection 10.0.216.90:53857 (1506 connections now open) 2016-03-13T08:37:04.182+0000 I NETWORK [conn4077269] end connection 10.0.216.90:53856 (1506 connections now open) 2016-03-13T08:37:04.183+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53880 #4077282 (1506 connections now open) 2016-03-13T08:37:04.183+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53881 #4077283 (1507 connections now open) 2016-03-13T08:37:04.600+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:04.600+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:04.600+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:04.600+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:37:04.962+0000 I NETWORK [conn4077271] end connection 10.0.20.53:47101 (1506 connections now open) 2016-03-13T08:37:04.963+0000 I NETWORK [conn4077270] end connection 10.0.20.53:47102 (1506 connections now open) 2016-03-13T08:37:04.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47113 #4077284 (1506 connections now open) 2016-03-13T08:37:04.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47114 #4077285 (1507 connections now open) 2016-03-13T08:37:05.101+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:05.102+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:05.102+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:05.102+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:37:05.603+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:05.604+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:05.604+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:05.604+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:37:06.105+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:06.105+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:06.105+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:06.105+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:37:06.606+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:06.607+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:06.607+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:06.607+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:37:07.108+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:07.108+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:07.108+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:07.108+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:37:07.461+0000 I NETWORK [conn4077272] end connection 10.0.179.209:36958 (1506 connections now open) 2016-03-13T08:37:07.461+0000 I NETWORK [conn4077273] end connection 10.0.179.209:36957 (1505 connections now open) 2016-03-13T08:37:07.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36970 #4077286 (1506 connections now open) 2016-03-13T08:37:07.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36971 #4077287 (1507 connections now open) 2016-03-13T08:37:07.609+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:07.610+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:07.610+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:07.610+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:37:07.932+0000 I NETWORK [conn4077274] end connection 10.0.26.69:38879 (1506 connections now open) 2016-03-13T08:37:07.932+0000 I NETWORK [conn4077275] end connection 10.0.26.69:38880 (1506 connections now open) 2016-03-13T08:37:07.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38895 #4077288 (1506 connections now open) 2016-03-13T08:37:07.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38896 #4077289 (1507 connections now open) 2016-03-13T08:37:08.111+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:08.112+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:08.112+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:08.112+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:37:08.613+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:08.613+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:08.613+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:08.613+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:37:09.114+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:09.115+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:09.115+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:09.115+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:37:09.616+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:09.616+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:09.617+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:09.617+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:37:10.117+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:10.118+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:10.118+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:10.118+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:37:10.118+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:37:10.118+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:37:10.118+0000-56e526b618fa188537623c86", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858230118), what: "balancer.round", ns: "", details: { executionTimeMillis: 10577, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:37:10.950+0000 I NETWORK [conn4077278] end connection 10.0.27.237:60461 (1506 connections now open) 2016-03-13T08:37:10.950+0000 I NETWORK [conn4077279] end connection 10.0.27.237:60460 (1506 connections now open) 2016-03-13T08:37:10.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60479 #4077290 (1506 connections now open) 2016-03-13T08:37:10.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60478 #4077291 (1507 connections now open) 2016-03-13T08:37:12.543+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:37:12.275+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:37:12.681+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:12.681+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:12.681+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:37:12.681+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:37:12.989+0000 I NETWORK [conn4077281] end connection 10.0.7.113:55827 (1506 connections now open) 2016-03-13T08:37:12.989+0000 I NETWORK [conn4077280] end connection 10.0.7.113:55828 (1506 connections now open) 2016-03-13T08:37:12.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55837 #4077292 (1506 connections now open) 2016-03-13T08:37:12.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55838 #4077293 (1507 connections now open) 2016-03-13T08:37:14.195+0000 I NETWORK [conn4077283] end connection 10.0.216.90:53881 (1506 connections now open) 2016-03-13T08:37:14.195+0000 I NETWORK [conn4077282] end connection 10.0.216.90:53880 (1506 connections now open) 2016-03-13T08:37:14.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53888 #4077294 (1507 connections now open) 2016-03-13T08:37:14.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53887 #4077295 (1507 connections now open) 2016-03-13T08:37:14.975+0000 I NETWORK [conn4077285] end connection 10.0.20.53:47114 (1506 connections now open) 2016-03-13T08:37:14.975+0000 I NETWORK [conn4077284] end connection 10.0.20.53:47113 (1506 connections now open) 2016-03-13T08:37:14.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47125 #4077296 (1506 connections now open) 2016-03-13T08:37:14.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47124 #4077297 (1507 connections now open) 2016-03-13T08:37:16.118+0000 I NETWORK [conn4067017] end connection 10.0.20.53:36363 (1506 connections now open) 2016-03-13T08:37:16.118+0000 I NETWORK [conn4065695] end connection 10.0.27.237:48547 (1506 connections now open) 2016-03-13T08:37:16.118+0000 I NETWORK [conn4069871] end connection 10.0.26.69:59159 (1506 connections now open) 2016-03-13T08:37:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60489 #4077298 (1505 connections now open) 2016-03-13T08:37:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47128 #4077299 (1506 connections now open) 2016-03-13T08:37:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38917 #4077300 (1507 connections now open) 2016-03-13T08:37:17.475+0000 I NETWORK [conn4077287] end connection 10.0.179.209:36971 (1506 connections now open) 2016-03-13T08:37:17.475+0000 I NETWORK [conn4077286] end connection 10.0.179.209:36970 (1506 connections now open) 2016-03-13T08:37:17.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36986 #4077301 (1506 connections now open) 2016-03-13T08:37:17.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:36985 #4077302 (1507 connections now open) 2016-03-13T08:37:17.946+0000 I NETWORK [conn4077289] end connection 10.0.26.69:38896 (1506 connections now open) 2016-03-13T08:37:17.946+0000 I NETWORK [conn4077288] end connection 10.0.26.69:38895 (1506 connections now open) 2016-03-13T08:37:17.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38919 #4077303 (1506 connections now open) 2016-03-13T08:37:17.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38918 #4077304 (1507 connections now open) 2016-03-13T08:37:20.174+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:20.175+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:20.175+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:20.175+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:37:20.676+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:20.677+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:20.677+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:20.677+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:37:20.963+0000 I NETWORK [conn4077290] end connection 10.0.27.237:60479 (1506 connections now open) 2016-03-13T08:37:20.963+0000 I NETWORK [conn4077291] end connection 10.0.27.237:60478 (1506 connections now open) 2016-03-13T08:37:20.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60491 #4077305 (1506 connections now open) 2016-03-13T08:37:20.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60490 #4077306 (1507 connections now open) 2016-03-13T08:37:21.177+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:21.178+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:21.178+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:21.178+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:37:21.679+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:21.680+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:21.680+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:21.680+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:37:22.181+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:22.181+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:22.181+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:22.181+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:37:22.682+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:22.683+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:22.683+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:22.683+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:37:22.683+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:37:22.683+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:37:22.683+0000-56e526c218fa188537623c87", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858242683), what: "balancer.round", ns: "", details: { executionTimeMillis: 2530, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:37:22.684+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:37:23.003+0000 I NETWORK [conn4077292] end connection 10.0.7.113:55837 (1506 connections now open) 2016-03-13T08:37:23.003+0000 I NETWORK [conn4077293] end connection 10.0.7.113:55838 (1505 connections now open) 2016-03-13T08:37:23.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55852 #4077307 (1506 connections now open) 2016-03-13T08:37:23.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55853 #4077308 (1507 connections now open) 2016-03-13T08:37:24.209+0000 I NETWORK [conn4077294] end connection 10.0.216.90:53888 (1506 connections now open) 2016-03-13T08:37:24.209+0000 I NETWORK [conn4077295] end connection 10.0.216.90:53887 (1506 connections now open) 2016-03-13T08:37:24.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53901 #4077309 (1506 connections now open) 2016-03-13T08:37:24.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53900 #4077310 (1507 connections now open) 2016-03-13T08:37:24.989+0000 I NETWORK [conn4077297] end connection 10.0.20.53:47124 (1506 connections now open) 2016-03-13T08:37:24.989+0000 I NETWORK [conn4077296] end connection 10.0.20.53:47125 (1506 connections now open) 2016-03-13T08:37:24.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47144 #4077311 (1506 connections now open) 2016-03-13T08:37:24.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47145 #4077312 (1507 connections now open) 2016-03-13T08:37:27.488+0000 I NETWORK [conn4077301] end connection 10.0.179.209:36986 (1506 connections now open) 2016-03-13T08:37:27.488+0000 I NETWORK [conn4077302] end connection 10.0.179.209:36985 (1506 connections now open) 2016-03-13T08:37:27.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37003 #4077313 (1506 connections now open) 2016-03-13T08:37:27.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37004 #4077314 (1507 connections now open) 2016-03-13T08:37:27.959+0000 I NETWORK [conn4077304] end connection 10.0.26.69:38918 (1506 connections now open) 2016-03-13T08:37:27.959+0000 I NETWORK [conn4077303] end connection 10.0.26.69:38919 (1506 connections now open) 2016-03-13T08:37:27.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38927 #4077315 (1506 connections now open) 2016-03-13T08:37:27.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38928 #4077316 (1507 connections now open) 2016-03-13T08:37:28.113+0000 I NETWORK [conn4067085] end connection 10.0.216.90:42950 (1506 connections now open) 2016-03-13T08:37:28.113+0000 I NETWORK [conn4068432] end connection 10.0.20.53:37843 (1506 connections now open) 2016-03-13T08:37:28.113+0000 I NETWORK [conn4069128] end connection 10.0.26.69:58363 (1506 connections now open) 2016-03-13T08:37:28.113+0000 I NETWORK [conn4067985] end connection 10.0.27.237:50839 (1504 connections now open) 2016-03-13T08:37:28.113+0000 I NETWORK [conn4069323] end connection 10.0.7.113:47325 (1502 connections now open) 2016-03-13T08:37:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53918 #4077317 (1503 connections now open) 2016-03-13T08:37:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47148 #4077318 (1504 connections now open) 2016-03-13T08:37:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38931 #4077319 (1505 connections now open) 2016-03-13T08:37:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60502 #4077320 (1506 connections now open) 2016-03-13T08:37:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55869 #4077321 (1507 connections now open) 2016-03-13T08:37:30.976+0000 I NETWORK [conn4077306] end connection 10.0.27.237:60490 (1506 connections now open) 2016-03-13T08:37:30.976+0000 I NETWORK [conn4077305] end connection 10.0.27.237:60491 (1506 connections now open) 2016-03-13T08:37:30.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60504 #4077322 (1506 connections now open) 2016-03-13T08:37:30.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60505 #4077323 (1507 connections now open) 2016-03-13T08:37:32.739+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:37:32.742+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:32.751+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:32.751+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:32.751+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:37:33.016+0000 I NETWORK [conn4077307] end connection 10.0.7.113:55852 (1506 connections now open) 2016-03-13T08:37:33.016+0000 I NETWORK [conn4077308] end connection 10.0.7.113:55853 (1506 connections now open) 2016-03-13T08:37:33.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55872 #4077324 (1506 connections now open) 2016-03-13T08:37:33.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55873 #4077325 (1507 connections now open) 2016-03-13T08:37:33.252+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:33.253+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:33.253+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:33.253+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:37:33.754+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:33.754+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:33.754+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:33.754+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:37:34.222+0000 I NETWORK [conn4077310] end connection 10.0.216.90:53900 (1506 connections now open) 2016-03-13T08:37:34.222+0000 I NETWORK [conn4077309] end connection 10.0.216.90:53901 (1506 connections now open) 2016-03-13T08:37:34.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53924 #4077326 (1506 connections now open) 2016-03-13T08:37:34.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53923 #4077327 (1507 connections now open) 2016-03-13T08:37:34.255+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:34.256+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:34.256+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:34.256+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:37:34.757+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:34.758+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:34.758+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:34.758+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:37:35.002+0000 I NETWORK [conn4077312] end connection 10.0.20.53:47145 (1506 connections now open) 2016-03-13T08:37:35.002+0000 I NETWORK [conn4077311] end connection 10.0.20.53:47144 (1505 connections now open) 2016-03-13T08:37:35.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47156 #4077328 (1506 connections now open) 2016-03-13T08:37:35.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47157 #4077329 (1507 connections now open) 2016-03-13T08:37:35.259+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:35.259+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:35.259+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:35.259+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:37:35.760+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:35.761+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:35.761+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:35.761+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:37:36.262+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:36.262+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:36.262+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:36.262+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:37:36.763+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:36.764+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:36.764+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:36.764+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:37:37.265+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:37.266+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:37.266+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:37.266+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:37:37.501+0000 I NETWORK [conn4077314] end connection 10.0.179.209:37004 (1506 connections now open) 2016-03-13T08:37:37.501+0000 I NETWORK [conn4077313] end connection 10.0.179.209:37003 (1506 connections now open) 2016-03-13T08:37:37.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37014 #4077330 (1506 connections now open) 2016-03-13T08:37:37.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37013 #4077331 (1507 connections now open) 2016-03-13T08:37:37.767+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:37.767+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:37.769+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:37.769+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:37:37.972+0000 I NETWORK [conn4077315] end connection 10.0.26.69:38927 (1506 connections now open) 2016-03-13T08:37:37.972+0000 I NETWORK [conn4077316] end connection 10.0.26.69:38928 (1506 connections now open) 2016-03-13T08:37:37.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38942 #4077332 (1506 connections now open) 2016-03-13T08:37:37.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38943 #4077333 (1507 connections now open) 2016-03-13T08:37:38.270+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:38.271+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:38.271+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:38.271+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:37:38.772+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:38.773+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:38.773+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:38.773+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:37:39.273+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:39.274+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:39.274+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:39.274+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:37:39.775+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:39.776+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:39.776+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:39.776+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:37:40.277+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:40.277+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:40.277+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:40.277+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:37:40.778+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:40.779+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:40.779+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:40.779+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:37:40.989+0000 I NETWORK [conn4077322] end connection 10.0.27.237:60504 (1506 connections now open) 2016-03-13T08:37:40.989+0000 I NETWORK [conn4077323] end connection 10.0.27.237:60505 (1505 connections now open) 2016-03-13T08:37:40.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60520 #4077334 (1506 connections now open) 2016-03-13T08:37:40.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60521 #4077335 (1507 connections now open) 2016-03-13T08:37:41.280+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:41.281+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:41.281+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:41.281+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:37:41.782+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:41.782+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:41.782+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:41.782+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:37:42.283+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:42.284+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:42.284+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:42.284+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:37:42.689+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:37:42.543+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:37:42.689+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:42.690+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:42.690+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:37:42.690+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:37:42.785+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:42.785+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:42.786+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:42.786+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:37:43.029+0000 I NETWORK [conn4077325] end connection 10.0.7.113:55873 (1506 connections now open) 2016-03-13T08:37:43.029+0000 I NETWORK [conn4077324] end connection 10.0.7.113:55872 (1505 connections now open) 2016-03-13T08:37:43.030+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55879 #4077336 (1506 connections now open) 2016-03-13T08:37:43.030+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55880 #4077337 (1507 connections now open) 2016-03-13T08:37:43.287+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:43.287+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:43.287+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:43.287+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:37:43.287+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:37:43.287+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:37:43.287+0000-56e526d718fa188537623c88", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858263287), what: "balancer.round", ns: "", details: { executionTimeMillis: 10582, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:37:44.235+0000 I NETWORK [conn4077327] end connection 10.0.216.90:53923 (1506 connections now open) 2016-03-13T08:37:44.235+0000 I NETWORK [conn4077326] end connection 10.0.216.90:53924 (1506 connections now open) 2016-03-13T08:37:44.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53932 #4077338 (1506 connections now open) 2016-03-13T08:37:44.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53933 #4077339 (1507 connections now open) 2016-03-13T08:37:45.015+0000 I NETWORK [conn4077329] end connection 10.0.20.53:47157 (1506 connections now open) 2016-03-13T08:37:45.016+0000 I NETWORK [conn4077328] end connection 10.0.20.53:47156 (1505 connections now open) 2016-03-13T08:37:45.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47167 #4077340 (1506 connections now open) 2016-03-13T08:37:45.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47168 #4077341 (1507 connections now open) 2016-03-13T08:37:46.117+0000 I NETWORK [conn4063599] end connection 10.0.179.209:51133 (1506 connections now open) 2016-03-13T08:37:46.117+0000 I NETWORK [conn4072747] end connection 10.0.20.53:42329 (1506 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [conn4073207] end connection 10.0.26.69:34492 (1504 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [conn4069043] end connection 10.0.216.90:45019 (1504 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [conn4067485] end connection 10.0.7.113:45374 (1502 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47170 #4077342 (1503 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37025 #4077343 (1504 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53937 #4077344 (1505 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38962 #4077345 (1506 connections now open) 2016-03-13T08:37:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55889 #4077346 (1507 connections now open) 2016-03-13T08:37:47.514+0000 I NETWORK [conn4077330] end connection 10.0.179.209:37014 (1506 connections now open) 2016-03-13T08:37:47.514+0000 I NETWORK [conn4077331] end connection 10.0.179.209:37013 (1506 connections now open) 2016-03-13T08:37:47.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37026 #4077347 (1506 connections now open) 2016-03-13T08:37:47.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37027 #4077348 (1507 connections now open) 2016-03-13T08:37:47.994+0000 I NETWORK [conn4077333] end connection 10.0.26.69:38943 (1506 connections now open) 2016-03-13T08:37:47.994+0000 I NETWORK [conn4077332] end connection 10.0.26.69:38942 (1506 connections now open) 2016-03-13T08:37:47.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38963 #4077349 (1507 connections now open) 2016-03-13T08:37:47.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38964 #4077350 (1507 connections now open) 2016-03-13T08:37:51.002+0000 I NETWORK [conn4077335] end connection 10.0.27.237:60521 (1506 connections now open) 2016-03-13T08:37:51.002+0000 I NETWORK [conn4077334] end connection 10.0.27.237:60520 (1506 connections now open) 2016-03-13T08:37:51.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60528 #4077351 (1506 connections now open) 2016-03-13T08:37:51.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60529 #4077352 (1507 connections now open) 2016-03-13T08:37:52.693+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:52.693+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:52.693+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:37:52.693+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:37:53.042+0000 I NETWORK [conn4077336] end connection 10.0.7.113:55879 (1506 connections now open) 2016-03-13T08:37:53.042+0000 I NETWORK [conn4077337] end connection 10.0.7.113:55880 (1506 connections now open) 2016-03-13T08:37:53.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55893 #4077353 (1506 connections now open) 2016-03-13T08:37:53.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55892 #4077354 (1507 connections now open) 2016-03-13T08:37:53.342+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:53.343+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:53.343+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:53.343+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:37:53.844+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:53.844+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:53.844+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:53.844+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:37:54.249+0000 I NETWORK [conn4077339] end connection 10.0.216.90:53933 (1506 connections now open) 2016-03-13T08:37:54.249+0000 I NETWORK [conn4077338] end connection 10.0.216.90:53932 (1506 connections now open) 2016-03-13T08:37:54.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53948 #4077355 (1506 connections now open) 2016-03-13T08:37:54.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53947 #4077356 (1507 connections now open) 2016-03-13T08:37:54.345+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:54.346+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:54.346+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:54.346+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:37:54.847+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:54.848+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:54.848+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:54.848+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:37:55.028+0000 I NETWORK [conn4077341] end connection 10.0.20.53:47168 (1506 connections now open) 2016-03-13T08:37:55.028+0000 I NETWORK [conn4077340] end connection 10.0.20.53:47167 (1506 connections now open) 2016-03-13T08:37:55.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47188 #4077357 (1506 connections now open) 2016-03-13T08:37:55.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47187 #4077358 (1507 connections now open) 2016-03-13T08:37:55.349+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:55.350+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:55.350+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:55.350+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:37:55.851+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:37:55.851+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:37:55.851+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:37:55.851+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:37:55.851+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:37:55.851+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:37:55.851+0000-56e526e318fa188537623c89", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858275851), what: "balancer.round", ns: "", details: { executionTimeMillis: 2547, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:37:57.527+0000 I NETWORK [conn4077348] end connection 10.0.179.209:37027 (1506 connections now open) 2016-03-13T08:37:57.527+0000 I NETWORK [conn4077347] end connection 10.0.179.209:37026 (1506 connections now open) 2016-03-13T08:37:57.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37044 #4077359 (1506 connections now open) 2016-03-13T08:37:57.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37043 #4077360 (1507 connections now open) 2016-03-13T08:37:58.007+0000 I NETWORK [conn4077350] end connection 10.0.26.69:38964 (1506 connections now open) 2016-03-13T08:37:58.007+0000 I NETWORK [conn4077349] end connection 10.0.26.69:38963 (1506 connections now open) 2016-03-13T08:37:58.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38972 #4077361 (1506 connections now open) 2016-03-13T08:37:58.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38971 #4077362 (1507 connections now open) 2016-03-13T08:37:58.113+0000 I NETWORK [conn4068576] end connection 10.0.179.209:56419 (1506 connections now open) 2016-03-13T08:37:58.113+0000 I NETWORK [conn4067128] end connection 10.0.216.90:42997 (1505 connections now open) 2016-03-13T08:37:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37048 #4077363 (1507 connections now open) 2016-03-13T08:37:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53964 #4077364 (1507 connections now open) 2016-03-13T08:38:01.024+0000 I NETWORK [conn4077351] end connection 10.0.27.237:60528 (1506 connections now open) 2016-03-13T08:38:01.024+0000 I NETWORK [conn4077352] end connection 10.0.27.237:60529 (1506 connections now open) 2016-03-13T08:38:01.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60542 #4077365 (1506 connections now open) 2016-03-13T08:38:01.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60543 #4077366 (1507 connections now open) 2016-03-13T08:38:02.696+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:38:03.065+0000 I NETWORK [conn4077354] end connection 10.0.7.113:55892 (1506 connections now open) 2016-03-13T08:38:03.065+0000 I NETWORK [conn4077353] end connection 10.0.7.113:55893 (1506 connections now open) 2016-03-13T08:38:03.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55917 #4077367 (1506 connections now open) 2016-03-13T08:38:03.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55916 #4077368 (1507 connections now open) 2016-03-13T08:38:04.270+0000 I NETWORK [conn4077356] end connection 10.0.216.90:53947 (1506 connections now open) 2016-03-13T08:38:04.270+0000 I NETWORK [conn4077355] end connection 10.0.216.90:53948 (1506 connections now open) 2016-03-13T08:38:04.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53969 #4077369 (1506 connections now open) 2016-03-13T08:38:04.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53970 #4077370 (1507 connections now open) 2016-03-13T08:38:05.050+0000 I NETWORK [conn4077358] end connection 10.0.20.53:47187 (1506 connections now open) 2016-03-13T08:38:05.050+0000 I NETWORK [conn4077357] end connection 10.0.20.53:47188 (1505 connections now open) 2016-03-13T08:38:05.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47198 #4077371 (1506 connections now open) 2016-03-13T08:38:05.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47199 #4077372 (1507 connections now open) 2016-03-13T08:38:05.905+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:38:05.907+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:05.915+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:05.915+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:05.915+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:38:06.416+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:06.417+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:06.417+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:06.417+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:38:06.918+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:06.919+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:06.919+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:06.919+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:38:07.419+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:07.420+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:07.420+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:07.420+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:38:07.548+0000 I NETWORK [conn4077359] end connection 10.0.179.209:37044 (1506 connections now open) 2016-03-13T08:38:07.548+0000 I NETWORK [conn4077360] end connection 10.0.179.209:37043 (1506 connections now open) 2016-03-13T08:38:07.549+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37055 #4077373 (1506 connections now open) 2016-03-13T08:38:07.549+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37056 #4077374 (1507 connections now open) 2016-03-13T08:38:07.921+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:07.922+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:07.922+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:07.922+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:38:08.021+0000 I NETWORK [conn4077362] end connection 10.0.26.69:38971 (1506 connections now open) 2016-03-13T08:38:08.021+0000 I NETWORK [conn4077361] end connection 10.0.26.69:38972 (1506 connections now open) 2016-03-13T08:38:08.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38989 #4077375 (1506 connections now open) 2016-03-13T08:38:08.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:38988 #4077376 (1507 connections now open) 2016-03-13T08:38:08.423+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:08.423+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:08.423+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:08.423+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:38:08.924+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:08.925+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:08.925+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:08.925+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:38:09.426+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:09.426+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:09.426+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:09.426+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:38:09.927+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:09.928+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:09.928+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:09.928+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:38:10.429+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:10.429+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:10.430+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:10.430+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:38:10.930+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:10.931+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:10.931+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:10.931+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:38:11.037+0000 I NETWORK [conn4077366] end connection 10.0.27.237:60543 (1506 connections now open) 2016-03-13T08:38:11.037+0000 I NETWORK [conn4077365] end connection 10.0.27.237:60542 (1506 connections now open) 2016-03-13T08:38:11.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60562 #4077377 (1506 connections now open) 2016-03-13T08:38:11.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60563 #4077378 (1507 connections now open) 2016-03-13T08:38:11.432+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:11.433+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:11.433+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:11.433+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:38:11.934+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:11.934+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:11.934+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:11.934+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:38:12.435+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:12.436+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:12.436+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:12.436+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:38:12.699+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:12.699+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:12.700+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:38:12.700+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:38:12.833+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:38:12.689+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:38:12.937+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:12.937+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:12.937+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:12.937+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:38:13.078+0000 I NETWORK [conn4077367] end connection 10.0.7.113:55917 (1506 connections now open) 2016-03-13T08:38:13.078+0000 I NETWORK [conn4077368] end connection 10.0.7.113:55916 (1505 connections now open) 2016-03-13T08:38:13.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55928 #4077379 (1506 connections now open) 2016-03-13T08:38:13.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55929 #4077380 (1507 connections now open) 2016-03-13T08:38:13.438+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:13.439+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:13.439+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:13.439+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:38:13.940+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:13.941+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:13.941+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:13.941+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:38:14.282+0000 I NETWORK [conn4077369] end connection 10.0.216.90:53969 (1506 connections now open) 2016-03-13T08:38:14.282+0000 I NETWORK [conn4077370] end connection 10.0.216.90:53970 (1506 connections now open) 2016-03-13T08:38:14.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53978 #4077381 (1506 connections now open) 2016-03-13T08:38:14.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53979 #4077382 (1507 connections now open) 2016-03-13T08:38:14.441+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:14.442+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:14.442+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:14.442+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:38:14.944+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:14.944+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:14.944+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:14.944+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:38:15.063+0000 I NETWORK [conn4077371] end connection 10.0.20.53:47198 (1506 connections now open) 2016-03-13T08:38:15.063+0000 I NETWORK [conn4077372] end connection 10.0.20.53:47199 (1505 connections now open) 2016-03-13T08:38:15.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47211 #4077383 (1506 connections now open) 2016-03-13T08:38:15.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47210 #4077384 (1507 connections now open) 2016-03-13T08:38:15.445+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:15.446+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:15.446+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:15.446+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:38:15.947+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:15.948+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:15.948+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:15.948+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:38:16.117+0000 I NETWORK [conn4072790] end connection 10.0.179.209:60622 (1506 connections now open) 2016-03-13T08:38:16.117+0000 I NETWORK [conn4068279] end connection 10.0.216.90:44212 (1506 connections now open) 2016-03-13T08:38:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53982 #4077385 (1506 connections now open) 2016-03-13T08:38:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37063 #4077386 (1507 connections now open) 2016-03-13T08:38:16.118+0000 I NETWORK [conn4069241] end connection 10.0.27.237:52149 (1506 connections now open) 2016-03-13T08:38:16.119+0000 I NETWORK [conn4065380] end connection 10.0.7.113:43124 (1505 connections now open) 2016-03-13T08:38:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60572 #4077387 (1506 connections now open) 2016-03-13T08:38:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55939 #4077388 (1507 connections now open) 2016-03-13T08:38:16.448+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:16.449+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:16.449+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:16.449+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:38:16.449+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:38:16.449+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:38:16.449+0000-56e526f818fa188537623c8a", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858296449), what: "balancer.round", ns: "", details: { executionTimeMillis: 10562, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:38:17.562+0000 I NETWORK [conn4077374] end connection 10.0.179.209:37056 (1506 connections now open) 2016-03-13T08:38:17.562+0000 I NETWORK [conn4077373] end connection 10.0.179.209:37055 (1506 connections now open) 2016-03-13T08:38:17.563+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37072 #4077389 (1506 connections now open) 2016-03-13T08:38:17.563+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37073 #4077390 (1507 connections now open) 2016-03-13T08:38:18.034+0000 I NETWORK [conn4077375] end connection 10.0.26.69:38989 (1506 connections now open) 2016-03-13T08:38:18.034+0000 I NETWORK [conn4077376] end connection 10.0.26.69:38988 (1506 connections now open) 2016-03-13T08:38:18.034+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39012 #4077391 (1506 connections now open) 2016-03-13T08:38:18.034+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39011 #4077392 (1507 connections now open) 2016-03-13T08:38:21.050+0000 I NETWORK [conn4077377] end connection 10.0.27.237:60562 (1506 connections now open) 2016-03-13T08:38:21.050+0000 I NETWORK [conn4077378] end connection 10.0.27.237:60563 (1505 connections now open) 2016-03-13T08:38:21.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60574 #4077393 (1506 connections now open) 2016-03-13T08:38:21.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60575 #4077394 (1507 connections now open) 2016-03-13T08:38:22.702+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:22.703+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:22.703+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:38:22.703+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:38:23.091+0000 I NETWORK [conn4077379] end connection 10.0.7.113:55928 (1506 connections now open) 2016-03-13T08:38:23.091+0000 I NETWORK [conn4077380] end connection 10.0.7.113:55929 (1506 connections now open) 2016-03-13T08:38:23.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55943 #4077395 (1506 connections now open) 2016-03-13T08:38:23.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55942 #4077396 (1507 connections now open) 2016-03-13T08:38:24.295+0000 I NETWORK [conn4077382] end connection 10.0.216.90:53979 (1506 connections now open) 2016-03-13T08:38:24.295+0000 I NETWORK [conn4077381] end connection 10.0.216.90:53978 (1506 connections now open) 2016-03-13T08:38:24.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53994 #4077397 (1506 connections now open) 2016-03-13T08:38:24.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:53993 #4077398 (1507 connections now open) 2016-03-13T08:38:25.076+0000 I NETWORK [conn4077384] end connection 10.0.20.53:47210 (1506 connections now open) 2016-03-13T08:38:25.076+0000 I NETWORK [conn4077383] end connection 10.0.20.53:47211 (1506 connections now open) 2016-03-13T08:38:25.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47229 #4077399 (1506 connections now open) 2016-03-13T08:38:25.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47230 #4077400 (1507 connections now open) 2016-03-13T08:38:26.516+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:26.516+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:26.517+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:26.517+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:38:27.017+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:27.018+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:27.018+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:27.018+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:38:27.519+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:27.519+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:27.520+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:27.520+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:38:27.575+0000 I NETWORK [conn4077390] end connection 10.0.179.209:37073 (1506 connections now open) 2016-03-13T08:38:27.575+0000 I NETWORK [conn4077389] end connection 10.0.179.209:37072 (1506 connections now open) 2016-03-13T08:38:27.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37089 #4077401 (1506 connections now open) 2016-03-13T08:38:27.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37090 #4077402 (1507 connections now open) 2016-03-13T08:38:28.020+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:28.021+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:28.021+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:28.021+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:38:28.047+0000 I NETWORK [conn4077391] end connection 10.0.26.69:39012 (1506 connections now open) 2016-03-13T08:38:28.047+0000 I NETWORK [conn4077392] end connection 10.0.26.69:39011 (1506 connections now open) 2016-03-13T08:38:28.047+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39021 #4077403 (1506 connections now open) 2016-03-13T08:38:28.047+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39020 #4077404 (1507 connections now open) 2016-03-13T08:38:28.113+0000 I NETWORK [conn4076718] end connection 10.0.26.69:38296 (1506 connections now open) 2016-03-13T08:38:28.113+0000 I NETWORK [conn4068392] end connection 10.0.216.90:44331 (1506 connections now open) 2016-03-13T08:38:28.113+0000 I NETWORK [conn4074987] end connection 10.0.27.237:58176 (1504 connections now open) 2016-03-13T08:38:28.113+0000 I NETWORK [conn4068435] end connection 10.0.7.113:46366 (1503 connections now open) 2016-03-13T08:38:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39024 #4077405 (1504 connections now open) 2016-03-13T08:38:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54010 #4077406 (1505 connections now open) 2016-03-13T08:38:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60584 #4077407 (1506 connections now open) 2016-03-13T08:38:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55961 #4077408 (1507 connections now open) 2016-03-13T08:38:28.522+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:28.523+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:28.523+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:28.523+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:38:29.024+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:29.024+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:29.024+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:29.024+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:38:29.024+0000 I SHARDING [Balancer] caught exception while doing balance: None of the hosts for replica set rsmydomain3 could be contacted. 2016-03-13T08:38:29.024+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:38:29.024+0000-56e5270518fa188537623c8b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858309024), what: "balancer.round", ns: "", details: { executionTimeMillis: 2540, errorOccured: true, errmsg: "None of the hosts for replica set rsmydomain3 could be contacted." } } 2016-03-13T08:38:31.063+0000 I NETWORK [conn4077393] end connection 10.0.27.237:60574 (1506 connections now open) 2016-03-13T08:38:31.063+0000 I NETWORK [conn4077394] end connection 10.0.27.237:60575 (1506 connections now open) 2016-03-13T08:38:31.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60586 #4077409 (1506 connections now open) 2016-03-13T08:38:31.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60587 #4077410 (1507 connections now open) 2016-03-13T08:38:32.705+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:38:33.104+0000 I NETWORK [conn4077396] end connection 10.0.7.113:55942 (1506 connections now open) 2016-03-13T08:38:33.104+0000 I NETWORK [conn4077395] end connection 10.0.7.113:55943 (1506 connections now open) 2016-03-13T08:38:33.105+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55964 #4077411 (1506 connections now open) 2016-03-13T08:38:33.105+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55965 #4077412 (1507 connections now open) 2016-03-13T08:38:34.308+0000 I NETWORK [conn4077398] end connection 10.0.216.90:53993 (1506 connections now open) 2016-03-13T08:38:34.308+0000 I NETWORK [conn4077397] end connection 10.0.216.90:53994 (1506 connections now open) 2016-03-13T08:38:34.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54016 #4077413 (1506 connections now open) 2016-03-13T08:38:34.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54017 #4077414 (1507 connections now open) 2016-03-13T08:38:35.089+0000 I NETWORK [conn4077399] end connection 10.0.20.53:47229 (1506 connections now open) 2016-03-13T08:38:35.089+0000 I NETWORK [conn4077400] end connection 10.0.20.53:47230 (1506 connections now open) 2016-03-13T08:38:35.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47242 #4077415 (1506 connections now open) 2016-03-13T08:38:35.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47243 #4077416 (1507 connections now open) 2016-03-13T08:38:37.588+0000 I NETWORK [conn4077402] end connection 10.0.179.209:37090 (1506 connections now open) 2016-03-13T08:38:37.588+0000 I NETWORK [conn4077401] end connection 10.0.179.209:37089 (1506 connections now open) 2016-03-13T08:38:37.589+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37099 #4077417 (1506 connections now open) 2016-03-13T08:38:37.589+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37098 #4077418 (1507 connections now open) 2016-03-13T08:38:38.060+0000 I NETWORK [conn4077403] end connection 10.0.26.69:39021 (1506 connections now open) 2016-03-13T08:38:38.060+0000 I NETWORK [conn4077404] end connection 10.0.26.69:39020 (1506 connections now open) 2016-03-13T08:38:38.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39034 #4077419 (1506 connections now open) 2016-03-13T08:38:38.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39035 #4077420 (1507 connections now open) 2016-03-13T08:38:39.086+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:38:39.096+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:39.097+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:39.097+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:39.097+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:38:39.598+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:39.598+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:39.598+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:39.598+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:38:40.099+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:40.100+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:40.100+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:40.100+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:38:40.601+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:40.602+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:40.602+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:40.602+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:38:41.076+0000 I NETWORK [conn4077409] end connection 10.0.27.237:60586 (1506 connections now open) 2016-03-13T08:38:41.076+0000 I NETWORK [conn4077410] end connection 10.0.27.237:60587 (1506 connections now open) 2016-03-13T08:38:41.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60604 #4077421 (1506 connections now open) 2016-03-13T08:38:41.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60603 #4077422 (1507 connections now open) 2016-03-13T08:38:41.103+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:41.103+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:41.103+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:41.103+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:38:41.604+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:41.605+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:41.605+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:41.605+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:38:42.106+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:42.106+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:42.106+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:42.106+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:38:42.607+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:42.608+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:42.608+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:42.608+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:38:42.709+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:42.709+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:42.709+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:38:42.709+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:38:43.109+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:43.109+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:43.109+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:43.109+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:38:43.111+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:38:42.833+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:38:43.118+0000 I NETWORK [conn4077412] end connection 10.0.7.113:55965 (1506 connections now open) 2016-03-13T08:38:43.118+0000 I NETWORK [conn4077411] end connection 10.0.7.113:55964 (1506 connections now open) 2016-03-13T08:38:43.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55971 #4077423 (1506 connections now open) 2016-03-13T08:38:43.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55972 #4077424 (1507 connections now open) 2016-03-13T08:38:43.610+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:43.611+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:43.611+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:43.611+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:38:44.112+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:44.113+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:44.113+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:44.113+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:38:44.322+0000 I NETWORK [conn4077414] end connection 10.0.216.90:54017 (1506 connections now open) 2016-03-13T08:38:44.322+0000 I NETWORK [conn4077413] end connection 10.0.216.90:54016 (1506 connections now open) 2016-03-13T08:38:44.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54024 #4077425 (1506 connections now open) 2016-03-13T08:38:44.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54025 #4077426 (1507 connections now open) 2016-03-13T08:38:44.614+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:44.614+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:44.614+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:44.614+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:38:45.102+0000 I NETWORK [conn4077416] end connection 10.0.20.53:47243 (1506 connections now open) 2016-03-13T08:38:45.102+0000 I NETWORK [conn4077415] end connection 10.0.20.53:47242 (1506 connections now open) 2016-03-13T08:38:45.103+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47254 #4077427 (1506 connections now open) 2016-03-13T08:38:45.103+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47255 #4077428 (1507 connections now open) 2016-03-13T08:38:45.115+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:45.116+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:45.116+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:45.116+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:38:45.617+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:45.617+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:45.617+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:45.617+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:38:46.118+0000 I NETWORK [conn4064150] end connection 10.0.179.209:51735 (1506 connections now open) 2016-03-13T08:38:46.118+0000 I NETWORK [conn4074257] end connection 10.0.27.237:57442 (1505 connections now open) 2016-03-13T08:38:46.118+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:46.118+0000 I NETWORK [conn4065908] end connection 10.0.26.69:54910 (1505 connections now open) 2016-03-13T08:38:46.118+0000 I NETWORK [conn4066349] end connection 10.0.7.113:44165 (1503 connections now open) 2016-03-13T08:38:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37106 #4077429 (1504 connections now open) 2016-03-13T08:38:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39052 #4077430 (1505 connections now open) 2016-03-13T08:38:46.119+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60611 #4077431 (1506 connections now open) 2016-03-13T08:38:46.119+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:46.119+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:38:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55983 #4077432 (1507 connections now open) 2016-03-13T08:38:46.620+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:46.620+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:46.620+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:46.621+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:38:47.121+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:47.122+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:47.122+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:47.122+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:38:47.601+0000 I NETWORK [conn4077418] end connection 10.0.179.209:37098 (1506 connections now open) 2016-03-13T08:38:47.601+0000 I NETWORK [conn4077417] end connection 10.0.179.209:37099 (1505 connections now open) 2016-03-13T08:38:47.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37112 #4077433 (1506 connections now open) 2016-03-13T08:38:47.602+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37113 #4077434 (1507 connections now open) 2016-03-13T08:38:47.623+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:47.624+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:47.624+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:47.624+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:38:48.082+0000 I NETWORK [conn4077419] end connection 10.0.26.69:39034 (1506 connections now open) 2016-03-13T08:38:48.082+0000 I NETWORK [conn4077420] end connection 10.0.26.69:39035 (1506 connections now open) 2016-03-13T08:38:48.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39054 #4077435 (1506 connections now open) 2016-03-13T08:38:48.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39055 #4077436 (1507 connections now open) 2016-03-13T08:38:48.125+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:48.125+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:48.125+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:48.125+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:38:48.626+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:48.627+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:48.627+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:48.627+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:38:49.128+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:49.129+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:49.129+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:49.129+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:38:49.630+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:49.630+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:49.630+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:38:49.630+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:38:49.630+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:38:49.630+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:38:49.630+0000-56e5271918fa188537623c8c", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858329630), what: "balancer.round", ns: "", details: { executionTimeMillis: 10575, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:38:51.089+0000 I NETWORK [conn4077422] end connection 10.0.27.237:60603 (1506 connections now open) 2016-03-13T08:38:51.089+0000 I NETWORK [conn4077421] end connection 10.0.27.237:60604 (1506 connections now open) 2016-03-13T08:38:51.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60614 #4077437 (1506 connections now open) 2016-03-13T08:38:51.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60613 #4077438 (1507 connections now open) 2016-03-13T08:38:52.712+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:38:52.713+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:38:52.713+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:38:52.713+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:38:53.131+0000 I NETWORK [conn4077423] end connection 10.0.7.113:55971 (1506 connections now open) 2016-03-13T08:38:53.131+0000 I NETWORK [conn4077424] end connection 10.0.7.113:55972 (1505 connections now open) 2016-03-13T08:38:53.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55987 #4077439 (1506 connections now open) 2016-03-13T08:38:53.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:55986 #4077440 (1507 connections now open) 2016-03-13T08:38:54.335+0000 I NETWORK [conn4077426] end connection 10.0.216.90:54025 (1506 connections now open) 2016-03-13T08:38:54.335+0000 I NETWORK [conn4077425] end connection 10.0.216.90:54024 (1506 connections now open) 2016-03-13T08:38:54.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54040 #4077441 (1506 connections now open) 2016-03-13T08:38:54.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54039 #4077442 (1507 connections now open) 2016-03-13T08:38:55.116+0000 I NETWORK [conn4077427] end connection 10.0.20.53:47254 (1506 connections now open) 2016-03-13T08:38:55.116+0000 I NETWORK [conn4077428] end connection 10.0.20.53:47255 (1506 connections now open) 2016-03-13T08:38:55.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47275 #4077443 (1506 connections now open) 2016-03-13T08:38:55.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47274 #4077444 (1507 connections now open) 2016-03-13T08:38:57.614+0000 I NETWORK [conn4077433] end connection 10.0.179.209:37112 (1506 connections now open) 2016-03-13T08:38:57.614+0000 I NETWORK [conn4077434] end connection 10.0.179.209:37113 (1505 connections now open) 2016-03-13T08:38:57.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37130 #4077445 (1506 connections now open) 2016-03-13T08:38:57.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37129 #4077446 (1507 connections now open) 2016-03-13T08:38:58.095+0000 I NETWORK [conn4077436] end connection 10.0.26.69:39055 (1506 connections now open) 2016-03-13T08:38:58.095+0000 I NETWORK [conn4077435] end connection 10.0.26.69:39054 (1506 connections now open) 2016-03-13T08:38:58.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39063 #4077447 (1506 connections now open) 2016-03-13T08:38:58.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39064 #4077448 (1507 connections now open) 2016-03-13T08:38:58.114+0000 I NETWORK [conn4067172] end connection 10.0.216.90:43042 (1506 connections now open) 2016-03-13T08:38:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54057 #4077449 (1507 connections now open) 2016-03-13T08:39:01.103+0000 I NETWORK [conn4077437] end connection 10.0.27.237:60614 (1506 connections now open) 2016-03-13T08:39:01.103+0000 I NETWORK [conn4077438] end connection 10.0.27.237:60613 (1506 connections now open) 2016-03-13T08:39:01.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60629 #4077450 (1506 connections now open) 2016-03-13T08:39:01.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60630 #4077451 (1507 connections now open) 2016-03-13T08:39:02.716+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:39:02.717+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:39:02.717+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:39:02.717+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:39:03.144+0000 I NETWORK [conn4077439] end connection 10.0.7.113:55987 (1506 connections now open) 2016-03-13T08:39:03.144+0000 I NETWORK [conn4077440] end connection 10.0.7.113:55986 (1506 connections now open) 2016-03-13T08:39:03.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56011 #4077452 (1506 connections now open) 2016-03-13T08:39:03.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56010 #4077453 (1507 connections now open) 2016-03-13T08:39:04.356+0000 I NETWORK [conn4077442] end connection 10.0.216.90:54039 (1506 connections now open) 2016-03-13T08:39:04.356+0000 I NETWORK [conn4077441] end connection 10.0.216.90:54040 (1506 connections now open) 2016-03-13T08:39:04.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54062 #4077454 (1506 connections now open) 2016-03-13T08:39:04.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54063 #4077455 (1507 connections now open) 2016-03-13T08:39:05.137+0000 I NETWORK [conn4077443] end connection 10.0.20.53:47275 (1506 connections now open) 2016-03-13T08:39:05.137+0000 I NETWORK [conn4077444] end connection 10.0.20.53:47274 (1505 connections now open) 2016-03-13T08:39:05.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47286 #4077456 (1506 connections now open) 2016-03-13T08:39:05.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47287 #4077457 (1507 connections now open) 2016-03-13T08:39:07.634+0000 I NETWORK [conn4077446] end connection 10.0.179.209:37129 (1506 connections now open) 2016-03-13T08:39:07.634+0000 I NETWORK [conn4077445] end connection 10.0.179.209:37130 (1506 connections now open) 2016-03-13T08:39:07.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37142 #4077458 (1506 connections now open) 2016-03-13T08:39:07.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37141 #4077459 (1507 connections now open) 2016-03-13T08:39:08.108+0000 I NETWORK [conn4077447] end connection 10.0.26.69:39063 (1506 connections now open) 2016-03-13T08:39:08.108+0000 I NETWORK [conn4077448] end connection 10.0.26.69:39064 (1505 connections now open) 2016-03-13T08:39:08.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39079 #4077460 (1506 connections now open) 2016-03-13T08:39:08.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39080 #4077461 (1507 connections now open) 2016-03-13T08:39:11.124+0000 I NETWORK [conn4077451] end connection 10.0.27.237:60630 (1506 connections now open) 2016-03-13T08:39:11.124+0000 I NETWORK [conn4077450] end connection 10.0.27.237:60629 (1506 connections now open) 2016-03-13T08:39:11.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60647 #4077462 (1506 connections now open) 2016-03-13T08:39:11.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60648 #4077463 (1507 connections now open) 2016-03-13T08:39:12.721+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:39:12.730+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:39:12.730+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:39:12.730+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:39:13.167+0000 I NETWORK [conn4077452] end connection 10.0.7.113:56011 (1506 connections now open) 2016-03-13T08:39:13.167+0000 I NETWORK [conn4077453] end connection 10.0.7.113:56010 (1506 connections now open) 2016-03-13T08:39:13.176+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56021 #4077464 (1506 connections now open) 2016-03-13T08:39:13.176+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56022 #4077465 (1507 connections now open) 2016-03-13T08:39:13.249+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:39:13.111+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:39:14.369+0000 I NETWORK [conn4077455] end connection 10.0.216.90:54063 (1506 connections now open) 2016-03-13T08:39:14.369+0000 I NETWORK [conn4077454] end connection 10.0.216.90:54062 (1506 connections now open) 2016-03-13T08:39:14.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54069 #4077466 (1506 connections now open) 2016-03-13T08:39:14.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54070 #4077467 (1507 connections now open) 2016-03-13T08:39:15.150+0000 I NETWORK [conn4077457] end connection 10.0.20.53:47287 (1506 connections now open) 2016-03-13T08:39:15.150+0000 I NETWORK [conn4077456] end connection 10.0.20.53:47286 (1506 connections now open) 2016-03-13T08:39:15.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47297 #4077468 (1506 connections now open) 2016-03-13T08:39:15.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47298 #4077469 (1507 connections now open) 2016-03-13T08:39:16.118+0000 I NETWORK [conn4071739] end connection 10.0.26.69:32939 (1506 connections now open) 2016-03-13T08:39:16.118+0000 I NETWORK [conn4069363] end connection 10.0.27.237:52285 (1505 connections now open) 2016-03-13T08:39:16.118+0000 I NETWORK [conn4068030] end connection 10.0.7.113:45936 (1504 connections now open) 2016-03-13T08:39:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39099 #4077470 (1505 connections now open) 2016-03-13T08:39:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56032 #4077471 (1506 connections now open) 2016-03-13T08:39:16.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60658 #4077472 (1507 connections now open) 2016-03-13T08:39:17.646+0000 I NETWORK [conn4077458] end connection 10.0.179.209:37142 (1506 connections now open) 2016-03-13T08:39:17.646+0000 I NETWORK [conn4077459] end connection 10.0.179.209:37141 (1505 connections now open) 2016-03-13T08:39:17.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37157 #4077473 (1506 connections now open) 2016-03-13T08:39:17.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37158 #4077474 (1507 connections now open) 2016-03-13T08:39:18.121+0000 I NETWORK [conn4077461] end connection 10.0.26.69:39080 (1506 connections now open) 2016-03-13T08:39:18.121+0000 I NETWORK [conn4077460] end connection 10.0.26.69:39079 (1506 connections now open) 2016-03-13T08:39:18.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39102 #4077475 (1507 connections now open) 2016-03-13T08:39:18.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39101 #4077476 (1507 connections now open) 2016-03-13T08:39:21.137+0000 I NETWORK [conn4077463] end connection 10.0.27.237:60648 (1506 connections now open) 2016-03-13T08:39:21.137+0000 I NETWORK [conn4077462] end connection 10.0.27.237:60647 (1505 connections now open) 2016-03-13T08:39:21.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60660 #4077477 (1506 connections now open) 2016-03-13T08:39:21.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60661 #4077478 (1507 connections now open) 2016-03-13T08:39:22.734+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:39:22.735+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:39:22.735+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:39:22.735+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:39:23.189+0000 I NETWORK [conn4077464] end connection 10.0.7.113:56021 (1506 connections now open) 2016-03-13T08:39:23.189+0000 I NETWORK [conn4077465] end connection 10.0.7.113:56022 (1506 connections now open) 2016-03-13T08:39:23.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56036 #4077479 (1507 connections now open) 2016-03-13T08:39:23.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56035 #4077480 (1507 connections now open) 2016-03-13T08:39:24.382+0000 I NETWORK [conn4077467] end connection 10.0.216.90:54070 (1506 connections now open) 2016-03-13T08:39:24.382+0000 I NETWORK [conn4077466] end connection 10.0.216.90:54069 (1506 connections now open) 2016-03-13T08:39:24.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54085 #4077481 (1506 connections now open) 2016-03-13T08:39:24.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54084 #4077482 (1507 connections now open) 2016-03-13T08:39:25.164+0000 I NETWORK [conn4077468] end connection 10.0.20.53:47297 (1506 connections now open) 2016-03-13T08:39:25.164+0000 I NETWORK [conn4077469] end connection 10.0.20.53:47298 (1506 connections now open) 2016-03-13T08:39:25.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47318 #4077483 (1506 connections now open) 2016-03-13T08:39:25.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47319 #4077484 (1507 connections now open) 2016-03-13T08:39:27.659+0000 I NETWORK [conn4077473] end connection 10.0.179.209:37157 (1506 connections now open) 2016-03-13T08:39:27.659+0000 I NETWORK [conn4077474] end connection 10.0.179.209:37158 (1506 connections now open) 2016-03-13T08:39:27.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37175 #4077485 (1506 connections now open) 2016-03-13T08:39:27.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37174 #4077486 (1507 connections now open) 2016-03-13T08:39:28.114+0000 I NETWORK [conn4074146] end connection 10.0.27.237:57335 (1506 connections now open) 2016-03-13T08:39:28.114+0000 I NETWORK [conn4068520] end connection 10.0.216.90:44465 (1506 connections now open) 2016-03-13T08:39:28.114+0000 I NETWORK [conn4068324] end connection 10.0.26.69:57498 (1506 connections now open) 2016-03-13T08:39:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54102 #4077487 (1505 connections now open) 2016-03-13T08:39:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60671 #4077488 (1506 connections now open) 2016-03-13T08:39:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39111 #4077489 (1507 connections now open) 2016-03-13T08:39:28.133+0000 I NETWORK [conn4077475] end connection 10.0.26.69:39102 (1506 connections now open) 2016-03-13T08:39:28.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39112 #4077490 (1507 connections now open) 2016-03-13T08:39:28.134+0000 I NETWORK [conn4077476] end connection 10.0.26.69:39101 (1505 connections now open) 2016-03-13T08:39:28.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39113 #4077491 (1507 connections now open) 2016-03-13T08:39:31.149+0000 I NETWORK [conn4077477] end connection 10.0.27.237:60660 (1506 connections now open) 2016-03-13T08:39:31.150+0000 I NETWORK [conn4077478] end connection 10.0.27.237:60661 (1505 connections now open) 2016-03-13T08:39:31.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60673 #4077492 (1506 connections now open) 2016-03-13T08:39:31.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60674 #4077493 (1507 connections now open) 2016-03-13T08:39:32.738+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:39:32.738+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:39:32.738+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:39:32.738+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:39:33.202+0000 I NETWORK [conn4077479] end connection 10.0.7.113:56036 (1506 connections now open) 2016-03-13T08:39:33.202+0000 I NETWORK [conn4077480] end connection 10.0.7.113:56035 (1506 connections now open) 2016-03-13T08:39:33.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56057 #4077494 (1506 connections now open) 2016-03-13T08:39:33.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56056 #4077495 (1507 connections now open) 2016-03-13T08:39:34.395+0000 I NETWORK [conn4077482] end connection 10.0.216.90:54084 (1506 connections now open) 2016-03-13T08:39:34.395+0000 I NETWORK [conn4077481] end connection 10.0.216.90:54085 (1506 connections now open) 2016-03-13T08:39:34.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54109 #4077496 (1506 connections now open) 2016-03-13T08:39:34.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54108 #4077497 (1507 connections now open) 2016-03-13T08:39:35.178+0000 I NETWORK [conn4077484] end connection 10.0.20.53:47319 (1506 connections now open) 2016-03-13T08:39:35.178+0000 I NETWORK [conn4077483] end connection 10.0.20.53:47318 (1506 connections now open) 2016-03-13T08:39:35.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47332 #4077498 (1506 connections now open) 2016-03-13T08:39:35.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47331 #4077499 (1507 connections now open) 2016-03-13T08:39:37.671+0000 I NETWORK [conn4077485] end connection 10.0.179.209:37175 (1506 connections now open) 2016-03-13T08:39:37.672+0000 I NETWORK [conn4077486] end connection 10.0.179.209:37174 (1505 connections now open) 2016-03-13T08:39:37.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37183 #4077500 (1506 connections now open) 2016-03-13T08:39:37.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37184 #4077501 (1507 connections now open) 2016-03-13T08:39:38.145+0000 I NETWORK [conn4077490] end connection 10.0.26.69:39112 (1506 connections now open) 2016-03-13T08:39:38.146+0000 I NETWORK [conn4077491] end connection 10.0.26.69:39113 (1505 connections now open) 2016-03-13T08:39:38.155+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39124 #4077502 (1506 connections now open) 2016-03-13T08:39:38.155+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39123 #4077503 (1507 connections now open) 2016-03-13T08:39:41.162+0000 I NETWORK [conn4077492] end connection 10.0.27.237:60673 (1506 connections now open) 2016-03-13T08:39:41.162+0000 I NETWORK [conn4077493] end connection 10.0.27.237:60674 (1506 connections now open) 2016-03-13T08:39:41.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60689 #4077504 (1506 connections now open) 2016-03-13T08:39:41.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60690 #4077505 (1507 connections now open) 2016-03-13T08:39:42.741+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:39:42.751+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:39:42.751+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:39:42.751+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:39:43.216+0000 I NETWORK [conn4077495] end connection 10.0.7.113:56056 (1506 connections now open) 2016-03-13T08:39:43.216+0000 I NETWORK [conn4077494] end connection 10.0.7.113:56057 (1506 connections now open) 2016-03-13T08:39:43.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56064 #4077506 (1506 connections now open) 2016-03-13T08:39:43.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56065 #4077507 (1507 connections now open) 2016-03-13T08:39:43.517+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:39:43.250+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:39:44.408+0000 I NETWORK [conn4077497] end connection 10.0.216.90:54108 (1506 connections now open) 2016-03-13T08:39:44.408+0000 I NETWORK [conn4077496] end connection 10.0.216.90:54109 (1506 connections now open) 2016-03-13T08:39:44.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54118 #4077508 (1506 connections now open) 2016-03-13T08:39:44.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54117 #4077509 (1507 connections now open) 2016-03-13T08:39:45.192+0000 I NETWORK [conn4077499] end connection 10.0.20.53:47331 (1506 connections now open) 2016-03-13T08:39:45.192+0000 I NETWORK [conn4077498] end connection 10.0.20.53:47332 (1506 connections now open) 2016-03-13T08:39:45.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47344 #4077510 (1506 connections now open) 2016-03-13T08:39:45.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47343 #4077511 (1507 connections now open) 2016-03-13T08:39:46.118+0000 I NETWORK [conn4067105] end connection 10.0.20.53:36448 (1506 connections now open) 2016-03-13T08:39:46.118+0000 I NETWORK [conn4068408] end connection 10.0.216.90:44348 (1506 connections now open) 2016-03-13T08:39:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54121 #4077512 (1506 connections now open) 2016-03-13T08:39:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47347 #4077513 (1507 connections now open) 2016-03-13T08:39:46.121+0000 I NETWORK [conn4073038] end connection 10.0.26.69:34318 (1506 connections now open) 2016-03-13T08:39:46.121+0000 I NETWORK [conn4071698] end connection 10.0.27.237:54767 (1506 connections now open) 2016-03-13T08:39:46.121+0000 I NETWORK [conn4068071] end connection 10.0.7.113:45979 (1505 connections now open) 2016-03-13T08:39:46.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39142 #4077514 (1505 connections now open) 2016-03-13T08:39:46.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60696 #4077515 (1506 connections now open) 2016-03-13T08:39:46.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56076 #4077516 (1507 connections now open) 2016-03-13T08:39:47.683+0000 I NETWORK [conn4077500] end connection 10.0.179.209:37183 (1506 connections now open) 2016-03-13T08:39:47.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37196 #4077517 (1507 connections now open) 2016-03-13T08:39:47.684+0000 I NETWORK [conn4077501] end connection 10.0.179.209:37184 (1506 connections now open) 2016-03-13T08:39:47.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37197 #4077518 (1507 connections now open) 2016-03-13T08:39:48.167+0000 I NETWORK [conn4077503] end connection 10.0.26.69:39123 (1506 connections now open) 2016-03-13T08:39:48.167+0000 I NETWORK [conn4077502] end connection 10.0.26.69:39124 (1506 connections now open) 2016-03-13T08:39:48.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39144 #4077519 (1507 connections now open) 2016-03-13T08:39:48.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39145 #4077520 (1507 connections now open) 2016-03-13T08:39:51.176+0000 I NETWORK [conn4077504] end connection 10.0.27.237:60689 (1506 connections now open) 2016-03-13T08:39:51.176+0000 I NETWORK [conn4077505] end connection 10.0.27.237:60690 (1506 connections now open) 2016-03-13T08:39:51.176+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60698 #4077521 (1506 connections now open) 2016-03-13T08:39:51.176+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60699 #4077522 (1507 connections now open) 2016-03-13T08:39:52.753+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:39:52.754+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:39:52.754+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:39:52.754+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:39:52.754+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:39:53.230+0000 I NETWORK [conn4077507] end connection 10.0.7.113:56065 (1506 connections now open) 2016-03-13T08:39:53.230+0000 I NETWORK [conn4077506] end connection 10.0.7.113:56064 (1506 connections now open) 2016-03-13T08:39:53.231+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56080 #4077523 (1507 connections now open) 2016-03-13T08:39:53.231+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56079 #4077524 (1507 connections now open) 2016-03-13T08:39:54.421+0000 I NETWORK [conn4077508] end connection 10.0.216.90:54118 (1506 connections now open) 2016-03-13T08:39:54.421+0000 I NETWORK [conn4077509] end connection 10.0.216.90:54117 (1506 connections now open) 2016-03-13T08:39:54.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54131 #4077525 (1506 connections now open) 2016-03-13T08:39:54.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54132 #4077526 (1507 connections now open) 2016-03-13T08:39:55.205+0000 I NETWORK [conn4077510] end connection 10.0.20.53:47344 (1506 connections now open) 2016-03-13T08:39:55.205+0000 I NETWORK [conn4077511] end connection 10.0.20.53:47343 (1506 connections now open) 2016-03-13T08:39:55.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47364 #4077527 (1506 connections now open) 2016-03-13T08:39:55.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47365 #4077528 (1507 connections now open) 2016-03-13T08:39:57.696+0000 I NETWORK [conn4077518] end connection 10.0.179.209:37197 (1506 connections now open) 2016-03-13T08:39:57.696+0000 I NETWORK [conn4077517] end connection 10.0.179.209:37196 (1506 connections now open) 2016-03-13T08:39:57.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37213 #4077529 (1506 connections now open) 2016-03-13T08:39:57.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37214 #4077530 (1507 connections now open) 2016-03-13T08:39:58.114+0000 I NETWORK [conn4070077] end connection 10.0.216.90:46132 (1506 connections now open) 2016-03-13T08:39:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54148 #4077531 (1507 connections now open) 2016-03-13T08:39:58.180+0000 I NETWORK [conn4077520] end connection 10.0.26.69:39145 (1506 connections now open) 2016-03-13T08:39:58.180+0000 I NETWORK [conn4077519] end connection 10.0.26.69:39144 (1506 connections now open) 2016-03-13T08:39:58.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39154 #4077532 (1506 connections now open) 2016-03-13T08:39:58.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39155 #4077533 (1507 connections now open) 2016-03-13T08:40:00.055+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:40:01.189+0000 I NETWORK [conn4077522] end connection 10.0.27.237:60699 (1506 connections now open) 2016-03-13T08:40:01.189+0000 I NETWORK [conn4077521] end connection 10.0.27.237:60698 (1505 connections now open) 2016-03-13T08:40:01.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60713 #4077534 (1506 connections now open) 2016-03-13T08:40:01.198+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60714 #4077535 (1507 connections now open) 2016-03-13T08:40:02.757+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:40:02.758+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:40:02.758+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:40:02.758+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:40:03.244+0000 I NETWORK [conn4077524] end connection 10.0.7.113:56079 (1506 connections now open) 2016-03-13T08:40:03.244+0000 I NETWORK [conn4077523] end connection 10.0.7.113:56080 (1506 connections now open) 2016-03-13T08:40:03.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56105 #4077536 (1506 connections now open) 2016-03-13T08:40:03.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56104 #4077537 (1507 connections now open) 2016-03-13T08:40:04.443+0000 I NETWORK [conn4077526] end connection 10.0.216.90:54132 (1506 connections now open) 2016-03-13T08:40:04.443+0000 I NETWORK [conn4077525] end connection 10.0.216.90:54131 (1506 connections now open) 2016-03-13T08:40:04.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54155 #4077538 (1506 connections now open) 2016-03-13T08:40:04.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54154 #4077539 (1507 connections now open) 2016-03-13T08:40:05.228+0000 I NETWORK [conn4077527] end connection 10.0.20.53:47364 (1506 connections now open) 2016-03-13T08:40:05.228+0000 I NETWORK [conn4077528] end connection 10.0.20.53:47365 (1506 connections now open) 2016-03-13T08:40:05.229+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47377 #4077540 (1506 connections now open) 2016-03-13T08:40:05.229+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47376 #4077541 (1507 connections now open) 2016-03-13T08:40:07.718+0000 I NETWORK [conn4077530] end connection 10.0.179.209:37214 (1506 connections now open) 2016-03-13T08:40:07.718+0000 I NETWORK [conn4077529] end connection 10.0.179.209:37213 (1506 connections now open) 2016-03-13T08:40:07.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37228 #4077542 (1506 connections now open) 2016-03-13T08:40:07.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37227 #4077543 (1507 connections now open) 2016-03-13T08:40:08.194+0000 I NETWORK [conn4077532] end connection 10.0.26.69:39154 (1506 connections now open) 2016-03-13T08:40:08.194+0000 I NETWORK [conn4077533] end connection 10.0.26.69:39155 (1506 connections now open) 2016-03-13T08:40:08.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39170 #4077544 (1506 connections now open) 2016-03-13T08:40:08.195+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39169 #4077545 (1507 connections now open) 2016-03-13T08:40:11.210+0000 I NETWORK [conn4077534] end connection 10.0.27.237:60713 (1506 connections now open) 2016-03-13T08:40:11.210+0000 I NETWORK [conn4077535] end connection 10.0.27.237:60714 (1506 connections now open) 2016-03-13T08:40:11.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60732 #4077546 (1506 connections now open) 2016-03-13T08:40:11.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60731 #4077547 (1507 connections now open) 2016-03-13T08:40:12.769+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:40:12.770+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:40:12.770+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:40:12.770+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:40:13.266+0000 I NETWORK [conn4077537] end connection 10.0.7.113:56104 (1506 connections now open) 2016-03-13T08:40:13.266+0000 I NETWORK [conn4077536] end connection 10.0.7.113:56105 (1506 connections now open) 2016-03-13T08:40:13.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56116 #4077548 (1506 connections now open) 2016-03-13T08:40:13.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56115 #4077549 (1507 connections now open) 2016-03-13T08:40:13.668+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:40:13.518+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:40:14.456+0000 I NETWORK [conn4077539] end connection 10.0.216.90:54154 (1506 connections now open) 2016-03-13T08:40:14.456+0000 I NETWORK [conn4077538] end connection 10.0.216.90:54155 (1506 connections now open) 2016-03-13T08:40:14.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54163 #4077550 (1506 connections now open) 2016-03-13T08:40:14.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54162 #4077551 (1507 connections now open) 2016-03-13T08:40:15.242+0000 I NETWORK [conn4077541] end connection 10.0.20.53:47376 (1506 connections now open) 2016-03-13T08:40:15.242+0000 I NETWORK [conn4077540] end connection 10.0.20.53:47377 (1506 connections now open) 2016-03-13T08:40:15.243+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47388 #4077552 (1506 connections now open) 2016-03-13T08:40:15.243+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47387 #4077553 (1507 connections now open) 2016-03-13T08:40:16.118+0000 I NETWORK [conn4069083] end connection 10.0.179.209:56927 (1506 connections now open) 2016-03-13T08:40:16.118+0000 I NETWORK [conn4072887] end connection 10.0.216.90:49145 (1506 connections now open) 2016-03-13T08:40:16.118+0000 I NETWORK [conn4066117] end connection 10.0.26.69:55136 (1504 connections now open) 2016-03-13T08:40:16.118+0000 I NETWORK [conn4069701] end connection 10.0.27.237:52642 (1503 connections now open) 2016-03-13T08:40:16.118+0000 I NETWORK [conn4066803] end connection 10.0.7.113:44668 (1503 connections now open) 2016-03-13T08:40:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54166 #4077554 (1503 connections now open) 2016-03-13T08:40:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39192 #4077555 (1504 connections now open) 2016-03-13T08:40:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37235 #4077556 (1505 connections now open) 2016-03-13T08:40:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60742 #4077557 (1506 connections now open) 2016-03-13T08:40:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56127 #4077558 (1507 connections now open) 2016-03-13T08:40:17.740+0000 I NETWORK [conn4077543] end connection 10.0.179.209:37227 (1506 connections now open) 2016-03-13T08:40:17.740+0000 I NETWORK [conn4077542] end connection 10.0.179.209:37228 (1506 connections now open) 2016-03-13T08:40:17.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37245 #4077559 (1506 connections now open) 2016-03-13T08:40:17.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37244 #4077560 (1507 connections now open) 2016-03-13T08:40:18.207+0000 I NETWORK [conn4077544] end connection 10.0.26.69:39170 (1506 connections now open) 2016-03-13T08:40:18.207+0000 I NETWORK [conn4077545] end connection 10.0.26.69:39169 (1506 connections now open) 2016-03-13T08:40:18.208+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39195 #4077561 (1507 connections now open) 2016-03-13T08:40:18.208+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39194 #4077562 (1507 connections now open) 2016-03-13T08:40:21.223+0000 I NETWORK [conn4077547] end connection 10.0.27.237:60731 (1506 connections now open) 2016-03-13T08:40:21.223+0000 I NETWORK [conn4077546] end connection 10.0.27.237:60732 (1506 connections now open) 2016-03-13T08:40:21.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60744 #4077563 (1506 connections now open) 2016-03-13T08:40:21.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60745 #4077564 (1507 connections now open) 2016-03-13T08:40:22.773+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:40:22.774+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:40:22.774+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:40:22.774+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:40:23.280+0000 I NETWORK [conn4077548] end connection 10.0.7.113:56116 (1506 connections now open) 2016-03-13T08:40:23.280+0000 I NETWORK [conn4077549] end connection 10.0.7.113:56115 (1506 connections now open) 2016-03-13T08:40:23.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56132 #4077565 (1507 connections now open) 2016-03-13T08:40:23.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56131 #4077566 (1507 connections now open) 2016-03-13T08:40:24.469+0000 I NETWORK [conn4077550] end connection 10.0.216.90:54163 (1506 connections now open) 2016-03-13T08:40:24.469+0000 I NETWORK [conn4077551] end connection 10.0.216.90:54162 (1506 connections now open) 2016-03-13T08:40:24.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54176 #4077567 (1506 connections now open) 2016-03-13T08:40:24.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54177 #4077568 (1507 connections now open) 2016-03-13T08:40:25.256+0000 I NETWORK [conn4077553] end connection 10.0.20.53:47387 (1506 connections now open) 2016-03-13T08:40:25.256+0000 I NETWORK [conn4077552] end connection 10.0.20.53:47388 (1506 connections now open) 2016-03-13T08:40:25.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47410 #4077569 (1506 connections now open) 2016-03-13T08:40:25.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47411 #4077570 (1507 connections now open) 2016-03-13T08:40:27.753+0000 I NETWORK [conn4077559] end connection 10.0.179.209:37245 (1506 connections now open) 2016-03-13T08:40:27.753+0000 I NETWORK [conn4077560] end connection 10.0.179.209:37244 (1506 connections now open) 2016-03-13T08:40:27.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37262 #4077571 (1506 connections now open) 2016-03-13T08:40:27.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37263 #4077572 (1507 connections now open) 2016-03-13T08:40:28.113+0000 I NETWORK [conn4069339] end connection 10.0.179.209:57185 (1506 connections now open) 2016-03-13T08:40:28.113+0000 I NETWORK [conn4073769] end connection 10.0.20.53:43432 (1505 connections now open) 2016-03-13T08:40:28.113+0000 I NETWORK [conn4070774] end connection 10.0.27.237:53783 (1504 connections now open) 2016-03-13T08:40:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37266 #4077573 (1505 connections now open) 2016-03-13T08:40:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47415 #4077574 (1506 connections now open) 2016-03-13T08:40:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60752 #4077575 (1507 connections now open) 2016-03-13T08:40:28.221+0000 I NETWORK [conn4077562] end connection 10.0.26.69:39194 (1506 connections now open) 2016-03-13T08:40:28.221+0000 I NETWORK [conn4077561] end connection 10.0.26.69:39195 (1506 connections now open) 2016-03-13T08:40:28.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39205 #4077576 (1506 connections now open) 2016-03-13T08:40:28.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39204 #4077577 (1507 connections now open) 2016-03-13T08:40:31.236+0000 I NETWORK [conn4077564] end connection 10.0.27.237:60745 (1506 connections now open) 2016-03-13T08:40:31.236+0000 I NETWORK [conn4077563] end connection 10.0.27.237:60744 (1506 connections now open) 2016-03-13T08:40:31.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60759 #4077578 (1506 connections now open) 2016-03-13T08:40:31.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60758 #4077579 (1507 connections now open) 2016-03-13T08:40:32.777+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:40:32.777+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:40:32.777+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:40:32.777+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:40:33.293+0000 I NETWORK [conn4077565] end connection 10.0.7.113:56132 (1506 connections now open) 2016-03-13T08:40:33.293+0000 I NETWORK [conn4077566] end connection 10.0.7.113:56131 (1506 connections now open) 2016-03-13T08:40:33.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56153 #4077580 (1506 connections now open) 2016-03-13T08:40:33.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56154 #4077581 (1507 connections now open) 2016-03-13T08:40:34.482+0000 I NETWORK [conn4077567] end connection 10.0.216.90:54176 (1506 connections now open) 2016-03-13T08:40:34.482+0000 I NETWORK [conn4077568] end connection 10.0.216.90:54177 (1506 connections now open) 2016-03-13T08:40:34.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54199 #4077582 (1506 connections now open) 2016-03-13T08:40:34.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54200 #4077583 (1507 connections now open) 2016-03-13T08:40:35.269+0000 I NETWORK [conn4077570] end connection 10.0.20.53:47411 (1506 connections now open) 2016-03-13T08:40:35.269+0000 I NETWORK [conn4077569] end connection 10.0.20.53:47410 (1506 connections now open) 2016-03-13T08:40:35.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47424 #4077584 (1506 connections now open) 2016-03-13T08:40:35.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47425 #4077585 (1507 connections now open) 2016-03-13T08:40:37.766+0000 I NETWORK [conn4077571] end connection 10.0.179.209:37262 (1506 connections now open) 2016-03-13T08:40:37.766+0000 I NETWORK [conn4077572] end connection 10.0.179.209:37263 (1506 connections now open) 2016-03-13T08:40:37.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37272 #4077586 (1506 connections now open) 2016-03-13T08:40:37.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37273 #4077587 (1507 connections now open) 2016-03-13T08:40:38.234+0000 I NETWORK [conn4077577] end connection 10.0.26.69:39204 (1506 connections now open) 2016-03-13T08:40:38.234+0000 I NETWORK [conn4077576] end connection 10.0.26.69:39205 (1506 connections now open) 2016-03-13T08:40:38.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39217 #4077588 (1506 connections now open) 2016-03-13T08:40:38.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39216 #4077589 (1507 connections now open) 2016-03-13T08:40:41.249+0000 I NETWORK [conn4077578] end connection 10.0.27.237:60759 (1506 connections now open) 2016-03-13T08:40:41.250+0000 I NETWORK [conn4077579] end connection 10.0.27.237:60758 (1505 connections now open) 2016-03-13T08:40:41.250+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60776 #4077590 (1506 connections now open) 2016-03-13T08:40:41.250+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60777 #4077591 (1507 connections now open) 2016-03-13T08:40:42.789+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:40:42.789+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:40:42.789+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:40:42.789+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:40:43.306+0000 I NETWORK [conn4077580] end connection 10.0.7.113:56153 (1506 connections now open) 2016-03-13T08:40:43.306+0000 I NETWORK [conn4077581] end connection 10.0.7.113:56154 (1505 connections now open) 2016-03-13T08:40:43.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56160 #4077592 (1506 connections now open) 2016-03-13T08:40:43.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56161 #4077593 (1507 connections now open) 2016-03-13T08:40:43.923+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:40:43.668+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:40:44.495+0000 I NETWORK [conn4077583] end connection 10.0.216.90:54200 (1506 connections now open) 2016-03-13T08:40:44.495+0000 I NETWORK [conn4077582] end connection 10.0.216.90:54199 (1506 connections now open) 2016-03-13T08:40:44.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54209 #4077594 (1506 connections now open) 2016-03-13T08:40:44.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54208 #4077595 (1507 connections now open) 2016-03-13T08:40:45.282+0000 I NETWORK [conn4077585] end connection 10.0.20.53:47425 (1506 connections now open) 2016-03-13T08:40:45.282+0000 I NETWORK [conn4077584] end connection 10.0.20.53:47424 (1506 connections now open) 2016-03-13T08:40:45.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47436 #4077596 (1506 connections now open) 2016-03-13T08:40:45.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47435 #4077597 (1507 connections now open) 2016-03-13T08:40:46.118+0000 I NETWORK [conn4068007] end connection 10.0.179.209:55848 (1506 connections now open) 2016-03-13T08:40:46.118+0000 I NETWORK [conn4065458] end connection 10.0.20.53:34809 (1506 connections now open) 2016-03-13T08:40:46.118+0000 I NETWORK [conn4066077] end connection 10.0.26.69:55094 (1504 connections now open) 2016-03-13T08:40:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37279 #4077598 (1505 connections now open) 2016-03-13T08:40:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47440 #4077599 (1506 connections now open) 2016-03-13T08:40:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39235 #4077600 (1507 connections now open) 2016-03-13T08:40:47.779+0000 I NETWORK [conn4077587] end connection 10.0.179.209:37273 (1506 connections now open) 2016-03-13T08:40:47.779+0000 I NETWORK [conn4077586] end connection 10.0.179.209:37272 (1506 connections now open) 2016-03-13T08:40:47.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37285 #4077601 (1506 connections now open) 2016-03-13T08:40:47.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37284 #4077602 (1507 connections now open) 2016-03-13T08:40:48.357+0000 I NETWORK [conn4077588] end connection 10.0.26.69:39217 (1506 connections now open) 2016-03-13T08:40:48.357+0000 I NETWORK [conn4077589] end connection 10.0.26.69:39216 (1506 connections now open) 2016-03-13T08:40:48.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39237 #4077603 (1506 connections now open) 2016-03-13T08:40:48.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39238 #4077604 (1507 connections now open) 2016-03-13T08:40:51.263+0000 I NETWORK [conn4077590] end connection 10.0.27.237:60776 (1506 connections now open) 2016-03-13T08:40:51.263+0000 I NETWORK [conn4077591] end connection 10.0.27.237:60777 (1506 connections now open) 2016-03-13T08:40:51.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60784 #4077605 (1506 connections now open) 2016-03-13T08:40:51.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60785 #4077606 (1507 connections now open) 2016-03-13T08:40:52.792+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:40:52.793+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:40:52.793+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:40:52.793+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:40:53.319+0000 I NETWORK [conn4077592] end connection 10.0.7.113:56160 (1506 connections now open) 2016-03-13T08:40:53.319+0000 I NETWORK [conn4077593] end connection 10.0.7.113:56161 (1506 connections now open) 2016-03-13T08:40:53.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56174 #4077607 (1506 connections now open) 2016-03-13T08:40:53.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56175 #4077608 (1507 connections now open) 2016-03-13T08:40:54.508+0000 I NETWORK [conn4077594] end connection 10.0.216.90:54209 (1506 connections now open) 2016-03-13T08:40:54.508+0000 I NETWORK [conn4077595] end connection 10.0.216.90:54208 (1506 connections now open) 2016-03-13T08:40:54.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54221 #4077609 (1506 connections now open) 2016-03-13T08:40:54.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54220 #4077610 (1507 connections now open) 2016-03-13T08:40:55.295+0000 I NETWORK [conn4077596] end connection 10.0.20.53:47436 (1506 connections now open) 2016-03-13T08:40:55.295+0000 I NETWORK [conn4077597] end connection 10.0.20.53:47435 (1506 connections now open) 2016-03-13T08:40:55.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47456 #4077611 (1506 connections now open) 2016-03-13T08:40:55.305+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47457 #4077612 (1507 connections now open) 2016-03-13T08:40:57.791+0000 I NETWORK [conn4077602] end connection 10.0.179.209:37284 (1506 connections now open) 2016-03-13T08:40:57.791+0000 I NETWORK [conn4077601] end connection 10.0.179.209:37285 (1506 connections now open) 2016-03-13T08:40:57.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37302 #4077613 (1506 connections now open) 2016-03-13T08:40:57.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37303 #4077614 (1507 connections now open) 2016-03-13T08:40:58.113+0000 I NETWORK [conn4077145] end connection 10.0.216.90:53736 (1506 connections now open) 2016-03-13T08:40:58.113+0000 I NETWORK [conn4071073] end connection 10.0.26.69:60459 (1506 connections now open) 2016-03-13T08:40:58.114+0000 I NETWORK [conn4070900] end connection 10.0.27.237:53922 (1504 connections now open) 2016-03-13T08:40:58.114+0000 I NETWORK [conn4065824] end connection 10.0.7.113:43612 (1503 connections now open) 2016-03-13T08:40:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54238 #4077615 (1504 connections now open) 2016-03-13T08:40:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39246 #4077616 (1505 connections now open) 2016-03-13T08:40:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60791 #4077617 (1506 connections now open) 2016-03-13T08:40:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56193 #4077618 (1507 connections now open) 2016-03-13T08:40:58.370+0000 I NETWORK [conn4077603] end connection 10.0.26.69:39237 (1506 connections now open) 2016-03-13T08:40:58.370+0000 I NETWORK [conn4077604] end connection 10.0.26.69:39238 (1506 connections now open) 2016-03-13T08:40:58.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39247 #4077619 (1506 connections now open) 2016-03-13T08:40:58.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39248 #4077620 (1507 connections now open) 2016-03-13T08:41:01.275+0000 I NETWORK [conn4077606] end connection 10.0.27.237:60785 (1506 connections now open) 2016-03-13T08:41:01.275+0000 I NETWORK [conn4077605] end connection 10.0.27.237:60784 (1505 connections now open) 2016-03-13T08:41:01.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60796 #4077621 (1506 connections now open) 2016-03-13T08:41:01.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60797 #4077622 (1507 connections now open) 2016-03-13T08:41:02.797+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:41:02.798+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:41:02.798+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:41:02.798+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:41:03.332+0000 I NETWORK [conn4077607] end connection 10.0.7.113:56174 (1506 connections now open) 2016-03-13T08:41:03.332+0000 I NETWORK [conn4077608] end connection 10.0.7.113:56175 (1506 connections now open) 2016-03-13T08:41:03.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56197 #4077623 (1506 connections now open) 2016-03-13T08:41:03.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56198 #4077624 (1507 connections now open) 2016-03-13T08:41:04.530+0000 I NETWORK [conn4077610] end connection 10.0.216.90:54220 (1506 connections now open) 2016-03-13T08:41:04.530+0000 I NETWORK [conn4077609] end connection 10.0.216.90:54221 (1506 connections now open) 2016-03-13T08:41:04.540+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54244 #4077625 (1506 connections now open) 2016-03-13T08:41:04.540+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54243 #4077626 (1507 connections now open) 2016-03-13T08:41:05.317+0000 I NETWORK [conn4077612] end connection 10.0.20.53:47457 (1506 connections now open) 2016-03-13T08:41:05.317+0000 I NETWORK [conn4077611] end connection 10.0.20.53:47456 (1506 connections now open) 2016-03-13T08:41:05.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47468 #4077627 (1506 connections now open) 2016-03-13T08:41:05.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47469 #4077628 (1507 connections now open) 2016-03-13T08:41:07.804+0000 I NETWORK [conn4077614] end connection 10.0.179.209:37303 (1506 connections now open) 2016-03-13T08:41:07.804+0000 I NETWORK [conn4077613] end connection 10.0.179.209:37302 (1506 connections now open) 2016-03-13T08:41:07.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37314 #4077629 (1506 connections now open) 2016-03-13T08:41:07.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37315 #4077630 (1507 connections now open) 2016-03-13T08:41:08.383+0000 I NETWORK [conn4077619] end connection 10.0.26.69:39247 (1506 connections now open) 2016-03-13T08:41:08.383+0000 I NETWORK [conn4077620] end connection 10.0.26.69:39248 (1506 connections now open) 2016-03-13T08:41:08.392+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39260 #4077631 (1506 connections now open) 2016-03-13T08:41:08.392+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39261 #4077632 (1507 connections now open) 2016-03-13T08:41:11.297+0000 I NETWORK [conn4077622] end connection 10.0.27.237:60797 (1506 connections now open) 2016-03-13T08:41:11.298+0000 I NETWORK [conn4077621] end connection 10.0.27.237:60796 (1505 connections now open) 2016-03-13T08:41:11.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60814 #4077633 (1506 connections now open) 2016-03-13T08:41:11.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60815 #4077634 (1507 connections now open) 2016-03-13T08:41:12.800+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:41:12.809+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:41:12.809+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:41:12.810+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:41:13.355+0000 I NETWORK [conn4077624] end connection 10.0.7.113:56198 (1506 connections now open) 2016-03-13T08:41:13.355+0000 I NETWORK [conn4077623] end connection 10.0.7.113:56197 (1506 connections now open) 2016-03-13T08:41:13.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56210 #4077635 (1506 connections now open) 2016-03-13T08:41:13.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56209 #4077636 (1507 connections now open) 2016-03-13T08:41:14.104+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:41:13.923+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:41:14.552+0000 I NETWORK [conn4077625] end connection 10.0.216.90:54244 (1506 connections now open) 2016-03-13T08:41:14.552+0000 I NETWORK [conn4077626] end connection 10.0.216.90:54243 (1506 connections now open) 2016-03-13T08:41:14.553+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54252 #4077637 (1506 connections now open) 2016-03-13T08:41:14.553+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54253 #4077638 (1507 connections now open) 2016-03-13T08:41:15.395+0000 I NETWORK [conn4077627] end connection 10.0.20.53:47468 (1506 connections now open) 2016-03-13T08:41:15.395+0000 I NETWORK [conn4077628] end connection 10.0.20.53:47469 (1506 connections now open) 2016-03-13T08:41:15.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47479 #4077639 (1506 connections now open) 2016-03-13T08:41:15.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47480 #4077640 (1507 connections now open) 2016-03-13T08:41:16.118+0000 I NETWORK [conn4068094] end connection 10.0.179.209:55933 (1506 connections now open) 2016-03-13T08:41:16.118+0000 I NETWORK [conn4066827] end connection 10.0.26.69:55918 (1506 connections now open) 2016-03-13T08:41:16.118+0000 I NETWORK [conn4068199] end connection 10.0.7.113:46114 (1504 connections now open) 2016-03-13T08:41:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37321 #4077641 (1505 connections now open) 2016-03-13T08:41:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39282 #4077642 (1506 connections now open) 2016-03-13T08:41:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56221 #4077643 (1507 connections now open) 2016-03-13T08:41:17.825+0000 I NETWORK [conn4077630] end connection 10.0.179.209:37315 (1506 connections now open) 2016-03-13T08:41:17.825+0000 I NETWORK [conn4077629] end connection 10.0.179.209:37314 (1506 connections now open) 2016-03-13T08:41:17.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37330 #4077644 (1507 connections now open) 2016-03-13T08:41:17.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37331 #4077645 (1507 connections now open) 2016-03-13T08:41:18.405+0000 I NETWORK [conn4077632] end connection 10.0.26.69:39261 (1506 connections now open) 2016-03-13T08:41:18.405+0000 I NETWORK [conn4077631] end connection 10.0.26.69:39260 (1506 connections now open) 2016-03-13T08:41:18.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39284 #4077646 (1506 connections now open) 2016-03-13T08:41:18.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39285 #4077647 (1507 connections now open) 2016-03-13T08:41:21.311+0000 I NETWORK [conn4077634] end connection 10.0.27.237:60815 (1506 connections now open) 2016-03-13T08:41:21.311+0000 I NETWORK [conn4077633] end connection 10.0.27.237:60814 (1506 connections now open) 2016-03-13T08:41:21.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60828 #4077648 (1506 connections now open) 2016-03-13T08:41:21.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60827 #4077649 (1507 connections now open) 2016-03-13T08:41:22.813+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:41:22.814+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:41:22.814+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:41:22.814+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 9 checks in a row. Polling will stop after 21 more failed checks 2016-03-13T08:41:23.368+0000 I NETWORK [conn4077636] end connection 10.0.7.113:56209 (1506 connections now open) 2016-03-13T08:41:23.368+0000 I NETWORK [conn4077635] end connection 10.0.7.113:56210 (1506 connections now open) 2016-03-13T08:41:23.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56224 #4077650 (1506 connections now open) 2016-03-13T08:41:23.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56225 #4077651 (1507 connections now open) 2016-03-13T08:41:24.565+0000 I NETWORK [conn4077638] end connection 10.0.216.90:54253 (1506 connections now open) 2016-03-13T08:41:24.566+0000 I NETWORK [conn4077637] end connection 10.0.216.90:54252 (1506 connections now open) 2016-03-13T08:41:24.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54267 #4077652 (1506 connections now open) 2016-03-13T08:41:24.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54266 #4077653 (1507 connections now open) 2016-03-13T08:41:25.408+0000 I NETWORK [conn4077640] end connection 10.0.20.53:47480 (1506 connections now open) 2016-03-13T08:41:25.408+0000 I NETWORK [conn4077639] end connection 10.0.20.53:47479 (1506 connections now open) 2016-03-13T08:41:25.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47501 #4077654 (1506 connections now open) 2016-03-13T08:41:25.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47502 #4077655 (1507 connections now open) 2016-03-13T08:41:27.838+0000 I NETWORK [conn4077644] end connection 10.0.179.209:37330 (1506 connections now open) 2016-03-13T08:41:27.838+0000 I NETWORK [conn4077645] end connection 10.0.179.209:37331 (1505 connections now open) 2016-03-13T08:41:27.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37338 #4077656 (1506 connections now open) 2016-03-13T08:41:27.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37339 #4077657 (1507 connections now open) 2016-03-13T08:41:28.114+0000 I NETWORK [conn4065738] end connection 10.0.20.53:35088 (1506 connections now open) 2016-03-13T08:41:28.114+0000 I NETWORK [conn4068916] end connection 10.0.26.69:58134 (1506 connections now open) 2016-03-13T08:41:28.114+0000 I NETWORK [conn4075326] end connection 10.0.7.113:53745 (1506 connections now open) 2016-03-13T08:41:28.114+0000 I NETWORK [conn4067215] end connection 10.0.216.90:43089 (1506 connections now open) 2016-03-13T08:41:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54284 #4077658 (1504 connections now open) 2016-03-13T08:41:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47506 #4077659 (1505 connections now open) 2016-03-13T08:41:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39293 #4077660 (1506 connections now open) 2016-03-13T08:41:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56241 #4077661 (1507 connections now open) 2016-03-13T08:41:28.418+0000 I NETWORK [conn4077647] end connection 10.0.26.69:39285 (1506 connections now open) 2016-03-13T08:41:28.418+0000 I NETWORK [conn4077646] end connection 10.0.26.69:39284 (1506 connections now open) 2016-03-13T08:41:28.419+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39294 #4077662 (1506 connections now open) 2016-03-13T08:41:28.419+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39295 #4077663 (1507 connections now open) 2016-03-13T08:41:31.324+0000 I NETWORK [conn4077649] end connection 10.0.27.237:60827 (1506 connections now open) 2016-03-13T08:41:31.324+0000 I NETWORK [conn4077648] end connection 10.0.27.237:60828 (1506 connections now open) 2016-03-13T08:41:31.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60842 #4077664 (1506 connections now open) 2016-03-13T08:41:31.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60841 #4077665 (1507 connections now open) 2016-03-13T08:41:32.817+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:41:32.817+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:41:32.817+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:41:32.817+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 10 checks in a row. Polling will stop after 20 more failed checks 2016-03-13T08:41:33.381+0000 I NETWORK [conn4077651] end connection 10.0.7.113:56225 (1506 connections now open) 2016-03-13T08:41:33.381+0000 I NETWORK [conn4077650] end connection 10.0.7.113:56224 (1506 connections now open) 2016-03-13T08:41:33.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56243 #4077666 (1506 connections now open) 2016-03-13T08:41:33.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56244 #4077667 (1507 connections now open) 2016-03-13T08:41:34.579+0000 I NETWORK [conn4077653] end connection 10.0.216.90:54266 (1506 connections now open) 2016-03-13T08:41:34.579+0000 I NETWORK [conn4077652] end connection 10.0.216.90:54267 (1506 connections now open) 2016-03-13T08:41:34.580+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54291 #4077668 (1506 connections now open) 2016-03-13T08:41:34.580+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54290 #4077669 (1507 connections now open) 2016-03-13T08:41:35.421+0000 I NETWORK [conn4077654] end connection 10.0.20.53:47501 (1506 connections now open) 2016-03-13T08:41:35.421+0000 I NETWORK [conn4077655] end connection 10.0.20.53:47502 (1505 connections now open) 2016-03-13T08:41:35.421+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47515 #4077670 (1506 connections now open) 2016-03-13T08:41:35.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47516 #4077671 (1507 connections now open) 2016-03-13T08:41:37.851+0000 I NETWORK [conn4077656] end connection 10.0.179.209:37338 (1506 connections now open) 2016-03-13T08:41:37.851+0000 I NETWORK [conn4077657] end connection 10.0.179.209:37339 (1505 connections now open) 2016-03-13T08:41:37.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37357 #4077672 (1506 connections now open) 2016-03-13T08:41:37.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37358 #4077673 (1507 connections now open) 2016-03-13T08:41:38.431+0000 I NETWORK [conn4077662] end connection 10.0.26.69:39294 (1506 connections now open) 2016-03-13T08:41:38.432+0000 I NETWORK [conn4077663] end connection 10.0.26.69:39295 (1505 connections now open) 2016-03-13T08:41:38.441+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39306 #4077674 (1506 connections now open) 2016-03-13T08:41:38.441+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39305 #4077675 (1507 connections now open) 2016-03-13T08:41:41.336+0000 I NETWORK [conn4077664] end connection 10.0.27.237:60842 (1506 connections now open) 2016-03-13T08:41:41.337+0000 I NETWORK [conn4077665] end connection 10.0.27.237:60841 (1506 connections now open) 2016-03-13T08:41:41.337+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60859 #4077676 (1506 connections now open) 2016-03-13T08:41:41.337+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60860 #4077677 (1507 connections now open) 2016-03-13T08:41:42.829+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:41:42.829+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:41:42.829+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:41:42.829+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 11 checks in a row. Polling will stop after 19 more failed checks 2016-03-13T08:41:43.395+0000 I NETWORK [conn4077666] end connection 10.0.7.113:56243 (1506 connections now open) 2016-03-13T08:41:43.395+0000 I NETWORK [conn4077667] end connection 10.0.7.113:56244 (1506 connections now open) 2016-03-13T08:41:43.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56251 #4077678 (1506 connections now open) 2016-03-13T08:41:43.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56252 #4077679 (1507 connections now open) 2016-03-13T08:41:44.246+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:41:44.104+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:41:44.592+0000 I NETWORK [conn4077669] end connection 10.0.216.90:54290 (1506 connections now open) 2016-03-13T08:41:44.592+0000 I NETWORK [conn4077668] end connection 10.0.216.90:54291 (1506 connections now open) 2016-03-13T08:41:44.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54298 #4077680 (1506 connections now open) 2016-03-13T08:41:44.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54299 #4077681 (1507 connections now open) 2016-03-13T08:41:45.434+0000 I NETWORK [conn4077671] end connection 10.0.20.53:47516 (1506 connections now open) 2016-03-13T08:41:45.434+0000 I NETWORK [conn4077670] end connection 10.0.20.53:47515 (1506 connections now open) 2016-03-13T08:41:45.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47525 #4077682 (1506 connections now open) 2016-03-13T08:41:45.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47526 #4077683 (1507 connections now open) 2016-03-13T08:41:46.118+0000 I NETWORK [conn4064493] end connection 10.0.179.209:52098 (1506 connections now open) 2016-03-13T08:41:46.118+0000 I NETWORK [conn4067899] end connection 10.0.20.53:37288 (1505 connections now open) 2016-03-13T08:41:46.118+0000 I NETWORK [conn4068577] end connection 10.0.216.90:44526 (1504 connections now open) 2016-03-13T08:41:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37364 #4077684 (1505 connections now open) 2016-03-13T08:41:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47528 #4077685 (1506 connections now open) 2016-03-13T08:41:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54301 #4077686 (1507 connections now open) 2016-03-13T08:41:47.864+0000 I NETWORK [conn4077672] end connection 10.0.179.209:37357 (1506 connections now open) 2016-03-13T08:41:47.864+0000 I NETWORK [conn4077673] end connection 10.0.179.209:37358 (1506 connections now open) 2016-03-13T08:41:47.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37366 #4077687 (1506 connections now open) 2016-03-13T08:41:47.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37365 #4077688 (1507 connections now open) 2016-03-13T08:41:48.453+0000 I NETWORK [conn4077675] end connection 10.0.26.69:39305 (1506 connections now open) 2016-03-13T08:41:48.453+0000 I NETWORK [conn4077674] end connection 10.0.26.69:39306 (1506 connections now open) 2016-03-13T08:41:48.454+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39328 #4077689 (1506 connections now open) 2016-03-13T08:41:48.454+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39327 #4077690 (1507 connections now open) 2016-03-13T08:41:51.349+0000 I NETWORK [conn4077676] end connection 10.0.27.237:60859 (1506 connections now open) 2016-03-13T08:41:51.349+0000 I NETWORK [conn4077677] end connection 10.0.27.237:60860 (1506 connections now open) 2016-03-13T08:41:51.350+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60867 #4077691 (1506 connections now open) 2016-03-13T08:41:51.350+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60868 #4077692 (1507 connections now open) 2016-03-13T08:41:52.832+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:41:52.833+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:41:52.833+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:41:52.833+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 12 checks in a row. Polling will stop after 18 more failed checks 2016-03-13T08:41:53.408+0000 I NETWORK [conn4077678] end connection 10.0.7.113:56251 (1506 connections now open) 2016-03-13T08:41:53.408+0000 I NETWORK [conn4077679] end connection 10.0.7.113:56252 (1506 connections now open) 2016-03-13T08:41:53.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56265 #4077693 (1506 connections now open) 2016-03-13T08:41:53.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56266 #4077694 (1507 connections now open) 2016-03-13T08:41:54.606+0000 I NETWORK [conn4077680] end connection 10.0.216.90:54298 (1506 connections now open) 2016-03-13T08:41:54.606+0000 I NETWORK [conn4077681] end connection 10.0.216.90:54299 (1506 connections now open) 2016-03-13T08:41:54.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54312 #4077695 (1506 connections now open) 2016-03-13T08:41:54.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54311 #4077696 (1507 connections now open) 2016-03-13T08:41:55.448+0000 I NETWORK [conn4077683] end connection 10.0.20.53:47526 (1506 connections now open) 2016-03-13T08:41:55.448+0000 I NETWORK [conn4077682] end connection 10.0.20.53:47525 (1506 connections now open) 2016-03-13T08:41:55.458+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47545 #4077697 (1506 connections now open) 2016-03-13T08:41:55.458+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47544 #4077698 (1507 connections now open) 2016-03-13T08:41:57.876+0000 I NETWORK [conn4077688] end connection 10.0.179.209:37365 (1506 connections now open) 2016-03-13T08:41:57.876+0000 I NETWORK [conn4077687] end connection 10.0.179.209:37366 (1506 connections now open) 2016-03-13T08:41:57.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37377 #4077699 (1507 connections now open) 2016-03-13T08:41:57.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37378 #4077700 (1507 connections now open) 2016-03-13T08:41:58.113+0000 I NETWORK [conn4067192] end connection 10.0.179.209:54975 (1506 connections now open) 2016-03-13T08:41:58.113+0000 I NETWORK [conn4072762] end connection 10.0.20.53:42350 (1505 connections now open) 2016-03-13T08:41:58.113+0000 I NETWORK [conn4070248] end connection 10.0.216.90:46304 (1505 connections now open) 2016-03-13T08:41:58.113+0000 I NETWORK [conn4070856] end connection 10.0.26.69:60230 (1503 connections now open) 2016-03-13T08:41:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37388 #4077701 (1505 connections now open) 2016-03-13T08:41:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54327 #4077702 (1505 connections now open) 2016-03-13T08:41:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47549 #4077703 (1506 connections now open) 2016-03-13T08:41:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39336 #4077704 (1507 connections now open) 2016-03-13T08:41:58.114+0000 I NETWORK [conn4071185] end connection 10.0.7.113:49323 (1506 connections now open) 2016-03-13T08:41:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56283 #4077705 (1507 connections now open) 2016-03-13T08:41:58.467+0000 I NETWORK [conn4077690] end connection 10.0.26.69:39327 (1506 connections now open) 2016-03-13T08:41:58.467+0000 I NETWORK [conn4077689] end connection 10.0.26.69:39328 (1506 connections now open) 2016-03-13T08:41:58.467+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39339 #4077706 (1506 connections now open) 2016-03-13T08:41:58.467+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39338 #4077707 (1507 connections now open) 2016-03-13T08:42:01.362+0000 I NETWORK [conn4077692] end connection 10.0.27.237:60868 (1506 connections now open) 2016-03-13T08:42:01.362+0000 I NETWORK [conn4077691] end connection 10.0.27.237:60867 (1506 connections now open) 2016-03-13T08:42:01.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60881 #4077708 (1506 connections now open) 2016-03-13T08:42:01.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60880 #4077709 (1507 connections now open) 2016-03-13T08:42:02.836+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:42:02.836+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:42:02.837+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:42:02.837+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 13 checks in a row. Polling will stop after 17 more failed checks 2016-03-13T08:42:03.422+0000 I NETWORK [conn4077693] end connection 10.0.7.113:56265 (1506 connections now open) 2016-03-13T08:42:03.422+0000 I NETWORK [conn4077694] end connection 10.0.7.113:56266 (1506 connections now open) 2016-03-13T08:42:03.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56288 #4077710 (1506 connections now open) 2016-03-13T08:42:03.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56287 #4077711 (1507 connections now open) 2016-03-13T08:42:04.628+0000 I NETWORK [conn4077695] end connection 10.0.216.90:54312 (1506 connections now open) 2016-03-13T08:42:04.628+0000 I NETWORK [conn4077696] end connection 10.0.216.90:54311 (1506 connections now open) 2016-03-13T08:42:04.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54333 #4077712 (1506 connections now open) 2016-03-13T08:42:04.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54334 #4077713 (1507 connections now open) 2016-03-13T08:42:05.470+0000 I NETWORK [conn4077697] end connection 10.0.20.53:47545 (1506 connections now open) 2016-03-13T08:42:05.470+0000 I NETWORK [conn4077698] end connection 10.0.20.53:47544 (1505 connections now open) 2016-03-13T08:42:05.471+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47554 #4077714 (1506 connections now open) 2016-03-13T08:42:05.471+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47555 #4077715 (1507 connections now open) 2016-03-13T08:42:07.889+0000 I NETWORK [conn4077699] end connection 10.0.179.209:37377 (1506 connections now open) 2016-03-13T08:42:07.889+0000 I NETWORK [conn4077700] end connection 10.0.179.209:37378 (1506 connections now open) 2016-03-13T08:42:07.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37399 #4077716 (1506 connections now open) 2016-03-13T08:42:07.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37398 #4077717 (1507 connections now open) 2016-03-13T08:42:08.480+0000 I NETWORK [conn4077706] end connection 10.0.26.69:39339 (1506 connections now open) 2016-03-13T08:42:08.480+0000 I NETWORK [conn4077707] end connection 10.0.26.69:39338 (1506 connections now open) 2016-03-13T08:42:08.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39351 #4077718 (1507 connections now open) 2016-03-13T08:42:08.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39350 #4077719 (1507 connections now open) 2016-03-13T08:42:11.375+0000 I NETWORK [conn4077708] end connection 10.0.27.237:60881 (1506 connections now open) 2016-03-13T08:42:11.376+0000 I NETWORK [conn4077709] end connection 10.0.27.237:60880 (1506 connections now open) 2016-03-13T08:42:11.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60899 #4077720 (1506 connections now open) 2016-03-13T08:42:11.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60900 #4077721 (1507 connections now open) 2016-03-13T08:42:12.840+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:42:12.849+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:42:12.849+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:42:12.849+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 14 checks in a row. Polling will stop after 16 more failed checks 2016-03-13T08:42:13.444+0000 I NETWORK [conn4077710] end connection 10.0.7.113:56288 (1506 connections now open) 2016-03-13T08:42:13.444+0000 I NETWORK [conn4077711] end connection 10.0.7.113:56287 (1505 connections now open) 2016-03-13T08:42:13.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56299 #4077722 (1506 connections now open) 2016-03-13T08:42:13.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56300 #4077723 (1507 connections now open) 2016-03-13T08:42:14.528+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:42:14.247+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:42:14.641+0000 I NETWORK [conn4077712] end connection 10.0.216.90:54333 (1506 connections now open) 2016-03-13T08:42:14.641+0000 I NETWORK [conn4077713] end connection 10.0.216.90:54334 (1506 connections now open) 2016-03-13T08:42:14.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54341 #4077724 (1506 connections now open) 2016-03-13T08:42:14.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54342 #4077725 (1507 connections now open) 2016-03-13T08:42:15.483+0000 I NETWORK [conn4077714] end connection 10.0.20.53:47554 (1506 connections now open) 2016-03-13T08:42:15.483+0000 I NETWORK [conn4077715] end connection 10.0.20.53:47555 (1506 connections now open) 2016-03-13T08:42:15.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47566 #4077726 (1506 connections now open) 2016-03-13T08:42:15.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47567 #4077727 (1507 connections now open) 2016-03-13T08:42:16.118+0000 I NETWORK [conn4068789] end connection 10.0.20.53:38223 (1506 connections now open) 2016-03-13T08:42:16.118+0000 I NETWORK [conn4065572] end connection 10.0.27.237:48421 (1505 connections now open) 2016-03-13T08:42:16.118+0000 I NETWORK [conn4064831] end connection 10.0.7.113:42523 (1504 connections now open) 2016-03-13T08:42:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47569 #4077728 (1505 connections now open) 2016-03-13T08:42:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56309 #4077729 (1506 connections now open) 2016-03-13T08:42:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60910 #4077730 (1507 connections now open) 2016-03-13T08:42:17.911+0000 I NETWORK [conn4077717] end connection 10.0.179.209:37398 (1506 connections now open) 2016-03-13T08:42:17.911+0000 I NETWORK [conn4077716] end connection 10.0.179.209:37399 (1506 connections now open) 2016-03-13T08:42:17.912+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37410 #4077731 (1506 connections now open) 2016-03-13T08:42:17.912+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37411 #4077732 (1507 connections now open) 2016-03-13T08:42:18.493+0000 I NETWORK [conn4077719] end connection 10.0.26.69:39350 (1506 connections now open) 2016-03-13T08:42:18.493+0000 I NETWORK [conn4077718] end connection 10.0.26.69:39351 (1506 connections now open) 2016-03-13T08:42:18.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39374 #4077733 (1506 connections now open) 2016-03-13T08:42:18.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39375 #4077734 (1507 connections now open) 2016-03-13T08:42:21.397+0000 I NETWORK [conn4077720] end connection 10.0.27.237:60899 (1506 connections now open) 2016-03-13T08:42:21.397+0000 I NETWORK [conn4077721] end connection 10.0.27.237:60900 (1506 connections now open) 2016-03-13T08:42:21.398+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60912 #4077735 (1506 connections now open) 2016-03-13T08:42:21.398+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60913 #4077736 (1507 connections now open) 2016-03-13T08:42:22.852+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:42:22.853+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:42:22.853+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:42:22.853+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 15 checks in a row. Polling will stop after 15 more failed checks 2016-03-13T08:42:23.457+0000 I NETWORK [conn4077723] end connection 10.0.7.113:56300 (1506 connections now open) 2016-03-13T08:42:23.457+0000 I NETWORK [conn4077722] end connection 10.0.7.113:56299 (1505 connections now open) 2016-03-13T08:42:23.458+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56312 #4077737 (1506 connections now open) 2016-03-13T08:42:23.458+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56313 #4077738 (1507 connections now open) 2016-03-13T08:42:24.654+0000 I NETWORK [conn4077725] end connection 10.0.216.90:54342 (1506 connections now open) 2016-03-13T08:42:24.654+0000 I NETWORK [conn4077724] end connection 10.0.216.90:54341 (1506 connections now open) 2016-03-13T08:42:24.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54353 #4077739 (1506 connections now open) 2016-03-13T08:42:24.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54354 #4077740 (1507 connections now open) 2016-03-13T08:42:25.496+0000 I NETWORK [conn4077726] end connection 10.0.20.53:47566 (1506 connections now open) 2016-03-13T08:42:25.496+0000 I NETWORK [conn4077727] end connection 10.0.20.53:47567 (1505 connections now open) 2016-03-13T08:42:25.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47584 #4077741 (1506 connections now open) 2016-03-13T08:42:25.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47585 #4077742 (1507 connections now open) 2016-03-13T08:42:27.924+0000 I NETWORK [conn4077732] end connection 10.0.179.209:37411 (1506 connections now open) 2016-03-13T08:42:27.924+0000 I NETWORK [conn4077731] end connection 10.0.179.209:37410 (1506 connections now open) 2016-03-13T08:42:27.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37422 #4077743 (1506 connections now open) 2016-03-13T08:42:27.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37423 #4077744 (1507 connections now open) 2016-03-13T08:42:28.113+0000 I NETWORK [conn4070289] end connection 10.0.216.90:46349 (1506 connections now open) 2016-03-13T08:42:28.113+0000 I NETWORK [conn4070947] end connection 10.0.27.237:53967 (1506 connections now open) 2016-03-13T08:42:28.113+0000 I NETWORK [conn4069932] end connection 10.0.26.69:59218 (1506 connections now open) 2016-03-13T08:42:28.113+0000 I NETWORK [conn4068691] end connection 10.0.7.113:46643 (1503 connections now open) 2016-03-13T08:42:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54361 #4077745 (1504 connections now open) 2016-03-13T08:42:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60919 #4077746 (1505 connections now open) 2016-03-13T08:42:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39382 #4077747 (1506 connections now open) 2016-03-13T08:42:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56330 #4077748 (1507 connections now open) 2016-03-13T08:42:28.505+0000 I NETWORK [conn4077734] end connection 10.0.26.69:39375 (1506 connections now open) 2016-03-13T08:42:28.506+0000 I NETWORK [conn4077733] end connection 10.0.26.69:39374 (1505 connections now open) 2016-03-13T08:42:28.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39384 #4077749 (1506 connections now open) 2016-03-13T08:42:28.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39385 #4077750 (1507 connections now open) 2016-03-13T08:42:31.410+0000 I NETWORK [conn4077735] end connection 10.0.27.237:60912 (1506 connections now open) 2016-03-13T08:42:31.410+0000 I NETWORK [conn4077736] end connection 10.0.27.237:60913 (1506 connections now open) 2016-03-13T08:42:31.411+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60924 #4077751 (1506 connections now open) 2016-03-13T08:42:31.411+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60925 #4077752 (1507 connections now open) 2016-03-13T08:42:32.855+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:42:32.856+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:42:32.856+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:42:32.856+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 16 checks in a row. Polling will stop after 14 more failed checks 2016-03-13T08:42:33.471+0000 I NETWORK [conn4077737] end connection 10.0.7.113:56312 (1506 connections now open) 2016-03-13T08:42:33.471+0000 I NETWORK [conn4077738] end connection 10.0.7.113:56313 (1506 connections now open) 2016-03-13T08:42:33.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56332 #4077753 (1506 connections now open) 2016-03-13T08:42:33.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56333 #4077754 (1507 connections now open) 2016-03-13T08:42:34.667+0000 I NETWORK [conn4077740] end connection 10.0.216.90:54354 (1506 connections now open) 2016-03-13T08:42:34.667+0000 I NETWORK [conn4077739] end connection 10.0.216.90:54353 (1506 connections now open) 2016-03-13T08:42:34.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54378 #4077755 (1506 connections now open) 2016-03-13T08:42:34.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54377 #4077756 (1507 connections now open) 2016-03-13T08:42:35.508+0000 I NETWORK [conn4077741] end connection 10.0.20.53:47584 (1506 connections now open) 2016-03-13T08:42:35.509+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47596 #4077757 (1507 connections now open) 2016-03-13T08:42:35.509+0000 I NETWORK [conn4077742] end connection 10.0.20.53:47585 (1506 connections now open) 2016-03-13T08:42:35.509+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47597 #4077758 (1507 connections now open) 2016-03-13T08:42:37.937+0000 I NETWORK [conn4077743] end connection 10.0.179.209:37422 (1506 connections now open) 2016-03-13T08:42:37.937+0000 I NETWORK [conn4077744] end connection 10.0.179.209:37423 (1506 connections now open) 2016-03-13T08:42:37.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37441 #4077759 (1506 connections now open) 2016-03-13T08:42:37.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37442 #4077760 (1507 connections now open) 2016-03-13T08:42:38.519+0000 I NETWORK [conn4077750] end connection 10.0.26.69:39385 (1506 connections now open) 2016-03-13T08:42:38.519+0000 I NETWORK [conn4077749] end connection 10.0.26.69:39384 (1506 connections now open) 2016-03-13T08:42:38.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39395 #4077761 (1506 connections now open) 2016-03-13T08:42:38.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39394 #4077762 (1507 connections now open) 2016-03-13T08:42:41.423+0000 I NETWORK [conn4077751] end connection 10.0.27.237:60924 (1506 connections now open) 2016-03-13T08:42:41.423+0000 I NETWORK [conn4077752] end connection 10.0.27.237:60925 (1506 connections now open) 2016-03-13T08:42:41.424+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60943 #4077763 (1506 connections now open) 2016-03-13T08:42:41.424+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60942 #4077764 (1507 connections now open) 2016-03-13T08:42:42.868+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:42:42.868+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:42:42.868+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:42:42.868+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 17 checks in a row. Polling will stop after 13 more failed checks 2016-03-13T08:42:43.485+0000 I NETWORK [conn4077754] end connection 10.0.7.113:56333 (1506 connections now open) 2016-03-13T08:42:43.485+0000 I NETWORK [conn4077753] end connection 10.0.7.113:56332 (1506 connections now open) 2016-03-13T08:42:43.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56340 #4077765 (1506 connections now open) 2016-03-13T08:42:43.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56341 #4077766 (1507 connections now open) 2016-03-13T08:42:44.680+0000 I NETWORK [conn4077756] end connection 10.0.216.90:54377 (1506 connections now open) 2016-03-13T08:42:44.680+0000 I NETWORK [conn4077755] end connection 10.0.216.90:54378 (1506 connections now open) 2016-03-13T08:42:44.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54387 #4077767 (1506 connections now open) 2016-03-13T08:42:44.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54386 #4077768 (1507 connections now open) 2016-03-13T08:42:44.793+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:42:44.528+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:42:45.521+0000 I NETWORK [conn4077757] end connection 10.0.20.53:47596 (1506 connections now open) 2016-03-13T08:42:45.521+0000 I NETWORK [conn4077758] end connection 10.0.20.53:47597 (1506 connections now open) 2016-03-13T08:42:45.522+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47608 #4077769 (1506 connections now open) 2016-03-13T08:42:45.522+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47607 #4077770 (1507 connections now open) 2016-03-13T08:42:46.118+0000 I NETWORK [conn4069788] end connection 10.0.27.237:52731 (1506 connections now open) 2016-03-13T08:42:46.118+0000 I NETWORK [conn4071287] end connection 10.0.7.113:49438 (1505 connections now open) 2016-03-13T08:42:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60950 #4077771 (1506 connections now open) 2016-03-13T08:42:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56347 #4077772 (1507 connections now open) 2016-03-13T08:42:47.950+0000 I NETWORK [conn4077760] end connection 10.0.179.209:37442 (1506 connections now open) 2016-03-13T08:42:47.950+0000 I NETWORK [conn4077759] end connection 10.0.179.209:37441 (1506 connections now open) 2016-03-13T08:42:47.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37449 #4077773 (1506 connections now open) 2016-03-13T08:42:47.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37450 #4077774 (1507 connections now open) 2016-03-13T08:42:48.540+0000 I NETWORK [conn4077761] end connection 10.0.26.69:39395 (1506 connections now open) 2016-03-13T08:42:48.541+0000 I NETWORK [conn4077762] end connection 10.0.26.69:39394 (1505 connections now open) 2016-03-13T08:42:48.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39415 #4077775 (1507 connections now open) 2016-03-13T08:42:48.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39416 #4077776 (1507 connections now open) 2016-03-13T08:42:51.437+0000 I NETWORK [conn4077764] end connection 10.0.27.237:60942 (1506 connections now open) 2016-03-13T08:42:51.437+0000 I NETWORK [conn4077763] end connection 10.0.27.237:60943 (1506 connections now open) 2016-03-13T08:42:51.438+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60952 #4077777 (1506 connections now open) 2016-03-13T08:42:51.438+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60953 #4077778 (1507 connections now open) 2016-03-13T08:42:52.871+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:42:52.872+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:42:52.872+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:42:52.872+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 18 checks in a row. Polling will stop after 12 more failed checks 2016-03-13T08:42:53.498+0000 I NETWORK [conn4077766] end connection 10.0.7.113:56341 (1506 connections now open) 2016-03-13T08:42:53.498+0000 I NETWORK [conn4077765] end connection 10.0.7.113:56340 (1506 connections now open) 2016-03-13T08:42:53.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56355 #4077779 (1506 connections now open) 2016-03-13T08:42:53.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56354 #4077780 (1507 connections now open) 2016-03-13T08:42:54.694+0000 I NETWORK [conn4077767] end connection 10.0.216.90:54387 (1506 connections now open) 2016-03-13T08:42:54.694+0000 I NETWORK [conn4077768] end connection 10.0.216.90:54386 (1506 connections now open) 2016-03-13T08:42:54.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54400 #4077781 (1506 connections now open) 2016-03-13T08:42:54.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54401 #4077782 (1507 connections now open) 2016-03-13T08:42:55.534+0000 I NETWORK [conn4077770] end connection 10.0.20.53:47607 (1506 connections now open) 2016-03-13T08:42:55.534+0000 I NETWORK [conn4077769] end connection 10.0.20.53:47608 (1506 connections now open) 2016-03-13T08:42:55.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47626 #4077783 (1506 connections now open) 2016-03-13T08:42:55.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47627 #4077784 (1507 connections now open) 2016-03-13T08:42:57.963+0000 I NETWORK [conn4077774] end connection 10.0.179.209:37450 (1506 connections now open) 2016-03-13T08:42:57.963+0000 I NETWORK [conn4077773] end connection 10.0.179.209:37449 (1506 connections now open) 2016-03-13T08:42:57.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37462 #4077785 (1506 connections now open) 2016-03-13T08:42:57.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37461 #4077786 (1507 connections now open) 2016-03-13T08:42:58.113+0000 I NETWORK [conn4068559] end connection 10.0.20.53:37974 (1506 connections now open) 2016-03-13T08:42:58.113+0000 I NETWORK [conn4066347] end connection 10.0.179.209:54093 (1506 connections now open) 2016-03-13T08:42:58.113+0000 I NETWORK [conn4070333] end connection 10.0.216.90:46393 (1504 connections now open) 2016-03-13T08:42:58.113+0000 I NETWORK [conn4071416] end connection 10.0.26.69:60824 (1504 connections now open) 2016-03-13T08:42:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47630 #4077787 (1504 connections now open) 2016-03-13T08:42:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39422 #4077788 (1505 connections now open) 2016-03-13T08:42:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37465 #4077789 (1506 connections now open) 2016-03-13T08:42:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54408 #4077790 (1507 connections now open) 2016-03-13T08:42:58.554+0000 I NETWORK [conn4077776] end connection 10.0.26.69:39416 (1506 connections now open) 2016-03-13T08:42:58.554+0000 I NETWORK [conn4077775] end connection 10.0.26.69:39415 (1506 connections now open) 2016-03-13T08:42:58.554+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39424 #4077791 (1506 connections now open) 2016-03-13T08:42:58.554+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39423 #4077792 (1507 connections now open) 2016-03-13T08:43:01.451+0000 I NETWORK [conn4077778] end connection 10.0.27.237:60953 (1506 connections now open) 2016-03-13T08:43:01.451+0000 I NETWORK [conn4077777] end connection 10.0.27.237:60952 (1506 connections now open) 2016-03-13T08:43:01.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60965 #4077793 (1506 connections now open) 2016-03-13T08:43:01.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60964 #4077794 (1507 connections now open) 2016-03-13T08:43:02.875+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:43:02.876+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:43:02.876+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:43:02.876+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 19 checks in a row. Polling will stop after 11 more failed checks 2016-03-13T08:43:03.511+0000 I NETWORK [conn4077779] end connection 10.0.7.113:56355 (1506 connections now open) 2016-03-13T08:43:03.511+0000 I NETWORK [conn4077780] end connection 10.0.7.113:56354 (1505 connections now open) 2016-03-13T08:43:03.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56375 #4077795 (1506 connections now open) 2016-03-13T08:43:03.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56376 #4077796 (1507 connections now open) 2016-03-13T08:43:04.709+0000 I NETWORK [conn4077781] end connection 10.0.216.90:54400 (1506 connections now open) 2016-03-13T08:43:04.709+0000 I NETWORK [conn4077782] end connection 10.0.216.90:54401 (1506 connections now open) 2016-03-13T08:43:04.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54424 #4077797 (1506 connections now open) 2016-03-13T08:43:04.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54423 #4077798 (1507 connections now open) 2016-03-13T08:43:05.556+0000 I NETWORK [conn4077783] end connection 10.0.20.53:47626 (1506 connections now open) 2016-03-13T08:43:05.556+0000 I NETWORK [conn4077784] end connection 10.0.20.53:47627 (1506 connections now open) 2016-03-13T08:43:05.556+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47636 #4077799 (1506 connections now open) 2016-03-13T08:43:05.556+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47635 #4077800 (1507 connections now open) 2016-03-13T08:43:07.977+0000 I NETWORK [conn4077786] end connection 10.0.179.209:37461 (1506 connections now open) 2016-03-13T08:43:07.977+0000 I NETWORK [conn4077785] end connection 10.0.179.209:37462 (1506 connections now open) 2016-03-13T08:43:07.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37483 #4077801 (1506 connections now open) 2016-03-13T08:43:07.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37484 #4077802 (1507 connections now open) 2016-03-13T08:43:08.567+0000 I NETWORK [conn4077792] end connection 10.0.26.69:39423 (1506 connections now open) 2016-03-13T08:43:08.567+0000 I NETWORK [conn4077791] end connection 10.0.26.69:39424 (1506 connections now open) 2016-03-13T08:43:08.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39438 #4077803 (1506 connections now open) 2016-03-13T08:43:08.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39437 #4077804 (1507 connections now open) 2016-03-13T08:43:11.464+0000 I NETWORK [conn4077793] end connection 10.0.27.237:60965 (1506 connections now open) 2016-03-13T08:43:11.464+0000 I NETWORK [conn4077794] end connection 10.0.27.237:60964 (1506 connections now open) 2016-03-13T08:43:11.474+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60984 #4077805 (1506 connections now open) 2016-03-13T08:43:11.474+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60983 #4077806 (1507 connections now open) 2016-03-13T08:43:12.887+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:43:12.888+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:43:12.888+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:43:12.888+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 20 checks in a row. Polling will stop after 10 more failed checks 2016-03-13T08:43:13.532+0000 I NETWORK [conn4077796] end connection 10.0.7.113:56376 (1506 connections now open) 2016-03-13T08:43:13.532+0000 I NETWORK [conn4077795] end connection 10.0.7.113:56375 (1505 connections now open) 2016-03-13T08:43:13.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56386 #4077807 (1506 connections now open) 2016-03-13T08:43:13.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56387 #4077808 (1507 connections now open) 2016-03-13T08:43:14.722+0000 I NETWORK [conn4077798] end connection 10.0.216.90:54423 (1506 connections now open) 2016-03-13T08:43:14.722+0000 I NETWORK [conn4077797] end connection 10.0.216.90:54424 (1505 connections now open) 2016-03-13T08:43:14.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54432 #4077809 (1506 connections now open) 2016-03-13T08:43:14.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54433 #4077810 (1507 connections now open) 2016-03-13T08:43:15.073+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:43:14.793+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:43:15.569+0000 I NETWORK [conn4077799] end connection 10.0.20.53:47636 (1506 connections now open) 2016-03-13T08:43:15.569+0000 I NETWORK [conn4077800] end connection 10.0.20.53:47635 (1506 connections now open) 2016-03-13T08:43:15.569+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47648 #4077811 (1506 connections now open) 2016-03-13T08:43:15.569+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47649 #4077812 (1507 connections now open) 2016-03-13T08:43:16.117+0000 I NETWORK [conn4066288] end connection 10.0.27.237:49141 (1506 connections now open) 2016-03-13T08:43:16.117+0000 I NETWORK [conn4069763] end connection 10.0.216.90:45791 (1506 connections now open) 2016-03-13T08:43:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54435 #4077813 (1506 connections now open) 2016-03-13T08:43:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60995 #4077814 (1507 connections now open) 2016-03-13T08:43:17.991+0000 I NETWORK [conn4077802] end connection 10.0.179.209:37484 (1506 connections now open) 2016-03-13T08:43:17.991+0000 I NETWORK [conn4077801] end connection 10.0.179.209:37483 (1506 connections now open) 2016-03-13T08:43:17.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37496 #4077815 (1506 connections now open) 2016-03-13T08:43:17.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37495 #4077816 (1507 connections now open) 2016-03-13T08:43:18.581+0000 I NETWORK [conn4077804] end connection 10.0.26.69:39437 (1506 connections now open) 2016-03-13T08:43:18.581+0000 I NETWORK [conn4077803] end connection 10.0.26.69:39438 (1506 connections now open) 2016-03-13T08:43:18.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39462 #4077817 (1507 connections now open) 2016-03-13T08:43:18.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39463 #4077818 (1507 connections now open) 2016-03-13T08:43:21.486+0000 I NETWORK [conn4077806] end connection 10.0.27.237:60983 (1506 connections now open) 2016-03-13T08:43:21.486+0000 I NETWORK [conn4077805] end connection 10.0.27.237:60984 (1506 connections now open) 2016-03-13T08:43:21.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60998 #4077819 (1506 connections now open) 2016-03-13T08:43:21.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:60997 #4077820 (1507 connections now open) 2016-03-13T08:43:22.891+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:43:22.892+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:43:22.892+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:43:22.892+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 21 checks in a row. Polling will stop after 9 more failed checks 2016-03-13T08:43:23.545+0000 I NETWORK [conn4077807] end connection 10.0.7.113:56386 (1506 connections now open) 2016-03-13T08:43:23.545+0000 I NETWORK [conn4077808] end connection 10.0.7.113:56387 (1506 connections now open) 2016-03-13T08:43:23.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56400 #4077821 (1506 connections now open) 2016-03-13T08:43:23.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56401 #4077822 (1507 connections now open) 2016-03-13T08:43:24.735+0000 I NETWORK [conn4077809] end connection 10.0.216.90:54432 (1506 connections now open) 2016-03-13T08:43:24.735+0000 I NETWORK [conn4077810] end connection 10.0.216.90:54433 (1506 connections now open) 2016-03-13T08:43:24.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54445 #4077823 (1506 connections now open) 2016-03-13T08:43:24.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54446 #4077824 (1507 connections now open) 2016-03-13T08:43:25.582+0000 I NETWORK [conn4077812] end connection 10.0.20.53:47649 (1506 connections now open) 2016-03-13T08:43:25.582+0000 I NETWORK [conn4077811] end connection 10.0.20.53:47648 (1506 connections now open) 2016-03-13T08:43:25.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47668 #4077825 (1506 connections now open) 2016-03-13T08:43:25.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47669 #4077826 (1507 connections now open) 2016-03-13T08:43:28.004+0000 I NETWORK [conn4077816] end connection 10.0.179.209:37495 (1506 connections now open) 2016-03-13T08:43:28.004+0000 I NETWORK [conn4077815] end connection 10.0.179.209:37496 (1506 connections now open) 2016-03-13T08:43:28.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37507 #4077827 (1506 connections now open) 2016-03-13T08:43:28.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37508 #4077828 (1507 connections now open) 2016-03-13T08:43:28.113+0000 I NETWORK [conn4069846] end connection 10.0.179.209:57695 (1506 connections now open) 2016-03-13T08:43:28.113+0000 I NETWORK [conn4069431] end connection 10.0.27.237:52347 (1505 connections now open) 2016-03-13T08:43:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37512 #4077829 (1506 connections now open) 2016-03-13T08:43:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32770 #4077830 (1507 connections now open) 2016-03-13T08:43:28.594+0000 I NETWORK [conn4077818] end connection 10.0.26.69:39463 (1506 connections now open) 2016-03-13T08:43:28.594+0000 I NETWORK [conn4077817] end connection 10.0.26.69:39462 (1506 connections now open) 2016-03-13T08:43:28.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39471 #4077831 (1506 connections now open) 2016-03-13T08:43:28.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39472 #4077832 (1507 connections now open) 2016-03-13T08:43:31.499+0000 I NETWORK [conn4077819] end connection 10.0.27.237:60998 (1506 connections now open) 2016-03-13T08:43:31.499+0000 I NETWORK [conn4077820] end connection 10.0.27.237:60997 (1506 connections now open) 2016-03-13T08:43:31.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32777 #4077833 (1506 connections now open) 2016-03-13T08:43:31.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32778 #4077834 (1507 connections now open) 2016-03-13T08:43:32.895+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:43:32.896+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:43:32.896+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:43:32.896+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 22 checks in a row. Polling will stop after 8 more failed checks 2016-03-13T08:43:33.558+0000 I NETWORK [conn4077822] end connection 10.0.7.113:56401 (1506 connections now open) 2016-03-13T08:43:33.558+0000 I NETWORK [conn4077821] end connection 10.0.7.113:56400 (1506 connections now open) 2016-03-13T08:43:33.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56421 #4077835 (1506 connections now open) 2016-03-13T08:43:33.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56422 #4077836 (1507 connections now open) 2016-03-13T08:43:34.749+0000 I NETWORK [conn4077823] end connection 10.0.216.90:54445 (1506 connections now open) 2016-03-13T08:43:34.749+0000 I NETWORK [conn4077824] end connection 10.0.216.90:54446 (1506 connections now open) 2016-03-13T08:43:34.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54469 #4077837 (1506 connections now open) 2016-03-13T08:43:34.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54470 #4077838 (1507 connections now open) 2016-03-13T08:43:35.595+0000 I NETWORK [conn4077825] end connection 10.0.20.53:47668 (1506 connections now open) 2016-03-13T08:43:35.595+0000 I NETWORK [conn4077826] end connection 10.0.20.53:47669 (1506 connections now open) 2016-03-13T08:43:35.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47680 #4077839 (1506 connections now open) 2016-03-13T08:43:35.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47679 #4077840 (1507 connections now open) 2016-03-13T08:43:38.017+0000 I NETWORK [conn4077828] end connection 10.0.179.209:37508 (1506 connections now open) 2016-03-13T08:43:38.017+0000 I NETWORK [conn4077827] end connection 10.0.179.209:37507 (1506 connections now open) 2016-03-13T08:43:38.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37529 #4077841 (1506 connections now open) 2016-03-13T08:43:38.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37528 #4077842 (1507 connections now open) 2016-03-13T08:43:38.607+0000 I NETWORK [conn4077832] end connection 10.0.26.69:39472 (1506 connections now open) 2016-03-13T08:43:38.607+0000 I NETWORK [conn4077831] end connection 10.0.26.69:39471 (1506 connections now open) 2016-03-13T08:43:38.608+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39483 #4077843 (1506 connections now open) 2016-03-13T08:43:38.608+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39482 #4077844 (1507 connections now open) 2016-03-13T08:43:41.512+0000 I NETWORK [conn4077833] end connection 10.0.27.237:32777 (1506 connections now open) 2016-03-13T08:43:41.512+0000 I NETWORK [conn4077834] end connection 10.0.27.237:32778 (1505 connections now open) 2016-03-13T08:43:41.512+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32793 #4077845 (1506 connections now open) 2016-03-13T08:43:41.513+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32794 #4077846 (1507 connections now open) 2016-03-13T08:43:42.899+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:43:42.907+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:43:42.907+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:43:42.907+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 23 checks in a row. Polling will stop after 7 more failed checks 2016-03-13T08:43:43.571+0000 I NETWORK [conn4077836] end connection 10.0.7.113:56422 (1506 connections now open) 2016-03-13T08:43:43.571+0000 I NETWORK [conn4077835] end connection 10.0.7.113:56421 (1506 connections now open) 2016-03-13T08:43:43.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56429 #4077847 (1506 connections now open) 2016-03-13T08:43:43.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56430 #4077848 (1507 connections now open) 2016-03-13T08:43:44.762+0000 I NETWORK [conn4077837] end connection 10.0.216.90:54469 (1506 connections now open) 2016-03-13T08:43:44.762+0000 I NETWORK [conn4077838] end connection 10.0.216.90:54470 (1506 connections now open) 2016-03-13T08:43:44.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54478 #4077849 (1506 connections now open) 2016-03-13T08:43:44.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54479 #4077850 (1507 connections now open) 2016-03-13T08:43:45.274+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:43:45.073+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:43:45.608+0000 I NETWORK [conn4077840] end connection 10.0.20.53:47679 (1506 connections now open) 2016-03-13T08:43:45.608+0000 I NETWORK [conn4077839] end connection 10.0.20.53:47680 (1506 connections now open) 2016-03-13T08:43:45.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47691 #4077851 (1506 connections now open) 2016-03-13T08:43:45.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47692 #4077852 (1507 connections now open) 2016-03-13T08:43:46.117+0000 I NETWORK [conn4073206] end connection 10.0.179.209:32814 (1506 connections now open) 2016-03-13T08:43:46.117+0000 I NETWORK [conn4069581] end connection 10.0.20.53:39040 (1506 connections now open) 2016-03-13T08:43:46.117+0000 I NETWORK [conn4071988] end connection 10.0.27.237:55089 (1504 connections now open) 2016-03-13T08:43:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37536 #4077853 (1505 connections now open) 2016-03-13T08:43:46.118+0000 I NETWORK [conn4065045] end connection 10.0.7.113:42752 (1504 connections now open) 2016-03-13T08:43:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47694 #4077854 (1505 connections now open) 2016-03-13T08:43:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32800 #4077855 (1506 connections now open) 2016-03-13T08:43:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56436 #4077856 (1507 connections now open) 2016-03-13T08:43:48.030+0000 I NETWORK [conn4077841] end connection 10.0.179.209:37529 (1506 connections now open) 2016-03-13T08:43:48.030+0000 I NETWORK [conn4077842] end connection 10.0.179.209:37528 (1506 connections now open) 2016-03-13T08:43:48.031+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37538 #4077857 (1506 connections now open) 2016-03-13T08:43:48.031+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37537 #4077858 (1507 connections now open) 2016-03-13T08:43:48.620+0000 I NETWORK [conn4077843] end connection 10.0.26.69:39483 (1506 connections now open) 2016-03-13T08:43:48.620+0000 I NETWORK [conn4077844] end connection 10.0.26.69:39482 (1506 connections now open) 2016-03-13T08:43:48.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39503 #4077859 (1506 connections now open) 2016-03-13T08:43:48.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39504 #4077860 (1507 connections now open) 2016-03-13T08:43:51.525+0000 I NETWORK [conn4077845] end connection 10.0.27.237:32793 (1506 connections now open) 2016-03-13T08:43:51.525+0000 I NETWORK [conn4077846] end connection 10.0.27.237:32794 (1505 connections now open) 2016-03-13T08:43:51.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32802 #4077861 (1506 connections now open) 2016-03-13T08:43:51.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32803 #4077862 (1507 connections now open) 2016-03-13T08:43:52.914+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:43:52.915+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:43:52.915+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:43:52.915+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 24 checks in a row. Polling will stop after 6 more failed checks 2016-03-13T08:43:53.585+0000 I NETWORK [conn4077848] end connection 10.0.7.113:56430 (1506 connections now open) 2016-03-13T08:43:53.585+0000 I NETWORK [conn4077847] end connection 10.0.7.113:56429 (1506 connections now open) 2016-03-13T08:43:53.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56444 #4077863 (1506 connections now open) 2016-03-13T08:43:53.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56445 #4077864 (1507 connections now open) 2016-03-13T08:43:54.776+0000 I NETWORK [conn4077850] end connection 10.0.216.90:54479 (1506 connections now open) 2016-03-13T08:43:54.776+0000 I NETWORK [conn4077849] end connection 10.0.216.90:54478 (1506 connections now open) 2016-03-13T08:43:54.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54491 #4077865 (1506 connections now open) 2016-03-13T08:43:54.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54492 #4077866 (1507 connections now open) 2016-03-13T08:43:55.621+0000 I NETWORK [conn4077852] end connection 10.0.20.53:47692 (1506 connections now open) 2016-03-13T08:43:55.621+0000 I NETWORK [conn4077851] end connection 10.0.20.53:47691 (1505 connections now open) 2016-03-13T08:43:55.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47710 #4077867 (1506 connections now open) 2016-03-13T08:43:55.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47711 #4077868 (1507 connections now open) 2016-03-13T08:43:58.043+0000 I NETWORK [conn4077857] end connection 10.0.179.209:37538 (1506 connections now open) 2016-03-13T08:43:58.043+0000 I NETWORK [conn4077858] end connection 10.0.179.209:37537 (1506 connections now open) 2016-03-13T08:43:58.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37549 #4077869 (1506 connections now open) 2016-03-13T08:43:58.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37550 #4077870 (1507 connections now open) 2016-03-13T08:43:58.113+0000 I NETWORK [conn4069929] end connection 10.0.179.209:57778 (1506 connections now open) 2016-03-13T08:43:58.113+0000 I NETWORK [conn4071457] end connection 10.0.26.69:60867 (1505 connections now open) 2016-03-13T08:43:58.113+0000 I NETWORK [conn4070374] end connection 10.0.216.90:46439 (1505 connections now open) 2016-03-13T08:43:58.113+0000 I NETWORK [conn4072638] end connection 10.0.27.237:55782 (1503 connections now open) 2016-03-13T08:43:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37553 #4077871 (1504 connections now open) 2016-03-13T08:43:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39513 #4077872 (1505 connections now open) 2016-03-13T08:43:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32808 #4077873 (1506 connections now open) 2016-03-13T08:43:58.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54498 #4077874 (1507 connections now open) 2016-03-13T08:43:58.634+0000 I NETWORK [conn4077860] end connection 10.0.26.69:39504 (1506 connections now open) 2016-03-13T08:43:58.634+0000 I NETWORK [conn4077859] end connection 10.0.26.69:39503 (1506 connections now open) 2016-03-13T08:43:58.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39515 #4077875 (1506 connections now open) 2016-03-13T08:43:58.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39514 #4077876 (1507 connections now open) 2016-03-13T08:44:01.538+0000 I NETWORK [conn4077861] end connection 10.0.27.237:32802 (1506 connections now open) 2016-03-13T08:44:01.538+0000 I NETWORK [conn4077862] end connection 10.0.27.237:32803 (1505 connections now open) 2016-03-13T08:44:01.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32814 #4077877 (1506 connections now open) 2016-03-13T08:44:01.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32815 #4077878 (1507 connections now open) 2016-03-13T08:44:02.918+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:44:02.919+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:44:02.919+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:44:02.919+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 25 checks in a row. Polling will stop after 5 more failed checks 2016-03-13T08:44:03.598+0000 I NETWORK [conn4077864] end connection 10.0.7.113:56445 (1506 connections now open) 2016-03-13T08:44:03.598+0000 I NETWORK [conn4077863] end connection 10.0.7.113:56444 (1506 connections now open) 2016-03-13T08:44:03.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56467 #4077879 (1506 connections now open) 2016-03-13T08:44:03.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56468 #4077880 (1507 connections now open) 2016-03-13T08:44:04.788+0000 I NETWORK [conn4077865] end connection 10.0.216.90:54491 (1506 connections now open) 2016-03-13T08:44:04.788+0000 I NETWORK [conn4077866] end connection 10.0.216.90:54492 (1506 connections now open) 2016-03-13T08:44:04.789+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54514 #4077881 (1506 connections now open) 2016-03-13T08:44:04.789+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54515 #4077882 (1507 connections now open) 2016-03-13T08:44:05.642+0000 I NETWORK [conn4077868] end connection 10.0.20.53:47711 (1506 connections now open) 2016-03-13T08:44:05.642+0000 I NETWORK [conn4077867] end connection 10.0.20.53:47710 (1506 connections now open) 2016-03-13T08:44:05.643+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47721 #4077883 (1506 connections now open) 2016-03-13T08:44:05.643+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47720 #4077884 (1507 connections now open) 2016-03-13T08:44:08.064+0000 I NETWORK [conn4077869] end connection 10.0.179.209:37549 (1506 connections now open) 2016-03-13T08:44:08.064+0000 I NETWORK [conn4077870] end connection 10.0.179.209:37550 (1506 connections now open) 2016-03-13T08:44:08.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37573 #4077885 (1506 connections now open) 2016-03-13T08:44:08.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37574 #4077886 (1507 connections now open) 2016-03-13T08:44:08.647+0000 I NETWORK [conn4077876] end connection 10.0.26.69:39514 (1506 connections now open) 2016-03-13T08:44:08.647+0000 I NETWORK [conn4077875] end connection 10.0.26.69:39515 (1506 connections now open) 2016-03-13T08:44:08.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39529 #4077887 (1506 connections now open) 2016-03-13T08:44:08.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39530 #4077888 (1507 connections now open) 2016-03-13T08:44:11.551+0000 I NETWORK [conn4077878] end connection 10.0.27.237:32815 (1506 connections now open) 2016-03-13T08:44:11.551+0000 I NETWORK [conn4077877] end connection 10.0.27.237:32814 (1506 connections now open) 2016-03-13T08:44:11.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32832 #4077889 (1506 connections now open) 2016-03-13T08:44:11.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32833 #4077890 (1507 connections now open) 2016-03-13T08:44:12.921+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:44:12.931+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:44:12.931+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:44:12.931+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 26 checks in a row. Polling will stop after 4 more failed checks 2016-03-13T08:44:13.612+0000 I NETWORK [conn4077879] end connection 10.0.7.113:56467 (1506 connections now open) 2016-03-13T08:44:13.612+0000 I NETWORK [conn4077880] end connection 10.0.7.113:56468 (1505 connections now open) 2016-03-13T08:44:13.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56479 #4077891 (1506 connections now open) 2016-03-13T08:44:13.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56480 #4077892 (1507 connections now open) 2016-03-13T08:44:14.802+0000 I NETWORK [conn4077881] end connection 10.0.216.90:54514 (1506 connections now open) 2016-03-13T08:44:14.802+0000 I NETWORK [conn4077882] end connection 10.0.216.90:54515 (1506 connections now open) 2016-03-13T08:44:14.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54521 #4077893 (1506 connections now open) 2016-03-13T08:44:14.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54522 #4077894 (1507 connections now open) 2016-03-13T08:44:15.525+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:44:15.274+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:44:15.655+0000 I NETWORK [conn4077884] end connection 10.0.20.53:47720 (1506 connections now open) 2016-03-13T08:44:15.655+0000 I NETWORK [conn4077883] end connection 10.0.20.53:47721 (1506 connections now open) 2016-03-13T08:44:15.656+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47733 #4077895 (1506 connections now open) 2016-03-13T08:44:15.656+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47732 #4077896 (1507 connections now open) 2016-03-13T08:44:16.118+0000 I NETWORK [conn4074086] end connection 10.0.179.209:33716 (1506 connections now open) 2016-03-13T08:44:16.118+0000 I NETWORK [conn4068790] end connection 10.0.216.90:44746 (1505 connections now open) 2016-03-13T08:44:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37580 #4077897 (1506 connections now open) 2016-03-13T08:44:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54524 #4077898 (1507 connections now open) 2016-03-13T08:44:18.077+0000 I NETWORK [conn4077886] end connection 10.0.179.209:37574 (1506 connections now open) 2016-03-13T08:44:18.077+0000 I NETWORK [conn4077885] end connection 10.0.179.209:37573 (1506 connections now open) 2016-03-13T08:44:18.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37584 #4077899 (1506 connections now open) 2016-03-13T08:44:18.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37585 #4077900 (1507 connections now open) 2016-03-13T08:44:18.660+0000 I NETWORK [conn4077888] end connection 10.0.26.69:39530 (1506 connections now open) 2016-03-13T08:44:18.660+0000 I NETWORK [conn4077887] end connection 10.0.26.69:39529 (1506 connections now open) 2016-03-13T08:44:18.670+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39553 #4077901 (1506 connections now open) 2016-03-13T08:44:18.670+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39554 #4077902 (1507 connections now open) 2016-03-13T08:44:21.572+0000 I NETWORK [conn4077889] end connection 10.0.27.237:32832 (1506 connections now open) 2016-03-13T08:44:21.573+0000 I NETWORK [conn4077890] end connection 10.0.27.237:32833 (1505 connections now open) 2016-03-13T08:44:21.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32844 #4077903 (1506 connections now open) 2016-03-13T08:44:21.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32845 #4077904 (1507 connections now open) 2016-03-13T08:44:22.934+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:44:22.934+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:44:22.934+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:44:22.934+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 27 checks in a row. Polling will stop after 3 more failed checks 2016-03-13T08:44:23.634+0000 I NETWORK [conn4077891] end connection 10.0.7.113:56479 (1506 connections now open) 2016-03-13T08:44:23.634+0000 I NETWORK [conn4077892] end connection 10.0.7.113:56480 (1506 connections now open) 2016-03-13T08:44:23.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56495 #4077905 (1506 connections now open) 2016-03-13T08:44:23.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56494 #4077906 (1507 connections now open) 2016-03-13T08:44:24.816+0000 I NETWORK [conn4077893] end connection 10.0.216.90:54521 (1506 connections now open) 2016-03-13T08:44:24.816+0000 I NETWORK [conn4077894] end connection 10.0.216.90:54522 (1506 connections now open) 2016-03-13T08:44:24.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54533 #4077907 (1506 connections now open) 2016-03-13T08:44:24.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54532 #4077908 (1507 connections now open) 2016-03-13T08:44:25.668+0000 I NETWORK [conn4077896] end connection 10.0.20.53:47732 (1506 connections now open) 2016-03-13T08:44:25.668+0000 I NETWORK [conn4077895] end connection 10.0.20.53:47733 (1506 connections now open) 2016-03-13T08:44:25.669+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47753 #4077909 (1506 connections now open) 2016-03-13T08:44:25.669+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47752 #4077910 (1507 connections now open) 2016-03-13T08:44:28.090+0000 I NETWORK [conn4077899] end connection 10.0.179.209:37584 (1506 connections now open) 2016-03-13T08:44:28.090+0000 I NETWORK [conn4077900] end connection 10.0.179.209:37585 (1506 connections now open) 2016-03-13T08:44:28.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37596 #4077911 (1506 connections now open) 2016-03-13T08:44:28.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37597 #4077912 (1507 connections now open) 2016-03-13T08:44:28.112+0000 I NETWORK [conn4068733] end connection 10.0.20.53:38151 (1506 connections now open) 2016-03-13T08:44:28.113+0000 I NETWORK [conn4067925] end connection 10.0.216.90:43829 (1505 connections now open) 2016-03-13T08:44:28.113+0000 I NETWORK [conn4077235] end connection 10.0.7.113:55779 (1504 connections now open) 2016-03-13T08:44:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47756 #4077913 (1505 connections now open) 2016-03-13T08:44:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54539 #4077914 (1506 connections now open) 2016-03-13T08:44:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56511 #4077915 (1507 connections now open) 2016-03-13T08:44:28.682+0000 I NETWORK [conn4077902] end connection 10.0.26.69:39554 (1506 connections now open) 2016-03-13T08:44:28.682+0000 I NETWORK [conn4077901] end connection 10.0.26.69:39553 (1506 connections now open) 2016-03-13T08:44:28.683+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39565 #4077916 (1506 connections now open) 2016-03-13T08:44:28.683+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39564 #4077917 (1507 connections now open) 2016-03-13T08:44:31.585+0000 I NETWORK [conn4077903] end connection 10.0.27.237:32844 (1506 connections now open) 2016-03-13T08:44:31.585+0000 I NETWORK [conn4077904] end connection 10.0.27.237:32845 (1505 connections now open) 2016-03-13T08:44:31.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32856 #4077918 (1506 connections now open) 2016-03-13T08:44:31.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32857 #4077919 (1507 connections now open) 2016-03-13T08:44:32.937+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:44:32.938+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:44:32.938+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:44:32.938+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 28 checks in a row. Polling will stop after 2 more failed checks 2016-03-13T08:44:33.647+0000 I NETWORK [conn4077906] end connection 10.0.7.113:56494 (1506 connections now open) 2016-03-13T08:44:33.648+0000 I NETWORK [conn4077905] end connection 10.0.7.113:56495 (1506 connections now open) 2016-03-13T08:44:33.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56515 #4077920 (1506 connections now open) 2016-03-13T08:44:33.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56514 #4077921 (1507 connections now open) 2016-03-13T08:44:34.830+0000 I NETWORK [conn4077907] end connection 10.0.216.90:54533 (1506 connections now open) 2016-03-13T08:44:34.830+0000 I NETWORK [conn4077908] end connection 10.0.216.90:54532 (1506 connections now open) 2016-03-13T08:44:34.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54557 #4077922 (1506 connections now open) 2016-03-13T08:44:34.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54556 #4077923 (1507 connections now open) 2016-03-13T08:44:35.681+0000 I NETWORK [conn4077910] end connection 10.0.20.53:47752 (1506 connections now open) 2016-03-13T08:44:35.681+0000 I NETWORK [conn4077909] end connection 10.0.20.53:47753 (1506 connections now open) 2016-03-13T08:44:35.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47764 #4077924 (1506 connections now open) 2016-03-13T08:44:35.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47765 #4077925 (1507 connections now open) 2016-03-13T08:44:38.103+0000 I NETWORK [conn4077912] end connection 10.0.179.209:37597 (1506 connections now open) 2016-03-13T08:44:38.103+0000 I NETWORK [conn4077911] end connection 10.0.179.209:37596 (1506 connections now open) 2016-03-13T08:44:38.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37617 #4077926 (1506 connections now open) 2016-03-13T08:44:38.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37618 #4077927 (1507 connections now open) 2016-03-13T08:44:38.696+0000 I NETWORK [conn4077917] end connection 10.0.26.69:39564 (1506 connections now open) 2016-03-13T08:44:38.696+0000 I NETWORK [conn4077916] end connection 10.0.26.69:39565 (1506 connections now open) 2016-03-13T08:44:38.696+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39576 #4077928 (1506 connections now open) 2016-03-13T08:44:38.696+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39577 #4077929 (1507 connections now open) 2016-03-13T08:44:41.598+0000 I NETWORK [conn4077919] end connection 10.0.27.237:32857 (1506 connections now open) 2016-03-13T08:44:41.598+0000 I NETWORK [conn4077918] end connection 10.0.27.237:32856 (1506 connections now open) 2016-03-13T08:44:41.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32873 #4077930 (1506 connections now open) 2016-03-13T08:44:41.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32872 #4077931 (1507 connections now open) 2016-03-13T08:44:42.950+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:44:42.950+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:44:42.950+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:44:42.950+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 29 checks in a row. Polling will stop after 1 more failed checks 2016-03-13T08:44:43.661+0000 I NETWORK [conn4077921] end connection 10.0.7.113:56514 (1506 connections now open) 2016-03-13T08:44:43.661+0000 I NETWORK [conn4077920] end connection 10.0.7.113:56515 (1506 connections now open) 2016-03-13T08:44:43.662+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56523 #4077932 (1506 connections now open) 2016-03-13T08:44:43.662+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56524 #4077933 (1507 connections now open) 2016-03-13T08:44:44.844+0000 I NETWORK [conn4077923] end connection 10.0.216.90:54556 (1506 connections now open) 2016-03-13T08:44:44.844+0000 I NETWORK [conn4077922] end connection 10.0.216.90:54557 (1506 connections now open) 2016-03-13T08:44:44.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54566 #4077934 (1506 connections now open) 2016-03-13T08:44:44.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54565 #4077935 (1507 connections now open) 2016-03-13T08:44:45.694+0000 I NETWORK [conn4077925] end connection 10.0.20.53:47765 (1506 connections now open) 2016-03-13T08:44:45.694+0000 I NETWORK [conn4077924] end connection 10.0.20.53:47764 (1506 connections now open) 2016-03-13T08:44:45.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47775 #4077936 (1506 connections now open) 2016-03-13T08:44:45.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47776 #4077937 (1507 connections now open) 2016-03-13T08:44:45.807+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:44:45.526+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:44:46.118+0000 I NETWORK [conn4068280] end connection 10.0.20.53:37687 (1506 connections now open) 2016-03-13T08:44:46.118+0000 I NETWORK [conn4071652] end connection 10.0.27.237:54724 (1505 connections now open) 2016-03-13T08:44:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47779 #4077938 (1506 connections now open) 2016-03-13T08:44:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32880 #4077939 (1507 connections now open) 2016-03-13T08:44:48.116+0000 I NETWORK [conn4077926] end connection 10.0.179.209:37617 (1506 connections now open) 2016-03-13T08:44:48.116+0000 I NETWORK [conn4077927] end connection 10.0.179.209:37618 (1506 connections now open) 2016-03-13T08:44:48.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37625 #4077940 (1506 connections now open) 2016-03-13T08:44:48.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37626 #4077941 (1507 connections now open) 2016-03-13T08:44:48.709+0000 I NETWORK [conn4077928] end connection 10.0.26.69:39576 (1506 connections now open) 2016-03-13T08:44:48.709+0000 I NETWORK [conn4077929] end connection 10.0.26.69:39577 (1506 connections now open) 2016-03-13T08:44:48.710+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39597 #4077942 (1506 connections now open) 2016-03-13T08:44:48.710+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39598 #4077943 (1507 connections now open) 2016-03-13T08:44:51.611+0000 I NETWORK [conn4077931] end connection 10.0.27.237:32872 (1506 connections now open) 2016-03-13T08:44:51.611+0000 I NETWORK [conn4077930] end connection 10.0.27.237:32873 (1506 connections now open) 2016-03-13T08:44:51.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32883 #4077944 (1506 connections now open) 2016-03-13T08:44:51.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32884 #4077945 (1507 connections now open) 2016-03-13T08:44:52.953+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:44:52.954+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:44:52.954+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:44:52.954+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 30 checks in a row. Polling will stop after 0 more failed checks 2016-03-13T08:44:52.954+0000 I NETWORK [ReplicaSetMonitorWatcher] Stopping periodic monitoring of set rsmydomain3 because none of the hosts could be contacted for an extended period of time. 2016-03-13T08:44:53.675+0000 I NETWORK [conn4077933] end connection 10.0.7.113:56524 (1506 connections now open) 2016-03-13T08:44:53.675+0000 I NETWORK [conn4077932] end connection 10.0.7.113:56523 (1506 connections now open) 2016-03-13T08:44:53.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56536 #4077946 (1506 connections now open) 2016-03-13T08:44:53.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56537 #4077947 (1507 connections now open) 2016-03-13T08:44:54.858+0000 I NETWORK [conn4077934] end connection 10.0.216.90:54566 (1506 connections now open) 2016-03-13T08:44:54.858+0000 I NETWORK [conn4077935] end connection 10.0.216.90:54565 (1506 connections now open) 2016-03-13T08:44:54.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54578 #4077948 (1506 connections now open) 2016-03-13T08:44:54.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54579 #4077949 (1507 connections now open) 2016-03-13T08:44:55.708+0000 I NETWORK [conn4077936] end connection 10.0.20.53:47775 (1506 connections now open) 2016-03-13T08:44:55.708+0000 I NETWORK [conn4077937] end connection 10.0.20.53:47776 (1506 connections now open) 2016-03-13T08:44:55.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47795 #4077950 (1506 connections now open) 2016-03-13T08:44:55.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47794 #4077951 (1507 connections now open) 2016-03-13T08:44:58.113+0000 I NETWORK [conn4070139] end connection 10.0.179.209:57990 (1506 connections now open) 2016-03-13T08:44:58.114+0000 I NETWORK [conn4070991] end connection 10.0.27.237:54014 (1505 connections now open) 2016-03-13T08:44:58.114+0000 I NETWORK [conn4069413] end connection 10.0.7.113:47417 (1504 connections now open) 2016-03-13T08:44:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32889 #4077952 (1505 connections now open) 2016-03-13T08:44:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56554 #4077953 (1506 connections now open) 2016-03-13T08:44:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37637 #4077954 (1507 connections now open) 2016-03-13T08:44:58.128+0000 I NETWORK [conn4077940] end connection 10.0.179.209:37625 (1506 connections now open) 2016-03-13T08:44:58.128+0000 I NETWORK [conn4077941] end connection 10.0.179.209:37626 (1505 connections now open) 2016-03-13T08:44:58.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37638 #4077955 (1506 connections now open) 2016-03-13T08:44:58.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37639 #4077956 (1507 connections now open) 2016-03-13T08:44:58.722+0000 I NETWORK [conn4077942] end connection 10.0.26.69:39597 (1506 connections now open) 2016-03-13T08:44:58.722+0000 I NETWORK [conn4077943] end connection 10.0.26.69:39598 (1506 connections now open) 2016-03-13T08:44:58.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39609 #4077957 (1506 connections now open) 2016-03-13T08:44:58.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39608 #4077958 (1507 connections now open) 2016-03-13T08:45:01.624+0000 I NETWORK [conn4077945] end connection 10.0.27.237:32884 (1506 connections now open) 2016-03-13T08:45:01.624+0000 I NETWORK [conn4077944] end connection 10.0.27.237:32883 (1506 connections now open) 2016-03-13T08:45:01.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32895 #4077959 (1506 connections now open) 2016-03-13T08:45:01.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32896 #4077960 (1507 connections now open) 2016-03-13T08:45:02.160+0000 I NETWORK [Balancer] Starting new replica set monitor for rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:45:02.957+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:45:02.958+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:45:02.958+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:45:02.958+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:45:03.688+0000 I NETWORK [conn4077947] end connection 10.0.7.113:56537 (1506 connections now open) 2016-03-13T08:45:03.688+0000 I NETWORK [conn4077946] end connection 10.0.7.113:56536 (1506 connections now open) 2016-03-13T08:45:03.689+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56561 #4077961 (1506 connections now open) 2016-03-13T08:45:03.689+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56560 #4077962 (1507 connections now open) 2016-03-13T08:45:04.871+0000 I NETWORK [conn4077948] end connection 10.0.216.90:54578 (1506 connections now open) 2016-03-13T08:45:04.871+0000 I NETWORK [conn4077949] end connection 10.0.216.90:54579 (1506 connections now open) 2016-03-13T08:45:04.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54605 #4077963 (1506 connections now open) 2016-03-13T08:45:04.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54604 #4077964 (1507 connections now open) 2016-03-13T08:45:05.729+0000 I NETWORK [conn4077950] end connection 10.0.20.53:47795 (1506 connections now open) 2016-03-13T08:45:05.729+0000 I NETWORK [conn4077951] end connection 10.0.20.53:47794 (1506 connections now open) 2016-03-13T08:45:05.730+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47804 #4077965 (1506 connections now open) 2016-03-13T08:45:05.730+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47805 #4077966 (1507 connections now open) 2016-03-13T08:45:08.141+0000 I NETWORK [conn4077955] end connection 10.0.179.209:37638 (1506 connections now open) 2016-03-13T08:45:08.141+0000 I NETWORK [conn4077956] end connection 10.0.179.209:37639 (1506 connections now open) 2016-03-13T08:45:08.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37660 #4077967 (1506 connections now open) 2016-03-13T08:45:08.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37659 #4077968 (1507 connections now open) 2016-03-13T08:45:08.735+0000 I NETWORK [conn4077958] end connection 10.0.26.69:39608 (1506 connections now open) 2016-03-13T08:45:08.735+0000 I NETWORK [conn4077957] end connection 10.0.26.69:39609 (1506 connections now open) 2016-03-13T08:45:08.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39626 #4077969 (1506 connections now open) 2016-03-13T08:45:08.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39625 #4077970 (1507 connections now open) 2016-03-13T08:45:11.637+0000 I NETWORK [conn4077960] end connection 10.0.27.237:32896 (1506 connections now open) 2016-03-13T08:45:11.637+0000 I NETWORK [conn4077959] end connection 10.0.27.237:32895 (1505 connections now open) 2016-03-13T08:45:11.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32915 #4077971 (1506 connections now open) 2016-03-13T08:45:11.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32916 #4077972 (1507 connections now open) 2016-03-13T08:45:12.969+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:45:12.970+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:45:12.970+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:45:12.970+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:45:13.702+0000 I NETWORK [conn4077961] end connection 10.0.7.113:56561 (1506 connections now open) 2016-03-13T08:45:13.702+0000 I NETWORK [conn4077962] end connection 10.0.7.113:56560 (1506 connections now open) 2016-03-13T08:45:13.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56572 #4077973 (1506 connections now open) 2016-03-13T08:45:13.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56573 #4077974 (1507 connections now open) 2016-03-13T08:45:14.894+0000 I NETWORK [conn4077963] end connection 10.0.216.90:54605 (1506 connections now open) 2016-03-13T08:45:14.894+0000 I NETWORK [conn4077964] end connection 10.0.216.90:54604 (1506 connections now open) 2016-03-13T08:45:14.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54611 #4077975 (1506 connections now open) 2016-03-13T08:45:14.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54612 #4077976 (1507 connections now open) 2016-03-13T08:45:15.743+0000 I NETWORK [conn4077965] end connection 10.0.20.53:47804 (1506 connections now open) 2016-03-13T08:45:15.743+0000 I NETWORK [conn4077966] end connection 10.0.20.53:47805 (1506 connections now open) 2016-03-13T08:45:15.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47815 #4077977 (1506 connections now open) 2016-03-13T08:45:15.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47816 #4077978 (1507 connections now open) 2016-03-13T08:45:16.075+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:45:15.807+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:45:16.117+0000 I NETWORK [conn4070590] end connection 10.0.20.53:40099 (1506 connections now open) 2016-03-13T08:45:16.117+0000 I NETWORK [conn4077258] end connection 10.0.26.69:38870 (1505 connections now open) 2016-03-13T08:45:16.118+0000 I NETWORK [conn4068915] end connection 10.0.216.90:44884 (1504 connections now open) 2016-03-13T08:45:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39647 #4077979 (1505 connections now open) 2016-03-13T08:45:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54616 #4077980 (1506 connections now open) 2016-03-13T08:45:16.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47819 #4077981 (1507 connections now open) 2016-03-13T08:45:18.154+0000 I NETWORK [conn4077967] end connection 10.0.179.209:37660 (1506 connections now open) 2016-03-13T08:45:18.154+0000 I NETWORK [conn4077968] end connection 10.0.179.209:37659 (1505 connections now open) 2016-03-13T08:45:18.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37672 #4077982 (1506 connections now open) 2016-03-13T08:45:18.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37673 #4077983 (1507 connections now open) 2016-03-13T08:45:18.749+0000 I NETWORK [conn4077970] end connection 10.0.26.69:39625 (1506 connections now open) 2016-03-13T08:45:18.749+0000 I NETWORK [conn4077969] end connection 10.0.26.69:39626 (1506 connections now open) 2016-03-13T08:45:18.758+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39650 #4077984 (1506 connections now open) 2016-03-13T08:45:18.758+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39649 #4077985 (1507 connections now open) 2016-03-13T08:45:21.651+0000 I NETWORK [conn4077971] end connection 10.0.27.237:32915 (1506 connections now open) 2016-03-13T08:45:21.651+0000 I NETWORK [conn4077972] end connection 10.0.27.237:32916 (1506 connections now open) 2016-03-13T08:45:21.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32929 #4077986 (1506 connections now open) 2016-03-13T08:45:21.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32928 #4077987 (1507 connections now open) 2016-03-13T08:45:22.973+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:45:22.973+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:45:22.974+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:45:22.974+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:45:23.723+0000 I NETWORK [conn4077974] end connection 10.0.7.113:56573 (1506 connections now open) 2016-03-13T08:45:23.724+0000 I NETWORK [conn4077973] end connection 10.0.7.113:56572 (1505 connections now open) 2016-03-13T08:45:23.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56586 #4077988 (1506 connections now open) 2016-03-13T08:45:23.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56587 #4077989 (1507 connections now open) 2016-03-13T08:45:24.908+0000 I NETWORK [conn4077976] end connection 10.0.216.90:54612 (1506 connections now open) 2016-03-13T08:45:24.908+0000 I NETWORK [conn4077975] end connection 10.0.216.90:54611 (1506 connections now open) 2016-03-13T08:45:24.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54625 #4077990 (1506 connections now open) 2016-03-13T08:45:24.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54626 #4077991 (1507 connections now open) 2016-03-13T08:45:25.756+0000 I NETWORK [conn4077977] end connection 10.0.20.53:47815 (1506 connections now open) 2016-03-13T08:45:25.756+0000 I NETWORK [conn4077978] end connection 10.0.20.53:47816 (1506 connections now open) 2016-03-13T08:45:25.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47836 #4077992 (1506 connections now open) 2016-03-13T08:45:25.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47835 #4077993 (1507 connections now open) 2016-03-13T08:45:28.113+0000 I NETWORK [conn4068899] end connection 10.0.20.53:38334 (1506 connections now open) 2016-03-13T08:45:28.113+0000 I NETWORK [conn4067776] end connection 10.0.26.69:56911 (1505 connections now open) 2016-03-13T08:45:28.113+0000 I NETWORK [conn4066551] end connection 10.0.27.237:49408 (1504 connections now open) 2016-03-13T08:45:28.113+0000 I NETWORK [conn4069156] end connection 10.0.7.113:47143 (1503 connections now open) 2016-03-13T08:45:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47840 #4077994 (1504 connections now open) 2016-03-13T08:45:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32935 #4077995 (1505 connections now open) 2016-03-13T08:45:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39659 #4077996 (1506 connections now open) 2016-03-13T08:45:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56602 #4077997 (1507 connections now open) 2016-03-13T08:45:28.166+0000 I NETWORK [conn4077982] end connection 10.0.179.209:37672 (1506 connections now open) 2016-03-13T08:45:28.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37686 #4077998 (1507 connections now open) 2016-03-13T08:45:28.167+0000 I NETWORK [conn4077983] end connection 10.0.179.209:37673 (1505 connections now open) 2016-03-13T08:45:28.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37687 #4077999 (1507 connections now open) 2016-03-13T08:45:28.771+0000 I NETWORK [conn4077984] end connection 10.0.26.69:39650 (1506 connections now open) 2016-03-13T08:45:28.771+0000 I NETWORK [conn4077985] end connection 10.0.26.69:39649 (1506 connections now open) 2016-03-13T08:45:28.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39661 #4078000 (1506 connections now open) 2016-03-13T08:45:28.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39660 #4078001 (1507 connections now open) 2016-03-13T08:45:31.672+0000 I NETWORK [conn4077986] end connection 10.0.27.237:32929 (1506 connections now open) 2016-03-13T08:45:31.672+0000 I NETWORK [conn4077987] end connection 10.0.27.237:32928 (1505 connections now open) 2016-03-13T08:45:31.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32942 #4078002 (1506 connections now open) 2016-03-13T08:45:31.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32943 #4078003 (1507 connections now open) 2016-03-13T08:45:32.976+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:45:32.977+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:45:32.977+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:45:32.977+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 4 checks in a row. Polling will stop after 26 more failed checks 2016-03-13T08:45:33.737+0000 I NETWORK [conn4077989] end connection 10.0.7.113:56587 (1506 connections now open) 2016-03-13T08:45:33.737+0000 I NETWORK [conn4077988] end connection 10.0.7.113:56586 (1506 connections now open) 2016-03-13T08:45:33.737+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56606 #4078004 (1506 connections now open) 2016-03-13T08:45:33.737+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56605 #4078005 (1507 connections now open) 2016-03-13T08:45:34.921+0000 I NETWORK [conn4077990] end connection 10.0.216.90:54625 (1506 connections now open) 2016-03-13T08:45:34.921+0000 I NETWORK [conn4077991] end connection 10.0.216.90:54626 (1506 connections now open) 2016-03-13T08:45:34.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54650 #4078006 (1506 connections now open) 2016-03-13T08:45:34.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54649 #4078007 (1507 connections now open) 2016-03-13T08:45:35.770+0000 I NETWORK [conn4077992] end connection 10.0.20.53:47836 (1506 connections now open) 2016-03-13T08:45:35.770+0000 I NETWORK [conn4077993] end connection 10.0.20.53:47835 (1506 connections now open) 2016-03-13T08:45:35.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47847 #4078008 (1506 connections now open) 2016-03-13T08:45:35.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47848 #4078009 (1507 connections now open) 2016-03-13T08:45:38.179+0000 I NETWORK [conn4077998] end connection 10.0.179.209:37686 (1506 connections now open) 2016-03-13T08:45:38.179+0000 I NETWORK [conn4077999] end connection 10.0.179.209:37687 (1505 connections now open) 2016-03-13T08:45:38.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37704 #4078010 (1506 connections now open) 2016-03-13T08:45:38.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37705 #4078011 (1507 connections now open) 2016-03-13T08:45:38.784+0000 I NETWORK [conn4078000] end connection 10.0.26.69:39661 (1506 connections now open) 2016-03-13T08:45:38.784+0000 I NETWORK [conn4078001] end connection 10.0.26.69:39660 (1506 connections now open) 2016-03-13T08:45:38.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39671 #4078012 (1506 connections now open) 2016-03-13T08:45:38.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39672 #4078013 (1507 connections now open) 2016-03-13T08:45:41.685+0000 I NETWORK [conn4078002] end connection 10.0.27.237:32942 (1506 connections now open) 2016-03-13T08:45:41.685+0000 I NETWORK [conn4078003] end connection 10.0.27.237:32943 (1505 connections now open) 2016-03-13T08:45:41.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32958 #4078014 (1506 connections now open) 2016-03-13T08:45:41.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32959 #4078015 (1507 connections now open) 2016-03-13T08:45:42.989+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:45:42.989+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:45:42.989+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:45:42.989+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 5 checks in a row. Polling will stop after 25 more failed checks 2016-03-13T08:45:43.750+0000 I NETWORK [conn4078004] end connection 10.0.7.113:56606 (1506 connections now open) 2016-03-13T08:45:43.750+0000 I NETWORK [conn4078005] end connection 10.0.7.113:56605 (1506 connections now open) 2016-03-13T08:45:43.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56615 #4078016 (1506 connections now open) 2016-03-13T08:45:43.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56614 #4078017 (1507 connections now open) 2016-03-13T08:45:44.934+0000 I NETWORK [conn4078007] end connection 10.0.216.90:54649 (1506 connections now open) 2016-03-13T08:45:44.934+0000 I NETWORK [conn4078006] end connection 10.0.216.90:54650 (1506 connections now open) 2016-03-13T08:45:44.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54658 #4078018 (1506 connections now open) 2016-03-13T08:45:44.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54657 #4078019 (1507 connections now open) 2016-03-13T08:45:45.784+0000 I NETWORK [conn4078009] end connection 10.0.20.53:47848 (1506 connections now open) 2016-03-13T08:45:45.784+0000 I NETWORK [conn4078008] end connection 10.0.20.53:47847 (1506 connections now open) 2016-03-13T08:45:45.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47860 #4078020 (1506 connections now open) 2016-03-13T08:45:45.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47859 #4078021 (1507 connections now open) 2016-03-13T08:45:46.118+0000 I NETWORK [conn4067341] end connection 10.0.27.237:50191 (1506 connections now open) 2016-03-13T08:45:46.118+0000 I NETWORK [conn4069215] end connection 10.0.216.90:45202 (1506 connections now open) 2016-03-13T08:45:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54661 #4078022 (1506 connections now open) 2016-03-13T08:45:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32964 #4078023 (1507 connections now open) 2016-03-13T08:45:46.289+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:45:46.075+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:45:48.191+0000 I NETWORK [conn4078010] end connection 10.0.179.209:37704 (1506 connections now open) 2016-03-13T08:45:48.192+0000 I NETWORK [conn4078011] end connection 10.0.179.209:37705 (1505 connections now open) 2016-03-13T08:45:48.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37712 #4078024 (1506 connections now open) 2016-03-13T08:45:48.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37713 #4078025 (1507 connections now open) 2016-03-13T08:45:48.798+0000 I NETWORK [conn4078013] end connection 10.0.26.69:39672 (1506 connections now open) 2016-03-13T08:45:48.798+0000 I NETWORK [conn4078012] end connection 10.0.26.69:39671 (1506 connections now open) 2016-03-13T08:45:48.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39692 #4078026 (1507 connections now open) 2016-03-13T08:45:48.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39691 #4078027 (1507 connections now open) 2016-03-13T08:45:51.697+0000 I NETWORK [conn4078015] end connection 10.0.27.237:32959 (1506 connections now open) 2016-03-13T08:45:51.697+0000 I NETWORK [conn4078014] end connection 10.0.27.237:32958 (1506 connections now open) 2016-03-13T08:45:51.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32966 #4078028 (1506 connections now open) 2016-03-13T08:45:51.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32967 #4078029 (1507 connections now open) 2016-03-13T08:45:52.992+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:45:52.993+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:45:52.993+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:45:52.993+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 6 checks in a row. Polling will stop after 24 more failed checks 2016-03-13T08:45:53.763+0000 I NETWORK [conn4078016] end connection 10.0.7.113:56615 (1506 connections now open) 2016-03-13T08:45:53.763+0000 I NETWORK [conn4078017] end connection 10.0.7.113:56614 (1506 connections now open) 2016-03-13T08:45:53.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56629 #4078030 (1506 connections now open) 2016-03-13T08:45:53.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56630 #4078031 (1507 connections now open) 2016-03-13T08:45:54.948+0000 I NETWORK [conn4078019] end connection 10.0.216.90:54657 (1506 connections now open) 2016-03-13T08:45:54.948+0000 I NETWORK [conn4078018] end connection 10.0.216.90:54658 (1506 connections now open) 2016-03-13T08:45:54.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54670 #4078032 (1506 connections now open) 2016-03-13T08:45:54.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54671 #4078033 (1507 connections now open) 2016-03-13T08:45:55.798+0000 I NETWORK [conn4078021] end connection 10.0.20.53:47859 (1506 connections now open) 2016-03-13T08:45:55.798+0000 I NETWORK [conn4078020] end connection 10.0.20.53:47860 (1506 connections now open) 2016-03-13T08:45:55.807+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47878 #4078034 (1506 connections now open) 2016-03-13T08:45:55.807+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47877 #4078035 (1507 connections now open) 2016-03-13T08:45:58.113+0000 I NETWORK [conn4070646] end connection 10.0.179.209:58500 (1506 connections now open) 2016-03-13T08:45:58.113+0000 I NETWORK [conn4070165] end connection 10.0.216.90:46212 (1506 connections now open) 2016-03-13T08:45:58.113+0000 I NETWORK [conn4074818] end connection 10.0.26.69:36235 (1504 connections now open) 2016-03-13T08:45:58.113+0000 I NETWORK [conn4073434] end connection 10.0.7.113:51718 (1503 connections now open) 2016-03-13T08:45:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54677 #4078036 (1504 connections now open) 2016-03-13T08:45:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56647 #4078037 (1505 connections now open) 2016-03-13T08:45:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39701 #4078038 (1506 connections now open) 2016-03-13T08:45:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37724 #4078039 (1507 connections now open) 2016-03-13T08:45:58.204+0000 I NETWORK [conn4078024] end connection 10.0.179.209:37712 (1506 connections now open) 2016-03-13T08:45:58.204+0000 I NETWORK [conn4078025] end connection 10.0.179.209:37713 (1505 connections now open) 2016-03-13T08:45:58.205+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37725 #4078040 (1506 connections now open) 2016-03-13T08:45:58.205+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37726 #4078041 (1507 connections now open) 2016-03-13T08:45:58.811+0000 I NETWORK [conn4078026] end connection 10.0.26.69:39692 (1506 connections now open) 2016-03-13T08:45:58.811+0000 I NETWORK [conn4078027] end connection 10.0.26.69:39691 (1506 connections now open) 2016-03-13T08:45:58.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39703 #4078042 (1506 connections now open) 2016-03-13T08:45:58.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39702 #4078043 (1507 connections now open) 2016-03-13T08:46:01.710+0000 I NETWORK [conn4078029] end connection 10.0.27.237:32967 (1506 connections now open) 2016-03-13T08:46:01.710+0000 I NETWORK [conn4078028] end connection 10.0.27.237:32966 (1506 connections now open) 2016-03-13T08:46:01.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32980 #4078044 (1506 connections now open) 2016-03-13T08:46:01.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32981 #4078045 (1507 connections now open) 2016-03-13T08:46:02.996+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:46:02.997+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:46:02.997+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:46:02.997+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 7 checks in a row. Polling will stop after 23 more failed checks 2016-03-13T08:46:03.777+0000 I NETWORK [conn4078031] end connection 10.0.7.113:56630 (1506 connections now open) 2016-03-13T08:46:03.777+0000 I NETWORK [conn4078030] end connection 10.0.7.113:56629 (1506 connections now open) 2016-03-13T08:46:03.778+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56652 #4078046 (1506 connections now open) 2016-03-13T08:46:03.778+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56653 #4078047 (1507 connections now open) 2016-03-13T08:46:04.962+0000 I NETWORK [conn4078033] end connection 10.0.216.90:54671 (1506 connections now open) 2016-03-13T08:46:04.962+0000 I NETWORK [conn4078032] end connection 10.0.216.90:54670 (1506 connections now open) 2016-03-13T08:46:04.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54693 #4078048 (1506 connections now open) 2016-03-13T08:46:04.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54694 #4078049 (1507 connections now open) 2016-03-13T08:46:05.820+0000 I NETWORK [conn4078034] end connection 10.0.20.53:47878 (1506 connections now open) 2016-03-13T08:46:05.820+0000 I NETWORK [conn4078035] end connection 10.0.20.53:47877 (1506 connections now open) 2016-03-13T08:46:05.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47887 #4078050 (1506 connections now open) 2016-03-13T08:46:05.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47888 #4078051 (1507 connections now open) 2016-03-13T08:46:08.218+0000 I NETWORK [conn4078041] end connection 10.0.179.209:37726 (1506 connections now open) 2016-03-13T08:46:08.218+0000 I NETWORK [conn4078040] end connection 10.0.179.209:37725 (1506 connections now open) 2016-03-13T08:46:08.218+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37745 #4078052 (1506 connections now open) 2016-03-13T08:46:08.218+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37744 #4078053 (1507 connections now open) 2016-03-13T08:46:08.825+0000 I NETWORK [conn4078043] end connection 10.0.26.69:39702 (1506 connections now open) 2016-03-13T08:46:08.825+0000 I NETWORK [conn4078042] end connection 10.0.26.69:39703 (1506 connections now open) 2016-03-13T08:46:08.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39717 #4078054 (1506 connections now open) 2016-03-13T08:46:08.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39716 #4078055 (1507 connections now open) 2016-03-13T08:46:11.723+0000 I NETWORK [conn4078045] end connection 10.0.27.237:32981 (1506 connections now open) 2016-03-13T08:46:11.723+0000 I NETWORK [conn4078044] end connection 10.0.27.237:32980 (1506 connections now open) 2016-03-13T08:46:11.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32997 #4078056 (1506 connections now open) 2016-03-13T08:46:11.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:32998 #4078057 (1507 connections now open) 2016-03-13T08:46:13.000+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:46:13.009+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:46:13.009+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:46:13.009+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 8 checks in a row. Polling will stop after 22 more failed checks 2016-03-13T08:46:13.791+0000 I NETWORK [conn4078046] end connection 10.0.7.113:56652 (1506 connections now open) 2016-03-13T08:46:13.791+0000 I NETWORK [conn4078047] end connection 10.0.7.113:56653 (1506 connections now open) 2016-03-13T08:46:13.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56664 #4078058 (1506 connections now open) 2016-03-13T08:46:13.801+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56663 #4078059 (1507 connections now open) 2016-03-13T08:46:14.984+0000 I NETWORK [conn4078049] end connection 10.0.216.90:54694 (1506 connections now open) 2016-03-13T08:46:14.984+0000 I NETWORK [conn4078048] end connection 10.0.216.90:54693 (1506 connections now open) 2016-03-13T08:46:14.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54702 #4078060 (1506 connections now open) 2016-03-13T08:46:14.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54701 #4078061 (1507 connections now open) 2016-03-13T08:46:15.833+0000 I NETWORK [conn4078050] end connection 10.0.20.53:47887 (1506 connections now open) 2016-03-13T08:46:15.833+0000 I NETWORK [conn4078051] end connection 10.0.20.53:47888 (1506 connections now open) 2016-03-13T08:46:15.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47898 #4078062 (1506 connections now open) 2016-03-13T08:46:15.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47899 #4078063 (1507 connections now open) 2016-03-13T08:46:16.118+0000 I NETWORK [conn4073414] end connection 10.0.179.209:33029 (1506 connections now open) 2016-03-13T08:46:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37753 #4078064 (1507 connections now open) 2016-03-13T08:46:16.422+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:46:16.289+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:46:18.231+0000 I NETWORK [conn4078052] end connection 10.0.179.209:37745 (1506 connections now open) 2016-03-13T08:46:18.231+0000 I NETWORK [conn4078053] end connection 10.0.179.209:37744 (1506 connections now open) 2016-03-13T08:46:18.232+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37757 #4078065 (1506 connections now open) 2016-03-13T08:46:18.232+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37758 #4078066 (1507 connections now open) 2016-03-13T08:46:18.838+0000 I NETWORK [conn4078054] end connection 10.0.26.69:39717 (1506 connections now open) 2016-03-13T08:46:18.838+0000 I NETWORK [conn4078055] end connection 10.0.26.69:39716 (1506 connections now open) 2016-03-13T08:46:18.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39741 #4078067 (1506 connections now open) 2016-03-13T08:46:18.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39740 #4078068 (1507 connections now open) 2016-03-13T08:46:21.736+0000 I NETWORK [conn4078057] end connection 10.0.27.237:32998 (1506 connections now open) 2016-03-13T08:46:21.736+0000 I NETWORK [conn4078056] end connection 10.0.27.237:32997 (1506 connections now open) 2016-03-13T08:46:21.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33009 #4078069 (1506 connections now open) 2016-03-13T08:46:21.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33010 #4078070 (1507 connections now open) 2016-03-13T08:46:23.012+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:46:23.013+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:46:23.813+0000 I NETWORK [conn4078058] end connection 10.0.7.113:56664 (1506 connections now open) 2016-03-13T08:46:23.813+0000 I NETWORK [conn4078059] end connection 10.0.7.113:56663 (1506 connections now open) 2016-03-13T08:46:23.814+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56678 #4078071 (1506 connections now open) 2016-03-13T08:46:23.814+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56677 #4078072 (1507 connections now open) 2016-03-13T08:46:24.999+0000 I NETWORK [conn4078060] end connection 10.0.216.90:54702 (1506 connections now open) 2016-03-13T08:46:24.999+0000 I NETWORK [conn4078061] end connection 10.0.216.90:54701 (1506 connections now open) 2016-03-13T08:46:24.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54714 #4078073 (1506 connections now open) 2016-03-13T08:46:24.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54715 #4078074 (1507 connections now open) 2016-03-13T08:46:25.847+0000 I NETWORK [conn4078062] end connection 10.0.20.53:47898 (1506 connections now open) 2016-03-13T08:46:25.847+0000 I NETWORK [conn4078063] end connection 10.0.20.53:47899 (1506 connections now open) 2016-03-13T08:46:25.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47918 #4078075 (1506 connections now open) 2016-03-13T08:46:25.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47919 #4078076 (1507 connections now open) 2016-03-13T08:46:28.113+0000 I NETWORK [conn4067773] end connection 10.0.179.209:55604 (1506 connections now open) 2016-03-13T08:46:28.113+0000 I NETWORK [conn4064728] end connection 10.0.20.53:34082 (1505 connections now open) 2016-03-13T08:46:28.113+0000 I NETWORK [conn4070097] end connection 10.0.27.237:53074 (1504 connections now open) 2016-03-13T08:46:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37769 #4078077 (1506 connections now open) 2016-03-13T08:46:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47922 #4078078 (1506 connections now open) 2016-03-13T08:46:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33016 #4078079 (1507 connections now open) 2016-03-13T08:46:28.114+0000 I NETWORK [conn4069197] end connection 10.0.7.113:47191 (1506 connections now open) 2016-03-13T08:46:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56685 #4078080 (1507 connections now open) 2016-03-13T08:46:28.244+0000 I NETWORK [conn4078065] end connection 10.0.179.209:37757 (1506 connections now open) 2016-03-13T08:46:28.244+0000 I NETWORK [conn4078066] end connection 10.0.179.209:37758 (1506 connections now open) 2016-03-13T08:46:28.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37770 #4078081 (1506 connections now open) 2016-03-13T08:46:28.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37771 #4078082 (1507 connections now open) 2016-03-13T08:46:28.860+0000 I NETWORK [conn4078068] end connection 10.0.26.69:39740 (1506 connections now open) 2016-03-13T08:46:28.860+0000 I NETWORK [conn4078067] end connection 10.0.26.69:39741 (1506 connections now open) 2016-03-13T08:46:28.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39751 #4078083 (1506 connections now open) 2016-03-13T08:46:28.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39752 #4078084 (1507 connections now open) 2016-03-13T08:46:31.757+0000 I NETWORK [conn4078069] end connection 10.0.27.237:33009 (1506 connections now open) 2016-03-13T08:46:31.757+0000 I NETWORK [conn4078070] end connection 10.0.27.237:33010 (1505 connections now open) 2016-03-13T08:46:31.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33023 #4078085 (1506 connections now open) 2016-03-13T08:46:31.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33022 #4078086 (1507 connections now open) 2016-03-13T08:46:33.017+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:46:33.017+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:46:33.827+0000 I NETWORK [conn4078072] end connection 10.0.7.113:56677 (1506 connections now open) 2016-03-13T08:46:33.827+0000 I NETWORK [conn4078071] end connection 10.0.7.113:56678 (1506 connections now open) 2016-03-13T08:46:33.828+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56699 #4078087 (1506 connections now open) 2016-03-13T08:46:33.828+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56698 #4078088 (1507 connections now open) 2016-03-13T08:46:35.012+0000 I NETWORK [conn4078073] end connection 10.0.216.90:54714 (1506 connections now open) 2016-03-13T08:46:35.012+0000 I NETWORK [conn4078074] end connection 10.0.216.90:54715 (1506 connections now open) 2016-03-13T08:46:35.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54736 #4078089 (1506 connections now open) 2016-03-13T08:46:35.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54737 #4078090 (1507 connections now open) 2016-03-13T08:46:35.860+0000 I NETWORK [conn4078076] end connection 10.0.20.53:47919 (1506 connections now open) 2016-03-13T08:46:35.860+0000 I NETWORK [conn4078075] end connection 10.0.20.53:47918 (1506 connections now open) 2016-03-13T08:46:35.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47930 #4078091 (1506 connections now open) 2016-03-13T08:46:35.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47931 #4078092 (1507 connections now open) 2016-03-13T08:46:38.257+0000 I NETWORK [conn4078082] end connection 10.0.179.209:37771 (1506 connections now open) 2016-03-13T08:46:38.257+0000 I NETWORK [conn4078081] end connection 10.0.179.209:37770 (1506 connections now open) 2016-03-13T08:46:38.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37787 #4078093 (1506 connections now open) 2016-03-13T08:46:38.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37788 #4078094 (1507 connections now open) 2016-03-13T08:46:38.873+0000 I NETWORK [conn4078084] end connection 10.0.26.69:39752 (1506 connections now open) 2016-03-13T08:46:38.873+0000 I NETWORK [conn4078083] end connection 10.0.26.69:39751 (1506 connections now open) 2016-03-13T08:46:38.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39764 #4078095 (1506 connections now open) 2016-03-13T08:46:38.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39765 #4078096 (1507 connections now open) 2016-03-13T08:46:41.770+0000 I NETWORK [conn4078086] end connection 10.0.27.237:33022 (1506 connections now open) 2016-03-13T08:46:41.770+0000 I NETWORK [conn4078085] end connection 10.0.27.237:33023 (1506 connections now open) 2016-03-13T08:46:41.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33040 #4078097 (1506 connections now open) 2016-03-13T08:46:41.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33041 #4078098 (1507 connections now open) 2016-03-13T08:46:43.020+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:46:43.841+0000 I NETWORK [conn4078088] end connection 10.0.7.113:56698 (1506 connections now open) 2016-03-13T08:46:43.841+0000 I NETWORK [conn4078087] end connection 10.0.7.113:56699 (1506 connections now open) 2016-03-13T08:46:43.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56707 #4078099 (1506 connections now open) 2016-03-13T08:46:43.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56706 #4078100 (1507 connections now open) 2016-03-13T08:46:45.026+0000 I NETWORK [conn4078089] end connection 10.0.216.90:54736 (1506 connections now open) 2016-03-13T08:46:45.026+0000 I NETWORK [conn4078090] end connection 10.0.216.90:54737 (1506 connections now open) 2016-03-13T08:46:45.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54745 #4078101 (1506 connections now open) 2016-03-13T08:46:45.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54746 #4078102 (1507 connections now open) 2016-03-13T08:46:45.874+0000 I NETWORK [conn4078091] end connection 10.0.20.53:47930 (1506 connections now open) 2016-03-13T08:46:45.874+0000 I NETWORK [conn4078092] end connection 10.0.20.53:47931 (1506 connections now open) 2016-03-13T08:46:45.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47942 #4078103 (1506 connections now open) 2016-03-13T08:46:45.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47943 #4078104 (1507 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [conn4073457] end connection 10.0.179.209:33074 (1506 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [conn4070120] end connection 10.0.20.53:39616 (1506 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [conn4067527] end connection 10.0.7.113:45419 (1506 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [conn4071607] end connection 10.0.27.237:54675 (1506 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47945 #4078105 (1504 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37796 #4078106 (1505 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33046 #4078107 (1506 connections now open) 2016-03-13T08:46:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56713 #4078108 (1507 connections now open) 2016-03-13T08:46:46.662+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:46:46.422+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:46:48.270+0000 I NETWORK [conn4078093] end connection 10.0.179.209:37787 (1506 connections now open) 2016-03-13T08:46:48.270+0000 I NETWORK [conn4078094] end connection 10.0.179.209:37788 (1506 connections now open) 2016-03-13T08:46:48.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37797 #4078109 (1506 connections now open) 2016-03-13T08:46:48.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37798 #4078110 (1507 connections now open) 2016-03-13T08:46:48.887+0000 I NETWORK [conn4078096] end connection 10.0.26.69:39765 (1506 connections now open) 2016-03-13T08:46:48.887+0000 I NETWORK [conn4078095] end connection 10.0.26.69:39764 (1506 connections now open) 2016-03-13T08:46:48.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39785 #4078111 (1506 connections now open) 2016-03-13T08:46:48.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39786 #4078112 (1507 connections now open) 2016-03-13T08:46:51.783+0000 I NETWORK [conn4078097] end connection 10.0.27.237:33040 (1506 connections now open) 2016-03-13T08:46:51.783+0000 I NETWORK [conn4078098] end connection 10.0.27.237:33041 (1506 connections now open) 2016-03-13T08:46:51.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33049 #4078113 (1506 connections now open) 2016-03-13T08:46:51.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33050 #4078114 (1507 connections now open) 2016-03-13T08:46:53.023+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:46:53.855+0000 I NETWORK [conn4078099] end connection 10.0.7.113:56707 (1506 connections now open) 2016-03-13T08:46:53.855+0000 I NETWORK [conn4078100] end connection 10.0.7.113:56706 (1506 connections now open) 2016-03-13T08:46:53.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56721 #4078115 (1506 connections now open) 2016-03-13T08:46:53.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56720 #4078116 (1507 connections now open) 2016-03-13T08:46:55.040+0000 I NETWORK [conn4078101] end connection 10.0.216.90:54745 (1506 connections now open) 2016-03-13T08:46:55.040+0000 I NETWORK [conn4078102] end connection 10.0.216.90:54746 (1506 connections now open) 2016-03-13T08:46:55.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54759 #4078117 (1506 connections now open) 2016-03-13T08:46:55.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54760 #4078118 (1507 connections now open) 2016-03-13T08:46:55.888+0000 I NETWORK [conn4078103] end connection 10.0.20.53:47942 (1506 connections now open) 2016-03-13T08:46:55.888+0000 I NETWORK [conn4078104] end connection 10.0.20.53:47943 (1506 connections now open) 2016-03-13T08:46:55.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47962 #4078119 (1506 connections now open) 2016-03-13T08:46:55.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47961 #4078120 (1507 connections now open) 2016-03-13T08:46:58.113+0000 I NETWORK [conn4071288] end connection 10.0.20.53:40818 (1506 connections now open) 2016-03-13T08:46:58.113+0000 I NETWORK [conn4070142] end connection 10.0.26.69:59454 (1505 connections now open) 2016-03-13T08:46:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47965 #4078121 (1506 connections now open) 2016-03-13T08:46:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39794 #4078122 (1507 connections now open) 2016-03-13T08:46:58.282+0000 I NETWORK [conn4078109] end connection 10.0.179.209:37797 (1506 connections now open) 2016-03-13T08:46:58.282+0000 I NETWORK [conn4078110] end connection 10.0.179.209:37798 (1505 connections now open) 2016-03-13T08:46:58.291+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37810 #4078123 (1506 connections now open) 2016-03-13T08:46:58.291+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37811 #4078124 (1507 connections now open) 2016-03-13T08:46:58.900+0000 I NETWORK [conn4078112] end connection 10.0.26.69:39786 (1506 connections now open) 2016-03-13T08:46:58.900+0000 I NETWORK [conn4078111] end connection 10.0.26.69:39785 (1506 connections now open) 2016-03-13T08:46:58.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39795 #4078125 (1506 connections now open) 2016-03-13T08:46:58.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39796 #4078126 (1507 connections now open) 2016-03-13T08:47:01.796+0000 I NETWORK [conn4078114] end connection 10.0.27.237:33050 (1506 connections now open) 2016-03-13T08:47:01.796+0000 I NETWORK [conn4078113] end connection 10.0.27.237:33049 (1506 connections now open) 2016-03-13T08:47:01.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33065 #4078127 (1506 connections now open) 2016-03-13T08:47:01.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33064 #4078128 (1507 connections now open) 2016-03-13T08:47:03.027+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:47:03.869+0000 I NETWORK [conn4078116] end connection 10.0.7.113:56720 (1506 connections now open) 2016-03-13T08:47:03.869+0000 I NETWORK [conn4078115] end connection 10.0.7.113:56721 (1506 connections now open) 2016-03-13T08:47:03.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56744 #4078129 (1506 connections now open) 2016-03-13T08:47:03.870+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56743 #4078130 (1507 connections now open) 2016-03-13T08:47:05.054+0000 I NETWORK [conn4078118] end connection 10.0.216.90:54760 (1506 connections now open) 2016-03-13T08:47:05.054+0000 I NETWORK [conn4078117] end connection 10.0.216.90:54759 (1506 connections now open) 2016-03-13T08:47:05.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54784 #4078131 (1506 connections now open) 2016-03-13T08:47:05.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54783 #4078132 (1507 connections now open) 2016-03-13T08:47:05.910+0000 I NETWORK [conn4078119] end connection 10.0.20.53:47962 (1506 connections now open) 2016-03-13T08:47:05.910+0000 I NETWORK [conn4078120] end connection 10.0.20.53:47961 (1506 connections now open) 2016-03-13T08:47:05.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47973 #4078133 (1506 connections now open) 2016-03-13T08:47:05.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47974 #4078134 (1507 connections now open) 2016-03-13T08:47:08.304+0000 I NETWORK [conn4078124] end connection 10.0.179.209:37811 (1506 connections now open) 2016-03-13T08:47:08.304+0000 I NETWORK [conn4078123] end connection 10.0.179.209:37810 (1506 connections now open) 2016-03-13T08:47:08.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37829 #4078135 (1506 connections now open) 2016-03-13T08:47:08.305+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37830 #4078136 (1507 connections now open) 2016-03-13T08:47:08.914+0000 I NETWORK [conn4078126] end connection 10.0.26.69:39796 (1506 connections now open) 2016-03-13T08:47:08.914+0000 I NETWORK [conn4078125] end connection 10.0.26.69:39795 (1506 connections now open) 2016-03-13T08:47:08.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39809 #4078137 (1506 connections now open) 2016-03-13T08:47:08.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39810 #4078138 (1507 connections now open) 2016-03-13T08:47:11.809+0000 I NETWORK [conn4078127] end connection 10.0.27.237:33065 (1506 connections now open) 2016-03-13T08:47:11.809+0000 I NETWORK [conn4078128] end connection 10.0.27.237:33064 (1506 connections now open) 2016-03-13T08:47:11.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33082 #4078139 (1506 connections now open) 2016-03-13T08:47:11.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33083 #4078140 (1507 connections now open) 2016-03-13T08:47:13.038+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:47:13.882+0000 I NETWORK [conn4078130] end connection 10.0.7.113:56743 (1506 connections now open) 2016-03-13T08:47:13.882+0000 I NETWORK [conn4078129] end connection 10.0.7.113:56744 (1506 connections now open) 2016-03-13T08:47:13.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56755 #4078141 (1506 connections now open) 2016-03-13T08:47:13.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56754 #4078142 (1507 connections now open) 2016-03-13T08:47:15.076+0000 I NETWORK [conn4078132] end connection 10.0.216.90:54783 (1506 connections now open) 2016-03-13T08:47:15.076+0000 I NETWORK [conn4078131] end connection 10.0.216.90:54784 (1506 connections now open) 2016-03-13T08:47:15.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54792 #4078143 (1506 connections now open) 2016-03-13T08:47:15.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54791 #4078144 (1507 connections now open) 2016-03-13T08:47:15.924+0000 I NETWORK [conn4078134] end connection 10.0.20.53:47974 (1506 connections now open) 2016-03-13T08:47:15.924+0000 I NETWORK [conn4078133] end connection 10.0.20.53:47973 (1506 connections now open) 2016-03-13T08:47:15.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47984 #4078145 (1506 connections now open) 2016-03-13T08:47:15.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47983 #4078146 (1507 connections now open) 2016-03-13T08:47:16.118+0000 I NETWORK [conn4076869] end connection 10.0.179.209:36551 (1506 connections now open) 2016-03-13T08:47:16.118+0000 I NETWORK [conn4071479] end connection 10.0.20.53:41029 (1506 connections now open) 2016-03-13T08:47:16.118+0000 I NETWORK [conn4065541] end connection 10.0.216.90:41285 (1505 connections now open) 2016-03-13T08:47:16.118+0000 I NETWORK [conn4067043] end connection 10.0.26.69:56143 (1503 connections now open) 2016-03-13T08:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37838 #4078147 (1504 connections now open) 2016-03-13T08:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:47986 #4078148 (1505 connections now open) 2016-03-13T08:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54796 #4078149 (1506 connections now open) 2016-03-13T08:47:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39830 #4078150 (1507 connections now open) 2016-03-13T08:47:16.789+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:47:16.662+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:47:18.317+0000 I NETWORK [conn4078135] end connection 10.0.179.209:37829 (1506 connections now open) 2016-03-13T08:47:18.317+0000 I NETWORK [conn4078136] end connection 10.0.179.209:37830 (1506 connections now open) 2016-03-13T08:47:18.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37842 #4078151 (1506 connections now open) 2016-03-13T08:47:18.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37843 #4078152 (1507 connections now open) 2016-03-13T08:47:18.927+0000 I NETWORK [conn4078137] end connection 10.0.26.69:39809 (1506 connections now open) 2016-03-13T08:47:18.927+0000 I NETWORK [conn4078138] end connection 10.0.26.69:39810 (1506 connections now open) 2016-03-13T08:47:18.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39833 #4078153 (1506 connections now open) 2016-03-13T08:47:18.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39832 #4078154 (1507 connections now open) 2016-03-13T08:47:21.822+0000 I NETWORK [conn4078140] end connection 10.0.27.237:33083 (1506 connections now open) 2016-03-13T08:47:21.822+0000 I NETWORK [conn4078139] end connection 10.0.27.237:33082 (1505 connections now open) 2016-03-13T08:47:21.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33096 #4078155 (1506 connections now open) 2016-03-13T08:47:21.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33095 #4078156 (1507 connections now open) 2016-03-13T08:47:23.042+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:47:23.905+0000 I NETWORK [conn4078141] end connection 10.0.7.113:56755 (1506 connections now open) 2016-03-13T08:47:23.905+0000 I NETWORK [conn4078142] end connection 10.0.7.113:56754 (1506 connections now open) 2016-03-13T08:47:23.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56769 #4078157 (1506 connections now open) 2016-03-13T08:47:23.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56768 #4078158 (1507 connections now open) 2016-03-13T08:47:25.089+0000 I NETWORK [conn4078144] end connection 10.0.216.90:54791 (1506 connections now open) 2016-03-13T08:47:25.089+0000 I NETWORK [conn4078143] end connection 10.0.216.90:54792 (1506 connections now open) 2016-03-13T08:47:25.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54806 #4078159 (1506 connections now open) 2016-03-13T08:47:25.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54805 #4078160 (1507 connections now open) 2016-03-13T08:47:25.937+0000 I NETWORK [conn4078145] end connection 10.0.20.53:47984 (1506 connections now open) 2016-03-13T08:47:25.937+0000 I NETWORK [conn4078146] end connection 10.0.20.53:47983 (1506 connections now open) 2016-03-13T08:47:25.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48002 #4078161 (1506 connections now open) 2016-03-13T08:47:25.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48003 #4078162 (1507 connections now open) 2016-03-13T08:47:28.113+0000 I NETWORK [conn4074729] end connection 10.0.20.53:44437 (1506 connections now open) 2016-03-13T08:47:28.113+0000 I NETWORK [conn4068111] end connection 10.0.179.209:55953 (1506 connections now open) 2016-03-13T08:47:28.114+0000 I NETWORK [conn4068010] end connection 10.0.216.90:43922 (1504 connections now open) 2016-03-13T08:47:28.114+0000 I NETWORK [conn4070079] end connection 10.0.7.113:48148 (1503 connections now open) 2016-03-13T08:47:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37853 #4078163 (1504 connections now open) 2016-03-13T08:47:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48005 #4078164 (1505 connections now open) 2016-03-13T08:47:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54812 #4078165 (1506 connections now open) 2016-03-13T08:47:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56776 #4078166 (1507 connections now open) 2016-03-13T08:47:28.331+0000 I NETWORK [conn4078152] end connection 10.0.179.209:37843 (1506 connections now open) 2016-03-13T08:47:28.331+0000 I NETWORK [conn4078151] end connection 10.0.179.209:37842 (1506 connections now open) 2016-03-13T08:47:28.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37855 #4078167 (1506 connections now open) 2016-03-13T08:47:28.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37854 #4078168 (1507 connections now open) 2016-03-13T08:47:28.941+0000 I NETWORK [conn4078154] end connection 10.0.26.69:39832 (1506 connections now open) 2016-03-13T08:47:28.941+0000 I NETWORK [conn4078153] end connection 10.0.26.69:39833 (1506 connections now open) 2016-03-13T08:47:28.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39843 #4078169 (1506 connections now open) 2016-03-13T08:47:28.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39844 #4078170 (1507 connections now open) 2016-03-13T08:47:31.844+0000 I NETWORK [conn4078155] end connection 10.0.27.237:33096 (1506 connections now open) 2016-03-13T08:47:31.844+0000 I NETWORK [conn4078156] end connection 10.0.27.237:33095 (1506 connections now open) 2016-03-13T08:47:31.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33108 #4078171 (1506 connections now open) 2016-03-13T08:47:31.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33109 #4078172 (1507 connections now open) 2016-03-13T08:47:33.045+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:47:33.917+0000 I NETWORK [conn4078157] end connection 10.0.7.113:56769 (1506 connections now open) 2016-03-13T08:47:33.917+0000 I NETWORK [conn4078158] end connection 10.0.7.113:56768 (1506 connections now open) 2016-03-13T08:47:33.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56789 #4078173 (1506 connections now open) 2016-03-13T08:47:33.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56790 #4078174 (1507 connections now open) 2016-03-13T08:47:35.102+0000 I NETWORK [conn4078160] end connection 10.0.216.90:54805 (1506 connections now open) 2016-03-13T08:47:35.102+0000 I NETWORK [conn4078159] end connection 10.0.216.90:54806 (1506 connections now open) 2016-03-13T08:47:35.103+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54828 #4078175 (1506 connections now open) 2016-03-13T08:47:35.103+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54829 #4078176 (1507 connections now open) 2016-03-13T08:47:35.951+0000 I NETWORK [conn4078162] end connection 10.0.20.53:48003 (1506 connections now open) 2016-03-13T08:47:35.951+0000 I NETWORK [conn4078161] end connection 10.0.20.53:48002 (1506 connections now open) 2016-03-13T08:47:35.952+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48015 #4078177 (1506 connections now open) 2016-03-13T08:47:35.952+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48016 #4078178 (1507 connections now open) 2016-03-13T08:47:38.344+0000 I NETWORK [conn4078167] end connection 10.0.179.209:37855 (1506 connections now open) 2016-03-13T08:47:38.344+0000 I NETWORK [conn4078168] end connection 10.0.179.209:37854 (1506 connections now open) 2016-03-13T08:47:38.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37873 #4078179 (1506 connections now open) 2016-03-13T08:47:38.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37872 #4078180 (1507 connections now open) 2016-03-13T08:47:38.954+0000 I NETWORK [conn4078169] end connection 10.0.26.69:39843 (1506 connections now open) 2016-03-13T08:47:38.955+0000 I NETWORK [conn4078170] end connection 10.0.26.69:39844 (1505 connections now open) 2016-03-13T08:47:38.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39856 #4078181 (1506 connections now open) 2016-03-13T08:47:38.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39857 #4078182 (1507 connections now open) 2016-03-13T08:47:41.857+0000 I NETWORK [conn4078172] end connection 10.0.27.237:33109 (1506 connections now open) 2016-03-13T08:47:41.857+0000 I NETWORK [conn4078171] end connection 10.0.27.237:33108 (1505 connections now open) 2016-03-13T08:47:41.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33117 #4078183 (1506 connections now open) 2016-03-13T08:47:41.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33118 #4078184 (1507 connections now open) 2016-03-13T08:47:43.049+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:47:43.931+0000 I NETWORK [conn4078173] end connection 10.0.7.113:56789 (1506 connections now open) 2016-03-13T08:47:43.931+0000 I NETWORK [conn4078174] end connection 10.0.7.113:56790 (1506 connections now open) 2016-03-13T08:47:43.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56797 #4078185 (1507 connections now open) 2016-03-13T08:47:43.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56796 #4078186 (1507 connections now open) 2016-03-13T08:47:45.116+0000 I NETWORK [conn4078175] end connection 10.0.216.90:54828 (1506 connections now open) 2016-03-13T08:47:45.116+0000 I NETWORK [conn4078176] end connection 10.0.216.90:54829 (1506 connections now open) 2016-03-13T08:47:45.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54837 #4078187 (1506 connections now open) 2016-03-13T08:47:45.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54838 #4078188 (1507 connections now open) 2016-03-13T08:47:45.965+0000 I NETWORK [conn4078178] end connection 10.0.20.53:48016 (1506 connections now open) 2016-03-13T08:47:45.965+0000 I NETWORK [conn4078177] end connection 10.0.20.53:48015 (1506 connections now open) 2016-03-13T08:47:45.965+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48027 #4078189 (1506 connections now open) 2016-03-13T08:47:45.965+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48026 #4078190 (1507 connections now open) 2016-03-13T08:47:46.118+0000 I NETWORK [conn4073622] end connection 10.0.179.209:33240 (1506 connections now open) 2016-03-13T08:47:46.118+0000 I NETWORK [conn4069553] end connection 10.0.216.90:45562 (1505 connections now open) 2016-03-13T08:47:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37880 #4078191 (1506 connections now open) 2016-03-13T08:47:46.118+0000 I NETWORK [conn4076997] end connection 10.0.7.113:55533 (1504 connections now open) 2016-03-13T08:47:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54842 #4078192 (1506 connections now open) 2016-03-13T08:47:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56803 #4078193 (1507 connections now open) 2016-03-13T08:47:47.057+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:47:46.789+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:47:48.357+0000 I NETWORK [conn4078179] end connection 10.0.179.209:37873 (1506 connections now open) 2016-03-13T08:47:48.357+0000 I NETWORK [conn4078180] end connection 10.0.179.209:37872 (1506 connections now open) 2016-03-13T08:47:48.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37882 #4078194 (1506 connections now open) 2016-03-13T08:47:48.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37883 #4078195 (1507 connections now open) 2016-03-13T08:47:48.967+0000 I NETWORK [conn4078181] end connection 10.0.26.69:39856 (1506 connections now open) 2016-03-13T08:47:48.967+0000 I NETWORK [conn4078182] end connection 10.0.26.69:39857 (1506 connections now open) 2016-03-13T08:47:48.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39876 #4078196 (1507 connections now open) 2016-03-13T08:47:48.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39875 #4078197 (1507 connections now open) 2016-03-13T08:47:51.870+0000 I NETWORK [conn4078184] end connection 10.0.27.237:33118 (1506 connections now open) 2016-03-13T08:47:51.870+0000 I NETWORK [conn4078183] end connection 10.0.27.237:33117 (1506 connections now open) 2016-03-13T08:47:51.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33136 #4078198 (1506 connections now open) 2016-03-13T08:47:51.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33135 #4078199 (1507 connections now open) 2016-03-13T08:47:53.052+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:47:53.945+0000 I NETWORK [conn4078186] end connection 10.0.7.113:56796 (1506 connections now open) 2016-03-13T08:47:53.945+0000 I NETWORK [conn4078185] end connection 10.0.7.113:56797 (1506 connections now open) 2016-03-13T08:47:53.945+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56810 #4078200 (1506 connections now open) 2016-03-13T08:47:53.945+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56811 #4078201 (1507 connections now open) 2016-03-13T08:47:55.130+0000 I NETWORK [conn4078187] end connection 10.0.216.90:54837 (1506 connections now open) 2016-03-13T08:47:55.130+0000 I NETWORK [conn4078188] end connection 10.0.216.90:54838 (1506 connections now open) 2016-03-13T08:47:55.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54853 #4078202 (1506 connections now open) 2016-03-13T08:47:55.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54852 #4078203 (1507 connections now open) 2016-03-13T08:47:55.978+0000 I NETWORK [conn4078189] end connection 10.0.20.53:48027 (1506 connections now open) 2016-03-13T08:47:55.978+0000 I NETWORK [conn4078190] end connection 10.0.20.53:48026 (1506 connections now open) 2016-03-13T08:47:55.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48047 #4078204 (1506 connections now open) 2016-03-13T08:47:55.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48046 #4078205 (1507 connections now open) 2016-03-13T08:47:58.113+0000 I NETWORK [conn4074061] end connection 10.0.179.209:33683 (1506 connections now open) 2016-03-13T08:47:58.114+0000 I NETWORK [conn4072804] end connection 10.0.26.69:34082 (1505 connections now open) 2016-03-13T08:47:58.114+0000 I NETWORK [conn4067549] end connection 10.0.216.90:43444 (1505 connections now open) 2016-03-13T08:47:58.114+0000 I NETWORK [conn4068368] end connection 10.0.27.237:51240 (1503 connections now open) 2016-03-13T08:47:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54859 #4078206 (1504 connections now open) 2016-03-13T08:47:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39885 #4078207 (1505 connections now open) 2016-03-13T08:47:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33142 #4078208 (1506 connections now open) 2016-03-13T08:47:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37894 #4078209 (1507 connections now open) 2016-03-13T08:47:58.371+0000 I NETWORK [conn4078195] end connection 10.0.179.209:37883 (1506 connections now open) 2016-03-13T08:47:58.371+0000 I NETWORK [conn4078194] end connection 10.0.179.209:37882 (1506 connections now open) 2016-03-13T08:47:58.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37895 #4078210 (1506 connections now open) 2016-03-13T08:47:58.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37896 #4078211 (1507 connections now open) 2016-03-13T08:47:58.981+0000 I NETWORK [conn4078197] end connection 10.0.26.69:39875 (1506 connections now open) 2016-03-13T08:47:58.981+0000 I NETWORK [conn4078196] end connection 10.0.26.69:39876 (1506 connections now open) 2016-03-13T08:47:58.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39886 #4078212 (1506 connections now open) 2016-03-13T08:47:58.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39887 #4078213 (1507 connections now open) 2016-03-13T08:48:01.883+0000 I NETWORK [conn4078198] end connection 10.0.27.237:33136 (1506 connections now open) 2016-03-13T08:48:01.883+0000 I NETWORK [conn4078199] end connection 10.0.27.237:33135 (1506 connections now open) 2016-03-13T08:48:01.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33146 #4078214 (1506 connections now open) 2016-03-13T08:48:01.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33147 #4078215 (1507 connections now open) 2016-03-13T08:48:03.056+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:48:03.959+0000 I NETWORK [conn4078200] end connection 10.0.7.113:56810 (1506 connections now open) 2016-03-13T08:48:03.959+0000 I NETWORK [conn4078201] end connection 10.0.7.113:56811 (1506 connections now open) 2016-03-13T08:48:03.960+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56834 #4078216 (1506 connections now open) 2016-03-13T08:48:03.960+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56835 #4078217 (1507 connections now open) 2016-03-13T08:48:05.144+0000 I NETWORK [conn4078203] end connection 10.0.216.90:54852 (1506 connections now open) 2016-03-13T08:48:05.144+0000 I NETWORK [conn4078202] end connection 10.0.216.90:54853 (1506 connections now open) 2016-03-13T08:48:05.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54874 #4078218 (1506 connections now open) 2016-03-13T08:48:05.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54875 #4078219 (1507 connections now open) 2016-03-13T08:48:06.000+0000 I NETWORK [conn4078205] end connection 10.0.20.53:48046 (1506 connections now open) 2016-03-13T08:48:06.000+0000 I NETWORK [conn4078204] end connection 10.0.20.53:48047 (1506 connections now open) 2016-03-13T08:48:06.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48057 #4078220 (1506 connections now open) 2016-03-13T08:48:06.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48056 #4078221 (1507 connections now open) 2016-03-13T08:48:08.384+0000 I NETWORK [conn4078210] end connection 10.0.179.209:37895 (1506 connections now open) 2016-03-13T08:48:08.384+0000 I NETWORK [conn4078211] end connection 10.0.179.209:37896 (1506 connections now open) 2016-03-13T08:48:08.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37915 #4078222 (1506 connections now open) 2016-03-13T08:48:08.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37914 #4078223 (1507 connections now open) 2016-03-13T08:48:08.994+0000 I NETWORK [conn4078213] end connection 10.0.26.69:39887 (1506 connections now open) 2016-03-13T08:48:08.994+0000 I NETWORK [conn4078212] end connection 10.0.26.69:39886 (1506 connections now open) 2016-03-13T08:48:08.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39899 #4078224 (1506 connections now open) 2016-03-13T08:48:08.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39898 #4078225 (1507 connections now open) 2016-03-13T08:48:10.502+0000 I NETWORK [conn4075309] end connection 10.0.91.115:52112 (1506 connections now open) 2016-03-13T08:48:10.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:53701 #4078226 (1507 connections now open) 2016-03-13T08:48:11.897+0000 I NETWORK [conn4078215] end connection 10.0.27.237:33147 (1506 connections now open) 2016-03-13T08:48:11.897+0000 I NETWORK [conn4078214] end connection 10.0.27.237:33146 (1506 connections now open) 2016-03-13T08:48:11.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33159 #4078227 (1506 connections now open) 2016-03-13T08:48:11.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33158 #4078228 (1507 connections now open) 2016-03-13T08:48:13.067+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:48:13.973+0000 I NETWORK [conn4078216] end connection 10.0.7.113:56834 (1506 connections now open) 2016-03-13T08:48:13.973+0000 I NETWORK [conn4078217] end connection 10.0.7.113:56835 (1506 connections now open) 2016-03-13T08:48:13.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56845 #4078229 (1506 connections now open) 2016-03-13T08:48:13.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56844 #4078230 (1507 connections now open) 2016-03-13T08:48:15.166+0000 I NETWORK [conn4078218] end connection 10.0.216.90:54874 (1506 connections now open) 2016-03-13T08:48:15.166+0000 I NETWORK [conn4078219] end connection 10.0.216.90:54875 (1506 connections now open) 2016-03-13T08:48:15.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54884 #4078231 (1506 connections now open) 2016-03-13T08:48:15.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54883 #4078232 (1507 connections now open) 2016-03-13T08:48:16.013+0000 I NETWORK [conn4078220] end connection 10.0.20.53:48057 (1506 connections now open) 2016-03-13T08:48:16.013+0000 I NETWORK [conn4078221] end connection 10.0.20.53:48056 (1505 connections now open) 2016-03-13T08:48:16.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48067 #4078233 (1506 connections now open) 2016-03-13T08:48:16.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48068 #4078234 (1507 connections now open) 2016-03-13T08:48:16.117+0000 I NETWORK [conn4071395] end connection 10.0.20.53:40940 (1506 connections now open) 2016-03-13T08:48:16.117+0000 I NETWORK [conn4067984] end connection 10.0.216.90:43893 (1505 connections now open) 2016-03-13T08:48:16.117+0000 I NETWORK [conn4067174] end connection 10.0.26.69:56277 (1504 connections now open) 2016-03-13T08:48:16.118+0000 I NETWORK [conn4068281] end connection 10.0.7.113:46203 (1503 connections now open) 2016-03-13T08:48:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48071 #4078235 (1504 connections now open) 2016-03-13T08:48:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54887 #4078236 (1505 connections now open) 2016-03-13T08:48:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56851 #4078237 (1506 connections now open) 2016-03-13T08:48:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39920 #4078238 (1507 connections now open) 2016-03-13T08:48:17.188+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:48:17.057+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:48:18.397+0000 I NETWORK [conn4078222] end connection 10.0.179.209:37915 (1506 connections now open) 2016-03-13T08:48:18.397+0000 I NETWORK [conn4078223] end connection 10.0.179.209:37914 (1506 connections now open) 2016-03-13T08:48:18.398+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37928 #4078239 (1506 connections now open) 2016-03-13T08:48:18.398+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37927 #4078240 (1507 connections now open) 2016-03-13T08:48:19.007+0000 I NETWORK [conn4078225] end connection 10.0.26.69:39898 (1506 connections now open) 2016-03-13T08:48:19.007+0000 I NETWORK [conn4078224] end connection 10.0.26.69:39899 (1505 connections now open) 2016-03-13T08:48:19.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39922 #4078241 (1507 connections now open) 2016-03-13T08:48:19.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39923 #4078242 (1507 connections now open) 2016-03-13T08:48:21.909+0000 I NETWORK [conn4078227] end connection 10.0.27.237:33159 (1506 connections now open) 2016-03-13T08:48:21.910+0000 I NETWORK [conn4078228] end connection 10.0.27.237:33158 (1505 connections now open) 2016-03-13T08:48:21.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33180 #4078243 (1506 connections now open) 2016-03-13T08:48:21.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33179 #4078244 (1507 connections now open) 2016-03-13T08:48:23.071+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:48:23.995+0000 I NETWORK [conn4078229] end connection 10.0.7.113:56845 (1506 connections now open) 2016-03-13T08:48:23.995+0000 I NETWORK [conn4078230] end connection 10.0.7.113:56844 (1506 connections now open) 2016-03-13T08:48:23.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56858 #4078245 (1506 connections now open) 2016-03-13T08:48:23.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56859 #4078246 (1507 connections now open) 2016-03-13T08:48:25.180+0000 I NETWORK [conn4078232] end connection 10.0.216.90:54883 (1506 connections now open) 2016-03-13T08:48:25.180+0000 I NETWORK [conn4078231] end connection 10.0.216.90:54884 (1506 connections now open) 2016-03-13T08:48:25.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54898 #4078247 (1506 connections now open) 2016-03-13T08:48:25.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54899 #4078248 (1507 connections now open) 2016-03-13T08:48:26.026+0000 I NETWORK [conn4078233] end connection 10.0.20.53:48067 (1506 connections now open) 2016-03-13T08:48:26.026+0000 I NETWORK [conn4078234] end connection 10.0.20.53:48068 (1506 connections now open) 2016-03-13T08:48:26.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48090 #4078249 (1506 connections now open) 2016-03-13T08:48:26.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48091 #4078250 (1507 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [conn4069214] end connection 10.0.20.53:38652 (1506 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [conn4072705] end connection 10.0.216.90:48952 (1505 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [conn4077103] end connection 10.0.27.237:60285 (1504 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [conn4069365] end connection 10.0.7.113:47374 (1503 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48094 #4078251 (1504 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54905 #4078252 (1505 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33186 #4078253 (1506 connections now open) 2016-03-13T08:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56865 #4078254 (1507 connections now open) 2016-03-13T08:48:28.410+0000 I NETWORK [conn4078240] end connection 10.0.179.209:37927 (1506 connections now open) 2016-03-13T08:48:28.410+0000 I NETWORK [conn4078239] end connection 10.0.179.209:37928 (1506 connections now open) 2016-03-13T08:48:28.411+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37940 #4078255 (1506 connections now open) 2016-03-13T08:48:28.411+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37941 #4078256 (1507 connections now open) 2016-03-13T08:48:29.019+0000 I NETWORK [conn4078241] end connection 10.0.26.69:39922 (1506 connections now open) 2016-03-13T08:48:29.019+0000 I NETWORK [conn4078242] end connection 10.0.26.69:39923 (1506 connections now open) 2016-03-13T08:48:29.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39931 #4078257 (1506 connections now open) 2016-03-13T08:48:29.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39932 #4078258 (1507 connections now open) 2016-03-13T08:48:31.931+0000 I NETWORK [conn4078243] end connection 10.0.27.237:33180 (1506 connections now open) 2016-03-13T08:48:31.931+0000 I NETWORK [conn4078244] end connection 10.0.27.237:33179 (1506 connections now open) 2016-03-13T08:48:31.932+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33189 #4078259 (1506 connections now open) 2016-03-13T08:48:31.932+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33188 #4078260 (1507 connections now open) 2016-03-13T08:48:33.074+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:48:33.703+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db1m1.mongo323.test.mydomain.com:27017 2016-03-13T08:48:33.713+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T08:48:33.755+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T08:48:34.008+0000 I NETWORK [conn4078246] end connection 10.0.7.113:56859 (1506 connections now open) 2016-03-13T08:48:34.008+0000 I NETWORK [conn4078245] end connection 10.0.7.113:56858 (1505 connections now open) 2016-03-13T08:48:34.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56878 #4078261 (1506 connections now open) 2016-03-13T08:48:34.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56879 #4078262 (1507 connections now open) 2016-03-13T08:48:34.039+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5296118fa188537623c8d 2016-03-13T08:48:35.194+0000 I NETWORK [conn4078248] end connection 10.0.216.90:54899 (1506 connections now open) 2016-03-13T08:48:35.194+0000 I NETWORK [conn4078247] end connection 10.0.216.90:54898 (1506 connections now open) 2016-03-13T08:48:35.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54922 #4078263 (1506 connections now open) 2016-03-13T08:48:35.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54921 #4078264 (1507 connections now open) 2016-03-13T08:48:36.040+0000 I NETWORK [conn4078250] end connection 10.0.20.53:48091 (1506 connections now open) 2016-03-13T08:48:36.040+0000 I NETWORK [conn4078249] end connection 10.0.20.53:48090 (1506 connections now open) 2016-03-13T08:48:36.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48104 #4078265 (1506 connections now open) 2016-03-13T08:48:36.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48105 #4078266 (1507 connections now open) 2016-03-13T08:48:36.704+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55c326d9d1c6750e2a397961')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c326d9d1c6750e2a397961') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c3c181d1c67514c5f99eaf') }, shard: "shmydomain3", lastmod: Timestamp 3000|0, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:48:36.704+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:48:36.868+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 3|0||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c326d9d1c6750e2a397961') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c3c181d1c67514c5f99eaf') }) shmydomain3 -> shmydomain2 2016-03-13T08:48:38.423+0000 I NETWORK [conn4078255] end connection 10.0.179.209:37940 (1506 connections now open) 2016-03-13T08:48:38.423+0000 I NETWORK [conn4078256] end connection 10.0.179.209:37941 (1506 connections now open) 2016-03-13T08:48:38.424+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37958 #4078267 (1506 connections now open) 2016-03-13T08:48:38.424+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37957 #4078268 (1507 connections now open) 2016-03-13T08:48:39.032+0000 I NETWORK [conn4078258] end connection 10.0.26.69:39932 (1506 connections now open) 2016-03-13T08:48:39.032+0000 I NETWORK [conn4078257] end connection 10.0.26.69:39931 (1506 connections now open) 2016-03-13T08:48:39.033+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39943 #4078269 (1506 connections now open) 2016-03-13T08:48:39.033+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39942 #4078270 (1507 connections now open) 2016-03-13T08:48:41.027+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 16713 version: 30518|1||563ba74d869758a4542b9075 based on: 30517|1||563ba74d869758a4542b9075 2016-03-13T08:48:41.217+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:48:41.217+0000-56e5296918fa188537623c8e", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858921217), what: "balancer.round", ns: "", details: { executionTimeMillis: 7672, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:48:41.534+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:48:41.945+0000 I NETWORK [conn4078260] end connection 10.0.27.237:33188 (1506 connections now open) 2016-03-13T08:48:41.945+0000 I NETWORK [conn4078259] end connection 10.0.27.237:33189 (1506 connections now open) 2016-03-13T08:48:41.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33201 #4078271 (1506 connections now open) 2016-03-13T08:48:41.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33202 #4078272 (1507 connections now open) 2016-03-13T08:48:42.879+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5296a18fa188537623c8f 2016-03-13T08:48:43.078+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:48:44.021+0000 I NETWORK [conn4078261] end connection 10.0.7.113:56878 (1506 connections now open) 2016-03-13T08:48:44.021+0000 I NETWORK [conn4078262] end connection 10.0.7.113:56879 (1506 connections now open) 2016-03-13T08:48:44.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56887 #4078273 (1506 connections now open) 2016-03-13T08:48:44.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56886 #4078274 (1507 connections now open) 2016-03-13T08:48:45.207+0000 I NETWORK [conn4078264] end connection 10.0.216.90:54921 (1506 connections now open) 2016-03-13T08:48:45.207+0000 I NETWORK [conn4078263] end connection 10.0.216.90:54922 (1506 connections now open) 2016-03-13T08:48:45.208+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54930 #4078275 (1506 connections now open) 2016-03-13T08:48:45.208+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54931 #4078276 (1507 connections now open) 2016-03-13T08:48:45.365+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55c4d4e2599fcd1d2cdf5f67')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c4d4e2599fcd1d2cdf5f67') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c5a25446f6040e4660bf7c') }, shard: "shmydomain3", lastmod: Timestamp 30518000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:48:45.365+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:48:45.518+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30518|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c4d4e2599fcd1d2cdf5f67') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c5a25446f6040e4660bf7c') }) shmydomain3 -> shmydomain2 2016-03-13T08:48:46.053+0000 I NETWORK [conn4078266] end connection 10.0.20.53:48105 (1506 connections now open) 2016-03-13T08:48:46.053+0000 I NETWORK [conn4078265] end connection 10.0.20.53:48104 (1506 connections now open) 2016-03-13T08:48:46.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48115 #4078277 (1506 connections now open) 2016-03-13T08:48:46.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48116 #4078278 (1507 connections now open) 2016-03-13T08:48:46.117+0000 I NETWORK [conn4067193] end connection 10.0.216.90:43056 (1506 connections now open) 2016-03-13T08:48:46.117+0000 I NETWORK [conn4070545] end connection 10.0.27.237:53550 (1505 connections now open) 2016-03-13T08:48:46.118+0000 I NETWORK [conn4066374] end connection 10.0.26.69:55408 (1504 connections now open) 2016-03-13T08:48:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54934 #4078279 (1505 connections now open) 2016-03-13T08:48:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33217 #4078280 (1506 connections now open) 2016-03-13T08:48:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39951 #4078281 (1507 connections now open) 2016-03-13T08:48:47.096+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 695ms sequenceNumber: 16714 version: 30519|1||563ba74d869758a4542b9075 based on: 30518|1||563ba74d869758a4542b9075 2016-03-13T08:48:47.377+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:48:47.377+0000-56e5296f18fa188537623c90", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858927377), what: "balancer.round", ns: "", details: { executionTimeMillis: 4797, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:48:47.491+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:48:47.188+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:48:47.524+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:48:48.436+0000 I NETWORK [conn4078268] end connection 10.0.179.209:37957 (1506 connections now open) 2016-03-13T08:48:48.436+0000 I NETWORK [conn4078267] end connection 10.0.179.209:37958 (1506 connections now open) 2016-03-13T08:48:48.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37968 #4078282 (1506 connections now open) 2016-03-13T08:48:48.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37967 #4078283 (1507 connections now open) 2016-03-13T08:48:48.926+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5297018fa188537623c91 2016-03-13T08:48:49.046+0000 I NETWORK [conn4078270] end connection 10.0.26.69:39942 (1506 connections now open) 2016-03-13T08:48:49.046+0000 I NETWORK [conn4078269] end connection 10.0.26.69:39943 (1506 connections now open) 2016-03-13T08:48:49.046+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39964 #4078284 (1506 connections now open) 2016-03-13T08:48:49.047+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39963 #4078285 (1507 connections now open) 2016-03-13T08:48:51.368+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55c605c346f6040e466a2afe')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c605c346f6040e466a2afe') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c636946a6cd60e38c0098d') }, shard: "shmydomain3", lastmod: Timestamp 30519000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:48:51.368+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:48:51.522+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30519|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c605c346f6040e466a2afe') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c636946a6cd60e38c0098d') }) shmydomain3 -> shmydomain2 2016-03-13T08:48:51.958+0000 I NETWORK [conn4078271] end connection 10.0.27.237:33201 (1506 connections now open) 2016-03-13T08:48:51.958+0000 I NETWORK [conn4078272] end connection 10.0.27.237:33202 (1506 connections now open) 2016-03-13T08:48:51.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33221 #4078286 (1506 connections now open) 2016-03-13T08:48:51.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33220 #4078287 (1507 connections now open) 2016-03-13T08:48:52.878+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 704ms sequenceNumber: 16715 version: 30520|1||563ba74d869758a4542b9075 based on: 30519|1||563ba74d869758a4542b9075 2016-03-13T08:48:53.071+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:48:53.071+0000-56e5297518fa188537623c92", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858933071), what: "balancer.round", ns: "", details: { executionTimeMillis: 4500, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:48:53.081+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:48:53.227+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:48:54.034+0000 I NETWORK [conn4078273] end connection 10.0.7.113:56887 (1506 connections now open) 2016-03-13T08:48:54.035+0000 I NETWORK [conn4078274] end connection 10.0.7.113:56886 (1505 connections now open) 2016-03-13T08:48:54.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56901 #4078288 (1506 connections now open) 2016-03-13T08:48:54.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56902 #4078289 (1507 connections now open) 2016-03-13T08:48:54.591+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5297618fa188537623c93 2016-03-13T08:48:55.220+0000 I NETWORK [conn4078275] end connection 10.0.216.90:54930 (1506 connections now open) 2016-03-13T08:48:55.221+0000 I NETWORK [conn4078276] end connection 10.0.216.90:54931 (1505 connections now open) 2016-03-13T08:48:55.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54946 #4078290 (1506 connections now open) 2016-03-13T08:48:55.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54945 #4078291 (1507 connections now open) 2016-03-13T08:48:56.067+0000 I NETWORK [conn4078277] end connection 10.0.20.53:48115 (1506 connections now open) 2016-03-13T08:48:56.067+0000 I NETWORK [conn4078278] end connection 10.0.20.53:48116 (1505 connections now open) 2016-03-13T08:48:56.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48137 #4078292 (1506 connections now open) 2016-03-13T08:48:56.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48136 #4078293 (1507 connections now open) 2016-03-13T08:48:57.073+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55c663806a6cd60e38c3f26e')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c663806a6cd60e38c3f26e') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c7044fa023a90e3a535714') }, shard: "shmydomain3", lastmod: Timestamp 30520000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:48:57.073+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:48:57.228+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30520|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c663806a6cd60e38c3f26e') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c7044fa023a90e3a535714') }) shmydomain3 -> shmydomain2 2016-03-13T08:48:58.114+0000 I NETWORK [conn4069385] end connection 10.0.179.209:57229 (1506 connections now open) 2016-03-13T08:48:58.114+0000 I NETWORK [conn4065278] end connection 10.0.20.53:34627 (1506 connections now open) 2016-03-13T08:48:58.114+0000 I NETWORK [conn4074256] end connection 10.0.216.90:50612 (1504 connections now open) 2016-03-13T08:48:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48140 #4078294 (1505 connections now open) 2016-03-13T08:48:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54951 #4078295 (1506 connections now open) 2016-03-13T08:48:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37979 #4078296 (1507 connections now open) 2016-03-13T08:48:58.449+0000 I NETWORK [conn4078283] end connection 10.0.179.209:37967 (1506 connections now open) 2016-03-13T08:48:58.449+0000 I NETWORK [conn4078282] end connection 10.0.179.209:37968 (1506 connections now open) 2016-03-13T08:48:58.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37980 #4078297 (1506 connections now open) 2016-03-13T08:48:58.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37981 #4078298 (1507 connections now open) 2016-03-13T08:48:58.874+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 690ms sequenceNumber: 16716 version: 30521|1||563ba74d869758a4542b9075 based on: 30520|1||563ba74d869758a4542b9075 2016-03-13T08:48:59.059+0000 I NETWORK [conn4078284] end connection 10.0.26.69:39964 (1506 connections now open) 2016-03-13T08:48:59.059+0000 I NETWORK [conn4078285] end connection 10.0.26.69:39963 (1506 connections now open) 2016-03-13T08:48:59.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39974 #4078299 (1506 connections now open) 2016-03-13T08:48:59.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39975 #4078300 (1507 connections now open) 2016-03-13T08:48:59.065+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:48:59.065+0000-56e5297b18fa188537623c94", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858939065), what: "balancer.round", ns: "", details: { executionTimeMillis: 4803, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:48:59.241+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:49:00.579+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5297c18fa188537623c95 2016-03-13T08:49:00.605+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db1m1.mongo323.test.mydomain.com:27017 2016-03-13T08:49:01.971+0000 I NETWORK [conn4078287] end connection 10.0.27.237:33220 (1506 connections now open) 2016-03-13T08:49:01.971+0000 I NETWORK [conn4078286] end connection 10.0.27.237:33221 (1506 connections now open) 2016-03-13T08:49:01.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33230 #4078301 (1506 connections now open) 2016-03-13T08:49:01.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33231 #4078302 (1507 connections now open) 2016-03-13T08:49:03.084+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:49:03.302+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55c75ed06a6cd60e36ca8258')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c75ed06a6cd60e36ca8258') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c7878a584ad153a99819ea') }, shard: "shmydomain3", lastmod: Timestamp 30521000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:49:03.302+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:49:03.456+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30521|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c75ed06a6cd60e36ca8258') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55c7878a584ad153a99819ea') }) shmydomain3 -> shmydomain2 2016-03-13T08:49:04.048+0000 I NETWORK [conn4078289] end connection 10.0.7.113:56902 (1506 connections now open) 2016-03-13T08:49:04.048+0000 I NETWORK [conn4078288] end connection 10.0.7.113:56901 (1506 connections now open) 2016-03-13T08:49:04.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56923 #4078303 (1506 connections now open) 2016-03-13T08:49:04.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56922 #4078304 (1507 connections now open) 2016-03-13T08:49:04.950+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 683ms sequenceNumber: 16717 version: 30522|1||563ba74d869758a4542b9075 based on: 30521|1||563ba74d869758a4542b9075 2016-03-13T08:49:05.143+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:49:05.143+0000-56e5298118fa188537623c96", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457858945143), what: "balancer.round", ns: "", details: { executionTimeMillis: 4868, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:49:05.234+0000 I NETWORK [conn4078291] end connection 10.0.216.90:54945 (1506 connections now open) 2016-03-13T08:49:05.234+0000 I NETWORK [conn4078290] end connection 10.0.216.90:54946 (1506 connections now open) 2016-03-13T08:49:05.243+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54967 #4078305 (1506 connections now open) 2016-03-13T08:49:05.243+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54966 #4078306 (1507 connections now open) 2016-03-13T08:49:05.291+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:49:06.089+0000 I NETWORK [conn4078293] end connection 10.0.20.53:48136 (1506 connections now open) 2016-03-13T08:49:06.089+0000 I NETWORK [conn4078292] end connection 10.0.20.53:48137 (1506 connections now open) 2016-03-13T08:49:06.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48148 #4078307 (1507 connections now open) 2016-03-13T08:49:06.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48147 #4078308 (1507 connections now open) 2016-03-13T08:49:08.462+0000 I NETWORK [conn4078297] end connection 10.0.179.209:37980 (1506 connections now open) 2016-03-13T08:49:08.462+0000 I NETWORK [conn4078298] end connection 10.0.179.209:37981 (1506 connections now open) 2016-03-13T08:49:08.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38000 #4078309 (1506 connections now open) 2016-03-13T08:49:08.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:37999 #4078310 (1507 connections now open) 2016-03-13T08:49:09.072+0000 I NETWORK [conn4078299] end connection 10.0.26.69:39974 (1506 connections now open) 2016-03-13T08:49:09.072+0000 I NETWORK [conn4078300] end connection 10.0.26.69:39975 (1506 connections now open) 2016-03-13T08:49:09.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39987 #4078311 (1506 connections now open) 2016-03-13T08:49:09.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:39988 #4078312 (1507 connections now open) 2016-03-13T08:49:11.984+0000 I NETWORK [conn4078302] end connection 10.0.27.237:33231 (1506 connections now open) 2016-03-13T08:49:11.984+0000 I NETWORK [conn4078301] end connection 10.0.27.237:33230 (1506 connections now open) 2016-03-13T08:49:11.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33243 #4078313 (1506 connections now open) 2016-03-13T08:49:11.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33242 #4078314 (1507 connections now open) 2016-03-13T08:49:13.097+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:49:14.061+0000 I NETWORK [conn4078303] end connection 10.0.7.113:56923 (1506 connections now open) 2016-03-13T08:49:14.061+0000 I NETWORK [conn4078304] end connection 10.0.7.113:56922 (1506 connections now open) 2016-03-13T08:49:14.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56935 #4078315 (1506 connections now open) 2016-03-13T08:49:14.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56934 #4078316 (1507 connections now open) 2016-03-13T08:49:15.256+0000 I NETWORK [conn4078306] end connection 10.0.216.90:54966 (1506 connections now open) 2016-03-13T08:49:15.256+0000 I NETWORK [conn4078305] end connection 10.0.216.90:54967 (1506 connections now open) 2016-03-13T08:49:15.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54975 #4078317 (1506 connections now open) 2016-03-13T08:49:15.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54976 #4078318 (1507 connections now open) 2016-03-13T08:49:16.103+0000 I NETWORK [conn4078307] end connection 10.0.20.53:48148 (1506 connections now open) 2016-03-13T08:49:16.103+0000 I NETWORK [conn4078308] end connection 10.0.20.53:48147 (1506 connections now open) 2016-03-13T08:49:16.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48158 #4078319 (1506 connections now open) 2016-03-13T08:49:16.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48159 #4078320 (1507 connections now open) 2016-03-13T08:49:16.118+0000 I NETWORK [conn4069152] end connection 10.0.20.53:38596 (1506 connections now open) 2016-03-13T08:49:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48162 #4078321 (1507 connections now open) 2016-03-13T08:49:17.769+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:49:17.492+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:49:17.769+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:49:17.906+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5298218fa188537623c97 2016-03-13T08:49:18.042+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5298c18fa188537623c98 2016-03-13T08:49:18.475+0000 I NETWORK [conn4078310] end connection 10.0.179.209:37999 (1506 connections now open) 2016-03-13T08:49:18.475+0000 I NETWORK [conn4078309] end connection 10.0.179.209:38000 (1506 connections now open) 2016-03-13T08:49:18.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38013 #4078322 (1506 connections now open) 2016-03-13T08:49:18.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38012 #4078323 (1507 connections now open) 2016-03-13T08:49:19.086+0000 I NETWORK [conn4078312] end connection 10.0.26.69:39988 (1506 connections now open) 2016-03-13T08:49:19.086+0000 I NETWORK [conn4078311] end connection 10.0.26.69:39987 (1506 connections now open) 2016-03-13T08:49:19.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40011 #4078324 (1506 connections now open) 2016-03-13T08:49:19.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40012 #4078325 (1507 connections now open) 2016-03-13T08:49:21.997+0000 I NETWORK [conn4078314] end connection 10.0.27.237:33242 (1506 connections now open) 2016-03-13T08:49:21.997+0000 I NETWORK [conn4078313] end connection 10.0.27.237:33243 (1506 connections now open) 2016-03-13T08:49:22.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33265 #4078326 (1506 connections now open) 2016-03-13T08:49:22.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33266 #4078327 (1507 connections now open) 2016-03-13T08:49:23.100+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:49:24.083+0000 I NETWORK [conn4078316] end connection 10.0.7.113:56934 (1506 connections now open) 2016-03-13T08:49:24.083+0000 I NETWORK [conn4078315] end connection 10.0.7.113:56935 (1506 connections now open) 2016-03-13T08:49:24.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56948 #4078328 (1506 connections now open) 2016-03-13T08:49:24.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56949 #4078329 (1507 connections now open) 2016-03-13T08:49:25.270+0000 I NETWORK [conn4078318] end connection 10.0.216.90:54976 (1506 connections now open) 2016-03-13T08:49:25.270+0000 I NETWORK [conn4078317] end connection 10.0.216.90:54975 (1506 connections now open) 2016-03-13T08:49:25.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54990 #4078330 (1506 connections now open) 2016-03-13T08:49:25.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54989 #4078331 (1507 connections now open) 2016-03-13T08:49:26.117+0000 I NETWORK [conn4078320] end connection 10.0.20.53:48159 (1506 connections now open) 2016-03-13T08:49:26.117+0000 I NETWORK [conn4078319] end connection 10.0.20.53:48158 (1506 connections now open) 2016-03-13T08:49:26.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48179 #4078332 (1506 connections now open) 2016-03-13T08:49:26.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48180 #4078333 (1507 connections now open) 2016-03-13T08:49:28.113+0000 I NETWORK [conn4066781] end connection 10.0.216.90:42631 (1506 connections now open) 2016-03-13T08:49:28.113+0000 I NETWORK [conn4067901] end connection 10.0.26.69:57045 (1505 connections now open) 2016-03-13T08:49:28.114+0000 I NETWORK [conn4066290] end connection 10.0.7.113:44097 (1504 connections now open) 2016-03-13T08:49:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:54996 #4078334 (1505 connections now open) 2016-03-13T08:49:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40020 #4078335 (1506 connections now open) 2016-03-13T08:49:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56956 #4078336 (1507 connections now open) 2016-03-13T08:49:28.487+0000 I NETWORK [conn4078322] end connection 10.0.179.209:38013 (1506 connections now open) 2016-03-13T08:49:28.487+0000 I NETWORK [conn4078323] end connection 10.0.179.209:38012 (1506 connections now open) 2016-03-13T08:49:28.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38026 #4078337 (1506 connections now open) 2016-03-13T08:49:28.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38025 #4078338 (1507 connections now open) 2016-03-13T08:49:29.106+0000 I NETWORK [conn4078324] end connection 10.0.26.69:40011 (1506 connections now open) 2016-03-13T08:49:29.107+0000 I NETWORK [conn4078325] end connection 10.0.26.69:40012 (1505 connections now open) 2016-03-13T08:49:29.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40021 #4078339 (1507 connections now open) 2016-03-13T08:49:29.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40022 #4078340 (1507 connections now open) 2016-03-13T08:49:32.020+0000 I NETWORK [conn4078326] end connection 10.0.27.237:33265 (1506 connections now open) 2016-03-13T08:49:32.020+0000 I NETWORK [conn4078327] end connection 10.0.27.237:33266 (1506 connections now open) 2016-03-13T08:49:32.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33274 #4078341 (1506 connections now open) 2016-03-13T08:49:32.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33273 #4078342 (1507 connections now open) 2016-03-13T08:49:33.103+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:49:34.097+0000 I NETWORK [conn4078328] end connection 10.0.7.113:56948 (1506 connections now open) 2016-03-13T08:49:34.097+0000 I NETWORK [conn4078329] end connection 10.0.7.113:56949 (1506 connections now open) 2016-03-13T08:49:34.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56969 #4078343 (1506 connections now open) 2016-03-13T08:49:34.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56970 #4078344 (1507 connections now open) 2016-03-13T08:49:35.283+0000 I NETWORK [conn4078331] end connection 10.0.216.90:54989 (1506 connections now open) 2016-03-13T08:49:35.283+0000 I NETWORK [conn4078330] end connection 10.0.216.90:54990 (1505 connections now open) 2016-03-13T08:49:35.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55012 #4078345 (1506 connections now open) 2016-03-13T08:49:35.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55013 #4078346 (1507 connections now open) 2016-03-13T08:49:36.130+0000 I NETWORK [conn4078332] end connection 10.0.20.53:48179 (1506 connections now open) 2016-03-13T08:49:36.130+0000 I NETWORK [conn4078333] end connection 10.0.20.53:48180 (1506 connections now open) 2016-03-13T08:49:36.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48192 #4078347 (1506 connections now open) 2016-03-13T08:49:36.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48193 #4078348 (1507 connections now open) 2016-03-13T08:49:38.500+0000 I NETWORK [conn4078337] end connection 10.0.179.209:38026 (1506 connections now open) 2016-03-13T08:49:38.500+0000 I NETWORK [conn4078338] end connection 10.0.179.209:38025 (1506 connections now open) 2016-03-13T08:49:38.501+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38043 #4078349 (1506 connections now open) 2016-03-13T08:49:38.501+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38044 #4078350 (1507 connections now open) 2016-03-13T08:49:39.119+0000 I NETWORK [conn4078339] end connection 10.0.26.69:40021 (1506 connections now open) 2016-03-13T08:49:39.119+0000 I NETWORK [conn4078340] end connection 10.0.26.69:40022 (1506 connections now open) 2016-03-13T08:49:39.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40031 #4078351 (1506 connections now open) 2016-03-13T08:49:39.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40032 #4078352 (1507 connections now open) 2016-03-13T08:49:42.033+0000 I NETWORK [conn4078342] end connection 10.0.27.237:33273 (1506 connections now open) 2016-03-13T08:49:42.033+0000 I NETWORK [conn4078341] end connection 10.0.27.237:33274 (1506 connections now open) 2016-03-13T08:49:42.034+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33285 #4078353 (1506 connections now open) 2016-03-13T08:49:42.034+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33286 #4078354 (1507 connections now open) 2016-03-13T08:49:43.107+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:49:44.110+0000 I NETWORK [conn4078343] end connection 10.0.7.113:56969 (1506 connections now open) 2016-03-13T08:49:44.110+0000 I NETWORK [conn4078344] end connection 10.0.7.113:56970 (1506 connections now open) 2016-03-13T08:49:44.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56978 #4078355 (1506 connections now open) 2016-03-13T08:49:44.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56979 #4078356 (1507 connections now open) 2016-03-13T08:49:45.296+0000 I NETWORK [conn4078345] end connection 10.0.216.90:55012 (1506 connections now open) 2016-03-13T08:49:45.296+0000 I NETWORK [conn4078346] end connection 10.0.216.90:55013 (1505 connections now open) 2016-03-13T08:49:45.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55021 #4078357 (1506 connections now open) 2016-03-13T08:49:45.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55022 #4078358 (1507 connections now open) 2016-03-13T08:49:46.118+0000 I NETWORK [conn4069412] end connection 10.0.27.237:52327 (1506 connections now open) 2016-03-13T08:49:46.118+0000 I NETWORK [conn4066591] end connection 10.0.216.90:42427 (1506 connections now open) 2016-03-13T08:49:46.118+0000 I NETWORK [conn4064621] end connection 10.0.7.113:42291 (1504 connections now open) 2016-03-13T08:49:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55024 #4078359 (1505 connections now open) 2016-03-13T08:49:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33301 #4078360 (1506 connections now open) 2016-03-13T08:49:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56986 #4078361 (1507 connections now open) 2016-03-13T08:49:46.143+0000 I NETWORK [conn4078348] end connection 10.0.20.53:48193 (1506 connections now open) 2016-03-13T08:49:46.143+0000 I NETWORK [conn4078347] end connection 10.0.20.53:48192 (1505 connections now open) 2016-03-13T08:49:46.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48204 #4078362 (1506 connections now open) 2016-03-13T08:49:46.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48205 #4078363 (1507 connections now open) 2016-03-13T08:49:48.170+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:49:48.042+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:49:48.170+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:49:48.238+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5299618fa188537623c99 2016-03-13T08:49:48.306+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e529a018fa188537623c9a 2016-03-13T08:49:48.375+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e529aa18fa188537623c9b 2016-03-13T08:49:48.513+0000 I NETWORK [conn4078349] end connection 10.0.179.209:38043 (1506 connections now open) 2016-03-13T08:49:48.513+0000 I NETWORK [conn4078350] end connection 10.0.179.209:38044 (1506 connections now open) 2016-03-13T08:49:48.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38053 #4078364 (1506 connections now open) 2016-03-13T08:49:48.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38054 #4078365 (1507 connections now open) 2016-03-13T08:49:49.132+0000 I NETWORK [conn4078352] end connection 10.0.26.69:40032 (1506 connections now open) 2016-03-13T08:49:49.132+0000 I NETWORK [conn4078351] end connection 10.0.26.69:40031 (1506 connections now open) 2016-03-13T08:49:49.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40051 #4078366 (1506 connections now open) 2016-03-13T08:49:49.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40050 #4078367 (1507 connections now open) 2016-03-13T08:49:52.046+0000 I NETWORK [conn4078354] end connection 10.0.27.237:33286 (1506 connections now open) 2016-03-13T08:49:52.046+0000 I NETWORK [conn4078353] end connection 10.0.27.237:33285 (1506 connections now open) 2016-03-13T08:49:52.047+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33303 #4078368 (1506 connections now open) 2016-03-13T08:49:52.047+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33304 #4078369 (1507 connections now open) 2016-03-13T08:49:53.110+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:49:54.124+0000 I NETWORK [conn4078355] end connection 10.0.7.113:56978 (1506 connections now open) 2016-03-13T08:49:54.124+0000 I NETWORK [conn4078356] end connection 10.0.7.113:56979 (1506 connections now open) 2016-03-13T08:49:54.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56993 #4078370 (1506 connections now open) 2016-03-13T08:49:54.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:56994 #4078371 (1507 connections now open) 2016-03-13T08:49:55.309+0000 I NETWORK [conn4078357] end connection 10.0.216.90:55021 (1506 connections now open) 2016-03-13T08:49:55.309+0000 I NETWORK [conn4078358] end connection 10.0.216.90:55022 (1505 connections now open) 2016-03-13T08:49:55.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55035 #4078372 (1506 connections now open) 2016-03-13T08:49:55.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55036 #4078373 (1507 connections now open) 2016-03-13T08:49:56.156+0000 I NETWORK [conn4078362] end connection 10.0.20.53:48204 (1506 connections now open) 2016-03-13T08:49:56.156+0000 I NETWORK [conn4078363] end connection 10.0.20.53:48205 (1506 connections now open) 2016-03-13T08:49:56.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48223 #4078374 (1506 connections now open) 2016-03-13T08:49:56.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48224 #4078375 (1507 connections now open) 2016-03-13T08:49:58.113+0000 I NETWORK [conn4071581] end connection 10.0.179.209:59432 (1506 connections now open) 2016-03-13T08:49:58.113+0000 I NETWORK [conn4069599] end connection 10.0.20.53:39058 (1505 connections now open) 2016-03-13T08:49:58.113+0000 I NETWORK [conn4066718] end connection 10.0.27.237:49575 (1504 connections now open) 2016-03-13T08:49:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48226 #4078376 (1505 connections now open) 2016-03-13T08:49:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33311 #4078377 (1506 connections now open) 2016-03-13T08:49:58.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38064 #4078378 (1507 connections now open) 2016-03-13T08:49:58.527+0000 I NETWORK [conn4078365] end connection 10.0.179.209:38054 (1506 connections now open) 2016-03-13T08:49:58.527+0000 I NETWORK [conn4078364] end connection 10.0.179.209:38053 (1506 connections now open) 2016-03-13T08:49:58.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38065 #4078379 (1506 connections now open) 2016-03-13T08:49:58.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38066 #4078380 (1507 connections now open) 2016-03-13T08:49:59.145+0000 I NETWORK [conn4078367] end connection 10.0.26.69:40050 (1506 connections now open) 2016-03-13T08:49:59.145+0000 I NETWORK [conn4078366] end connection 10.0.26.69:40051 (1506 connections now open) 2016-03-13T08:49:59.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40061 #4078381 (1506 connections now open) 2016-03-13T08:49:59.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40062 #4078382 (1507 connections now open) 2016-03-13T08:50:02.059+0000 I NETWORK [conn4078368] end connection 10.0.27.237:33303 (1506 connections now open) 2016-03-13T08:50:02.059+0000 I NETWORK [conn4078369] end connection 10.0.27.237:33304 (1506 connections now open) 2016-03-13T08:50:02.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33316 #4078383 (1506 connections now open) 2016-03-13T08:50:02.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33317 #4078384 (1507 connections now open) 2016-03-13T08:50:03.114+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:50:04.137+0000 I NETWORK [conn4078370] end connection 10.0.7.113:56993 (1506 connections now open) 2016-03-13T08:50:04.137+0000 I NETWORK [conn4078371] end connection 10.0.7.113:56994 (1506 connections now open) 2016-03-13T08:50:04.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57015 #4078385 (1506 connections now open) 2016-03-13T08:50:04.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57016 #4078386 (1507 connections now open) 2016-03-13T08:50:05.322+0000 I NETWORK [conn4078373] end connection 10.0.216.90:55036 (1506 connections now open) 2016-03-13T08:50:05.322+0000 I NETWORK [conn4078372] end connection 10.0.216.90:55035 (1506 connections now open) 2016-03-13T08:50:05.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55059 #4078387 (1506 connections now open) 2016-03-13T08:50:05.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55058 #4078388 (1507 connections now open) 2016-03-13T08:50:06.179+0000 I NETWORK [conn4078374] end connection 10.0.20.53:48223 (1506 connections now open) 2016-03-13T08:50:06.179+0000 I NETWORK [conn4078375] end connection 10.0.20.53:48224 (1506 connections now open) 2016-03-13T08:50:06.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48236 #4078389 (1506 connections now open) 2016-03-13T08:50:06.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48235 #4078390 (1507 connections now open) 2016-03-13T08:50:07.088+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e529be18fa188537623c9d 2016-03-13T08:50:08.540+0000 I NETWORK [conn4078380] end connection 10.0.179.209:38066 (1506 connections now open) 2016-03-13T08:50:08.540+0000 I NETWORK [conn4078379] end connection 10.0.179.209:38065 (1506 connections now open) 2016-03-13T08:50:08.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38085 #4078391 (1506 connections now open) 2016-03-13T08:50:08.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38086 #4078392 (1507 connections now open) 2016-03-13T08:50:09.158+0000 I NETWORK [conn4078382] end connection 10.0.26.69:40062 (1506 connections now open) 2016-03-13T08:50:09.158+0000 I NETWORK [conn4078381] end connection 10.0.26.69:40061 (1506 connections now open) 2016-03-13T08:50:09.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40075 #4078393 (1506 connections now open) 2016-03-13T08:50:09.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40076 #4078394 (1507 connections now open) 2016-03-13T08:50:10.495+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 707ms sequenceNumber: 16718 version: 30531|1||563ba74d869758a4542b9075 based on: 30522|1||563ba74d869758a4542b9075 2016-03-13T08:50:10.814+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55d0147a4a50b90e3ae67c37')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d0147a4a50b90e3ae67c37') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d0a44891f6e30e39a23195') }, shard: "shmydomain3", lastmod: Timestamp 30531000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:50:10.815+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:50:10.968+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30531|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d0147a4a50b90e3ae67c37') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d0a44891f6e30e39a23195') }) shmydomain3 -> shmydomain2 2016-03-13T08:50:12.073+0000 I NETWORK [conn4078384] end connection 10.0.27.237:33317 (1506 connections now open) 2016-03-13T08:50:12.073+0000 I NETWORK [conn4078383] end connection 10.0.27.237:33316 (1505 connections now open) 2016-03-13T08:50:12.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33328 #4078395 (1506 connections now open) 2016-03-13T08:50:12.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33329 #4078396 (1507 connections now open) 2016-03-13T08:50:12.527+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 699ms sequenceNumber: 16719 version: 30532|1||563ba74d869758a4542b9075 based on: 30531|1||563ba74d869758a4542b9075 2016-03-13T08:50:12.720+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:50:12.720+0000-56e529c418fa188537623c9e", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859012720), what: "balancer.round", ns: "", details: { executionTimeMillis: 5896, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:50:12.896+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:50:13.125+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:50:14.150+0000 I NETWORK [conn4078386] end connection 10.0.7.113:57016 (1506 connections now open) 2016-03-13T08:50:14.150+0000 I NETWORK [conn4078385] end connection 10.0.7.113:57015 (1506 connections now open) 2016-03-13T08:50:14.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57027 #4078397 (1506 connections now open) 2016-03-13T08:50:14.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57028 #4078398 (1507 connections now open) 2016-03-13T08:50:14.227+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e529c518fa188537623c9f 2016-03-13T08:50:15.335+0000 I NETWORK [conn4078388] end connection 10.0.216.90:55058 (1506 connections now open) 2016-03-13T08:50:15.335+0000 I NETWORK [conn4078387] end connection 10.0.216.90:55059 (1505 connections now open) 2016-03-13T08:50:15.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55067 #4078399 (1506 connections now open) 2016-03-13T08:50:15.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55068 #4078400 (1507 connections now open) 2016-03-13T08:50:16.117+0000 I NETWORK [conn4068304] end connection 10.0.179.209:56150 (1506 connections now open) 2016-03-13T08:50:16.117+0000 I NETWORK [conn4075098] end connection 10.0.20.53:44822 (1505 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [conn4070587] end connection 10.0.26.69:59942 (1505 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [conn4066845] end connection 10.0.216.90:42700 (1504 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38094 #4078401 (1504 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [conn4071205] end connection 10.0.7.113:49348 (1503 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55071 #4078402 (1504 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40087 #4078403 (1505 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48248 #4078404 (1506 connections now open) 2016-03-13T08:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57035 #4078405 (1507 connections now open) 2016-03-13T08:50:16.192+0000 I NETWORK [conn4078390] end connection 10.0.20.53:48235 (1506 connections now open) 2016-03-13T08:50:16.192+0000 I NETWORK [conn4078389] end connection 10.0.20.53:48236 (1506 connections now open) 2016-03-13T08:50:16.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48250 #4078406 (1506 connections now open) 2016-03-13T08:50:16.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48249 #4078407 (1507 connections now open) 2016-03-13T08:50:17.539+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55d0da97c334246d0a7c5ee9')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d0da97c334246d0a7c5ee9') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d19cb3c334246d0ccaf322') }, shard: "shmydomain3", lastmod: Timestamp 30532000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:50:17.540+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:50:17.696+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30532|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d0da97c334246d0a7c5ee9') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d19cb3c334246d0ccaf322') }) shmydomain3 -> shmydomain2 2016-03-13T08:50:18.553+0000 I NETWORK [conn4078391] end connection 10.0.179.209:38085 (1506 connections now open) 2016-03-13T08:50:18.553+0000 I NETWORK [conn4078392] end connection 10.0.179.209:38086 (1506 connections now open) 2016-03-13T08:50:18.554+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38099 #4078408 (1506 connections now open) 2016-03-13T08:50:18.554+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38100 #4078409 (1507 connections now open) 2016-03-13T08:50:18.627+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:50:18.375+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:50:18.627+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:50:18.764+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e529b418fa188537623c9c 2016-03-13T08:50:19.171+0000 I NETWORK [conn4078393] end connection 10.0.26.69:40075 (1506 connections now open) 2016-03-13T08:50:19.171+0000 I NETWORK [conn4078394] end connection 10.0.26.69:40076 (1506 connections now open) 2016-03-13T08:50:19.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40098 #4078410 (1506 connections now open) 2016-03-13T08:50:19.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40099 #4078411 (1507 connections now open) 2016-03-13T08:50:19.227+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 701ms sequenceNumber: 16720 version: 30533|1||563ba74d869758a4542b9075 based on: 30532|1||563ba74d869758a4542b9075 2016-03-13T08:50:19.424+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:50:19.424+0000-56e529cb18fa188537623ca0", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859019424), what: "balancer.round", ns: "", details: { executionTimeMillis: 5493, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:50:19.591+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:50:20.824+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e529cc18fa188537623ca1 2016-03-13T08:50:22.085+0000 I NETWORK [conn4078395] end connection 10.0.27.237:33328 (1506 connections now open) 2016-03-13T08:50:22.085+0000 I NETWORK [conn4078396] end connection 10.0.27.237:33329 (1505 connections now open) 2016-03-13T08:50:22.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33349 #4078412 (1506 connections now open) 2016-03-13T08:50:22.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33350 #4078413 (1507 connections now open) 2016-03-13T08:50:23.129+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:50:23.499+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55d231c65e7f4b0e3224b68b')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d231c65e7f4b0e3224b68b') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d24dda584ad10a1c820da6') }, shard: "shmydomain3", lastmod: Timestamp 30533000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:50:23.499+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:50:23.653+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30533|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d231c65e7f4b0e3224b68b') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d24dda584ad10a1c820da6') }) shmydomain3 -> shmydomain2 2016-03-13T08:50:24.164+0000 I NETWORK [conn4078397] end connection 10.0.7.113:57027 (1506 connections now open) 2016-03-13T08:50:24.164+0000 I NETWORK [conn4078398] end connection 10.0.7.113:57028 (1506 connections now open) 2016-03-13T08:50:24.174+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57042 #4078414 (1506 connections now open) 2016-03-13T08:50:24.174+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57043 #4078415 (1507 connections now open) 2016-03-13T08:50:25.357+0000 I NETWORK [conn4078399] end connection 10.0.216.90:55067 (1506 connections now open) 2016-03-13T08:50:25.357+0000 I NETWORK [conn4078400] end connection 10.0.216.90:55068 (1506 connections now open) 2016-03-13T08:50:25.357+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55082 #4078416 (1506 connections now open) 2016-03-13T08:50:25.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55083 #4078417 (1507 connections now open) 2016-03-13T08:50:26.205+0000 I NETWORK [conn4078406] end connection 10.0.20.53:48250 (1506 connections now open) 2016-03-13T08:50:26.205+0000 I NETWORK [conn4078407] end connection 10.0.20.53:48249 (1506 connections now open) 2016-03-13T08:50:26.206+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48269 #4078418 (1506 connections now open) 2016-03-13T08:50:26.206+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48268 #4078419 (1507 connections now open) 2016-03-13T08:50:28.113+0000 I NETWORK [conn4071669] end connection 10.0.179.209:59517 (1506 connections now open) 2016-03-13T08:50:28.113+0000 I NETWORK [conn4070223] end connection 10.0.27.237:53210 (1505 connections now open) 2016-03-13T08:50:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38111 #4078420 (1506 connections now open) 2016-03-13T08:50:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33356 #4078421 (1507 connections now open) 2016-03-13T08:50:28.567+0000 I NETWORK [conn4078409] end connection 10.0.179.209:38100 (1506 connections now open) 2016-03-13T08:50:28.567+0000 I NETWORK [conn4078408] end connection 10.0.179.209:38099 (1506 connections now open) 2016-03-13T08:50:28.567+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38114 #4078422 (1506 connections now open) 2016-03-13T08:50:28.567+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38113 #4078423 (1507 connections now open) 2016-03-13T08:50:28.907+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 702ms sequenceNumber: 16721 version: 30534|1||563ba74d869758a4542b9075 based on: 30533|1||563ba74d869758a4542b9075 2016-03-13T08:50:29.100+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:50:29.100+0000-56e529d518fa188537623ca2", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859029100), what: "balancer.round", ns: "", details: { executionTimeMillis: 8475, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:50:29.194+0000 I NETWORK [conn4078410] end connection 10.0.26.69:40098 (1506 connections now open) 2016-03-13T08:50:29.194+0000 I NETWORK [conn4078411] end connection 10.0.26.69:40099 (1506 connections now open) 2016-03-13T08:50:29.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40108 #4078424 (1507 connections now open) 2016-03-13T08:50:29.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40109 #4078425 (1507 connections now open) 2016-03-13T08:50:29.261+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:50:30.564+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e529d618fa188537623ca3 2016-03-13T08:50:32.106+0000 I NETWORK [conn4078412] end connection 10.0.27.237:33349 (1506 connections now open) 2016-03-13T08:50:32.106+0000 I NETWORK [conn4078413] end connection 10.0.27.237:33350 (1505 connections now open) 2016-03-13T08:50:32.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33358 #4078426 (1506 connections now open) 2016-03-13T08:50:32.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33359 #4078427 (1507 connections now open) 2016-03-13T08:50:33.053+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55d2e8d5584ad10a1c8fd617')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d2e8d5584ad10a1c8fd617') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d3876a9b00ba0e4405054f') }, shard: "shmydomain3", lastmod: Timestamp 30534000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:50:33.053+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:50:33.132+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:50:33.206+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30534|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d2e8d5584ad10a1c8fd617') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d3876a9b00ba0e4405054f') }) shmydomain3 -> shmydomain2 2016-03-13T08:50:34.187+0000 I NETWORK [conn4078415] end connection 10.0.7.113:57043 (1506 connections now open) 2016-03-13T08:50:34.187+0000 I NETWORK [conn4078414] end connection 10.0.7.113:57042 (1506 connections now open) 2016-03-13T08:50:34.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57063 #4078428 (1506 connections now open) 2016-03-13T08:50:34.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57064 #4078429 (1507 connections now open) 2016-03-13T08:50:34.621+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 707ms sequenceNumber: 16722 version: 30535|1||563ba74d869758a4542b9075 based on: 30534|1||563ba74d869758a4542b9075 2016-03-13T08:50:34.818+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:50:34.818+0000-56e529da18fa188537623ca4", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859034818), what: "balancer.round", ns: "", details: { executionTimeMillis: 4522, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:50:34.971+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:50:35.370+0000 I NETWORK [conn4078416] end connection 10.0.216.90:55082 (1506 connections now open) 2016-03-13T08:50:35.370+0000 I NETWORK [conn4078417] end connection 10.0.216.90:55083 (1506 connections now open) 2016-03-13T08:50:35.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55106 #4078430 (1506 connections now open) 2016-03-13T08:50:35.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55105 #4078431 (1507 connections now open) 2016-03-13T08:50:35.842+0000 I NETWORK [conn4076866] end connection 10.0.53.235:40900 (1506 connections now open) 2016-03-13T08:50:35.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.53.235:41750 #4078432 (1507 connections now open) 2016-03-13T08:50:36.219+0000 I NETWORK [conn4078418] end connection 10.0.20.53:48269 (1506 connections now open) 2016-03-13T08:50:36.219+0000 I NETWORK [conn4078419] end connection 10.0.20.53:48268 (1506 connections now open) 2016-03-13T08:50:36.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48282 #4078433 (1506 connections now open) 2016-03-13T08:50:36.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48283 #4078434 (1507 connections now open) 2016-03-13T08:50:36.240+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e529dc18fa188537623ca5 2016-03-13T08:50:38.580+0000 I NETWORK [conn4078422] end connection 10.0.179.209:38114 (1506 connections now open) 2016-03-13T08:50:38.580+0000 I NETWORK [conn4078423] end connection 10.0.179.209:38113 (1506 connections now open) 2016-03-13T08:50:38.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38131 #4078435 (1506 connections now open) 2016-03-13T08:50:38.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38132 #4078436 (1507 connections now open) 2016-03-13T08:50:39.207+0000 I NETWORK [conn4078425] end connection 10.0.26.69:40109 (1506 connections now open) 2016-03-13T08:50:39.207+0000 I NETWORK [conn4078424] end connection 10.0.26.69:40108 (1506 connections now open) 2016-03-13T08:50:39.208+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40120 #4078437 (1506 connections now open) 2016-03-13T08:50:39.208+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40119 #4078438 (1507 connections now open) 2016-03-13T08:50:39.498+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55d3b2c3b82cde0e646797c7')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d3b2c3b82cde0e646797c7') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d4a6fd14ae5b1dc98d2f9b') }, shard: "shmydomain3", lastmod: Timestamp 30535000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:50:39.498+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:50:39.653+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30535|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d3b2c3b82cde0e646797c7') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d4a6fd14ae5b1dc98d2f9b') }) shmydomain3 -> shmydomain2 2016-03-13T08:50:41.078+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 701ms sequenceNumber: 16723 version: 30536|1||563ba74d869758a4542b9075 based on: 30535|1||563ba74d869758a4542b9075 2016-03-13T08:50:41.272+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:50:41.272+0000-56e529e118fa188537623ca6", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859041272), what: "balancer.round", ns: "", details: { executionTimeMillis: 5265, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:50:41.433+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:50:41.523+0000 I NETWORK [conn4075310] end connection 10.0.91.115:52113 (1506 connections now open) 2016-03-13T08:50:41.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:53823 #4078439 (1507 connections now open) 2016-03-13T08:50:42.119+0000 I NETWORK [conn4078426] end connection 10.0.27.237:33358 (1506 connections now open) 2016-03-13T08:50:42.119+0000 I NETWORK [conn4078427] end connection 10.0.27.237:33359 (1506 connections now open) 2016-03-13T08:50:42.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33370 #4078440 (1506 connections now open) 2016-03-13T08:50:42.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33371 #4078441 (1507 connections now open) 2016-03-13T08:50:42.766+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e529e218fa188537623ca7 2016-03-13T08:50:43.136+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:50:44.200+0000 I NETWORK [conn4078428] end connection 10.0.7.113:57063 (1506 connections now open) 2016-03-13T08:50:44.200+0000 I NETWORK [conn4078429] end connection 10.0.7.113:57064 (1505 connections now open) 2016-03-13T08:50:44.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57071 #4078442 (1506 connections now open) 2016-03-13T08:50:44.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57072 #4078443 (1507 connections now open) 2016-03-13T08:50:45.241+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55d54c33584ad15f44fed13f')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d54c33584ad15f44fed13f') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d6238e197abf11dd15c908') }, shard: "shmydomain3", lastmod: Timestamp 30536000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:50:45.241+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:50:45.383+0000 I NETWORK [conn4078430] end connection 10.0.216.90:55106 (1506 connections now open) 2016-03-13T08:50:45.383+0000 I NETWORK [conn4078431] end connection 10.0.216.90:55105 (1506 connections now open) 2016-03-13T08:50:45.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55114 #4078444 (1506 connections now open) 2016-03-13T08:50:45.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55115 #4078445 (1507 connections now open) 2016-03-13T08:50:45.394+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30536|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d54c33584ad15f44fed13f') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55d6238e197abf11dd15c908') }) shmydomain3 -> shmydomain2 2016-03-13T08:50:46.117+0000 I NETWORK [conn4075475] end connection 10.0.20.53:45226 (1506 connections now open) 2016-03-13T08:50:46.117+0000 I NETWORK [conn4067260] end connection 10.0.179.209:55044 (1506 connections now open) 2016-03-13T08:50:46.117+0000 I NETWORK [conn4074817] end connection 10.0.216.90:51214 (1506 connections now open) 2016-03-13T08:50:46.118+0000 I NETWORK [conn4067086] end connection 10.0.27.237:49941 (1505 connections now open) 2016-03-13T08:50:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38138 #4078446 (1507 connections now open) 2016-03-13T08:50:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55118 #4078447 (1505 connections now open) 2016-03-13T08:50:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48293 #4078448 (1506 connections now open) 2016-03-13T08:50:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33387 #4078449 (1507 connections now open) 2016-03-13T08:50:46.233+0000 I NETWORK [conn4078434] end connection 10.0.20.53:48283 (1506 connections now open) 2016-03-13T08:50:46.233+0000 I NETWORK [conn4078433] end connection 10.0.20.53:48282 (1506 connections now open) 2016-03-13T08:50:46.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48295 #4078450 (1506 connections now open) 2016-03-13T08:50:46.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48294 #4078451 (1507 connections now open) 2016-03-13T08:50:46.862+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 16724 version: 30537|1||563ba74d869758a4542b9075 based on: 30536|1||563ba74d869758a4542b9075 2016-03-13T08:50:47.054+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:50:47.054+0000-56e529e718fa188537623ca8", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859047054), what: "balancer.round", ns: "", details: { executionTimeMillis: 4587, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:50:47.230+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:50:48.593+0000 I NETWORK [conn4078435] end connection 10.0.179.209:38131 (1506 connections now open) 2016-03-13T08:50:48.593+0000 I NETWORK [conn4078436] end connection 10.0.179.209:38132 (1506 connections now open) 2016-03-13T08:50:48.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38140 #4078452 (1506 connections now open) 2016-03-13T08:50:48.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38141 #4078453 (1507 connections now open) 2016-03-13T08:50:48.972+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:50:48.764+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:50:48.972+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:50:49.040+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e529e818fa188537623ca9 2016-03-13T08:50:49.220+0000 I NETWORK [conn4078438] end connection 10.0.26.69:40119 (1506 connections now open) 2016-03-13T08:50:49.220+0000 I NETWORK [conn4078437] end connection 10.0.26.69:40120 (1506 connections now open) 2016-03-13T08:50:49.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40139 #4078454 (1506 connections now open) 2016-03-13T08:50:49.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40140 #4078455 (1507 connections now open) 2016-03-13T08:50:52.132+0000 I NETWORK [conn4078440] end connection 10.0.27.237:33370 (1506 connections now open) 2016-03-13T08:50:52.132+0000 I NETWORK [conn4078441] end connection 10.0.27.237:33371 (1506 connections now open) 2016-03-13T08:50:52.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33390 #4078456 (1506 connections now open) 2016-03-13T08:50:52.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33389 #4078457 (1507 connections now open) 2016-03-13T08:50:53.139+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:50:54.214+0000 I NETWORK [conn4078443] end connection 10.0.7.113:57072 (1506 connections now open) 2016-03-13T08:50:54.214+0000 I NETWORK [conn4078442] end connection 10.0.7.113:57071 (1506 connections now open) 2016-03-13T08:50:54.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57084 #4078458 (1506 connections now open) 2016-03-13T08:50:54.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57085 #4078459 (1507 connections now open) 2016-03-13T08:50:55.396+0000 I NETWORK [conn4078444] end connection 10.0.216.90:55114 (1506 connections now open) 2016-03-13T08:50:55.396+0000 I NETWORK [conn4078445] end connection 10.0.216.90:55115 (1506 connections now open) 2016-03-13T08:50:55.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55129 #4078460 (1506 connections now open) 2016-03-13T08:50:55.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55130 #4078461 (1507 connections now open) 2016-03-13T08:50:56.246+0000 I NETWORK [conn4078451] end connection 10.0.20.53:48294 (1506 connections now open) 2016-03-13T08:50:56.246+0000 I NETWORK [conn4078450] end connection 10.0.20.53:48295 (1505 connections now open) 2016-03-13T08:50:56.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48314 #4078462 (1506 connections now open) 2016-03-13T08:50:56.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48313 #4078463 (1507 connections now open) 2016-03-13T08:50:58.113+0000 I NETWORK [conn4067649] end connection 10.0.179.209:55465 (1506 connections now open) 2016-03-13T08:50:58.113+0000 I NETWORK [conn4070249] end connection 10.0.7.113:48332 (1505 connections now open) 2016-03-13T08:50:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57092 #4078464 (1506 connections now open) 2016-03-13T08:50:58.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38151 #4078465 (1507 connections now open) 2016-03-13T08:50:58.607+0000 I NETWORK [conn4078453] end connection 10.0.179.209:38141 (1506 connections now open) 2016-03-13T08:50:58.607+0000 I NETWORK [conn4078452] end connection 10.0.179.209:38140 (1506 connections now open) 2016-03-13T08:50:58.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38154 #4078466 (1506 connections now open) 2016-03-13T08:50:58.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38153 #4078467 (1507 connections now open) 2016-03-13T08:50:59.233+0000 I NETWORK [conn4078454] end connection 10.0.26.69:40139 (1506 connections now open) 2016-03-13T08:50:59.233+0000 I NETWORK [conn4078455] end connection 10.0.26.69:40140 (1506 connections now open) 2016-03-13T08:50:59.234+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40149 #4078468 (1506 connections now open) 2016-03-13T08:50:59.234+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40150 #4078469 (1507 connections now open) 2016-03-13T08:51:02.145+0000 I NETWORK [conn4078457] end connection 10.0.27.237:33389 (1506 connections now open) 2016-03-13T08:51:02.145+0000 I NETWORK [conn4078456] end connection 10.0.27.237:33390 (1506 connections now open) 2016-03-13T08:51:02.145+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33403 #4078470 (1506 connections now open) 2016-03-13T08:51:02.145+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33402 #4078471 (1507 connections now open) 2016-03-13T08:51:03.143+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:04.227+0000 I NETWORK [conn4078459] end connection 10.0.7.113:57085 (1506 connections now open) 2016-03-13T08:51:04.227+0000 I NETWORK [conn4078458] end connection 10.0.7.113:57084 (1505 connections now open) 2016-03-13T08:51:04.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57108 #4078472 (1506 connections now open) 2016-03-13T08:51:04.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57107 #4078473 (1507 connections now open) 2016-03-13T08:51:05.409+0000 I NETWORK [conn4078461] end connection 10.0.216.90:55130 (1506 connections now open) 2016-03-13T08:51:05.409+0000 I NETWORK [conn4078460] end connection 10.0.216.90:55129 (1506 connections now open) 2016-03-13T08:51:05.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55150 #4078474 (1506 connections now open) 2016-03-13T08:51:05.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55151 #4078475 (1507 connections now open) 2016-03-13T08:51:06.269+0000 I NETWORK [conn4078462] end connection 10.0.20.53:48314 (1506 connections now open) 2016-03-13T08:51:06.269+0000 I NETWORK [conn4078463] end connection 10.0.20.53:48313 (1506 connections now open) 2016-03-13T08:51:06.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48325 #4078476 (1506 connections now open) 2016-03-13T08:51:06.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48326 #4078477 (1507 connections now open) 2016-03-13T08:51:08.620+0000 I NETWORK [conn4078467] end connection 10.0.179.209:38153 (1506 connections now open) 2016-03-13T08:51:08.620+0000 I NETWORK [conn4078466] end connection 10.0.179.209:38154 (1506 connections now open) 2016-03-13T08:51:08.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38173 #4078478 (1506 connections now open) 2016-03-13T08:51:08.621+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38174 #4078479 (1507 connections now open) 2016-03-13T08:51:09.247+0000 I NETWORK [conn4078469] end connection 10.0.26.69:40150 (1506 connections now open) 2016-03-13T08:51:09.247+0000 I NETWORK [conn4078468] end connection 10.0.26.69:40149 (1506 connections now open) 2016-03-13T08:51:09.247+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40164 #4078480 (1507 connections now open) 2016-03-13T08:51:09.247+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40163 #4078481 (1507 connections now open) 2016-03-13T08:51:12.158+0000 I NETWORK [conn4078471] end connection 10.0.27.237:33402 (1506 connections now open) 2016-03-13T08:51:12.158+0000 I NETWORK [conn4078470] end connection 10.0.27.237:33403 (1506 connections now open) 2016-03-13T08:51:12.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33413 #4078482 (1506 connections now open) 2016-03-13T08:51:12.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33414 #4078483 (1507 connections now open) 2016-03-13T08:51:13.145+0000 I NETWORK [ReplicaSetMonitorWatcher] Socket closed remotely, no longer connected (idle 10 secs, remote host 10.0.167.43:27017) 2016-03-13T08:51:14.240+0000 I NETWORK [conn4078473] end connection 10.0.7.113:57107 (1506 connections now open) 2016-03-13T08:51:14.240+0000 I NETWORK [conn4078472] end connection 10.0.7.113:57108 (1506 connections now open) 2016-03-13T08:51:14.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57118 #4078484 (1506 connections now open) 2016-03-13T08:51:14.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57119 #4078485 (1507 connections now open) 2016-03-13T08:51:14.648+0000 I NETWORK [ReplicaSetMonitorWatcher] Socket recv() errno:104 Connection reset by peer 10.0.167.43:27017 2016-03-13T08:51:14.648+0000 I NETWORK [ReplicaSetMonitorWatcher] SocketException: remote: (NONE):0 error: 9001 socket exception [RECV_ERROR] server [10.0.167.43:27017] 2016-03-13T08:51:14.657+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:14.657+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:51:14.657+0000 I NETWORK [ReplicaSetMonitorWatcher] All nodes for set rsmydomain3 are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks 2016-03-13T08:51:15.422+0000 I NETWORK [conn4078475] end connection 10.0.216.90:55151 (1506 connections now open) 2016-03-13T08:51:15.422+0000 I NETWORK [conn4078474] end connection 10.0.216.90:55150 (1506 connections now open) 2016-03-13T08:51:15.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55159 #4078486 (1506 connections now open) 2016-03-13T08:51:15.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55158 #4078487 (1507 connections now open) 2016-03-13T08:51:16.117+0000 I NETWORK [conn4065024] end connection 10.0.179.209:52663 (1506 connections now open) 2016-03-13T08:51:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38181 #4078488 (1507 connections now open) 2016-03-13T08:51:16.118+0000 I NETWORK [conn4070096] end connection 10.0.216.90:46146 (1505 connections now open) 2016-03-13T08:51:16.118+0000 I NETWORK [conn4072369] end connection 10.0.27.237:55493 (1505 connections now open) 2016-03-13T08:51:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55162 #4078489 (1506 connections now open) 2016-03-13T08:51:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33433 #4078490 (1507 connections now open) 2016-03-13T08:51:16.282+0000 I NETWORK [conn4078476] end connection 10.0.20.53:48325 (1506 connections now open) 2016-03-13T08:51:16.282+0000 I NETWORK [conn4078477] end connection 10.0.20.53:48326 (1506 connections now open) 2016-03-13T08:51:16.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48338 #4078491 (1506 connections now open) 2016-03-13T08:51:16.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48337 #4078492 (1507 connections now open) 2016-03-13T08:51:18.501+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:51:18.502+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:18.502+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:18.502+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 2 checks in a row. Polling will stop after 28 more failed checks 2016-03-13T08:51:18.633+0000 I NETWORK [conn4078478] end connection 10.0.179.209:38173 (1506 connections now open) 2016-03-13T08:51:18.633+0000 I NETWORK [conn4078479] end connection 10.0.179.209:38174 (1506 connections now open) 2016-03-13T08:51:18.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38186 #4078493 (1506 connections now open) 2016-03-13T08:51:18.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38187 #4078494 (1507 connections now open) 2016-03-13T08:51:19.003+0000 W NETWORK [Balancer] Failed to connect to 10.0.167.43:27017, reason: errno:111 Connection refused 2016-03-13T08:51:19.004+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:19.004+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:19.004+0000 I NETWORK [Balancer] All nodes for set rsmydomain3 are down. This has happened for 3 checks in a row. Polling will stop after 27 more failed checks 2016-03-13T08:51:19.260+0000 I NETWORK [conn4078481] end connection 10.0.26.69:40163 (1506 connections now open) 2016-03-13T08:51:19.260+0000 I NETWORK [conn4078480] end connection 10.0.26.69:40164 (1506 connections now open) 2016-03-13T08:51:19.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40185 #4078495 (1506 connections now open) 2016-03-13T08:51:19.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40186 #4078496 (1507 connections now open) 2016-03-13T08:51:19.317+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:51:19.040+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:51:19.317+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:51:19.453+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e529f218fa188537623caa 2016-03-13T08:51:19.506+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:19.506+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:19.590+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e529fc18fa188537623cab 2016-03-13T08:51:20.008+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:20.008+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:20.510+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:20.510+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:21.011+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:21.011+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:21.512+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:21.513+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:22.014+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:22.014+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:22.171+0000 I NETWORK [conn4078483] end connection 10.0.27.237:33414 (1506 connections now open) 2016-03-13T08:51:22.171+0000 I NETWORK [conn4078482] end connection 10.0.27.237:33413 (1506 connections now open) 2016-03-13T08:51:22.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33437 #4078497 (1506 connections now open) 2016-03-13T08:51:22.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33436 #4078498 (1507 connections now open) 2016-03-13T08:51:22.515+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:22.515+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:23.017+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:23.017+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:23.518+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:23.518+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:24.019+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:24.019+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:24.254+0000 I NETWORK [conn4078485] end connection 10.0.7.113:57119 (1506 connections now open) 2016-03-13T08:51:24.254+0000 I NETWORK [conn4078484] end connection 10.0.7.113:57118 (1506 connections now open) 2016-03-13T08:51:24.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57134 #4078499 (1506 connections now open) 2016-03-13T08:51:24.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57133 #4078500 (1507 connections now open) 2016-03-13T08:51:24.521+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:24.521+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:24.660+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:24.660+0000 W NETWORK [ReplicaSetMonitorWatcher] No primary detected for set rsmydomain3 2016-03-13T08:51:25.022+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:25.022+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:25.446+0000 I NETWORK [conn4078486] end connection 10.0.216.90:55159 (1506 connections now open) 2016-03-13T08:51:25.446+0000 I NETWORK [conn4078487] end connection 10.0.216.90:55158 (1506 connections now open) 2016-03-13T08:51:25.447+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55173 #4078501 (1506 connections now open) 2016-03-13T08:51:25.447+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55174 #4078502 (1507 connections now open) 2016-03-13T08:51:25.524+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:25.524+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:26.025+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:26.025+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:26.296+0000 I NETWORK [conn4078492] end connection 10.0.20.53:48337 (1506 connections now open) 2016-03-13T08:51:26.296+0000 I NETWORK [conn4078491] end connection 10.0.20.53:48338 (1506 connections now open) 2016-03-13T08:51:26.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48355 #4078503 (1506 connections now open) 2016-03-13T08:51:26.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48356 #4078504 (1507 connections now open) 2016-03-13T08:51:26.527+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:26.527+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:27.028+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:27.028+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:27.530+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:27.530+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:28.031+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:28.031+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:28.114+0000 I NETWORK [conn4068224] end connection 10.0.216.90:44147 (1506 connections now open) 2016-03-13T08:51:28.114+0000 I NETWORK [conn4066178] end connection 10.0.27.237:49029 (1505 connections now open) 2016-03-13T08:51:28.114+0000 I NETWORK [conn4069827] end connection 10.0.7.113:47874 (1504 connections now open) 2016-03-13T08:51:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55176 #4078505 (1505 connections now open) 2016-03-13T08:51:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33443 #4078506 (1506 connections now open) 2016-03-13T08:51:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57141 #4078507 (1507 connections now open) 2016-03-13T08:51:28.532+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:28.532+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:28.646+0000 I NETWORK [conn4078494] end connection 10.0.179.209:38187 (1506 connections now open) 2016-03-13T08:51:28.646+0000 I NETWORK [conn4078493] end connection 10.0.179.209:38186 (1506 connections now open) 2016-03-13T08:51:28.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38200 #4078508 (1506 connections now open) 2016-03-13T08:51:28.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38199 #4078509 (1507 connections now open) 2016-03-13T08:51:29.034+0000 W NETWORK [Balancer] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:29.034+0000 W NETWORK [Balancer] No primary detected for set rsmydomain3 2016-03-13T08:51:29.034+0000 I SHARDING [Balancer] caught exception while doing balance: could not find host matching read preference { mode: "primary" } for set rsmydomain3 2016-03-13T08:51:29.034+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:51:29.034+0000-56e52a1118fa188537623cac", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859089034), what: "balancer.round", ns: "", details: { executionTimeMillis: 10566, errorOccured: true, errmsg: "could not find host matching read preference { mode: "primary" } for set rsmydomain3" } } 2016-03-13T08:51:29.280+0000 I NETWORK [conn4078496] end connection 10.0.26.69:40186 (1506 connections now open) 2016-03-13T08:51:29.280+0000 I NETWORK [conn4078495] end connection 10.0.26.69:40185 (1506 connections now open) 2016-03-13T08:51:29.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40196 #4078510 (1506 connections now open) 2016-03-13T08:51:29.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40195 #4078511 (1507 connections now open) 2016-03-13T08:51:32.191+0000 I NETWORK [conn4078498] end connection 10.0.27.237:33436 (1506 connections now open) 2016-03-13T08:51:32.191+0000 I NETWORK [conn4078497] end connection 10.0.27.237:33437 (1506 connections now open) 2016-03-13T08:51:32.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33444 #4078512 (1506 connections now open) 2016-03-13T08:51:32.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33445 #4078513 (1507 connections now open) 2016-03-13T08:51:34.276+0000 I NETWORK [conn4078500] end connection 10.0.7.113:57133 (1506 connections now open) 2016-03-13T08:51:34.276+0000 I NETWORK [conn4078499] end connection 10.0.7.113:57134 (1506 connections now open) 2016-03-13T08:51:34.277+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57154 #4078514 (1506 connections now open) 2016-03-13T08:51:34.277+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57155 #4078515 (1507 connections now open) 2016-03-13T08:51:34.663+0000 W NETWORK [ReplicaSetMonitorWatcher] Failed to connect to 10.0.90.157:27017, reason: errno:111 Connection refused 2016-03-13T08:51:35.459+0000 I NETWORK [conn4078501] end connection 10.0.216.90:55173 (1506 connections now open) 2016-03-13T08:51:35.459+0000 I NETWORK [conn4078502] end connection 10.0.216.90:55174 (1506 connections now open) 2016-03-13T08:51:35.460+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55196 #4078516 (1506 connections now open) 2016-03-13T08:51:35.460+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55195 #4078517 (1507 connections now open) 2016-03-13T08:51:36.310+0000 I NETWORK [conn4078504] end connection 10.0.20.53:48356 (1506 connections now open) 2016-03-13T08:51:36.310+0000 I NETWORK [conn4078503] end connection 10.0.20.53:48355 (1506 connections now open) 2016-03-13T08:51:36.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48367 #4078518 (1506 connections now open) 2016-03-13T08:51:36.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48368 #4078519 (1507 connections now open) 2016-03-13T08:51:38.659+0000 I NETWORK [conn4078509] end connection 10.0.179.209:38199 (1506 connections now open) 2016-03-13T08:51:38.659+0000 I NETWORK [conn4078508] end connection 10.0.179.209:38200 (1506 connections now open) 2016-03-13T08:51:38.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38218 #4078520 (1506 connections now open) 2016-03-13T08:51:38.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38219 #4078521 (1507 connections now open) 2016-03-13T08:51:39.293+0000 I NETWORK [conn4078510] end connection 10.0.26.69:40196 (1506 connections now open) 2016-03-13T08:51:39.293+0000 I NETWORK [conn4078511] end connection 10.0.26.69:40195 (1506 connections now open) 2016-03-13T08:51:39.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40206 #4078522 (1506 connections now open) 2016-03-13T08:51:39.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40207 #4078523 (1507 connections now open) 2016-03-13T08:51:40.259+0000 I ASIO [Balancer] dropping unhealthy pooled connection to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T08:51:40.259+0000 I ASIO [Balancer] after drop, pool was empty, going to spawn some connections 2016-03-13T08:51:40.260+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T08:51:40.584+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52a1c18fa188537623cad 2016-03-13T08:51:42.204+0000 I NETWORK [conn4078512] end connection 10.0.27.237:33444 (1506 connections now open) 2016-03-13T08:51:42.204+0000 I NETWORK [conn4078513] end connection 10.0.27.237:33445 (1506 connections now open) 2016-03-13T08:51:42.205+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33456 #4078524 (1506 connections now open) 2016-03-13T08:51:42.205+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33457 #4078525 (1507 connections now open) 2016-03-13T08:51:43.937+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 703ms sequenceNumber: 16725 version: 30541|1||563ba74d869758a4542b9075 based on: 30537|1||563ba74d869758a4542b9075 2016-03-13T08:51:44.169+0000 I SHARDING [Balancer] chunk { _id: "mydomain.Sessions-a_ObjectId('5211e594645cff218c1b662b')_id_ObjectId('55da23aca83e820e42945d61')", ns: "mydomain.Sessions", min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55da23aca83e820e42945d61') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55da5345584ad15e53727597') }, shard: "shmydomain3", lastmod: Timestamp 30541000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } is not on a shard with the right tag: mydomainSessions 2016-03-13T08:51:44.169+0000 I SHARDING [Balancer] going to move to: shmydomain2 2016-03-13T08:51:44.289+0000 I NETWORK [conn4078514] end connection 10.0.7.113:57154 (1506 connections now open) 2016-03-13T08:51:44.289+0000 I NETWORK [conn4078515] end connection 10.0.7.113:57155 (1505 connections now open) 2016-03-13T08:51:44.290+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57163 #4078526 (1506 connections now open) 2016-03-13T08:51:44.290+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57162 #4078527 (1507 connections now open) 2016-03-13T08:51:44.326+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain3, lastmod: 30541|1||563ba74d869758a4542b9075, min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55da23aca83e820e42945d61') }, max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55da5345584ad15e53727597') }) shmydomain3 -> shmydomain2 2016-03-13T08:51:44.327+0000 W SHARDING [Balancer] could not move chunk min: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55da23aca83e820e42945d61') } max: { a: ObjectId('5211e594645cff218c1b662b'), _id: ObjectId('55da5345584ad15e53727597') }, continuing balancing round :: caused by :: 6 network error while attempting to run command 'moveChunk' on host 'in.db3m1.mongo323.test.mydomain.com:27017' 2016-03-13T08:51:44.327+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:51:44.327+0000-56e52a2018fa188537623cae", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859104327), what: "balancer.round", ns: "", details: { executionTimeMillis: 4090, errorOccured: false, candidateChunks: 1, chunksMoved: 0 } } 2016-03-13T08:51:44.429+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:51:44.666+0000 I NETWORK [ReplicaSetMonitorWatcher] changing hosts to rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017 from rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 2016-03-13T08:51:44.667+0000 I SHARDING [ReplicaSetMonitorWatcher] Updating ShardRegistry connection string for shard shmydomain3 from: rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017,in.db3m2.mongo323.test.mydomain.com:27017 to: rsmydomain3/in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T08:51:45.472+0000 I NETWORK [conn4078517] end connection 10.0.216.90:55195 (1506 connections now open) 2016-03-13T08:51:45.472+0000 I NETWORK [conn4078516] end connection 10.0.216.90:55196 (1506 connections now open) 2016-03-13T08:51:45.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55205 #4078528 (1506 connections now open) 2016-03-13T08:51:45.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55204 #4078529 (1507 connections now open) 2016-03-13T08:51:46.117+0000 I NETWORK [conn4074517] end connection 10.0.216.90:50892 (1506 connections now open) 2016-03-13T08:51:46.118+0000 I NETWORK [conn4074885] end connection 10.0.27.237:58074 (1505 connections now open) 2016-03-13T08:51:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55208 #4078530 (1506 connections now open) 2016-03-13T08:51:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33472 #4078531 (1507 connections now open) 2016-03-13T08:51:46.323+0000 I NETWORK [conn4078518] end connection 10.0.20.53:48367 (1506 connections now open) 2016-03-13T08:51:46.323+0000 I NETWORK [conn4078519] end connection 10.0.20.53:48368 (1506 connections now open) 2016-03-13T08:51:46.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48381 #4078532 (1506 connections now open) 2016-03-13T08:51:46.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48382 #4078533 (1507 connections now open) 2016-03-13T08:51:48.672+0000 I NETWORK [conn4078521] end connection 10.0.179.209:38219 (1506 connections now open) 2016-03-13T08:51:48.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38227 #4078534 (1507 connections now open) 2016-03-13T08:51:48.672+0000 I NETWORK [conn4078520] end connection 10.0.179.209:38218 (1506 connections now open) 2016-03-13T08:51:48.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38228 #4078535 (1507 connections now open) 2016-03-13T08:51:49.307+0000 I NETWORK [conn4078522] end connection 10.0.26.69:40206 (1506 connections now open) 2016-03-13T08:51:49.307+0000 I NETWORK [conn4078523] end connection 10.0.26.69:40207 (1506 connections now open) 2016-03-13T08:51:49.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40227 #4078536 (1506 connections now open) 2016-03-13T08:51:49.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40226 #4078537 (1507 connections now open) 2016-03-13T08:51:49.800+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:51:49.590+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:51:52.218+0000 I NETWORK [conn4078524] end connection 10.0.27.237:33456 (1506 connections now open) 2016-03-13T08:51:52.218+0000 I NETWORK [conn4078525] end connection 10.0.27.237:33457 (1506 connections now open) 2016-03-13T08:51:52.218+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33474 #4078538 (1506 connections now open) 2016-03-13T08:51:52.218+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33475 #4078539 (1507 connections now open) 2016-03-13T08:51:54.303+0000 I NETWORK [conn4078527] end connection 10.0.7.113:57162 (1506 connections now open) 2016-03-13T08:51:54.303+0000 I NETWORK [conn4078526] end connection 10.0.7.113:57163 (1506 connections now open) 2016-03-13T08:51:54.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57176 #4078540 (1506 connections now open) 2016-03-13T08:51:54.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57177 #4078541 (1507 connections now open) 2016-03-13T08:51:55.485+0000 I NETWORK [conn4078529] end connection 10.0.216.90:55204 (1506 connections now open) 2016-03-13T08:51:55.485+0000 I NETWORK [conn4078528] end connection 10.0.216.90:55205 (1506 connections now open) 2016-03-13T08:51:55.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55218 #4078542 (1506 connections now open) 2016-03-13T08:51:55.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55217 #4078543 (1507 connections now open) 2016-03-13T08:51:56.337+0000 I NETWORK [conn4078533] end connection 10.0.20.53:48382 (1506 connections now open) 2016-03-13T08:51:56.337+0000 I NETWORK [conn4078532] end connection 10.0.20.53:48381 (1506 connections now open) 2016-03-13T08:51:56.346+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48398 #4078544 (1506 connections now open) 2016-03-13T08:51:56.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48399 #4078545 (1507 connections now open) 2016-03-13T08:51:58.113+0000 I NETWORK [conn4065951] end connection 10.0.20.53:35292 (1506 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [conn4071140] end connection 10.0.216.90:47262 (1505 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [conn4075449] end connection 10.0.26.69:36916 (1505 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [conn4070478] end connection 10.0.27.237:53475 (1503 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [conn4069584] end connection 10.0.7.113:47600 (1502 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48402 #4078546 (1504 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55221 #4078547 (1504 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40235 #4078548 (1505 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33481 #4078549 (1506 connections now open) 2016-03-13T08:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57184 #4078550 (1507 connections now open) 2016-03-13T08:51:58.685+0000 I NETWORK [conn4078535] end connection 10.0.179.209:38228 (1506 connections now open) 2016-03-13T08:51:58.685+0000 I NETWORK [conn4078534] end connection 10.0.179.209:38227 (1506 connections now open) 2016-03-13T08:51:58.692+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38239 #4078551 (1506 connections now open) 2016-03-13T08:51:58.692+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38240 #4078552 (1507 connections now open) 2016-03-13T08:51:59.320+0000 I NETWORK [conn4078537] end connection 10.0.26.69:40226 (1506 connections now open) 2016-03-13T08:51:59.320+0000 I NETWORK [conn4078536] end connection 10.0.26.69:40227 (1506 connections now open) 2016-03-13T08:51:59.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40238 #4078553 (1507 connections now open) 2016-03-13T08:51:59.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40237 #4078554 (1507 connections now open) 2016-03-13T08:52:02.231+0000 I NETWORK [conn4078539] end connection 10.0.27.237:33475 (1506 connections now open) 2016-03-13T08:52:02.231+0000 I NETWORK [conn4078538] end connection 10.0.27.237:33474 (1506 connections now open) 2016-03-13T08:52:02.232+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33485 #4078555 (1506 connections now open) 2016-03-13T08:52:02.232+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33486 #4078556 (1507 connections now open) 2016-03-13T08:52:04.317+0000 I NETWORK [conn4078541] end connection 10.0.7.113:57177 (1506 connections now open) 2016-03-13T08:52:04.317+0000 I NETWORK [conn4078540] end connection 10.0.7.113:57176 (1506 connections now open) 2016-03-13T08:52:04.317+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57198 #4078557 (1506 connections now open) 2016-03-13T08:52:04.317+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57199 #4078558 (1507 connections now open) 2016-03-13T08:52:05.498+0000 I NETWORK [conn4078542] end connection 10.0.216.90:55218 (1506 connections now open) 2016-03-13T08:52:05.498+0000 I NETWORK [conn4078543] end connection 10.0.216.90:55217 (1506 connections now open) 2016-03-13T08:52:05.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55239 #4078559 (1506 connections now open) 2016-03-13T08:52:05.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55240 #4078560 (1507 connections now open) 2016-03-13T08:52:06.359+0000 I NETWORK [conn4078544] end connection 10.0.20.53:48398 (1506 connections now open) 2016-03-13T08:52:06.359+0000 I NETWORK [conn4078545] end connection 10.0.20.53:48399 (1506 connections now open) 2016-03-13T08:52:06.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48408 #4078561 (1506 connections now open) 2016-03-13T08:52:06.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48409 #4078562 (1507 connections now open) 2016-03-13T08:52:08.705+0000 I NETWORK [conn4078551] end connection 10.0.179.209:38239 (1506 connections now open) 2016-03-13T08:52:08.705+0000 I NETWORK [conn4078552] end connection 10.0.179.209:38240 (1506 connections now open) 2016-03-13T08:52:08.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38261 #4078563 (1506 connections now open) 2016-03-13T08:52:08.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38260 #4078564 (1507 connections now open) 2016-03-13T08:52:09.334+0000 I NETWORK [conn4078553] end connection 10.0.26.69:40238 (1506 connections now open) 2016-03-13T08:52:09.334+0000 I NETWORK [conn4078554] end connection 10.0.26.69:40237 (1506 connections now open) 2016-03-13T08:52:09.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40250 #4078565 (1506 connections now open) 2016-03-13T08:52:09.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40249 #4078566 (1507 connections now open) 2016-03-13T08:52:12.244+0000 I NETWORK [conn4078556] end connection 10.0.27.237:33486 (1506 connections now open) 2016-03-13T08:52:12.244+0000 I NETWORK [conn4078555] end connection 10.0.27.237:33485 (1506 connections now open) 2016-03-13T08:52:12.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33497 #4078567 (1506 connections now open) 2016-03-13T08:52:12.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33498 #4078568 (1507 connections now open) 2016-03-13T08:52:14.330+0000 I NETWORK [conn4078557] end connection 10.0.7.113:57198 (1506 connections now open) 2016-03-13T08:52:14.330+0000 I NETWORK [conn4078558] end connection 10.0.7.113:57199 (1506 connections now open) 2016-03-13T08:52:14.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57209 #4078569 (1506 connections now open) 2016-03-13T08:52:14.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57210 #4078570 (1507 connections now open) 2016-03-13T08:52:15.511+0000 I NETWORK [conn4078560] end connection 10.0.216.90:55240 (1506 connections now open) 2016-03-13T08:52:15.511+0000 I NETWORK [conn4078559] end connection 10.0.216.90:55239 (1506 connections now open) 2016-03-13T08:52:15.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55248 #4078571 (1506 connections now open) 2016-03-13T08:52:15.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55249 #4078572 (1507 connections now open) 2016-03-13T08:52:16.118+0000 I NETWORK [conn4075056] end connection 10.0.20.53:44779 (1506 connections now open) 2016-03-13T08:52:16.118+0000 I NETWORK [conn4070521] end connection 10.0.216.90:46606 (1506 connections now open) 2016-03-13T08:52:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48421 #4078573 (1506 connections now open) 2016-03-13T08:52:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55252 #4078574 (1507 connections now open) 2016-03-13T08:52:16.119+0000 I NETWORK [conn4070922] end connection 10.0.27.237:53951 (1506 connections now open) 2016-03-13T08:52:16.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33516 #4078575 (1507 connections now open) 2016-03-13T08:52:16.372+0000 I NETWORK [conn4078561] end connection 10.0.20.53:48408 (1506 connections now open) 2016-03-13T08:52:16.372+0000 I NETWORK [conn4078562] end connection 10.0.20.53:48409 (1506 connections now open) 2016-03-13T08:52:16.373+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48422 #4078576 (1506 connections now open) 2016-03-13T08:52:16.373+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48423 #4078577 (1507 connections now open) 2016-03-13T08:52:18.719+0000 I NETWORK [conn4078563] end connection 10.0.179.209:38261 (1506 connections now open) 2016-03-13T08:52:18.719+0000 I NETWORK [conn4078564] end connection 10.0.179.209:38260 (1506 connections now open) 2016-03-13T08:52:18.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38273 #4078578 (1506 connections now open) 2016-03-13T08:52:18.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38274 #4078579 (1507 connections now open) 2016-03-13T08:52:19.347+0000 I NETWORK [conn4078566] end connection 10.0.26.69:40249 (1506 connections now open) 2016-03-13T08:52:19.347+0000 I NETWORK [conn4078565] end connection 10.0.26.69:40250 (1506 connections now open) 2016-03-13T08:52:19.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40273 #4078580 (1506 connections now open) 2016-03-13T08:52:19.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40274 #4078581 (1507 connections now open) 2016-03-13T08:52:20.067+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:52:19.800+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:52:20.068+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:52:20.204+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a2a18fa188537623caf 2016-03-13T08:52:20.341+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a3418fa188537623cb0 2016-03-13T08:52:20.487+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a3e18fa188537623cb1 2016-03-13T08:52:22.257+0000 I NETWORK [conn4078568] end connection 10.0.27.237:33498 (1506 connections now open) 2016-03-13T08:52:22.257+0000 I NETWORK [conn4078567] end connection 10.0.27.237:33497 (1506 connections now open) 2016-03-13T08:52:22.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33518 #4078582 (1506 connections now open) 2016-03-13T08:52:22.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33519 #4078583 (1507 connections now open) 2016-03-13T08:52:24.344+0000 I NETWORK [conn4078570] end connection 10.0.7.113:57210 (1506 connections now open) 2016-03-13T08:52:24.344+0000 I NETWORK [conn4078569] end connection 10.0.7.113:57209 (1506 connections now open) 2016-03-13T08:52:24.353+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57222 #4078584 (1506 connections now open) 2016-03-13T08:52:24.353+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57223 #4078585 (1507 connections now open) 2016-03-13T08:52:24.780+0000 I SHARDING [Balancer] could not acquire lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' (another update won) 2016-03-13T08:52:24.780+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' was not acquired. 2016-03-13T08:52:25.531+0000 I NETWORK [conn4078571] end connection 10.0.216.90:55248 (1506 connections now open) 2016-03-13T08:52:25.531+0000 I NETWORK [conn4078572] end connection 10.0.216.90:55249 (1506 connections now open) 2016-03-13T08:52:25.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55262 #4078586 (1506 connections now open) 2016-03-13T08:52:25.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55263 #4078587 (1507 connections now open) 2016-03-13T08:52:26.386+0000 I NETWORK [conn4078577] end connection 10.0.20.53:48423 (1506 connections now open) 2016-03-13T08:52:26.386+0000 I NETWORK [conn4078576] end connection 10.0.20.53:48422 (1506 connections now open) 2016-03-13T08:52:26.386+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48441 #4078588 (1506 connections now open) 2016-03-13T08:52:26.386+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48440 #4078589 (1507 connections now open) 2016-03-13T08:52:28.113+0000 I NETWORK [conn4069974] end connection 10.0.26.69:59270 (1506 connections now open) 2016-03-13T08:52:28.113+0000 I NETWORK [conn4073598] end connection 10.0.27.237:56795 (1506 connections now open) 2016-03-13T08:52:28.113+0000 I NETWORK [conn4070461] end connection 10.0.7.113:48565 (1504 connections now open) 2016-03-13T08:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40281 #4078590 (1505 connections now open) 2016-03-13T08:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33526 #4078591 (1506 connections now open) 2016-03-13T08:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57230 #4078592 (1507 connections now open) 2016-03-13T08:52:28.732+0000 I NETWORK [conn4078579] end connection 10.0.179.209:38274 (1506 connections now open) 2016-03-13T08:52:28.732+0000 I NETWORK [conn4078578] end connection 10.0.179.209:38273 (1505 connections now open) 2016-03-13T08:52:28.732+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38286 #4078593 (1506 connections now open) 2016-03-13T08:52:28.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38287 #4078594 (1507 connections now open) 2016-03-13T08:52:29.368+0000 I NETWORK [conn4078580] end connection 10.0.26.69:40273 (1506 connections now open) 2016-03-13T08:52:29.368+0000 I NETWORK [conn4078581] end connection 10.0.26.69:40274 (1506 connections now open) 2016-03-13T08:52:29.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40283 #4078595 (1506 connections now open) 2016-03-13T08:52:29.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40282 #4078596 (1507 connections now open) 2016-03-13T08:52:32.279+0000 I NETWORK [conn4078583] end connection 10.0.27.237:33519 (1506 connections now open) 2016-03-13T08:52:32.279+0000 I NETWORK [conn4078582] end connection 10.0.27.237:33518 (1506 connections now open) 2016-03-13T08:52:32.280+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33528 #4078597 (1506 connections now open) 2016-03-13T08:52:32.280+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33529 #4078598 (1507 connections now open) 2016-03-13T08:52:34.366+0000 I NETWORK [conn4078585] end connection 10.0.7.113:57223 (1506 connections now open) 2016-03-13T08:52:34.366+0000 I NETWORK [conn4078584] end connection 10.0.7.113:57222 (1505 connections now open) 2016-03-13T08:52:34.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57242 #4078599 (1507 connections now open) 2016-03-13T08:52:34.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57243 #4078600 (1507 connections now open) 2016-03-13T08:52:35.544+0000 I NETWORK [conn4078587] end connection 10.0.216.90:55263 (1506 connections now open) 2016-03-13T08:52:35.544+0000 I NETWORK [conn4078586] end connection 10.0.216.90:55262 (1506 connections now open) 2016-03-13T08:52:35.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55287 #4078601 (1506 connections now open) 2016-03-13T08:52:35.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55288 #4078602 (1507 connections now open) 2016-03-13T08:52:36.399+0000 I NETWORK [conn4078588] end connection 10.0.20.53:48441 (1506 connections now open) 2016-03-13T08:52:36.399+0000 I NETWORK [conn4078589] end connection 10.0.20.53:48440 (1506 connections now open) 2016-03-13T08:52:36.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48454 #4078603 (1506 connections now open) 2016-03-13T08:52:36.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48453 #4078604 (1507 connections now open) 2016-03-13T08:52:38.745+0000 I NETWORK [conn4078593] end connection 10.0.179.209:38286 (1506 connections now open) 2016-03-13T08:52:38.745+0000 I NETWORK [conn4078594] end connection 10.0.179.209:38287 (1506 connections now open) 2016-03-13T08:52:38.746+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38304 #4078605 (1506 connections now open) 2016-03-13T08:52:38.746+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38305 #4078606 (1507 connections now open) 2016-03-13T08:52:39.381+0000 I NETWORK [conn4078596] end connection 10.0.26.69:40282 (1506 connections now open) 2016-03-13T08:52:39.381+0000 I NETWORK [conn4078595] end connection 10.0.26.69:40283 (1506 connections now open) 2016-03-13T08:52:39.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40294 #4078607 (1506 connections now open) 2016-03-13T08:52:39.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40295 #4078608 (1507 connections now open) 2016-03-13T08:52:42.293+0000 I NETWORK [conn4078598] end connection 10.0.27.237:33529 (1506 connections now open) 2016-03-13T08:52:42.293+0000 I NETWORK [conn4078597] end connection 10.0.27.237:33528 (1506 connections now open) 2016-03-13T08:52:42.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33540 #4078609 (1506 connections now open) 2016-03-13T08:52:42.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33541 #4078610 (1507 connections now open) 2016-03-13T08:52:44.380+0000 I NETWORK [conn4078600] end connection 10.0.7.113:57243 (1506 connections now open) 2016-03-13T08:52:44.380+0000 I NETWORK [conn4078599] end connection 10.0.7.113:57242 (1506 connections now open) 2016-03-13T08:52:44.380+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57250 #4078611 (1506 connections now open) 2016-03-13T08:52:44.380+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57249 #4078612 (1507 connections now open) 2016-03-13T08:52:45.557+0000 I NETWORK [conn4078602] end connection 10.0.216.90:55288 (1506 connections now open) 2016-03-13T08:52:45.557+0000 I NETWORK [conn4078601] end connection 10.0.216.90:55287 (1506 connections now open) 2016-03-13T08:52:45.558+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55296 #4078613 (1506 connections now open) 2016-03-13T08:52:45.558+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55295 #4078614 (1507 connections now open) 2016-03-13T08:52:46.120+0000 I NETWORK [conn4067523] end connection 10.0.20.53:36895 (1506 connections now open) 2016-03-13T08:52:46.120+0000 I NETWORK [conn4066869] end connection 10.0.26.69:55960 (1506 connections now open) 2016-03-13T08:52:46.120+0000 I NETWORK [conn4075307] end connection 10.0.27.237:58493 (1506 connections now open) 2016-03-13T08:52:46.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33557 #4078615 (1505 connections now open) 2016-03-13T08:52:46.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48466 #4078616 (1506 connections now open) 2016-03-13T08:52:46.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40303 #4078617 (1507 connections now open) 2016-03-13T08:52:46.412+0000 I NETWORK [conn4078603] end connection 10.0.20.53:48454 (1506 connections now open) 2016-03-13T08:52:46.412+0000 I NETWORK [conn4078604] end connection 10.0.20.53:48453 (1506 connections now open) 2016-03-13T08:52:46.413+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48467 #4078618 (1506 connections now open) 2016-03-13T08:52:46.413+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48468 #4078619 (1507 connections now open) 2016-03-13T08:52:48.759+0000 I NETWORK [conn4078606] end connection 10.0.179.209:38305 (1506 connections now open) 2016-03-13T08:52:48.759+0000 I NETWORK [conn4078605] end connection 10.0.179.209:38304 (1506 connections now open) 2016-03-13T08:52:48.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38314 #4078620 (1506 connections now open) 2016-03-13T08:52:48.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38315 #4078621 (1507 connections now open) 2016-03-13T08:52:49.394+0000 I NETWORK [conn4078607] end connection 10.0.26.69:40294 (1506 connections now open) 2016-03-13T08:52:49.394+0000 I NETWORK [conn4078608] end connection 10.0.26.69:40295 (1506 connections now open) 2016-03-13T08:52:49.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40315 #4078622 (1507 connections now open) 2016-03-13T08:52:49.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40314 #4078623 (1507 connections now open) 2016-03-13T08:52:50.750+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:52:50.487+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:52:50.750+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:52:50.896+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a4818fa188537623cb2 2016-03-13T08:52:51.008+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a5418fa188537623cb3 2016-03-13T08:52:51.076+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a5e18fa188537623cb4 2016-03-13T08:52:52.306+0000 I NETWORK [conn4078609] end connection 10.0.27.237:33540 (1506 connections now open) 2016-03-13T08:52:52.306+0000 I NETWORK [conn4078610] end connection 10.0.27.237:33541 (1506 connections now open) 2016-03-13T08:52:52.306+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33560 #4078624 (1506 connections now open) 2016-03-13T08:52:52.306+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33559 #4078625 (1507 connections now open) 2016-03-13T08:52:54.393+0000 I NETWORK [conn4078611] end connection 10.0.7.113:57250 (1506 connections now open) 2016-03-13T08:52:54.393+0000 I NETWORK [conn4078612] end connection 10.0.7.113:57249 (1506 connections now open) 2016-03-13T08:52:54.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57263 #4078626 (1507 connections now open) 2016-03-13T08:52:54.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57262 #4078627 (1507 connections now open) 2016-03-13T08:52:55.570+0000 I NETWORK [conn4078613] end connection 10.0.216.90:55296 (1506 connections now open) 2016-03-13T08:52:55.570+0000 I NETWORK [conn4078614] end connection 10.0.216.90:55295 (1506 connections now open) 2016-03-13T08:52:55.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55310 #4078628 (1506 connections now open) 2016-03-13T08:52:55.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55309 #4078629 (1507 connections now open) 2016-03-13T08:52:56.425+0000 I NETWORK [conn4078619] end connection 10.0.20.53:48468 (1506 connections now open) 2016-03-13T08:52:56.425+0000 I NETWORK [conn4078618] end connection 10.0.20.53:48467 (1506 connections now open) 2016-03-13T08:52:56.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48486 #4078630 (1506 connections now open) 2016-03-13T08:52:56.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48485 #4078631 (1507 connections now open) 2016-03-13T08:52:56.666+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52a6818fa188537623cb5 2016-03-13T08:52:58.113+0000 I NETWORK [conn4072090] end connection 10.0.179.209:59941 (1506 connections now open) 2016-03-13T08:52:58.113+0000 I NETWORK [conn4069085] end connection 10.0.26.69:58313 (1505 connections now open) 2016-03-13T08:52:58.113+0000 I NETWORK [conn4071757] end connection 10.0.27.237:54828 (1504 connections now open) 2016-03-13T08:52:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40323 #4078632 (1505 connections now open) 2016-03-13T08:52:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33566 #4078633 (1506 connections now open) 2016-03-13T08:52:58.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38325 #4078634 (1507 connections now open) 2016-03-13T08:52:58.772+0000 I NETWORK [conn4078621] end connection 10.0.179.209:38315 (1506 connections now open) 2016-03-13T08:52:58.772+0000 I NETWORK [conn4078620] end connection 10.0.179.209:38314 (1505 connections now open) 2016-03-13T08:52:58.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38326 #4078635 (1506 connections now open) 2016-03-13T08:52:58.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38327 #4078636 (1507 connections now open) 2016-03-13T08:52:59.407+0000 I NETWORK [conn4078623] end connection 10.0.26.69:40314 (1506 connections now open) 2016-03-13T08:52:59.407+0000 I NETWORK [conn4078622] end connection 10.0.26.69:40315 (1505 connections now open) 2016-03-13T08:52:59.408+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40325 #4078637 (1507 connections now open) 2016-03-13T08:52:59.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40324 #4078638 (1507 connections now open) 2016-03-13T08:52:59.729+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 16726 version: 30551|1||563ba74d869758a4542b9075 based on: 30541|1||563ba74d869758a4542b9075 2016-03-13T08:53:00.043+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633e62ca2a4330ea28bb45b')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633e62ca2a4330ea28bb45b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633e750584ad10f212b6727') }, shard: "shmydomain1", lastmod: Timestamp 30517000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:53:00.198+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30517|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633e62ca2a4330ea28bb45b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633e750584ad10f212b6727') }) shmydomain1 -> shmydomain3 2016-03-13T08:53:00.696+0000 I SHARDING [Balancer] moveChunk result: { ok: 0.0, errmsg: "socket exception [SEND_ERROR] for 10.0.167.43:27017", code: 9001 } 2016-03-13T08:53:00.697+0000 I SHARDING [Balancer] balancer move failed: { ok: 0.0, errmsg: "socket exception [SEND_ERROR] for 10.0.167.43:27017", code: 9001 } from: shmydomain1 to: shmydomain3 chunk: min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633e62ca2a4330ea28bb45b') } max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633e750584ad10f212b6727') } 2016-03-13T08:53:00.697+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:53:00.697+0000-56e52a6c18fa188537623cb6", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859180697), what: "balancer.round", ns: "", details: { executionTimeMillis: 4325, errorOccured: false, candidateChunks: 1, chunksMoved: 0 } } 2016-03-13T08:53:00.798+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:53:02.319+0000 I NETWORK [conn4078624] end connection 10.0.27.237:33560 (1506 connections now open) 2016-03-13T08:53:02.319+0000 I NETWORK [conn4078625] end connection 10.0.27.237:33559 (1506 connections now open) 2016-03-13T08:53:02.319+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33571 #4078639 (1506 connections now open) 2016-03-13T08:53:02.319+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33570 #4078640 (1507 connections now open) 2016-03-13T08:53:04.406+0000 I NETWORK [conn4078626] end connection 10.0.7.113:57263 (1506 connections now open) 2016-03-13T08:53:04.406+0000 I NETWORK [conn4078627] end connection 10.0.7.113:57262 (1506 connections now open) 2016-03-13T08:53:04.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57284 #4078641 (1507 connections now open) 2016-03-13T08:53:04.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57283 #4078642 (1507 connections now open) 2016-03-13T08:53:05.583+0000 I NETWORK [conn4078629] end connection 10.0.216.90:55309 (1506 connections now open) 2016-03-13T08:53:05.583+0000 I NETWORK [conn4078628] end connection 10.0.216.90:55310 (1506 connections now open) 2016-03-13T08:53:05.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55332 #4078643 (1506 connections now open) 2016-03-13T08:53:05.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55333 #4078644 (1507 connections now open) 2016-03-13T08:53:06.448+0000 I NETWORK [conn4078631] end connection 10.0.20.53:48485 (1506 connections now open) 2016-03-13T08:53:06.448+0000 I NETWORK [conn4078630] end connection 10.0.20.53:48486 (1506 connections now open) 2016-03-13T08:53:06.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48498 #4078645 (1506 connections now open) 2016-03-13T08:53:06.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48497 #4078646 (1507 connections now open) 2016-03-13T08:53:08.785+0000 I NETWORK [conn4078635] end connection 10.0.179.209:38326 (1506 connections now open) 2016-03-13T08:53:08.785+0000 I NETWORK [conn4078636] end connection 10.0.179.209:38327 (1506 connections now open) 2016-03-13T08:53:08.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38346 #4078647 (1506 connections now open) 2016-03-13T08:53:08.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38347 #4078648 (1507 connections now open) 2016-03-13T08:53:09.423+0000 I NETWORK [conn4078638] end connection 10.0.26.69:40324 (1506 connections now open) 2016-03-13T08:53:09.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40337 #4078649 (1507 connections now open) 2016-03-13T08:53:09.424+0000 I NETWORK [conn4078637] end connection 10.0.26.69:40325 (1506 connections now open) 2016-03-13T08:53:09.425+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40338 #4078650 (1507 connections now open) 2016-03-13T08:53:12.332+0000 I NETWORK [conn4078640] end connection 10.0.27.237:33570 (1506 connections now open) 2016-03-13T08:53:12.332+0000 I NETWORK [conn4078639] end connection 10.0.27.237:33571 (1506 connections now open) 2016-03-13T08:53:12.332+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33583 #4078651 (1506 connections now open) 2016-03-13T08:53:12.332+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33582 #4078652 (1507 connections now open) 2016-03-13T08:53:14.419+0000 I NETWORK [conn4078642] end connection 10.0.7.113:57283 (1506 connections now open) 2016-03-13T08:53:14.419+0000 I NETWORK [conn4078641] end connection 10.0.7.113:57284 (1506 connections now open) 2016-03-13T08:53:14.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57293 #4078653 (1506 connections now open) 2016-03-13T08:53:14.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57294 #4078654 (1507 connections now open) 2016-03-13T08:53:15.596+0000 I NETWORK [conn4078643] end connection 10.0.216.90:55332 (1506 connections now open) 2016-03-13T08:53:15.596+0000 I NETWORK [conn4078644] end connection 10.0.216.90:55333 (1506 connections now open) 2016-03-13T08:53:15.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55342 #4078655 (1506 connections now open) 2016-03-13T08:53:15.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55341 #4078656 (1507 connections now open) 2016-03-13T08:53:16.118+0000 I NETWORK [conn4073956] end connection 10.0.179.209:33586 (1506 connections now open) 2016-03-13T08:53:16.118+0000 I NETWORK [conn4072706] end connection 10.0.27.237:55861 (1505 connections now open) 2016-03-13T08:53:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38354 #4078657 (1506 connections now open) 2016-03-13T08:53:16.118+0000 I NETWORK [conn4067986] end connection 10.0.7.113:45889 (1505 connections now open) 2016-03-13T08:53:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33602 #4078658 (1506 connections now open) 2016-03-13T08:53:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57299 #4078659 (1507 connections now open) 2016-03-13T08:53:16.462+0000 I NETWORK [conn4078645] end connection 10.0.20.53:48498 (1506 connections now open) 2016-03-13T08:53:16.462+0000 I NETWORK [conn4078646] end connection 10.0.20.53:48497 (1506 connections now open) 2016-03-13T08:53:16.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48511 #4078660 (1506 connections now open) 2016-03-13T08:53:16.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48510 #4078661 (1507 connections now open) 2016-03-13T08:53:18.799+0000 I NETWORK [conn4078648] end connection 10.0.179.209:38347 (1506 connections now open) 2016-03-13T08:53:18.799+0000 I NETWORK [conn4078647] end connection 10.0.179.209:38346 (1506 connections now open) 2016-03-13T08:53:18.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38359 #4078662 (1506 connections now open) 2016-03-13T08:53:18.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38358 #4078663 (1507 connections now open) 2016-03-13T08:53:19.435+0000 I NETWORK [conn4078649] end connection 10.0.26.69:40337 (1506 connections now open) 2016-03-13T08:53:19.437+0000 I NETWORK [conn4078650] end connection 10.0.26.69:40338 (1505 connections now open) 2016-03-13T08:53:19.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40359 #4078664 (1506 connections now open) 2016-03-13T08:53:19.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40360 #4078665 (1507 connections now open) 2016-03-13T08:53:21.077+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 675ms sequenceNumber: 16727 version: 30552|1||563ba74d869758a4542b9075 based on: 30551|1||563ba74d869758a4542b9075 2016-03-13T08:53:21.323+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:53:21.076+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:53:21.323+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:53:21.461+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a7618fa188537623cb7 2016-03-13T08:53:21.596+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a8018fa188537623cb8 2016-03-13T08:53:22.345+0000 I NETWORK [conn4078651] end connection 10.0.27.237:33583 (1506 connections now open) 2016-03-13T08:53:22.345+0000 I NETWORK [conn4078652] end connection 10.0.27.237:33582 (1506 connections now open) 2016-03-13T08:53:22.346+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33606 #4078666 (1506 connections now open) 2016-03-13T08:53:22.346+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33605 #4078667 (1507 connections now open) 2016-03-13T08:53:24.433+0000 I NETWORK [conn4078653] end connection 10.0.7.113:57293 (1506 connections now open) 2016-03-13T08:53:24.433+0000 I NETWORK [conn4078654] end connection 10.0.7.113:57294 (1505 connections now open) 2016-03-13T08:53:24.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57306 #4078668 (1506 connections now open) 2016-03-13T08:53:24.444+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57305 #4078669 (1507 connections now open) 2016-03-13T08:53:25.618+0000 I NETWORK [conn4078656] end connection 10.0.216.90:55341 (1506 connections now open) 2016-03-13T08:53:25.618+0000 I NETWORK [conn4078655] end connection 10.0.216.90:55342 (1506 connections now open) 2016-03-13T08:53:25.618+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55355 #4078670 (1506 connections now open) 2016-03-13T08:53:25.618+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55356 #4078671 (1507 connections now open) 2016-03-13T08:53:26.475+0000 I NETWORK [conn4078660] end connection 10.0.20.53:48511 (1506 connections now open) 2016-03-13T08:53:26.475+0000 I NETWORK [conn4078661] end connection 10.0.20.53:48510 (1506 connections now open) 2016-03-13T08:53:26.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48529 #4078672 (1506 connections now open) 2016-03-13T08:53:26.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48530 #4078673 (1507 connections now open) 2016-03-13T08:53:28.114+0000 I NETWORK [conn4068367] end connection 10.0.26.69:57546 (1506 connections now open) 2016-03-13T08:53:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40368 #4078674 (1507 connections now open) 2016-03-13T08:53:28.812+0000 I NETWORK [conn4078663] end connection 10.0.179.209:38358 (1506 connections now open) 2016-03-13T08:53:28.812+0000 I NETWORK [conn4078662] end connection 10.0.179.209:38359 (1506 connections now open) 2016-03-13T08:53:28.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38372 #4078675 (1506 connections now open) 2016-03-13T08:53:28.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38373 #4078676 (1507 connections now open) 2016-03-13T08:53:29.457+0000 I NETWORK [conn4078665] end connection 10.0.26.69:40360 (1506 connections now open) 2016-03-13T08:53:29.457+0000 I NETWORK [conn4078664] end connection 10.0.26.69:40359 (1506 connections now open) 2016-03-13T08:53:29.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40371 #4078677 (1507 connections now open) 2016-03-13T08:53:29.458+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40370 #4078678 (1507 connections now open) 2016-03-13T08:53:32.358+0000 I NETWORK [conn4078667] end connection 10.0.27.237:33605 (1506 connections now open) 2016-03-13T08:53:32.358+0000 I NETWORK [conn4078666] end connection 10.0.27.237:33606 (1506 connections now open) 2016-03-13T08:53:32.359+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33614 #4078679 (1506 connections now open) 2016-03-13T08:53:32.359+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33615 #4078680 (1507 connections now open) 2016-03-13T08:53:34.455+0000 I NETWORK [conn4078669] end connection 10.0.7.113:57305 (1506 connections now open) 2016-03-13T08:53:34.455+0000 I NETWORK [conn4078668] end connection 10.0.7.113:57306 (1506 connections now open) 2016-03-13T08:53:34.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57326 #4078681 (1506 connections now open) 2016-03-13T08:53:34.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57327 #4078682 (1507 connections now open) 2016-03-13T08:53:35.501+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16728 version: 30553|1||563ba74d869758a4542b9075 based on: 30552|1||563ba74d869758a4542b9075 2016-03-13T08:53:35.631+0000 I NETWORK [conn4078670] end connection 10.0.216.90:55355 (1506 connections now open) 2016-03-13T08:53:35.631+0000 I NETWORK [conn4078671] end connection 10.0.216.90:55356 (1506 connections now open) 2016-03-13T08:53:35.631+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55376 #4078683 (1506 connections now open) 2016-03-13T08:53:35.631+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55377 #4078684 (1507 connections now open) 2016-03-13T08:53:36.489+0000 I NETWORK [conn4078673] end connection 10.0.20.53:48530 (1506 connections now open) 2016-03-13T08:53:36.489+0000 I NETWORK [conn4078672] end connection 10.0.20.53:48529 (1506 connections now open) 2016-03-13T08:53:36.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48541 #4078685 (1506 connections now open) 2016-03-13T08:53:36.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48542 #4078686 (1507 connections now open) 2016-03-13T08:53:38.826+0000 I NETWORK [conn4078676] end connection 10.0.179.209:38373 (1506 connections now open) 2016-03-13T08:53:38.826+0000 I NETWORK [conn4078675] end connection 10.0.179.209:38372 (1506 connections now open) 2016-03-13T08:53:38.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38392 #4078687 (1506 connections now open) 2016-03-13T08:53:38.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38391 #4078688 (1507 connections now open) 2016-03-13T08:53:39.470+0000 I NETWORK [conn4078678] end connection 10.0.26.69:40370 (1506 connections now open) 2016-03-13T08:53:39.470+0000 I NETWORK [conn4078677] end connection 10.0.26.69:40371 (1506 connections now open) 2016-03-13T08:53:39.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40382 #4078689 (1507 connections now open) 2016-03-13T08:53:39.471+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40381 #4078690 (1507 connections now open) 2016-03-13T08:53:42.371+0000 I NETWORK [conn4078680] end connection 10.0.27.237:33615 (1506 connections now open) 2016-03-13T08:53:42.371+0000 I NETWORK [conn4078679] end connection 10.0.27.237:33614 (1506 connections now open) 2016-03-13T08:53:42.372+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33627 #4078691 (1506 connections now open) 2016-03-13T08:53:42.372+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33626 #4078692 (1507 connections now open) 2016-03-13T08:53:44.469+0000 I NETWORK [conn4078681] end connection 10.0.7.113:57326 (1506 connections now open) 2016-03-13T08:53:44.469+0000 I NETWORK [conn4078682] end connection 10.0.7.113:57327 (1506 connections now open) 2016-03-13T08:53:44.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57332 #4078693 (1506 connections now open) 2016-03-13T08:53:44.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57333 #4078694 (1507 connections now open) 2016-03-13T08:53:45.644+0000 I NETWORK [conn4078684] end connection 10.0.216.90:55377 (1506 connections now open) 2016-03-13T08:53:45.644+0000 I NETWORK [conn4078683] end connection 10.0.216.90:55376 (1506 connections now open) 2016-03-13T08:53:45.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55384 #4078695 (1506 connections now open) 2016-03-13T08:53:45.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55383 #4078696 (1507 connections now open) 2016-03-13T08:53:46.502+0000 I NETWORK [conn4078685] end connection 10.0.20.53:48541 (1506 connections now open) 2016-03-13T08:53:46.502+0000 I NETWORK [conn4078686] end connection 10.0.20.53:48542 (1506 connections now open) 2016-03-13T08:53:46.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48555 #4078697 (1506 connections now open) 2016-03-13T08:53:46.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48554 #4078698 (1507 connections now open) 2016-03-13T08:53:48.839+0000 I NETWORK [conn4078688] end connection 10.0.179.209:38391 (1506 connections now open) 2016-03-13T08:53:48.839+0000 I NETWORK [conn4078687] end connection 10.0.179.209:38392 (1506 connections now open) 2016-03-13T08:53:48.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38400 #4078699 (1506 connections now open) 2016-03-13T08:53:48.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38401 #4078700 (1507 connections now open) 2016-03-13T08:53:49.483+0000 I NETWORK [conn4078690] end connection 10.0.26.69:40381 (1506 connections now open) 2016-03-13T08:53:49.483+0000 I NETWORK [conn4078689] end connection 10.0.26.69:40382 (1506 connections now open) 2016-03-13T08:53:49.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40400 #4078701 (1507 connections now open) 2016-03-13T08:53:49.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40401 #4078702 (1507 connections now open) 2016-03-13T08:53:51.803+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:53:51.596+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:53:51.803+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:53:51.871+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a8a18fa188537623cb9 2016-03-13T08:53:51.940+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a9518fa188537623cba 2016-03-13T08:53:52.008+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52a9f18fa188537623cbb 2016-03-13T08:53:52.385+0000 I NETWORK [conn4078692] end connection 10.0.27.237:33626 (1506 connections now open) 2016-03-13T08:53:52.385+0000 I NETWORK [conn4078691] end connection 10.0.27.237:33627 (1506 connections now open) 2016-03-13T08:53:52.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33646 #4078703 (1506 connections now open) 2016-03-13T08:53:52.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33647 #4078704 (1507 connections now open) 2016-03-13T08:53:54.483+0000 I NETWORK [conn4078694] end connection 10.0.7.113:57333 (1506 connections now open) 2016-03-13T08:53:54.483+0000 I NETWORK [conn4078693] end connection 10.0.7.113:57332 (1506 connections now open) 2016-03-13T08:53:54.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57345 #4078705 (1507 connections now open) 2016-03-13T08:53:54.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57346 #4078706 (1507 connections now open) 2016-03-13T08:53:55.657+0000 I NETWORK [conn4078696] end connection 10.0.216.90:55383 (1506 connections now open) 2016-03-13T08:53:55.657+0000 I NETWORK [conn4078695] end connection 10.0.216.90:55384 (1506 connections now open) 2016-03-13T08:53:55.657+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55397 #4078707 (1506 connections now open) 2016-03-13T08:53:55.657+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55398 #4078708 (1507 connections now open) 2016-03-13T08:53:56.515+0000 I NETWORK [conn4078697] end connection 10.0.20.53:48555 (1506 connections now open) 2016-03-13T08:53:56.515+0000 I NETWORK [conn4078698] end connection 10.0.20.53:48554 (1506 connections now open) 2016-03-13T08:53:56.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48573 #4078709 (1506 connections now open) 2016-03-13T08:53:56.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48574 #4078710 (1507 connections now open) 2016-03-13T08:53:58.114+0000 I NETWORK [conn4072176] end connection 10.0.26.69:33408 (1506 connections now open) 2016-03-13T08:53:58.114+0000 I NETWORK [conn4069785] end connection 10.0.7.113:47832 (1505 connections now open) 2016-03-13T08:53:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40409 #4078711 (1506 connections now open) 2016-03-13T08:53:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57353 #4078712 (1507 connections now open) 2016-03-13T08:53:58.852+0000 I NETWORK [conn4078700] end connection 10.0.179.209:38401 (1506 connections now open) 2016-03-13T08:53:58.852+0000 I NETWORK [conn4078699] end connection 10.0.179.209:38400 (1506 connections now open) 2016-03-13T08:53:58.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38415 #4078713 (1506 connections now open) 2016-03-13T08:53:58.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38414 #4078714 (1507 connections now open) 2016-03-13T08:53:59.497+0000 I NETWORK [conn4078701] end connection 10.0.26.69:40400 (1506 connections now open) 2016-03-13T08:53:59.497+0000 I NETWORK [conn4078702] end connection 10.0.26.69:40401 (1506 connections now open) 2016-03-13T08:53:59.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40411 #4078715 (1507 connections now open) 2016-03-13T08:53:59.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40412 #4078716 (1507 connections now open) 2016-03-13T08:54:02.398+0000 I NETWORK [conn4078704] end connection 10.0.27.237:33647 (1506 connections now open) 2016-03-13T08:54:02.398+0000 I NETWORK [conn4078703] end connection 10.0.27.237:33646 (1506 connections now open) 2016-03-13T08:54:02.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33658 #4078717 (1506 connections now open) 2016-03-13T08:54:02.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33659 #4078718 (1507 connections now open) 2016-03-13T08:54:04.497+0000 I NETWORK [conn4078705] end connection 10.0.7.113:57345 (1506 connections now open) 2016-03-13T08:54:04.497+0000 I NETWORK [conn4078706] end connection 10.0.7.113:57346 (1506 connections now open) 2016-03-13T08:54:04.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57367 #4078719 (1507 connections now open) 2016-03-13T08:54:04.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57368 #4078720 (1507 connections now open) 2016-03-13T08:54:05.670+0000 I NETWORK [conn4078707] end connection 10.0.216.90:55397 (1506 connections now open) 2016-03-13T08:54:05.670+0000 I NETWORK [conn4078708] end connection 10.0.216.90:55398 (1506 connections now open) 2016-03-13T08:54:05.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55418 #4078721 (1506 connections now open) 2016-03-13T08:54:05.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55417 #4078722 (1507 connections now open) 2016-03-13T08:54:05.696+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 677ms sequenceNumber: 16729 version: 30554|1||563ba74d869758a4542b9075 based on: 30553|1||563ba74d869758a4542b9075 2016-03-13T08:54:06.538+0000 I NETWORK [conn4078710] end connection 10.0.20.53:48574 (1506 connections now open) 2016-03-13T08:54:06.538+0000 I NETWORK [conn4078709] end connection 10.0.20.53:48573 (1506 connections now open) 2016-03-13T08:54:06.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48583 #4078723 (1506 connections now open) 2016-03-13T08:54:06.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48584 #4078724 (1507 connections now open) 2016-03-13T08:54:08.872+0000 I NETWORK [conn4078713] end connection 10.0.179.209:38415 (1506 connections now open) 2016-03-13T08:54:08.873+0000 I NETWORK [conn4078714] end connection 10.0.179.209:38414 (1505 connections now open) 2016-03-13T08:54:08.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38434 #4078725 (1506 connections now open) 2016-03-13T08:54:08.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38435 #4078726 (1507 connections now open) 2016-03-13T08:54:09.510+0000 I NETWORK [conn4078716] end connection 10.0.26.69:40412 (1506 connections now open) 2016-03-13T08:54:09.510+0000 I NETWORK [conn4078715] end connection 10.0.26.69:40411 (1506 connections now open) 2016-03-13T08:54:09.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40424 #4078727 (1506 connections now open) 2016-03-13T08:54:09.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40425 #4078728 (1507 connections now open) 2016-03-13T08:54:12.411+0000 I NETWORK [conn4078717] end connection 10.0.27.237:33658 (1506 connections now open) 2016-03-13T08:54:12.411+0000 I NETWORK [conn4078718] end connection 10.0.27.237:33659 (1506 connections now open) 2016-03-13T08:54:12.412+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33670 #4078729 (1506 connections now open) 2016-03-13T08:54:12.412+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33671 #4078730 (1507 connections now open) 2016-03-13T08:54:14.510+0000 I NETWORK [conn4078719] end connection 10.0.7.113:57367 (1506 connections now open) 2016-03-13T08:54:14.511+0000 I NETWORK [conn4078720] end connection 10.0.7.113:57368 (1505 connections now open) 2016-03-13T08:54:14.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57378 #4078731 (1506 connections now open) 2016-03-13T08:54:14.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57379 #4078732 (1507 connections now open) 2016-03-13T08:54:15.683+0000 I NETWORK [conn4078722] end connection 10.0.216.90:55417 (1506 connections now open) 2016-03-13T08:54:15.683+0000 I NETWORK [conn4078721] end connection 10.0.216.90:55418 (1506 connections now open) 2016-03-13T08:54:15.691+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55425 #4078733 (1506 connections now open) 2016-03-13T08:54:15.691+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55424 #4078734 (1507 connections now open) 2016-03-13T08:54:16.117+0000 I NETWORK [conn4064830] end connection 10.0.179.209:52461 (1506 connections now open) 2016-03-13T08:54:16.117+0000 I NETWORK [conn4072411] end connection 10.0.20.53:41976 (1505 connections now open) 2016-03-13T08:54:16.117+0000 I NETWORK [conn4070563] end connection 10.0.216.90:46648 (1505 connections now open) 2016-03-13T08:54:16.117+0000 I NETWORK [conn4070669] end connection 10.0.26.69:60036 (1503 connections now open) 2016-03-13T08:54:16.118+0000 I NETWORK [conn4071227] end connection 10.0.27.237:54272 (1502 connections now open) 2016-03-13T08:54:16.118+0000 I NETWORK [conn4066934] end connection 10.0.7.113:44805 (1501 connections now open) 2016-03-13T08:54:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38441 #4078735 (1503 connections now open) 2016-03-13T08:54:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48596 #4078736 (1503 connections now open) 2016-03-13T08:54:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55428 #4078737 (1504 connections now open) 2016-03-13T08:54:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40437 #4078738 (1505 connections now open) 2016-03-13T08:54:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57385 #4078739 (1506 connections now open) 2016-03-13T08:54:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33688 #4078740 (1507 connections now open) 2016-03-13T08:54:16.551+0000 I NETWORK [conn4078724] end connection 10.0.20.53:48584 (1506 connections now open) 2016-03-13T08:54:16.551+0000 I NETWORK [conn4078723] end connection 10.0.20.53:48583 (1505 connections now open) 2016-03-13T08:54:16.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48597 #4078741 (1506 connections now open) 2016-03-13T08:54:16.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48598 #4078742 (1507 connections now open) 2016-03-13T08:54:18.886+0000 I NETWORK [conn4078726] end connection 10.0.179.209:38435 (1506 connections now open) 2016-03-13T08:54:18.886+0000 I NETWORK [conn4078725] end connection 10.0.179.209:38434 (1506 connections now open) 2016-03-13T08:54:18.886+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38446 #4078743 (1506 connections now open) 2016-03-13T08:54:18.886+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38447 #4078744 (1507 connections now open) 2016-03-13T08:54:19.524+0000 I NETWORK [conn4078728] end connection 10.0.26.69:40425 (1506 connections now open) 2016-03-13T08:54:19.524+0000 I NETWORK [conn4078727] end connection 10.0.26.69:40424 (1506 connections now open) 2016-03-13T08:54:19.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40448 #4078745 (1506 connections now open) 2016-03-13T08:54:19.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40449 #4078746 (1507 connections now open) 2016-03-13T08:54:20.946+0000 I NETWORK [conn4078226] end connection 10.0.91.115:53701 (1506 connections now open) 2016-03-13T08:54:20.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:53998 #4078747 (1507 connections now open) 2016-03-13T08:54:21.004+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16730 version: 30555|1||563ba74d869758a4542b9075 based on: 30554|1||563ba74d869758a4542b9075 2016-03-13T08:54:21.783+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52abd18fa188537623cbe 2016-03-13T08:54:22.327+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:54:22.008+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:54:22.327+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:54:22.425+0000 I NETWORK [conn4078730] end connection 10.0.27.237:33671 (1506 connections now open) 2016-03-13T08:54:22.425+0000 I NETWORK [conn4078729] end connection 10.0.27.237:33670 (1506 connections now open) 2016-03-13T08:54:22.434+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33691 #4078748 (1506 connections now open) 2016-03-13T08:54:22.434+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33690 #4078749 (1507 connections now open) 2016-03-13T08:54:22.464+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52aa918fa188537623cbc 2016-03-13T08:54:22.602+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ab318fa188537623cbd 2016-03-13T08:54:24.345+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633eaaace48b40ea367394a')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633eaaace48b40ea367394a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ebc499bdd00ea5191cf8') }, shard: "shmydomain1", lastmod: Timestamp 30555000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:54:24.498+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30555|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633eaaace48b40ea367394a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ebc499bdd00ea5191cf8') }) shmydomain1 -> shmydomain3 2016-03-13T08:54:24.532+0000 I NETWORK [conn4078732] end connection 10.0.7.113:57379 (1506 connections now open) 2016-03-13T08:54:24.533+0000 I NETWORK [conn4078731] end connection 10.0.7.113:57378 (1505 connections now open) 2016-03-13T08:54:24.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57392 #4078750 (1506 connections now open) 2016-03-13T08:54:24.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57393 #4078751 (1507 connections now open) 2016-03-13T08:54:25.703+0000 I NETWORK [conn4078734] end connection 10.0.216.90:55424 (1506 connections now open) 2016-03-13T08:54:25.703+0000 I NETWORK [conn4078733] end connection 10.0.216.90:55425 (1506 connections now open) 2016-03-13T08:54:25.704+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55437 #4078752 (1506 connections now open) 2016-03-13T08:54:25.704+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55438 #4078753 (1507 connections now open) 2016-03-13T08:54:26.564+0000 I NETWORK [conn4078741] end connection 10.0.20.53:48597 (1506 connections now open) 2016-03-13T08:54:26.564+0000 I NETWORK [conn4078742] end connection 10.0.20.53:48598 (1506 connections now open) 2016-03-13T08:54:26.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48614 #4078754 (1506 connections now open) 2016-03-13T08:54:26.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48615 #4078755 (1507 connections now open) 2016-03-13T08:54:28.114+0000 I NETWORK [conn4076844] end connection 10.0.26.69:38426 (1506 connections now open) 2016-03-13T08:54:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40456 #4078756 (1507 connections now open) 2016-03-13T08:54:28.900+0000 I NETWORK [conn4078743] end connection 10.0.179.209:38446 (1506 connections now open) 2016-03-13T08:54:28.900+0000 I NETWORK [conn4078744] end connection 10.0.179.209:38447 (1506 connections now open) 2016-03-13T08:54:28.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38460 #4078757 (1506 connections now open) 2016-03-13T08:54:28.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38461 #4078758 (1507 connections now open) 2016-03-13T08:54:29.537+0000 I NETWORK [conn4078745] end connection 10.0.26.69:40448 (1506 connections now open) 2016-03-13T08:54:29.537+0000 I NETWORK [conn4078746] end connection 10.0.26.69:40449 (1506 connections now open) 2016-03-13T08:54:29.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40457 #4078759 (1507 connections now open) 2016-03-13T08:54:29.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40458 #4078760 (1507 connections now open) 2016-03-13T08:54:32.447+0000 I NETWORK [conn4078749] end connection 10.0.27.237:33690 (1506 connections now open) 2016-03-13T08:54:32.447+0000 I NETWORK [conn4078748] end connection 10.0.27.237:33691 (1506 connections now open) 2016-03-13T08:54:32.447+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33699 #4078761 (1506 connections now open) 2016-03-13T08:54:32.448+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33700 #4078762 (1507 connections now open) 2016-03-13T08:54:33.533+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16731 version: 30556|1||563ba74d869758a4542b9075 based on: 30555|1||563ba74d869758a4542b9075 2016-03-13T08:54:33.805+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:54:33.805+0000-56e52ac918fa188537623cbf", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859273805), what: "balancer.round", ns: "", details: { executionTimeMillis: 12350, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:54:33.988+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:54:34.554+0000 I NETWORK [conn4078751] end connection 10.0.7.113:57393 (1506 connections now open) 2016-03-13T08:54:34.554+0000 I NETWORK [conn4078750] end connection 10.0.7.113:57392 (1506 connections now open) 2016-03-13T08:54:34.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57412 #4078763 (1506 connections now open) 2016-03-13T08:54:34.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57413 #4078764 (1507 connections now open) 2016-03-13T08:54:35.320+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52acb18fa188537623cc0 2016-03-13T08:54:35.716+0000 I NETWORK [conn4078753] end connection 10.0.216.90:55438 (1506 connections now open) 2016-03-13T08:54:35.716+0000 I NETWORK [conn4078752] end connection 10.0.216.90:55437 (1505 connections now open) 2016-03-13T08:54:35.717+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55459 #4078765 (1506 connections now open) 2016-03-13T08:54:35.717+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55458 #4078766 (1507 connections now open) 2016-03-13T08:54:36.577+0000 I NETWORK [conn4078754] end connection 10.0.20.53:48614 (1506 connections now open) 2016-03-13T08:54:36.577+0000 I NETWORK [conn4078755] end connection 10.0.20.53:48615 (1506 connections now open) 2016-03-13T08:54:36.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48627 #4078767 (1506 connections now open) 2016-03-13T08:54:36.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48628 #4078768 (1507 connections now open) 2016-03-13T08:54:37.846+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633ebc499bdd00ea5191cf8')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ebc499bdd00ea5191cf8') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ecdc584ad10f212bee45') }, shard: "shmydomain1", lastmod: Timestamp 30556000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:54:38.000+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30556|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ebc499bdd00ea5191cf8') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ecdc584ad10f212bee45') }) shmydomain1 -> shmydomain3 2016-03-13T08:54:38.914+0000 I NETWORK [conn4078758] end connection 10.0.179.209:38461 (1506 connections now open) 2016-03-13T08:54:38.914+0000 I NETWORK [conn4078757] end connection 10.0.179.209:38460 (1506 connections now open) 2016-03-13T08:54:38.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38477 #4078769 (1506 connections now open) 2016-03-13T08:54:38.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38478 #4078770 (1507 connections now open) 2016-03-13T08:54:39.550+0000 I NETWORK [conn4078759] end connection 10.0.26.69:40457 (1506 connections now open) 2016-03-13T08:54:39.550+0000 I NETWORK [conn4078760] end connection 10.0.26.69:40458 (1506 connections now open) 2016-03-13T08:54:39.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40467 #4078771 (1506 connections now open) 2016-03-13T08:54:39.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40468 #4078772 (1507 connections now open) 2016-03-13T08:54:42.460+0000 I NETWORK [conn4078761] end connection 10.0.27.237:33699 (1506 connections now open) 2016-03-13T08:54:42.460+0000 I NETWORK [conn4078762] end connection 10.0.27.237:33700 (1506 connections now open) 2016-03-13T08:54:42.460+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33710 #4078773 (1506 connections now open) 2016-03-13T08:54:42.461+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33711 #4078774 (1507 connections now open) 2016-03-13T08:54:44.568+0000 I NETWORK [conn4078763] end connection 10.0.7.113:57412 (1506 connections now open) 2016-03-13T08:54:44.568+0000 I NETWORK [conn4078764] end connection 10.0.7.113:57413 (1506 connections now open) 2016-03-13T08:54:44.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57419 #4078775 (1506 connections now open) 2016-03-13T08:54:44.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57418 #4078776 (1507 connections now open) 2016-03-13T08:54:45.729+0000 I NETWORK [conn4078766] end connection 10.0.216.90:55458 (1506 connections now open) 2016-03-13T08:54:45.729+0000 I NETWORK [conn4078765] end connection 10.0.216.90:55459 (1506 connections now open) 2016-03-13T08:54:45.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55466 #4078777 (1506 connections now open) 2016-03-13T08:54:45.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55465 #4078778 (1507 connections now open) 2016-03-13T08:54:46.117+0000 I NETWORK [conn4068986] end connection 10.0.179.209:56831 (1506 connections now open) 2016-03-13T08:54:46.117+0000 I NETWORK [conn4067690] end connection 10.0.216.90:43585 (1505 connections now open) 2016-03-13T08:54:46.118+0000 I NETWORK [conn4076571] end connection 10.0.26.69:38138 (1504 connections now open) 2016-03-13T08:54:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38485 #4078779 (1505 connections now open) 2016-03-13T08:54:46.118+0000 I NETWORK [conn4068662] end connection 10.0.7.113:46605 (1504 connections now open) 2016-03-13T08:54:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55469 #4078780 (1506 connections now open) 2016-03-13T08:54:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40475 #4078781 (1506 connections now open) 2016-03-13T08:54:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57425 #4078782 (1507 connections now open) 2016-03-13T08:54:46.591+0000 I NETWORK [conn4078767] end connection 10.0.20.53:48627 (1506 connections now open) 2016-03-13T08:54:46.591+0000 I NETWORK [conn4078768] end connection 10.0.20.53:48628 (1506 connections now open) 2016-03-13T08:54:46.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48643 #4078783 (1506 connections now open) 2016-03-13T08:54:46.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48642 #4078784 (1507 connections now open) 2016-03-13T08:54:47.837+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16732 version: 30557|1||563ba74d869758a4542b9075 based on: 30556|1||563ba74d869758a4542b9075 2016-03-13T08:54:48.027+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:54:48.027+0000-56e52ad818fa188537623cc1", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859288027), what: "balancer.round", ns: "", details: { executionTimeMillis: 13003, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:54:48.150+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:54:48.927+0000 I NETWORK [conn4078770] end connection 10.0.179.209:38478 (1506 connections now open) 2016-03-13T08:54:48.927+0000 I NETWORK [conn4078769] end connection 10.0.179.209:38477 (1506 connections now open) 2016-03-13T08:54:48.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38487 #4078785 (1506 connections now open) 2016-03-13T08:54:48.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38488 #4078786 (1507 connections now open) 2016-03-13T08:54:49.525+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52ad918fa188537623cc2 2016-03-13T08:54:49.564+0000 I NETWORK [conn4078771] end connection 10.0.26.69:40467 (1506 connections now open) 2016-03-13T08:54:49.564+0000 I NETWORK [conn4078772] end connection 10.0.26.69:40468 (1506 connections now open) 2016-03-13T08:54:49.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40486 #4078787 (1506 connections now open) 2016-03-13T08:54:49.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40487 #4078788 (1507 connections now open) 2016-03-13T08:54:52.055+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633ecdc584ad10f212bee45')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ecdc584ad10f212bee45') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633edf5584ad10f212c07aa') }, shard: "shmydomain1", lastmod: Timestamp 30557000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:54:52.205+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30557|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ecdc584ad10f212bee45') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633edf5584ad10f212c07aa') }) shmydomain1 -> shmydomain3 2016-03-13T08:54:52.473+0000 I NETWORK [conn4078773] end connection 10.0.27.237:33710 (1506 connections now open) 2016-03-13T08:54:52.473+0000 I NETWORK [conn4078774] end connection 10.0.27.237:33711 (1506 connections now open) 2016-03-13T08:54:52.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33730 #4078789 (1506 connections now open) 2016-03-13T08:54:52.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33729 #4078790 (1507 connections now open) 2016-03-13T08:54:52.886+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:54:52.602+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:54:54.581+0000 I NETWORK [conn4078776] end connection 10.0.7.113:57418 (1506 connections now open) 2016-03-13T08:54:54.581+0000 I NETWORK [conn4078775] end connection 10.0.7.113:57419 (1506 connections now open) 2016-03-13T08:54:54.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57432 #4078791 (1506 connections now open) 2016-03-13T08:54:54.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57433 #4078792 (1507 connections now open) 2016-03-13T08:54:55.742+0000 I NETWORK [conn4078778] end connection 10.0.216.90:55465 (1506 connections now open) 2016-03-13T08:54:55.742+0000 I NETWORK [conn4078777] end connection 10.0.216.90:55466 (1506 connections now open) 2016-03-13T08:54:55.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55478 #4078793 (1506 connections now open) 2016-03-13T08:54:55.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55479 #4078794 (1507 connections now open) 2016-03-13T08:54:56.605+0000 I NETWORK [conn4078783] end connection 10.0.20.53:48643 (1506 connections now open) 2016-03-13T08:54:56.605+0000 I NETWORK [conn4078784] end connection 10.0.20.53:48642 (1506 connections now open) 2016-03-13T08:54:56.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48662 #4078795 (1506 connections now open) 2016-03-13T08:54:56.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48661 #4078796 (1507 connections now open) 2016-03-13T08:54:58.113+0000 I NETWORK [conn4074944] end connection 10.0.179.209:34613 (1506 connections now open) 2016-03-13T08:54:58.113+0000 I NETWORK [conn4073179] end connection 10.0.20.53:42804 (1505 connections now open) 2016-03-13T08:54:58.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48665 #4078797 (1506 connections now open) 2016-03-13T08:54:58.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38499 #4078798 (1507 connections now open) 2016-03-13T08:54:58.940+0000 I NETWORK [conn4078785] end connection 10.0.179.209:38487 (1506 connections now open) 2016-03-13T08:54:58.940+0000 I NETWORK [conn4078786] end connection 10.0.179.209:38488 (1506 connections now open) 2016-03-13T08:54:58.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38500 #4078799 (1506 connections now open) 2016-03-13T08:54:58.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38501 #4078800 (1507 connections now open) 2016-03-13T08:54:59.577+0000 I NETWORK [conn4078788] end connection 10.0.26.69:40487 (1506 connections now open) 2016-03-13T08:54:59.577+0000 I NETWORK [conn4078787] end connection 10.0.26.69:40486 (1506 connections now open) 2016-03-13T08:54:59.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40494 #4078801 (1506 connections now open) 2016-03-13T08:54:59.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40495 #4078802 (1507 connections now open) 2016-03-13T08:55:01.149+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 675ms sequenceNumber: 16733 version: 30558|1||563ba74d869758a4542b9075 based on: 30557|1||563ba74d869758a4542b9075 2016-03-13T08:55:01.425+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:55:01.425+0000-56e52ae518fa188537623cc3", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859301425), what: "balancer.round", ns: "", details: { executionTimeMillis: 12163, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:55:01.520+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:55:02.486+0000 I NETWORK [conn4078790] end connection 10.0.27.237:33729 (1506 connections now open) 2016-03-13T08:55:02.486+0000 I NETWORK [conn4078789] end connection 10.0.27.237:33730 (1506 connections now open) 2016-03-13T08:55:02.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33740 #4078803 (1506 connections now open) 2016-03-13T08:55:02.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33741 #4078804 (1507 connections now open) 2016-03-13T08:55:02.913+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52ae618fa188537623cc4 2016-03-13T08:55:04.594+0000 I NETWORK [conn4078792] end connection 10.0.7.113:57433 (1506 connections now open) 2016-03-13T08:55:04.594+0000 I NETWORK [conn4078791] end connection 10.0.7.113:57432 (1506 connections now open) 2016-03-13T08:55:04.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57453 #4078805 (1506 connections now open) 2016-03-13T08:55:04.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57454 #4078806 (1507 connections now open) 2016-03-13T08:55:05.730+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633edf5584ad10f212c07aa')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633edf5584ad10f212c07aa') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ef0099bdd00ea5196936') }, shard: "shmydomain1", lastmod: Timestamp 30558000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:55:05.755+0000 I NETWORK [conn4078793] end connection 10.0.216.90:55478 (1506 connections now open) 2016-03-13T08:55:05.755+0000 I NETWORK [conn4078794] end connection 10.0.216.90:55479 (1506 connections now open) 2016-03-13T08:55:05.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55502 #4078807 (1506 connections now open) 2016-03-13T08:55:05.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55501 #4078808 (1507 connections now open) 2016-03-13T08:55:05.882+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30558|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633edf5584ad10f212c07aa') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ef0099bdd00ea5196936') }) shmydomain1 -> shmydomain3 2016-03-13T08:55:06.619+0000 I NETWORK [conn4078796] end connection 10.0.20.53:48661 (1506 connections now open) 2016-03-13T08:55:06.619+0000 I NETWORK [conn4078795] end connection 10.0.20.53:48662 (1506 connections now open) 2016-03-13T08:55:06.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48673 #4078809 (1506 connections now open) 2016-03-13T08:55:06.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48674 #4078810 (1507 connections now open) 2016-03-13T08:55:08.954+0000 I NETWORK [conn4078799] end connection 10.0.179.209:38500 (1506 connections now open) 2016-03-13T08:55:08.954+0000 I NETWORK [conn4078800] end connection 10.0.179.209:38501 (1506 connections now open) 2016-03-13T08:55:08.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38520 #4078811 (1506 connections now open) 2016-03-13T08:55:08.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38521 #4078812 (1507 connections now open) 2016-03-13T08:55:09.590+0000 I NETWORK [conn4078801] end connection 10.0.26.69:40494 (1506 connections now open) 2016-03-13T08:55:09.590+0000 I NETWORK [conn4078802] end connection 10.0.26.69:40495 (1505 connections now open) 2016-03-13T08:55:09.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40507 #4078813 (1506 connections now open) 2016-03-13T08:55:09.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40508 #4078814 (1507 connections now open) 2016-03-13T08:55:12.499+0000 I NETWORK [conn4078803] end connection 10.0.27.237:33740 (1506 connections now open) 2016-03-13T08:55:12.499+0000 I NETWORK [conn4078804] end connection 10.0.27.237:33741 (1506 connections now open) 2016-03-13T08:55:12.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33754 #4078815 (1506 connections now open) 2016-03-13T08:55:12.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33755 #4078816 (1507 connections now open) 2016-03-13T08:55:14.608+0000 I NETWORK [conn4078805] end connection 10.0.7.113:57453 (1506 connections now open) 2016-03-13T08:55:14.608+0000 I NETWORK [conn4078806] end connection 10.0.7.113:57454 (1505 connections now open) 2016-03-13T08:55:14.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57463 #4078817 (1506 connections now open) 2016-03-13T08:55:14.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57464 #4078818 (1507 connections now open) 2016-03-13T08:55:14.749+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 674ms sequenceNumber: 16734 version: 30559|1||563ba74d869758a4542b9075 based on: 30558|1||563ba74d869758a4542b9075 2016-03-13T08:55:15.021+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:55:15.021+0000-56e52af318fa188537623cc5", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859315021), what: "balancer.round", ns: "", details: { executionTimeMillis: 12467, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:55:15.204+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:55:15.768+0000 I NETWORK [conn4078807] end connection 10.0.216.90:55502 (1506 connections now open) 2016-03-13T08:55:15.768+0000 I NETWORK [conn4078808] end connection 10.0.216.90:55501 (1506 connections now open) 2016-03-13T08:55:15.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55511 #4078819 (1506 connections now open) 2016-03-13T08:55:15.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55510 #4078820 (1507 connections now open) 2016-03-13T08:55:16.117+0000 I NETWORK [conn4070225] end connection 10.0.216.90:46282 (1506 connections now open) 2016-03-13T08:55:16.117+0000 I NETWORK [conn4070920] end connection 10.0.20.53:40450 (1506 connections now open) 2016-03-13T08:55:16.118+0000 I NETWORK [conn4072912] end connection 10.0.27.237:56086 (1504 connections now open) 2016-03-13T08:55:16.118+0000 I NETWORK [conn4068960] end connection 10.0.7.113:46926 (1503 connections now open) 2016-03-13T08:55:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48681 #4078821 (1504 connections now open) 2016-03-13T08:55:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33764 #4078822 (1505 connections now open) 2016-03-13T08:55:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57469 #4078823 (1506 connections now open) 2016-03-13T08:55:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55514 #4078824 (1507 connections now open) 2016-03-13T08:55:16.537+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52af418fa188537623cc6 2016-03-13T08:55:16.633+0000 I NETWORK [conn4078809] end connection 10.0.20.53:48673 (1506 connections now open) 2016-03-13T08:55:16.633+0000 I NETWORK [conn4078810] end connection 10.0.20.53:48674 (1506 connections now open) 2016-03-13T08:55:16.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48688 #4078825 (1506 connections now open) 2016-03-13T08:55:16.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48687 #4078826 (1507 connections now open) 2016-03-13T08:55:18.977+0000 I NETWORK [conn4078812] end connection 10.0.179.209:38521 (1506 connections now open) 2016-03-13T08:55:18.977+0000 I NETWORK [conn4078811] end connection 10.0.179.209:38520 (1506 connections now open) 2016-03-13T08:55:18.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38535 #4078827 (1506 connections now open) 2016-03-13T08:55:18.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38534 #4078828 (1507 connections now open) 2016-03-13T08:55:19.080+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633ef0099bdd00ea5196936')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ef0099bdd00ea5196936') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f011584ad10f212c3f27') }, shard: "shmydomain1", lastmod: Timestamp 30559000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:55:19.234+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30559|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ef0099bdd00ea5196936') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f011584ad10f212c3f27') }) shmydomain1 -> shmydomain3 2016-03-13T08:55:19.603+0000 I NETWORK [conn4078813] end connection 10.0.26.69:40507 (1506 connections now open) 2016-03-13T08:55:19.603+0000 I NETWORK [conn4078814] end connection 10.0.26.69:40508 (1506 connections now open) 2016-03-13T08:55:19.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40531 #4078829 (1506 connections now open) 2016-03-13T08:55:19.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40530 #4078830 (1507 connections now open) 2016-03-13T08:55:22.512+0000 I NETWORK [conn4078816] end connection 10.0.27.237:33755 (1506 connections now open) 2016-03-13T08:55:22.512+0000 I NETWORK [conn4078815] end connection 10.0.27.237:33754 (1506 connections now open) 2016-03-13T08:55:22.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33776 #4078831 (1506 connections now open) 2016-03-13T08:55:22.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33777 #4078832 (1507 connections now open) 2016-03-13T08:55:23.146+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:55:22.886+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:55:24.621+0000 I NETWORK [conn4078818] end connection 10.0.7.113:57464 (1506 connections now open) 2016-03-13T08:55:24.621+0000 I NETWORK [conn4078817] end connection 10.0.7.113:57463 (1506 connections now open) 2016-03-13T08:55:24.631+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57476 #4078833 (1506 connections now open) 2016-03-13T08:55:24.631+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57477 #4078834 (1507 connections now open) 2016-03-13T08:55:25.782+0000 I NETWORK [conn4078820] end connection 10.0.216.90:55510 (1506 connections now open) 2016-03-13T08:55:25.782+0000 I NETWORK [conn4078819] end connection 10.0.216.90:55511 (1506 connections now open) 2016-03-13T08:55:25.782+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55523 #4078835 (1506 connections now open) 2016-03-13T08:55:25.782+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55524 #4078836 (1507 connections now open) 2016-03-13T08:55:26.646+0000 I NETWORK [conn4078826] end connection 10.0.20.53:48687 (1506 connections now open) 2016-03-13T08:55:26.646+0000 I NETWORK [conn4078825] end connection 10.0.20.53:48688 (1506 connections now open) 2016-03-13T08:55:26.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48707 #4078837 (1506 connections now open) 2016-03-13T08:55:26.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48706 #4078838 (1507 connections now open) 2016-03-13T08:55:28.087+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16735 version: 30560|1||563ba74d869758a4542b9075 based on: 30559|1||563ba74d869758a4542b9075 2016-03-13T08:55:28.113+0000 I NETWORK [conn4066739] end connection 10.0.20.53:36091 (1506 connections now open) 2016-03-13T08:55:28.113+0000 I NETWORK [conn4068774] end connection 10.0.216.90:44734 (1505 connections now open) 2016-03-13T08:55:28.113+0000 I NETWORK [conn4070588] end connection 10.0.7.113:48696 (1504 connections now open) 2016-03-13T08:55:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48710 #4078839 (1505 connections now open) 2016-03-13T08:55:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55527 #4078840 (1506 connections now open) 2016-03-13T08:55:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57483 #4078841 (1507 connections now open) 2016-03-13T08:55:28.358+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:55:28.358+0000-56e52b0018fa188537623cc7", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859328358), what: "balancer.round", ns: "", details: { executionTimeMillis: 12122, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:55:28.589+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:55:28.990+0000 I NETWORK [conn4078828] end connection 10.0.179.209:38534 (1506 connections now open) 2016-03-13T08:55:28.990+0000 I NETWORK [conn4078827] end connection 10.0.179.209:38535 (1506 connections now open) 2016-03-13T08:55:28.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38548 #4078842 (1506 connections now open) 2016-03-13T08:55:28.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38549 #4078843 (1507 connections now open) 2016-03-13T08:55:29.625+0000 I NETWORK [conn4078830] end connection 10.0.26.69:40530 (1506 connections now open) 2016-03-13T08:55:29.625+0000 I NETWORK [conn4078829] end connection 10.0.26.69:40531 (1506 connections now open) 2016-03-13T08:55:29.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40539 #4078844 (1506 connections now open) 2016-03-13T08:55:29.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40540 #4078845 (1507 connections now open) 2016-03-13T08:55:29.926+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52b0118fa188537623cc8 2016-03-13T08:55:32.477+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633f011584ad10f212c3f27')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f011584ad10f212c3f27') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f12f7bc3500e94580fe6') }, shard: "shmydomain1", lastmod: Timestamp 30560000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:55:32.534+0000 I NETWORK [conn4078832] end connection 10.0.27.237:33777 (1506 connections now open) 2016-03-13T08:55:32.534+0000 I NETWORK [conn4078831] end connection 10.0.27.237:33776 (1506 connections now open) 2016-03-13T08:55:32.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33787 #4078846 (1506 connections now open) 2016-03-13T08:55:32.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33786 #4078847 (1507 connections now open) 2016-03-13T08:55:32.633+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30560|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f011584ad10f212c3f27') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f12f7bc3500e94580fe6') }) shmydomain1 -> shmydomain3 2016-03-13T08:55:34.644+0000 I NETWORK [conn4078834] end connection 10.0.7.113:57477 (1506 connections now open) 2016-03-13T08:55:34.644+0000 I NETWORK [conn4078833] end connection 10.0.7.113:57476 (1506 connections now open) 2016-03-13T08:55:34.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57496 #4078848 (1507 connections now open) 2016-03-13T08:55:34.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57495 #4078849 (1507 connections now open) 2016-03-13T08:55:35.795+0000 I NETWORK [conn4078836] end connection 10.0.216.90:55524 (1506 connections now open) 2016-03-13T08:55:35.795+0000 I NETWORK [conn4078835] end connection 10.0.216.90:55523 (1506 connections now open) 2016-03-13T08:55:35.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55545 #4078850 (1506 connections now open) 2016-03-13T08:55:35.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55546 #4078851 (1507 connections now open) 2016-03-13T08:55:36.660+0000 I NETWORK [conn4078838] end connection 10.0.20.53:48706 (1506 connections now open) 2016-03-13T08:55:36.660+0000 I NETWORK [conn4078837] end connection 10.0.20.53:48707 (1506 connections now open) 2016-03-13T08:55:36.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48719 #4078852 (1506 connections now open) 2016-03-13T08:55:36.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48720 #4078853 (1507 connections now open) 2016-03-13T08:55:39.003+0000 I NETWORK [conn4078842] end connection 10.0.179.209:38548 (1506 connections now open) 2016-03-13T08:55:39.003+0000 I NETWORK [conn4078843] end connection 10.0.179.209:38549 (1506 connections now open) 2016-03-13T08:55:39.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38566 #4078854 (1506 connections now open) 2016-03-13T08:55:39.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38567 #4078855 (1507 connections now open) 2016-03-13T08:55:39.639+0000 I NETWORK [conn4078844] end connection 10.0.26.69:40539 (1506 connections now open) 2016-03-13T08:55:39.639+0000 I NETWORK [conn4078845] end connection 10.0.26.69:40540 (1506 connections now open) 2016-03-13T08:55:39.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40551 #4078856 (1506 connections now open) 2016-03-13T08:55:39.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40552 #4078857 (1507 connections now open) 2016-03-13T08:55:41.529+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 679ms sequenceNumber: 16736 version: 30561|1||563ba74d869758a4542b9075 based on: 30560|1||563ba74d869758a4542b9075 2016-03-13T08:55:41.801+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:55:41.801+0000-56e52b0d18fa188537623cc9", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859341801), what: "balancer.round", ns: "", details: { executionTimeMillis: 12176, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:55:41.944+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:55:42.547+0000 I NETWORK [conn4078846] end connection 10.0.27.237:33787 (1506 connections now open) 2016-03-13T08:55:42.547+0000 I NETWORK [conn4078847] end connection 10.0.27.237:33786 (1506 connections now open) 2016-03-13T08:55:42.548+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33800 #4078858 (1506 connections now open) 2016-03-13T08:55:42.548+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33799 #4078859 (1507 connections now open) 2016-03-13T08:55:43.311+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52b0f18fa188537623cca 2016-03-13T08:55:44.657+0000 I NETWORK [conn4078849] end connection 10.0.7.113:57495 (1506 connections now open) 2016-03-13T08:55:44.657+0000 I NETWORK [conn4078848] end connection 10.0.7.113:57496 (1506 connections now open) 2016-03-13T08:55:44.657+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57503 #4078860 (1506 connections now open) 2016-03-13T08:55:44.657+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57504 #4078861 (1507 connections now open) 2016-03-13T08:55:45.808+0000 I NETWORK [conn4078851] end connection 10.0.216.90:55546 (1506 connections now open) 2016-03-13T08:55:45.808+0000 I NETWORK [conn4078850] end connection 10.0.216.90:55545 (1506 connections now open) 2016-03-13T08:55:45.808+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55553 #4078862 (1506 connections now open) 2016-03-13T08:55:45.808+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55554 #4078863 (1507 connections now open) 2016-03-13T08:55:45.852+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633f12f7bc3500e94580fe6')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f12f7bc3500e94580fe6') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f244584ad10f212c76e8') }, shard: "shmydomain1", lastmod: Timestamp 30561000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:55:46.010+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30561|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f12f7bc3500e94580fe6') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f244584ad10f212c76e8') }) shmydomain1 -> shmydomain3 2016-03-13T08:55:46.117+0000 I NETWORK [conn4068732] end connection 10.0.179.209:56577 (1506 connections now open) 2016-03-13T08:55:46.117+0000 I NETWORK [conn4070311] end connection 10.0.216.90:46371 (1505 connections now open) 2016-03-13T08:55:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38573 #4078864 (1506 connections now open) 2016-03-13T08:55:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55557 #4078865 (1507 connections now open) 2016-03-13T08:55:46.118+0000 I NETWORK [conn4072748] end connection 10.0.26.69:34027 (1506 connections now open) 2016-03-13T08:55:46.118+0000 I NETWORK [conn4068452] end connection 10.0.7.113:46380 (1505 connections now open) 2016-03-13T08:55:46.118+0000 I NETWORK [conn4070716] end connection 10.0.27.237:53729 (1506 connections now open) 2016-03-13T08:55:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40560 #4078866 (1505 connections now open) 2016-03-13T08:55:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57510 #4078867 (1506 connections now open) 2016-03-13T08:55:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33807 #4078868 (1507 connections now open) 2016-03-13T08:55:46.673+0000 I NETWORK [conn4078853] end connection 10.0.20.53:48720 (1506 connections now open) 2016-03-13T08:55:46.673+0000 I NETWORK [conn4078852] end connection 10.0.20.53:48719 (1506 connections now open) 2016-03-13T08:55:46.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48734 #4078869 (1507 connections now open) 2016-03-13T08:55:46.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48735 #4078870 (1507 connections now open) 2016-03-13T08:55:49.016+0000 I NETWORK [conn4078854] end connection 10.0.179.209:38566 (1506 connections now open) 2016-03-13T08:55:49.016+0000 I NETWORK [conn4078855] end connection 10.0.179.209:38567 (1505 connections now open) 2016-03-13T08:55:49.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38576 #4078871 (1506 connections now open) 2016-03-13T08:55:49.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38577 #4078872 (1507 connections now open) 2016-03-13T08:55:49.652+0000 I NETWORK [conn4078857] end connection 10.0.26.69:40552 (1506 connections now open) 2016-03-13T08:55:49.652+0000 I NETWORK [conn4078856] end connection 10.0.26.69:40551 (1506 connections now open) 2016-03-13T08:55:49.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40571 #4078873 (1506 connections now open) 2016-03-13T08:55:49.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40572 #4078874 (1507 connections now open) 2016-03-13T08:55:52.560+0000 I NETWORK [conn4078859] end connection 10.0.27.237:33799 (1506 connections now open) 2016-03-13T08:55:52.560+0000 I NETWORK [conn4078858] end connection 10.0.27.237:33800 (1506 connections now open) 2016-03-13T08:55:52.561+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33820 #4078875 (1506 connections now open) 2016-03-13T08:55:52.561+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33819 #4078876 (1507 connections now open) 2016-03-13T08:55:53.352+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:55:53.146+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:55:54.670+0000 I NETWORK [conn4078861] end connection 10.0.7.113:57504 (1506 connections now open) 2016-03-13T08:55:54.670+0000 I NETWORK [conn4078860] end connection 10.0.7.113:57503 (1506 connections now open) 2016-03-13T08:55:54.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57518 #4078877 (1506 connections now open) 2016-03-13T08:55:54.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57517 #4078878 (1507 connections now open) 2016-03-13T08:55:55.821+0000 I NETWORK [conn4078862] end connection 10.0.216.90:55553 (1506 connections now open) 2016-03-13T08:55:55.821+0000 I NETWORK [conn4078863] end connection 10.0.216.90:55554 (1506 connections now open) 2016-03-13T08:55:55.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55566 #4078879 (1506 connections now open) 2016-03-13T08:55:55.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55567 #4078880 (1507 connections now open) 2016-03-13T08:55:56.687+0000 I NETWORK [conn4078869] end connection 10.0.20.53:48734 (1506 connections now open) 2016-03-13T08:55:56.687+0000 I NETWORK [conn4078870] end connection 10.0.20.53:48735 (1506 connections now open) 2016-03-13T08:55:56.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48750 #4078881 (1506 connections now open) 2016-03-13T08:55:56.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48751 #4078882 (1507 connections now open) 2016-03-13T08:55:58.113+0000 I NETWORK [conn4069928] end connection 10.0.20.53:39412 (1506 connections now open) 2016-03-13T08:55:58.113+0000 I NETWORK [conn4069470] end connection 10.0.26.69:58728 (1506 connections now open) 2016-03-13T08:55:58.113+0000 I NETWORK [conn4072256] end connection 10.0.179.209:60103 (1506 connections now open) 2016-03-13T08:55:58.113+0000 I NETWORK [conn4075576] end connection 10.0.7.113:54022 (1505 connections now open) 2016-03-13T08:55:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40579 #4078883 (1507 connections now open) 2016-03-13T08:55:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57525 #4078884 (1505 connections now open) 2016-03-13T08:55:58.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48754 #4078885 (1506 connections now open) 2016-03-13T08:55:58.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38588 #4078886 (1507 connections now open) 2016-03-13T08:55:58.192+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16737 version: 30562|1||563ba74d869758a4542b9075 based on: 30561|1||563ba74d869758a4542b9075 2016-03-13T08:55:58.463+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:55:58.463+0000-56e52b1e18fa188537623ccb", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859358463), what: "balancer.round", ns: "", details: { executionTimeMillis: 15483, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:55:58.735+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:55:59.029+0000 I NETWORK [conn4078871] end connection 10.0.179.209:38576 (1506 connections now open) 2016-03-13T08:55:59.029+0000 I NETWORK [conn4078872] end connection 10.0.179.209:38577 (1505 connections now open) 2016-03-13T08:55:59.030+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38589 #4078887 (1506 connections now open) 2016-03-13T08:55:59.030+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38590 #4078888 (1507 connections now open) 2016-03-13T08:55:59.665+0000 I NETWORK [conn4078874] end connection 10.0.26.69:40572 (1506 connections now open) 2016-03-13T08:55:59.665+0000 I NETWORK [conn4078873] end connection 10.0.26.69:40571 (1506 connections now open) 2016-03-13T08:55:59.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40582 #4078889 (1506 connections now open) 2016-03-13T08:55:59.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40581 #4078890 (1507 connections now open) 2016-03-13T08:56:00.175+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52b1f18fa188537623ccc 2016-03-13T08:56:02.573+0000 I NETWORK [conn4078876] end connection 10.0.27.237:33819 (1506 connections now open) 2016-03-13T08:56:02.573+0000 I NETWORK [conn4078875] end connection 10.0.27.237:33820 (1505 connections now open) 2016-03-13T08:56:02.574+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33832 #4078891 (1506 connections now open) 2016-03-13T08:56:02.574+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33833 #4078892 (1507 connections now open) 2016-03-13T08:56:03.179+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633f244584ad10f212c76e8')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f244584ad10f212c76e8') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f354ce48b40ea18ddbfa') }, shard: "shmydomain1", lastmod: Timestamp 30562000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:56:03.335+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30562|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f244584ad10f212c76e8') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f354ce48b40ea18ddbfa') }) shmydomain1 -> shmydomain3 2016-03-13T08:56:04.684+0000 I NETWORK [conn4078878] end connection 10.0.7.113:57517 (1506 connections now open) 2016-03-13T08:56:04.684+0000 I NETWORK [conn4078877] end connection 10.0.7.113:57518 (1506 connections now open) 2016-03-13T08:56:04.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57540 #4078893 (1506 connections now open) 2016-03-13T08:56:04.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57539 #4078894 (1507 connections now open) 2016-03-13T08:56:05.834+0000 I NETWORK [conn4078880] end connection 10.0.216.90:55567 (1506 connections now open) 2016-03-13T08:56:05.834+0000 I NETWORK [conn4078879] end connection 10.0.216.90:55566 (1506 connections now open) 2016-03-13T08:56:05.843+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55589 #4078895 (1506 connections now open) 2016-03-13T08:56:05.843+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55588 #4078896 (1507 connections now open) 2016-03-13T08:56:06.700+0000 I NETWORK [conn4078881] end connection 10.0.20.53:48750 (1506 connections now open) 2016-03-13T08:56:06.700+0000 I NETWORK [conn4078882] end connection 10.0.20.53:48751 (1506 connections now open) 2016-03-13T08:56:06.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48762 #4078897 (1506 connections now open) 2016-03-13T08:56:06.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48763 #4078898 (1507 connections now open) 2016-03-13T08:56:09.042+0000 I NETWORK [conn4078887] end connection 10.0.179.209:38589 (1506 connections now open) 2016-03-13T08:56:09.042+0000 I NETWORK [conn4078888] end connection 10.0.179.209:38590 (1506 connections now open) 2016-03-13T08:56:09.043+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38608 #4078899 (1506 connections now open) 2016-03-13T08:56:09.043+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38609 #4078900 (1507 connections now open) 2016-03-13T08:56:09.678+0000 I NETWORK [conn4078889] end connection 10.0.26.69:40582 (1506 connections now open) 2016-03-13T08:56:09.678+0000 I NETWORK [conn4078890] end connection 10.0.26.69:40581 (1506 connections now open) 2016-03-13T08:56:09.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40593 #4078901 (1506 connections now open) 2016-03-13T08:56:09.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40594 #4078902 (1507 connections now open) 2016-03-13T08:56:12.586+0000 I NETWORK [conn4078892] end connection 10.0.27.237:33833 (1506 connections now open) 2016-03-13T08:56:12.586+0000 I NETWORK [conn4078891] end connection 10.0.27.237:33832 (1506 connections now open) 2016-03-13T08:56:12.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33850 #4078903 (1506 connections now open) 2016-03-13T08:56:12.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33849 #4078904 (1507 connections now open) 2016-03-13T08:56:13.173+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16738 version: 30563|1||563ba74d869758a4542b9075 based on: 30562|1||563ba74d869758a4542b9075 2016-03-13T08:56:13.445+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:56:13.445+0000-56e52b2d18fa188537623ccd", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859373445), what: "balancer.round", ns: "", details: { executionTimeMillis: 13676, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:56:13.627+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:56:14.697+0000 I NETWORK [conn4078893] end connection 10.0.7.113:57540 (1506 connections now open) 2016-03-13T08:56:14.697+0000 I NETWORK [conn4078894] end connection 10.0.7.113:57539 (1506 connections now open) 2016-03-13T08:56:14.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57550 #4078905 (1506 connections now open) 2016-03-13T08:56:14.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57551 #4078906 (1507 connections now open) 2016-03-13T08:56:14.958+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52b2e18fa188537623cce 2016-03-13T08:56:15.855+0000 I NETWORK [conn4078895] end connection 10.0.216.90:55589 (1506 connections now open) 2016-03-13T08:56:15.855+0000 I NETWORK [conn4078896] end connection 10.0.216.90:55588 (1506 connections now open) 2016-03-13T08:56:15.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55595 #4078907 (1506 connections now open) 2016-03-13T08:56:15.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55596 #4078908 (1507 connections now open) 2016-03-13T08:56:16.117+0000 I NETWORK [conn4070288] end connection 10.0.179.209:58131 (1506 connections now open) 2016-03-13T08:56:16.118+0000 I NETWORK [conn4070353] end connection 10.0.216.90:46417 (1505 connections now open) 2016-03-13T08:56:16.118+0000 I NETWORK [conn4071311] end connection 10.0.26.69:60720 (1505 connections now open) 2016-03-13T08:56:16.118+0000 I NETWORK [conn4066887] end connection 10.0.7.113:44757 (1503 connections now open) 2016-03-13T08:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38615 #4078909 (1505 connections now open) 2016-03-13T08:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40604 #4078910 (1505 connections now open) 2016-03-13T08:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57557 #4078911 (1506 connections now open) 2016-03-13T08:56:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55598 #4078912 (1507 connections now open) 2016-03-13T08:56:16.721+0000 I NETWORK [conn4078898] end connection 10.0.20.53:48763 (1506 connections now open) 2016-03-13T08:56:16.721+0000 I NETWORK [conn4078897] end connection 10.0.20.53:48762 (1506 connections now open) 2016-03-13T08:56:16.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48770 #4078913 (1506 connections now open) 2016-03-13T08:56:16.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48771 #4078914 (1507 connections now open) 2016-03-13T08:56:17.496+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633f354ce48b40ea18ddbfa')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f354ce48b40ea18ddbfa') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f45e99bdd00ea519ea10') }, shard: "shmydomain1", lastmod: Timestamp 30563000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:56:17.650+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30563|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f354ce48b40ea18ddbfa') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633f45e99bdd00ea519ea10') }) shmydomain1 -> shmydomain3 2016-03-13T08:56:19.055+0000 I NETWORK [conn4078899] end connection 10.0.179.209:38608 (1506 connections now open) 2016-03-13T08:56:19.055+0000 I NETWORK [conn4078900] end connection 10.0.179.209:38609 (1506 connections now open) 2016-03-13T08:56:19.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38622 #4078915 (1506 connections now open) 2016-03-13T08:56:19.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38621 #4078916 (1507 connections now open) 2016-03-13T08:56:19.701+0000 I NETWORK [conn4078902] end connection 10.0.26.69:40594 (1506 connections now open) 2016-03-13T08:56:19.701+0000 I NETWORK [conn4078901] end connection 10.0.26.69:40593 (1505 connections now open) 2016-03-13T08:56:19.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40615 #4078917 (1506 connections now open) 2016-03-13T08:56:19.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40616 #4078918 (1507 connections now open) 2016-03-13T08:56:22.599+0000 I NETWORK [conn4078903] end connection 10.0.27.237:33850 (1506 connections now open) 2016-03-13T08:56:22.599+0000 I NETWORK [conn4078904] end connection 10.0.27.237:33849 (1505 connections now open) 2016-03-13T08:56:22.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33868 #4078919 (1506 connections now open) 2016-03-13T08:56:22.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33867 #4078920 (1507 connections now open) 2016-03-13T08:56:23.528+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:56:23.352+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:56:24.711+0000 I NETWORK [conn4078905] end connection 10.0.7.113:57550 (1506 connections now open) 2016-03-13T08:56:24.711+0000 I NETWORK [conn4078906] end connection 10.0.7.113:57551 (1506 connections now open) 2016-03-13T08:56:24.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57565 #4078921 (1506 connections now open) 2016-03-13T08:56:24.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57564 #4078922 (1507 connections now open) 2016-03-13T08:56:25.869+0000 I NETWORK [conn4078908] end connection 10.0.216.90:55596 (1506 connections now open) 2016-03-13T08:56:25.869+0000 I NETWORK [conn4078907] end connection 10.0.216.90:55595 (1506 connections now open) 2016-03-13T08:56:25.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55608 #4078923 (1506 connections now open) 2016-03-13T08:56:25.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55609 #4078924 (1507 connections now open) 2016-03-13T08:56:26.528+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 674ms sequenceNumber: 16739 version: 30564|1||563ba74d869758a4542b9075 based on: 30563|1||563ba74d869758a4542b9075 2016-03-13T08:56:26.734+0000 I NETWORK [conn4078913] end connection 10.0.20.53:48770 (1506 connections now open) 2016-03-13T08:56:26.734+0000 I NETWORK [conn4078914] end connection 10.0.20.53:48771 (1506 connections now open) 2016-03-13T08:56:26.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48782 #4078925 (1506 connections now open) 2016-03-13T08:56:26.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48783 #4078926 (1507 connections now open) 2016-03-13T08:56:26.801+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:56:26.801+0000-56e52b3a18fa188537623ccf", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859386801), what: "balancer.round", ns: "", details: { executionTimeMillis: 12140, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:56:26.977+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:56:28.113+0000 I NETWORK [conn4077101] end connection 10.0.216.90:53692 (1506 connections now open) 2016-03-13T08:56:28.113+0000 I NETWORK [conn4069045] end connection 10.0.26.69:58272 (1506 connections now open) 2016-03-13T08:56:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55612 #4078927 (1506 connections now open) 2016-03-13T08:56:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40625 #4078928 (1507 connections now open) 2016-03-13T08:56:29.068+0000 I NETWORK [conn4078916] end connection 10.0.179.209:38621 (1506 connections now open) 2016-03-13T08:56:29.068+0000 I NETWORK [conn4078915] end connection 10.0.179.209:38622 (1506 connections now open) 2016-03-13T08:56:29.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38635 #4078929 (1506 connections now open) 2016-03-13T08:56:29.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38636 #4078930 (1507 connections now open) 2016-03-13T08:56:29.722+0000 I NETWORK [conn4078918] end connection 10.0.26.69:40616 (1506 connections now open) 2016-03-13T08:56:29.722+0000 I NETWORK [conn4078917] end connection 10.0.26.69:40615 (1506 connections now open) 2016-03-13T08:56:29.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40627 #4078931 (1507 connections now open) 2016-03-13T08:56:29.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40628 #4078932 (1507 connections now open) 2016-03-13T08:56:32.621+0000 I NETWORK [conn4078920] end connection 10.0.27.237:33867 (1506 connections now open) 2016-03-13T08:56:32.621+0000 I NETWORK [conn4078919] end connection 10.0.27.237:33868 (1506 connections now open) 2016-03-13T08:56:32.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33875 #4078933 (1506 connections now open) 2016-03-13T08:56:32.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33876 #4078934 (1507 connections now open) 2016-03-13T08:56:34.733+0000 I NETWORK [conn4078922] end connection 10.0.7.113:57564 (1506 connections now open) 2016-03-13T08:56:34.733+0000 I NETWORK [conn4078921] end connection 10.0.7.113:57565 (1506 connections now open) 2016-03-13T08:56:34.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57584 #4078935 (1508 connections now open) 2016-03-13T08:56:34.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57585 #4078936 (1507 connections now open) 2016-03-13T08:56:35.882+0000 I NETWORK [conn4078923] end connection 10.0.216.90:55608 (1506 connections now open) 2016-03-13T08:56:35.882+0000 I NETWORK [conn4078924] end connection 10.0.216.90:55609 (1506 connections now open) 2016-03-13T08:56:35.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55631 #4078937 (1506 connections now open) 2016-03-13T08:56:35.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55630 #4078938 (1507 connections now open) 2016-03-13T08:56:36.748+0000 I NETWORK [conn4078926] end connection 10.0.20.53:48783 (1506 connections now open) 2016-03-13T08:56:36.748+0000 I NETWORK [conn4078925] end connection 10.0.20.53:48782 (1505 connections now open) 2016-03-13T08:56:36.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48805 #4078939 (1506 connections now open) 2016-03-13T08:56:36.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48806 #4078940 (1507 connections now open) 2016-03-13T08:56:39.082+0000 I NETWORK [conn4078929] end connection 10.0.179.209:38635 (1506 connections now open) 2016-03-13T08:56:39.082+0000 I NETWORK [conn4078930] end connection 10.0.179.209:38636 (1506 connections now open) 2016-03-13T08:56:39.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38653 #4078941 (1506 connections now open) 2016-03-13T08:56:39.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38652 #4078942 (1507 connections now open) 2016-03-13T08:56:39.735+0000 I NETWORK [conn4078932] end connection 10.0.26.69:40628 (1506 connections now open) 2016-03-13T08:56:39.735+0000 I NETWORK [conn4078931] end connection 10.0.26.69:40627 (1506 connections now open) 2016-03-13T08:56:39.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40638 #4078943 (1507 connections now open) 2016-03-13T08:56:39.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40639 #4078944 (1507 connections now open) 2016-03-13T08:56:40.244+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16740 version: 30565|1||563ba74d869758a4542b9075 based on: 30564|1||563ba74d869758a4542b9075 2016-03-13T08:56:42.635+0000 I NETWORK [conn4078934] end connection 10.0.27.237:33876 (1506 connections now open) 2016-03-13T08:56:42.635+0000 I NETWORK [conn4078933] end connection 10.0.27.237:33875 (1506 connections now open) 2016-03-13T08:56:42.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33887 #4078945 (1506 connections now open) 2016-03-13T08:56:42.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33888 #4078946 (1507 connections now open) 2016-03-13T08:56:44.747+0000 I NETWORK [conn4078935] end connection 10.0.7.113:57584 (1506 connections now open) 2016-03-13T08:56:44.747+0000 I NETWORK [conn4078936] end connection 10.0.7.113:57585 (1506 connections now open) 2016-03-13T08:56:44.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57592 #4078947 (1507 connections now open) 2016-03-13T08:56:44.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57593 #4078948 (1507 connections now open) 2016-03-13T08:56:45.895+0000 I NETWORK [conn4078938] end connection 10.0.216.90:55630 (1506 connections now open) 2016-03-13T08:56:45.895+0000 I NETWORK [conn4078937] end connection 10.0.216.90:55631 (1506 connections now open) 2016-03-13T08:56:45.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55638 #4078949 (1506 connections now open) 2016-03-13T08:56:45.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55639 #4078950 (1507 connections now open) 2016-03-13T08:56:46.118+0000 I NETWORK [conn4074668] end connection 10.0.20.53:44377 (1506 connections now open) 2016-03-13T08:56:46.118+0000 I NETWORK [conn4069025] end connection 10.0.179.209:56874 (1506 connections now open) 2016-03-13T08:56:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48813 #4078951 (1506 connections now open) 2016-03-13T08:56:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38659 #4078952 (1507 connections now open) 2016-03-13T08:56:46.761+0000 I NETWORK [conn4078940] end connection 10.0.20.53:48806 (1506 connections now open) 2016-03-13T08:56:46.761+0000 I NETWORK [conn4078939] end connection 10.0.20.53:48805 (1506 connections now open) 2016-03-13T08:56:46.762+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48815 #4078953 (1506 connections now open) 2016-03-13T08:56:46.762+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48814 #4078954 (1507 connections now open) 2016-03-13T08:56:49.096+0000 I NETWORK [conn4078941] end connection 10.0.179.209:38653 (1506 connections now open) 2016-03-13T08:56:49.096+0000 I NETWORK [conn4078942] end connection 10.0.179.209:38652 (1506 connections now open) 2016-03-13T08:56:49.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38662 #4078955 (1506 connections now open) 2016-03-13T08:56:49.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38663 #4078956 (1507 connections now open) 2016-03-13T08:56:49.751+0000 I NETWORK [conn4078943] end connection 10.0.26.69:40638 (1506 connections now open) 2016-03-13T08:56:49.751+0000 I NETWORK [conn4078944] end connection 10.0.26.69:40639 (1505 connections now open) 2016-03-13T08:56:49.752+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40648 #4078957 (1506 connections now open) 2016-03-13T08:56:49.752+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40649 #4078958 (1507 connections now open) 2016-03-13T08:56:52.648+0000 I NETWORK [conn4078945] end connection 10.0.27.237:33887 (1506 connections now open) 2016-03-13T08:56:52.648+0000 I NETWORK [conn4078946] end connection 10.0.27.237:33888 (1506 connections now open) 2016-03-13T08:56:52.649+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33908 #4078959 (1506 connections now open) 2016-03-13T08:56:52.649+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33907 #4078960 (1507 connections now open) 2016-03-13T08:56:53.791+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:56:53.529+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:56:53.791+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:56:53.927+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b3c18fa188537623cd0 2016-03-13T08:56:54.066+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b4618fa188537623cd1 2016-03-13T08:56:54.202+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b5018fa188537623cd2 2016-03-13T08:56:54.760+0000 I NETWORK [conn4078947] end connection 10.0.7.113:57592 (1506 connections now open) 2016-03-13T08:56:54.760+0000 I NETWORK [conn4078948] end connection 10.0.7.113:57593 (1506 connections now open) 2016-03-13T08:56:54.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57606 #4078961 (1507 connections now open) 2016-03-13T08:56:54.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57605 #4078962 (1507 connections now open) 2016-03-13T08:56:54.809+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16741 version: 30566|1||563ba74d869758a4542b9075 based on: 30565|1||563ba74d869758a4542b9075 2016-03-13T08:56:55.908+0000 I NETWORK [conn4078949] end connection 10.0.216.90:55638 (1506 connections now open) 2016-03-13T08:56:55.908+0000 I NETWORK [conn4078950] end connection 10.0.216.90:55639 (1506 connections now open) 2016-03-13T08:56:55.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55650 #4078963 (1506 connections now open) 2016-03-13T08:56:55.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55649 #4078964 (1507 connections now open) 2016-03-13T08:56:56.775+0000 I NETWORK [conn4078954] end connection 10.0.20.53:48814 (1506 connections now open) 2016-03-13T08:56:56.775+0000 I NETWORK [conn4078953] end connection 10.0.20.53:48815 (1506 connections now open) 2016-03-13T08:56:56.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48826 #4078965 (1506 connections now open) 2016-03-13T08:56:56.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48827 #4078966 (1507 connections now open) 2016-03-13T08:56:58.113+0000 I NETWORK [conn4070055] end connection 10.0.20.53:39544 (1506 connections now open) 2016-03-13T08:56:58.113+0000 I NETWORK [conn4076889] end connection 10.0.216.90:53461 (1505 connections now open) 2016-03-13T08:56:58.113+0000 I NETWORK [conn4070608] end connection 10.0.27.237:53604 (1504 connections now open) 2016-03-13T08:56:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55653 #4078967 (1505 connections now open) 2016-03-13T08:56:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33915 #4078968 (1506 connections now open) 2016-03-13T08:56:58.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48840 #4078969 (1507 connections now open) 2016-03-13T08:56:59.109+0000 I NETWORK [conn4078955] end connection 10.0.179.209:38662 (1506 connections now open) 2016-03-13T08:56:59.109+0000 I NETWORK [conn4078956] end connection 10.0.179.209:38663 (1506 connections now open) 2016-03-13T08:56:59.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38676 #4078970 (1506 connections now open) 2016-03-13T08:56:59.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38677 #4078971 (1507 connections now open) 2016-03-13T08:56:59.764+0000 I NETWORK [conn4078957] end connection 10.0.26.69:40648 (1506 connections now open) 2016-03-13T08:56:59.764+0000 I NETWORK [conn4078958] end connection 10.0.26.69:40649 (1506 connections now open) 2016-03-13T08:56:59.765+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40667 #4078972 (1507 connections now open) 2016-03-13T08:56:59.765+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40666 #4078973 (1507 connections now open) 2016-03-13T08:57:02.661+0000 I NETWORK [conn4078959] end connection 10.0.27.237:33908 (1506 connections now open) 2016-03-13T08:57:02.661+0000 I NETWORK [conn4078960] end connection 10.0.27.237:33907 (1506 connections now open) 2016-03-13T08:57:02.662+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33919 #4078974 (1506 connections now open) 2016-03-13T08:57:02.662+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33918 #4078975 (1507 connections now open) 2016-03-13T08:57:04.773+0000 I NETWORK [conn4078961] end connection 10.0.7.113:57606 (1506 connections now open) 2016-03-13T08:57:04.773+0000 I NETWORK [conn4078962] end connection 10.0.7.113:57605 (1506 connections now open) 2016-03-13T08:57:04.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57628 #4078976 (1506 connections now open) 2016-03-13T08:57:04.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57627 #4078977 (1507 connections now open) 2016-03-13T08:57:05.921+0000 I NETWORK [conn4078963] end connection 10.0.216.90:55650 (1506 connections now open) 2016-03-13T08:57:05.921+0000 I NETWORK [conn4078964] end connection 10.0.216.90:55649 (1506 connections now open) 2016-03-13T08:57:05.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55670 #4078978 (1506 connections now open) 2016-03-13T08:57:05.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55671 #4078979 (1507 connections now open) 2016-03-13T08:57:06.789+0000 I NETWORK [conn4078965] end connection 10.0.20.53:48826 (1506 connections now open) 2016-03-13T08:57:06.789+0000 I NETWORK [conn4078966] end connection 10.0.20.53:48827 (1506 connections now open) 2016-03-13T08:57:06.789+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48848 #4078980 (1506 connections now open) 2016-03-13T08:57:06.789+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48849 #4078981 (1507 connections now open) 2016-03-13T08:57:08.415+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16742 version: 30567|1||563ba74d869758a4542b9075 based on: 30566|1||563ba74d869758a4542b9075 2016-03-13T08:57:09.131+0000 I NETWORK [conn4078971] end connection 10.0.179.209:38677 (1506 connections now open) 2016-03-13T08:57:09.131+0000 I NETWORK [conn4078970] end connection 10.0.179.209:38676 (1506 connections now open) 2016-03-13T08:57:09.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38697 #4078982 (1506 connections now open) 2016-03-13T08:57:09.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38696 #4078983 (1507 connections now open) 2016-03-13T08:57:09.778+0000 I NETWORK [conn4078972] end connection 10.0.26.69:40667 (1506 connections now open) 2016-03-13T08:57:09.778+0000 I NETWORK [conn4078973] end connection 10.0.26.69:40666 (1506 connections now open) 2016-03-13T08:57:09.778+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40676 #4078984 (1506 connections now open) 2016-03-13T08:57:09.778+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40677 #4078985 (1507 connections now open) 2016-03-13T08:57:12.675+0000 I NETWORK [conn4078974] end connection 10.0.27.237:33919 (1506 connections now open) 2016-03-13T08:57:12.675+0000 I NETWORK [conn4078975] end connection 10.0.27.237:33918 (1505 connections now open) 2016-03-13T08:57:12.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33931 #4078986 (1506 connections now open) 2016-03-13T08:57:12.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33932 #4078987 (1507 connections now open) 2016-03-13T08:57:14.786+0000 I NETWORK [conn4078977] end connection 10.0.7.113:57627 (1506 connections now open) 2016-03-13T08:57:14.786+0000 I NETWORK [conn4078976] end connection 10.0.7.113:57628 (1506 connections now open) 2016-03-13T08:57:14.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57639 #4078988 (1507 connections now open) 2016-03-13T08:57:14.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57640 #4078989 (1507 connections now open) 2016-03-13T08:57:15.934+0000 I NETWORK [conn4078979] end connection 10.0.216.90:55671 (1506 connections now open) 2016-03-13T08:57:15.934+0000 I NETWORK [conn4078978] end connection 10.0.216.90:55670 (1506 connections now open) 2016-03-13T08:57:15.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55679 #4078990 (1506 connections now open) 2016-03-13T08:57:15.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55680 #4078991 (1507 connections now open) 2016-03-13T08:57:16.117+0000 I NETWORK [conn4071226] end connection 10.0.20.53:40759 (1506 connections now open) 2016-03-13T08:57:16.117+0000 I NETWORK [conn4068349] end connection 10.0.27.237:51220 (1505 connections now open) 2016-03-13T08:57:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48857 #4078992 (1506 connections now open) 2016-03-13T08:57:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33938 #4078993 (1507 connections now open) 2016-03-13T08:57:16.802+0000 I NETWORK [conn4078980] end connection 10.0.20.53:48848 (1506 connections now open) 2016-03-13T08:57:16.802+0000 I NETWORK [conn4078981] end connection 10.0.20.53:48849 (1506 connections now open) 2016-03-13T08:57:16.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48858 #4078994 (1506 connections now open) 2016-03-13T08:57:16.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48859 #4078995 (1507 connections now open) 2016-03-13T08:57:19.145+0000 I NETWORK [conn4078982] end connection 10.0.179.209:38697 (1506 connections now open) 2016-03-13T08:57:19.145+0000 I NETWORK [conn4078983] end connection 10.0.179.209:38696 (1506 connections now open) 2016-03-13T08:57:19.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38710 #4078996 (1506 connections now open) 2016-03-13T08:57:19.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38709 #4078997 (1507 connections now open) 2016-03-13T08:57:19.791+0000 I NETWORK [conn4078985] end connection 10.0.26.69:40677 (1506 connections now open) 2016-03-13T08:57:19.791+0000 I NETWORK [conn4078984] end connection 10.0.26.69:40676 (1506 connections now open) 2016-03-13T08:57:19.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40693 #4078998 (1506 connections now open) 2016-03-13T08:57:19.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40694 #4078999 (1507 connections now open) 2016-03-13T08:57:22.688+0000 I NETWORK [conn4078986] end connection 10.0.27.237:33931 (1506 connections now open) 2016-03-13T08:57:22.688+0000 I NETWORK [conn4078987] end connection 10.0.27.237:33932 (1506 connections now open) 2016-03-13T08:57:22.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33950 #4079000 (1506 connections now open) 2016-03-13T08:57:22.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33949 #4079001 (1507 connections now open) 2016-03-13T08:57:24.160+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16743 version: 30568|1||563ba74d869758a4542b9075 based on: 30567|1||563ba74d869758a4542b9075 2016-03-13T08:57:24.418+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:57:24.202+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:57:24.418+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:57:24.486+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b5a18fa188537623cd3 2016-03-13T08:57:24.589+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b6418fa188537623cd4 2016-03-13T08:57:24.727+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b6e18fa188537623cd5 2016-03-13T08:57:24.800+0000 I NETWORK [conn4078989] end connection 10.0.7.113:57640 (1506 connections now open) 2016-03-13T08:57:24.800+0000 I NETWORK [conn4078988] end connection 10.0.7.113:57639 (1506 connections now open) 2016-03-13T08:57:24.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57652 #4079002 (1506 connections now open) 2016-03-13T08:57:24.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57651 #4079003 (1507 connections now open) 2016-03-13T08:57:25.947+0000 I NETWORK [conn4078990] end connection 10.0.216.90:55679 (1506 connections now open) 2016-03-13T08:57:25.947+0000 I NETWORK [conn4078991] end connection 10.0.216.90:55680 (1506 connections now open) 2016-03-13T08:57:25.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55692 #4079004 (1506 connections now open) 2016-03-13T08:57:25.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55693 #4079005 (1507 connections now open) 2016-03-13T08:57:26.816+0000 I NETWORK [conn4078994] end connection 10.0.20.53:48858 (1506 connections now open) 2016-03-13T08:57:26.816+0000 I NETWORK [conn4078995] end connection 10.0.20.53:48859 (1506 connections now open) 2016-03-13T08:57:26.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48871 #4079006 (1506 connections now open) 2016-03-13T08:57:26.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48870 #4079007 (1507 connections now open) 2016-03-13T08:57:28.113+0000 I NETWORK [conn4074685] end connection 10.0.20.53:44392 (1506 connections now open) 2016-03-13T08:57:28.113+0000 I NETWORK [conn4070183] end connection 10.0.179.209:58029 (1506 connections now open) 2016-03-13T08:57:28.113+0000 I NETWORK [conn4070990] end connection 10.0.26.69:60368 (1504 connections now open) 2016-03-13T08:57:28.114+0000 I NETWORK [conn4070691] end connection 10.0.27.237:53695 (1503 connections now open) 2016-03-13T08:57:28.114+0000 I NETWORK [conn4067799] end connection 10.0.7.113:45699 (1502 connections now open) 2016-03-13T08:57:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48884 #4079008 (1503 connections now open) 2016-03-13T08:57:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38721 #4079009 (1504 connections now open) 2016-03-13T08:57:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40710 #4079010 (1505 connections now open) 2016-03-13T08:57:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57659 #4079011 (1506 connections now open) 2016-03-13T08:57:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33958 #4079012 (1507 connections now open) 2016-03-13T08:57:29.158+0000 I NETWORK [conn4078997] end connection 10.0.179.209:38709 (1506 connections now open) 2016-03-13T08:57:29.158+0000 I NETWORK [conn4078996] end connection 10.0.179.209:38710 (1506 connections now open) 2016-03-13T08:57:29.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38724 #4079013 (1506 connections now open) 2016-03-13T08:57:29.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38723 #4079014 (1507 connections now open) 2016-03-13T08:57:29.812+0000 I NETWORK [conn4078999] end connection 10.0.26.69:40694 (1506 connections now open) 2016-03-13T08:57:29.812+0000 I NETWORK [conn4078998] end connection 10.0.26.69:40693 (1506 connections now open) 2016-03-13T08:57:29.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40711 #4079015 (1506 connections now open) 2016-03-13T08:57:29.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40712 #4079016 (1507 connections now open) 2016-03-13T08:57:32.711+0000 I NETWORK [conn4079001] end connection 10.0.27.237:33949 (1506 connections now open) 2016-03-13T08:57:32.711+0000 I NETWORK [conn4079000] end connection 10.0.27.237:33950 (1506 connections now open) 2016-03-13T08:57:32.712+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33959 #4079017 (1506 connections now open) 2016-03-13T08:57:32.712+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33960 #4079018 (1507 connections now open) 2016-03-13T08:57:34.822+0000 I NETWORK [conn4079002] end connection 10.0.7.113:57652 (1506 connections now open) 2016-03-13T08:57:34.822+0000 I NETWORK [conn4079003] end connection 10.0.7.113:57651 (1506 connections now open) 2016-03-13T08:57:34.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57672 #4079019 (1507 connections now open) 2016-03-13T08:57:34.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57671 #4079020 (1507 connections now open) 2016-03-13T08:57:35.970+0000 I NETWORK [conn4079004] end connection 10.0.216.90:55692 (1506 connections now open) 2016-03-13T08:57:35.970+0000 I NETWORK [conn4079005] end connection 10.0.216.90:55693 (1506 connections now open) 2016-03-13T08:57:35.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55714 #4079021 (1506 connections now open) 2016-03-13T08:57:35.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55715 #4079022 (1507 connections now open) 2016-03-13T08:57:36.829+0000 I NETWORK [conn4079007] end connection 10.0.20.53:48870 (1506 connections now open) 2016-03-13T08:57:36.829+0000 I NETWORK [conn4079006] end connection 10.0.20.53:48871 (1506 connections now open) 2016-03-13T08:57:36.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48895 #4079023 (1506 connections now open) 2016-03-13T08:57:36.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48894 #4079024 (1507 connections now open) 2016-03-13T08:57:39.045+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 674ms sequenceNumber: 16744 version: 30569|1||563ba74d869758a4542b9075 based on: 30568|1||563ba74d869758a4542b9075 2016-03-13T08:57:39.172+0000 I NETWORK [conn4079014] end connection 10.0.179.209:38723 (1506 connections now open) 2016-03-13T08:57:39.172+0000 I NETWORK [conn4079013] end connection 10.0.179.209:38724 (1506 connections now open) 2016-03-13T08:57:39.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38742 #4079025 (1506 connections now open) 2016-03-13T08:57:39.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38741 #4079026 (1507 connections now open) 2016-03-13T08:57:39.826+0000 I NETWORK [conn4079015] end connection 10.0.26.69:40711 (1506 connections now open) 2016-03-13T08:57:39.826+0000 I NETWORK [conn4079016] end connection 10.0.26.69:40712 (1506 connections now open) 2016-03-13T08:57:39.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40717 #4079027 (1507 connections now open) 2016-03-13T08:57:39.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40718 #4079028 (1507 connections now open) 2016-03-13T08:57:42.724+0000 I NETWORK [conn4079017] end connection 10.0.27.237:33959 (1506 connections now open) 2016-03-13T08:57:42.724+0000 I NETWORK [conn4079018] end connection 10.0.27.237:33960 (1506 connections now open) 2016-03-13T08:57:42.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33971 #4079029 (1506 connections now open) 2016-03-13T08:57:42.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33970 #4079030 (1507 connections now open) 2016-03-13T08:57:44.835+0000 I NETWORK [conn4079019] end connection 10.0.7.113:57672 (1506 connections now open) 2016-03-13T08:57:44.835+0000 I NETWORK [conn4079020] end connection 10.0.7.113:57671 (1506 connections now open) 2016-03-13T08:57:44.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57681 #4079031 (1507 connections now open) 2016-03-13T08:57:44.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57680 #4079032 (1507 connections now open) 2016-03-13T08:57:45.983+0000 I NETWORK [conn4079021] end connection 10.0.216.90:55714 (1506 connections now open) 2016-03-13T08:57:45.983+0000 I NETWORK [conn4079022] end connection 10.0.216.90:55715 (1506 connections now open) 2016-03-13T08:57:45.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55722 #4079033 (1506 connections now open) 2016-03-13T08:57:45.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55721 #4079034 (1507 connections now open) 2016-03-13T08:57:46.117+0000 I NETWORK [conn4067567] end connection 10.0.20.53:36940 (1506 connections now open) 2016-03-13T08:57:46.117+0000 I NETWORK [conn4071161] end connection 10.0.216.90:47287 (1505 connections now open) 2016-03-13T08:57:46.117+0000 I NETWORK [conn4066958] end connection 10.0.26.69:56052 (1504 connections now open) 2016-03-13T08:57:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48902 #4079035 (1505 connections now open) 2016-03-13T08:57:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55725 #4079036 (1506 connections now open) 2016-03-13T08:57:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40730 #4079037 (1507 connections now open) 2016-03-13T08:57:46.843+0000 I NETWORK [conn4079024] end connection 10.0.20.53:48894 (1506 connections now open) 2016-03-13T08:57:46.843+0000 I NETWORK [conn4079023] end connection 10.0.20.53:48895 (1506 connections now open) 2016-03-13T08:57:46.843+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48904 #4079038 (1506 connections now open) 2016-03-13T08:57:46.843+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48903 #4079039 (1507 connections now open) 2016-03-13T08:57:49.185+0000 I NETWORK [conn4079026] end connection 10.0.179.209:38741 (1506 connections now open) 2016-03-13T08:57:49.185+0000 I NETWORK [conn4079025] end connection 10.0.179.209:38742 (1506 connections now open) 2016-03-13T08:57:49.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38752 #4079040 (1506 connections now open) 2016-03-13T08:57:49.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38751 #4079041 (1507 connections now open) 2016-03-13T08:57:49.839+0000 I NETWORK [conn4079027] end connection 10.0.26.69:40717 (1506 connections now open) 2016-03-13T08:57:49.839+0000 I NETWORK [conn4079028] end connection 10.0.26.69:40718 (1506 connections now open) 2016-03-13T08:57:49.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40733 #4079042 (1506 connections now open) 2016-03-13T08:57:49.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40732 #4079043 (1507 connections now open) 2016-03-13T08:57:52.737+0000 I NETWORK [conn4079030] end connection 10.0.27.237:33970 (1506 connections now open) 2016-03-13T08:57:52.737+0000 I NETWORK [conn4079029] end connection 10.0.27.237:33971 (1506 connections now open) 2016-03-13T08:57:52.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33990 #4079044 (1506 connections now open) 2016-03-13T08:57:52.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33989 #4079045 (1507 connections now open) 2016-03-13T08:57:54.848+0000 I NETWORK [conn4079031] end connection 10.0.7.113:57681 (1506 connections now open) 2016-03-13T08:57:54.848+0000 I NETWORK [conn4079032] end connection 10.0.7.113:57680 (1506 connections now open) 2016-03-13T08:57:54.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57694 #4079046 (1507 connections now open) 2016-03-13T08:57:54.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57693 #4079047 (1507 connections now open) 2016-03-13T08:57:55.996+0000 I NETWORK [conn4079034] end connection 10.0.216.90:55721 (1506 connections now open) 2016-03-13T08:57:55.996+0000 I NETWORK [conn4079033] end connection 10.0.216.90:55722 (1506 connections now open) 2016-03-13T08:57:55.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55733 #4079048 (1506 connections now open) 2016-03-13T08:57:55.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55734 #4079049 (1507 connections now open) 2016-03-13T08:57:56.576+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:57:54.727+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:57:56.576+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T08:57:56.824+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b7818fa188537623cd6 2016-03-13T08:57:56.856+0000 I NETWORK [conn4079038] end connection 10.0.20.53:48904 (1506 connections now open) 2016-03-13T08:57:56.856+0000 I NETWORK [conn4079039] end connection 10.0.20.53:48903 (1506 connections now open) 2016-03-13T08:57:56.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48914 #4079050 (1506 connections now open) 2016-03-13T08:57:56.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48915 #4079051 (1507 connections now open) 2016-03-13T08:57:57.030+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b8218fa188537623cd7 2016-03-13T08:57:57.164+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52b8c18fa188537623cd8 2016-03-13T08:57:57.322+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16745 version: 30570|1||563ba74d869758a4542b9075 based on: 30569|1||563ba74d869758a4542b9075 2016-03-13T08:57:58.114+0000 I NETWORK [conn4073391] end connection 10.0.27.237:56579 (1506 connections now open) 2016-03-13T08:57:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:33997 #4079052 (1507 connections now open) 2016-03-13T08:57:58.944+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52b9618fa188537623cd9 2016-03-13T08:57:59.198+0000 I NETWORK [conn4079040] end connection 10.0.179.209:38752 (1506 connections now open) 2016-03-13T08:57:59.198+0000 I NETWORK [conn4079041] end connection 10.0.179.209:38751 (1506 connections now open) 2016-03-13T08:57:59.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38764 #4079053 (1506 connections now open) 2016-03-13T08:57:59.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38765 #4079054 (1507 connections now open) 2016-03-13T08:57:59.852+0000 I NETWORK [conn4079043] end connection 10.0.26.69:40732 (1506 connections now open) 2016-03-13T08:57:59.852+0000 I NETWORK [conn4079042] end connection 10.0.26.69:40733 (1506 connections now open) 2016-03-13T08:57:59.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40753 #4079055 (1507 connections now open) 2016-03-13T08:57:59.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40752 #4079056 (1507 connections now open) 2016-03-13T08:58:01.435+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633faaace48b40ea18e8bb6')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633faaace48b40ea18e8bb6') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fbb4ce48b40ea368d54f') }, shard: "shmydomain1", lastmod: Timestamp 30570000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:58:01.589+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30570|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633faaace48b40ea18e8bb6') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fbb4ce48b40ea368d54f') }) shmydomain1 -> shmydomain3 2016-03-13T08:58:02.751+0000 I NETWORK [conn4079045] end connection 10.0.27.237:33989 (1506 connections now open) 2016-03-13T08:58:02.751+0000 I NETWORK [conn4079044] end connection 10.0.27.237:33990 (1506 connections now open) 2016-03-13T08:58:02.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34001 #4079057 (1506 connections now open) 2016-03-13T08:58:02.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34000 #4079058 (1507 connections now open) 2016-03-13T08:58:04.862+0000 I NETWORK [conn4079047] end connection 10.0.7.113:57693 (1506 connections now open) 2016-03-13T08:58:04.862+0000 I NETWORK [conn4079046] end connection 10.0.7.113:57694 (1506 connections now open) 2016-03-13T08:58:04.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57717 #4079059 (1507 connections now open) 2016-03-13T08:58:04.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57716 #4079060 (1507 connections now open) 2016-03-13T08:58:06.009+0000 I NETWORK [conn4079048] end connection 10.0.216.90:55733 (1506 connections now open) 2016-03-13T08:58:06.009+0000 I NETWORK [conn4079049] end connection 10.0.216.90:55734 (1506 connections now open) 2016-03-13T08:58:06.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55754 #4079061 (1506 connections now open) 2016-03-13T08:58:06.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55755 #4079062 (1507 connections now open) 2016-03-13T08:58:06.869+0000 I NETWORK [conn4079051] end connection 10.0.20.53:48915 (1506 connections now open) 2016-03-13T08:58:06.869+0000 I NETWORK [conn4079050] end connection 10.0.20.53:48914 (1506 connections now open) 2016-03-13T08:58:06.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48935 #4079063 (1506 connections now open) 2016-03-13T08:58:06.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48936 #4079064 (1507 connections now open) 2016-03-13T08:58:09.219+0000 I NETWORK [conn4079053] end connection 10.0.179.209:38764 (1506 connections now open) 2016-03-13T08:58:09.219+0000 I NETWORK [conn4079054] end connection 10.0.179.209:38765 (1506 connections now open) 2016-03-13T08:58:09.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38783 #4079065 (1506 connections now open) 2016-03-13T08:58:09.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38784 #4079066 (1507 connections now open) 2016-03-13T08:58:09.866+0000 I NETWORK [conn4079056] end connection 10.0.26.69:40752 (1506 connections now open) 2016-03-13T08:58:09.866+0000 I NETWORK [conn4079055] end connection 10.0.26.69:40753 (1506 connections now open) 2016-03-13T08:58:09.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40760 #4079067 (1506 connections now open) 2016-03-13T08:58:09.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40761 #4079068 (1507 connections now open) 2016-03-13T08:58:11.539+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16746 version: 30571|1||563ba74d869758a4542b9075 based on: 30570|1||563ba74d869758a4542b9075 2016-03-13T08:58:11.811+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:58:11.811+0000-56e52ba318fa188537623cda", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859491811), what: "balancer.round", ns: "", details: { executionTimeMillis: 13162, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:58:11.925+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:58:12.764+0000 I NETWORK [conn4079057] end connection 10.0.27.237:34001 (1506 connections now open) 2016-03-13T08:58:12.764+0000 I NETWORK [conn4079058] end connection 10.0.27.237:34000 (1506 connections now open) 2016-03-13T08:58:12.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34015 #4079069 (1506 connections now open) 2016-03-13T08:58:12.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34016 #4079070 (1507 connections now open) 2016-03-13T08:58:13.152+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52ba418fa188537623cdb 2016-03-13T08:58:14.875+0000 I NETWORK [conn4079059] end connection 10.0.7.113:57717 (1506 connections now open) 2016-03-13T08:58:14.875+0000 I NETWORK [conn4079060] end connection 10.0.7.113:57716 (1506 connections now open) 2016-03-13T08:58:14.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57726 #4079071 (1507 connections now open) 2016-03-13T08:58:14.876+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57727 #4079072 (1507 connections now open) 2016-03-13T08:58:15.681+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633fbb4ce48b40ea368d54f')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fbb4ce48b40ea368d54f') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fcb7ce48b40ea18eb8ce') }, shard: "shmydomain1", lastmod: Timestamp 30571000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:58:15.837+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30571|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fbb4ce48b40ea368d54f') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fcb7ce48b40ea18eb8ce') }) shmydomain1 -> shmydomain3 2016-03-13T08:58:16.022+0000 I NETWORK [conn4079061] end connection 10.0.216.90:55754 (1506 connections now open) 2016-03-13T08:58:16.022+0000 I NETWORK [conn4079062] end connection 10.0.216.90:55755 (1506 connections now open) 2016-03-13T08:58:16.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55762 #4079073 (1506 connections now open) 2016-03-13T08:58:16.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55763 #4079074 (1507 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [conn4070396] end connection 10.0.216.90:46463 (1506 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [conn4071309] end connection 10.0.20.53:40850 (1506 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [conn4073163] end connection 10.0.27.237:56350 (1504 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [conn4070544] end connection 10.0.26.69:59898 (1504 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [conn4067151] end connection 10.0.7.113:45024 (1503 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55766 #4079075 (1503 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48943 #4079076 (1504 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34022 #4079077 (1505 connections now open) 2016-03-13T08:58:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40775 #4079078 (1506 connections now open) 2016-03-13T08:58:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57732 #4079079 (1507 connections now open) 2016-03-13T08:58:16.947+0000 I NETWORK [conn4079063] end connection 10.0.20.53:48935 (1506 connections now open) 2016-03-13T08:58:16.947+0000 I NETWORK [conn4079064] end connection 10.0.20.53:48936 (1505 connections now open) 2016-03-13T08:58:16.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48944 #4079080 (1506 connections now open) 2016-03-13T08:58:16.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48945 #4079081 (1507 connections now open) 2016-03-13T08:58:19.232+0000 I NETWORK [conn4079065] end connection 10.0.179.209:38783 (1506 connections now open) 2016-03-13T08:58:19.232+0000 I NETWORK [conn4079066] end connection 10.0.179.209:38784 (1506 connections now open) 2016-03-13T08:58:19.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38796 #4079082 (1506 connections now open) 2016-03-13T08:58:19.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38795 #4079083 (1507 connections now open) 2016-03-13T08:58:19.879+0000 I NETWORK [conn4079068] end connection 10.0.26.69:40761 (1506 connections now open) 2016-03-13T08:58:19.879+0000 I NETWORK [conn4079067] end connection 10.0.26.69:40760 (1506 connections now open) 2016-03-13T08:58:19.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40777 #4079084 (1506 connections now open) 2016-03-13T08:58:19.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40778 #4079085 (1507 connections now open) 2016-03-13T08:58:22.777+0000 I NETWORK [conn4079069] end connection 10.0.27.237:34015 (1506 connections now open) 2016-03-13T08:58:22.777+0000 I NETWORK [conn4079070] end connection 10.0.27.237:34016 (1506 connections now open) 2016-03-13T08:58:22.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34035 #4079086 (1506 connections now open) 2016-03-13T08:58:22.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34034 #4079087 (1507 connections now open) 2016-03-13T08:58:24.888+0000 I NETWORK [conn4079072] end connection 10.0.7.113:57727 (1506 connections now open) 2016-03-13T08:58:24.888+0000 I NETWORK [conn4079071] end connection 10.0.7.113:57726 (1506 connections now open) 2016-03-13T08:58:24.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57738 #4079088 (1506 connections now open) 2016-03-13T08:58:24.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57739 #4079089 (1507 connections now open) 2016-03-13T08:58:25.950+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16747 version: 30572|1||563ba74d869758a4542b9075 based on: 30571|1||563ba74d869758a4542b9075 2016-03-13T08:58:26.035+0000 I NETWORK [conn4079073] end connection 10.0.216.90:55762 (1506 connections now open) 2016-03-13T08:58:26.035+0000 I NETWORK [conn4079074] end connection 10.0.216.90:55763 (1505 connections now open) 2016-03-13T08:58:26.044+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55774 #4079090 (1506 connections now open) 2016-03-13T08:58:26.044+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55775 #4079091 (1507 connections now open) 2016-03-13T08:58:26.220+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:58:26.220+0000-56e52bb218fa188537623cdc", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859506220), what: "balancer.round", ns: "", details: { executionTimeMillis: 13263, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:58:26.392+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:58:26.960+0000 I NETWORK [conn4079081] end connection 10.0.20.53:48945 (1506 connections now open) 2016-03-13T08:58:26.960+0000 I NETWORK [conn4079080] end connection 10.0.20.53:48944 (1506 connections now open) 2016-03-13T08:58:26.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48957 #4079092 (1506 connections now open) 2016-03-13T08:58:26.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48956 #4079093 (1507 connections now open) 2016-03-13T08:58:27.415+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:58:27.165+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:58:27.725+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52bb318fa188537623cdd 2016-03-13T08:58:28.113+0000 I NETWORK [conn4070988] end connection 10.0.179.209:58839 (1506 connections now open) 2016-03-13T08:58:28.113+0000 I NETWORK [conn4072428] end connection 10.0.26.69:33677 (1506 connections now open) 2016-03-13T08:58:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38806 #4079094 (1506 connections now open) 2016-03-13T08:58:28.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40794 #4079095 (1507 connections now open) 2016-03-13T08:58:29.245+0000 I NETWORK [conn4079082] end connection 10.0.179.209:38796 (1506 connections now open) 2016-03-13T08:58:29.245+0000 I NETWORK [conn4079083] end connection 10.0.179.209:38795 (1506 connections now open) 2016-03-13T08:58:29.246+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38808 #4079096 (1506 connections now open) 2016-03-13T08:58:29.246+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38809 #4079097 (1507 connections now open) 2016-03-13T08:58:29.893+0000 I NETWORK [conn4079084] end connection 10.0.26.69:40777 (1506 connections now open) 2016-03-13T08:58:29.893+0000 I NETWORK [conn4079085] end connection 10.0.26.69:40778 (1505 connections now open) 2016-03-13T08:58:29.893+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40797 #4079098 (1507 connections now open) 2016-03-13T08:58:29.893+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40796 #4079099 (1507 connections now open) 2016-03-13T08:58:30.272+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633fcb7ce48b40ea18eb8ce')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fcb7ce48b40ea18eb8ce') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fdb9ce48b40ea369059d') }, shard: "shmydomain1", lastmod: Timestamp 30572000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:58:30.427+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30572|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fcb7ce48b40ea18eb8ce') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fdb9ce48b40ea369059d') }) shmydomain1 -> shmydomain3 2016-03-13T08:58:32.798+0000 I NETWORK [conn4079087] end connection 10.0.27.237:34034 (1506 connections now open) 2016-03-13T08:58:32.798+0000 I NETWORK [conn4079086] end connection 10.0.27.237:34035 (1506 connections now open) 2016-03-13T08:58:32.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34043 #4079100 (1506 connections now open) 2016-03-13T08:58:32.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34044 #4079101 (1507 connections now open) 2016-03-13T08:58:34.903+0000 I NETWORK [conn4079088] end connection 10.0.7.113:57738 (1506 connections now open) 2016-03-13T08:58:34.903+0000 I NETWORK [conn4079089] end connection 10.0.7.113:57739 (1506 connections now open) 2016-03-13T08:58:34.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57758 #4079102 (1507 connections now open) 2016-03-13T08:58:34.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57759 #4079103 (1507 connections now open) 2016-03-13T08:58:36.056+0000 I NETWORK [conn4079090] end connection 10.0.216.90:55774 (1506 connections now open) 2016-03-13T08:58:36.057+0000 I NETWORK [conn4079091] end connection 10.0.216.90:55775 (1505 connections now open) 2016-03-13T08:58:36.057+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55787 #4079104 (1506 connections now open) 2016-03-13T08:58:36.057+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55788 #4079105 (1507 connections now open) 2016-03-13T08:58:36.974+0000 I NETWORK [conn4079092] end connection 10.0.20.53:48957 (1506 connections now open) 2016-03-13T08:58:36.974+0000 I NETWORK [conn4079093] end connection 10.0.20.53:48956 (1506 connections now open) 2016-03-13T08:58:36.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48982 #4079106 (1506 connections now open) 2016-03-13T08:58:36.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48981 #4079107 (1507 connections now open) 2016-03-13T08:58:39.259+0000 I NETWORK [conn4079097] end connection 10.0.179.209:38809 (1506 connections now open) 2016-03-13T08:58:39.259+0000 I NETWORK [conn4079096] end connection 10.0.179.209:38808 (1506 connections now open) 2016-03-13T08:58:39.260+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38826 #4079108 (1506 connections now open) 2016-03-13T08:58:39.260+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38825 #4079109 (1507 connections now open) 2016-03-13T08:58:39.906+0000 I NETWORK [conn4079099] end connection 10.0.26.69:40796 (1506 connections now open) 2016-03-13T08:58:39.906+0000 I NETWORK [conn4079098] end connection 10.0.26.69:40797 (1506 connections now open) 2016-03-13T08:58:39.907+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40804 #4079110 (1507 connections now open) 2016-03-13T08:58:39.907+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40803 #4079111 (1507 connections now open) 2016-03-13T08:58:40.403+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16748 version: 30573|1||563ba74d869758a4542b9075 based on: 30572|1||563ba74d869758a4542b9075 2016-03-13T08:58:40.675+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:58:40.675+0000-56e52bc018fa188537623cde", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859520675), what: "balancer.round", ns: "", details: { executionTimeMillis: 13248, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:58:40.785+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:58:42.223+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52bc118fa188537623cdf 2016-03-13T08:58:42.811+0000 I NETWORK [conn4079101] end connection 10.0.27.237:34044 (1506 connections now open) 2016-03-13T08:58:42.811+0000 I NETWORK [conn4079100] end connection 10.0.27.237:34043 (1506 connections now open) 2016-03-13T08:58:42.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34055 #4079112 (1506 connections now open) 2016-03-13T08:58:42.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34054 #4079113 (1507 connections now open) 2016-03-13T08:58:44.822+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633fdb9ce48b40ea369059d')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fdb9ce48b40ea369059d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fec0ce48b40ea18eeb51') }, shard: "shmydomain1", lastmod: Timestamp 30573000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:58:44.916+0000 I NETWORK [conn4079102] end connection 10.0.7.113:57758 (1506 connections now open) 2016-03-13T08:58:44.916+0000 I NETWORK [conn4079103] end connection 10.0.7.113:57759 (1506 connections now open) 2016-03-13T08:58:44.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57765 #4079114 (1507 connections now open) 2016-03-13T08:58:44.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57766 #4079115 (1507 connections now open) 2016-03-13T08:58:44.976+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30573|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fdb9ce48b40ea369059d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fec0ce48b40ea18eeb51') }) shmydomain1 -> shmydomain3 2016-03-13T08:58:46.069+0000 I NETWORK [conn4079104] end connection 10.0.216.90:55787 (1506 connections now open) 2016-03-13T08:58:46.069+0000 I NETWORK [conn4079105] end connection 10.0.216.90:55788 (1505 connections now open) 2016-03-13T08:58:46.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55802 #4079116 (1506 connections now open) 2016-03-13T08:58:46.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55803 #4079117 (1507 connections now open) 2016-03-13T08:58:46.118+0000 I NETWORK [conn4073249] end connection 10.0.27.237:56440 (1506 connections now open) 2016-03-13T08:58:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34062 #4079118 (1507 connections now open) 2016-03-13T08:58:46.987+0000 I NETWORK [conn4079107] end connection 10.0.20.53:48981 (1506 connections now open) 2016-03-13T08:58:46.987+0000 I NETWORK [conn4079106] end connection 10.0.20.53:48982 (1506 connections now open) 2016-03-13T08:58:46.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48989 #4079119 (1506 connections now open) 2016-03-13T08:58:46.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:48990 #4079120 (1507 connections now open) 2016-03-13T08:58:49.272+0000 I NETWORK [conn4079108] end connection 10.0.179.209:38826 (1506 connections now open) 2016-03-13T08:58:49.272+0000 I NETWORK [conn4079109] end connection 10.0.179.209:38825 (1506 connections now open) 2016-03-13T08:58:49.273+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38834 #4079121 (1506 connections now open) 2016-03-13T08:58:49.273+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38835 #4079122 (1507 connections now open) 2016-03-13T08:58:49.920+0000 I NETWORK [conn4079111] end connection 10.0.26.69:40803 (1506 connections now open) 2016-03-13T08:58:49.920+0000 I NETWORK [conn4079110] end connection 10.0.26.69:40804 (1506 connections now open) 2016-03-13T08:58:49.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40817 #4079123 (1507 connections now open) 2016-03-13T08:58:49.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40818 #4079124 (1507 connections now open) 2016-03-13T08:58:52.824+0000 I NETWORK [conn4079112] end connection 10.0.27.237:34055 (1506 connections now open) 2016-03-13T08:58:52.824+0000 I NETWORK [conn4079113] end connection 10.0.27.237:34054 (1506 connections now open) 2016-03-13T08:58:52.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34075 #4079125 (1506 connections now open) 2016-03-13T08:58:52.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34074 #4079126 (1507 connections now open) 2016-03-13T08:58:54.834+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16749 version: 30574|1||563ba74d869758a4542b9075 based on: 30573|1||563ba74d869758a4542b9075 2016-03-13T08:58:54.929+0000 I NETWORK [conn4079115] end connection 10.0.7.113:57766 (1506 connections now open) 2016-03-13T08:58:54.929+0000 I NETWORK [conn4079114] end connection 10.0.7.113:57765 (1506 connections now open) 2016-03-13T08:58:54.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57780 #4079127 (1507 connections now open) 2016-03-13T08:58:54.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57779 #4079128 (1507 connections now open) 2016-03-13T08:58:55.105+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:58:55.105+0000-56e52bcf18fa188537623ce0", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859535105), what: "balancer.round", ns: "", details: { executionTimeMillis: 13284, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:58:55.286+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:58:56.082+0000 I NETWORK [conn4079116] end connection 10.0.216.90:55802 (1506 connections now open) 2016-03-13T08:58:56.082+0000 I NETWORK [conn4079117] end connection 10.0.216.90:55803 (1506 connections now open) 2016-03-13T08:58:56.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55812 #4079129 (1506 connections now open) 2016-03-13T08:58:56.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55813 #4079130 (1507 connections now open) 2016-03-13T08:58:57.000+0000 I NETWORK [conn4079120] end connection 10.0.20.53:48990 (1506 connections now open) 2016-03-13T08:58:57.000+0000 I NETWORK [conn4079119] end connection 10.0.20.53:48989 (1506 connections now open) 2016-03-13T08:58:57.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49003 #4079131 (1506 connections now open) 2016-03-13T08:58:57.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49002 #4079132 (1507 connections now open) 2016-03-13T08:58:57.882+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52bd018fa188537623ce1 2016-03-13T08:58:58.114+0000 I NETWORK [conn4074947] end connection 10.0.216.90:51354 (1506 connections now open) 2016-03-13T08:58:58.114+0000 I NETWORK [conn4069002] end connection 10.0.26.69:58225 (1506 connections now open) 2016-03-13T08:58:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55820 #4079133 (1506 connections now open) 2016-03-13T08:58:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40834 #4079134 (1507 connections now open) 2016-03-13T08:58:59.117+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:58:57.416+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:58:59.285+0000 I NETWORK [conn4079122] end connection 10.0.179.209:38835 (1506 connections now open) 2016-03-13T08:58:59.285+0000 I NETWORK [conn4079121] end connection 10.0.179.209:38834 (1506 connections now open) 2016-03-13T08:58:59.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38847 #4079135 (1506 connections now open) 2016-03-13T08:58:59.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38846 #4079136 (1507 connections now open) 2016-03-13T08:58:59.934+0000 I NETWORK [conn4079123] end connection 10.0.26.69:40817 (1506 connections now open) 2016-03-13T08:58:59.934+0000 I NETWORK [conn4079124] end connection 10.0.26.69:40818 (1506 connections now open) 2016-03-13T08:58:59.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40835 #4079137 (1506 connections now open) 2016-03-13T08:58:59.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40836 #4079138 (1507 connections now open) 2016-03-13T08:59:00.613+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633fec0ce48b40ea18eeb51')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fec0ce48b40ea18eeb51') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ffc6584ad10f212db43d') }, shard: "shmydomain1", lastmod: Timestamp 30574000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:59:00.769+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30574|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633fec0ce48b40ea18eeb51') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ffc6584ad10f212db43d') }) shmydomain1 -> shmydomain3 2016-03-13T08:59:02.837+0000 I NETWORK [conn4079125] end connection 10.0.27.237:34075 (1506 connections now open) 2016-03-13T08:59:02.837+0000 I NETWORK [conn4079126] end connection 10.0.27.237:34074 (1506 connections now open) 2016-03-13T08:59:02.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34087 #4079139 (1506 connections now open) 2016-03-13T08:59:02.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34086 #4079140 (1507 connections now open) 2016-03-13T08:59:04.943+0000 I NETWORK [conn4079128] end connection 10.0.7.113:57779 (1506 connections now open) 2016-03-13T08:59:04.943+0000 I NETWORK [conn4079127] end connection 10.0.7.113:57780 (1506 connections now open) 2016-03-13T08:59:04.943+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57801 #4079141 (1507 connections now open) 2016-03-13T08:59:04.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57802 #4079142 (1507 connections now open) 2016-03-13T08:59:06.095+0000 I NETWORK [conn4079129] end connection 10.0.216.90:55812 (1506 connections now open) 2016-03-13T08:59:06.095+0000 I NETWORK [conn4079130] end connection 10.0.216.90:55813 (1506 connections now open) 2016-03-13T08:59:06.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55828 #4079143 (1506 connections now open) 2016-03-13T08:59:06.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55829 #4079144 (1507 connections now open) 2016-03-13T08:59:07.013+0000 I NETWORK [conn4079132] end connection 10.0.20.53:49002 (1506 connections now open) 2016-03-13T08:59:07.013+0000 I NETWORK [conn4079131] end connection 10.0.20.53:49003 (1505 connections now open) 2016-03-13T08:59:07.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49025 #4079145 (1506 connections now open) 2016-03-13T08:59:07.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49026 #4079146 (1507 connections now open) 2016-03-13T08:59:09.308+0000 I NETWORK [conn4079135] end connection 10.0.179.209:38847 (1506 connections now open) 2016-03-13T08:59:09.308+0000 I NETWORK [conn4079136] end connection 10.0.179.209:38846 (1506 connections now open) 2016-03-13T08:59:09.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38866 #4079147 (1506 connections now open) 2016-03-13T08:59:09.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38865 #4079148 (1507 connections now open) 2016-03-13T08:59:09.948+0000 I NETWORK [conn4079137] end connection 10.0.26.69:40835 (1506 connections now open) 2016-03-13T08:59:09.948+0000 I NETWORK [conn4079138] end connection 10.0.26.69:40836 (1506 connections now open) 2016-03-13T08:59:09.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40844 #4079149 (1507 connections now open) 2016-03-13T08:59:09.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40845 #4079150 (1507 connections now open) 2016-03-13T08:59:10.685+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 709ms sequenceNumber: 16750 version: 30575|1||563ba74d869758a4542b9075 based on: 30574|1||563ba74d869758a4542b9075 2016-03-13T08:59:10.961+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:59:10.961+0000-56e52bde18fa188537623ce2", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859550961), what: "balancer.round", ns: "", details: { executionTimeMillis: 14640, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:59:11.120+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:59:12.521+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52be018fa188537623ce3 2016-03-13T08:59:12.851+0000 I NETWORK [conn4079140] end connection 10.0.27.237:34086 (1506 connections now open) 2016-03-13T08:59:12.851+0000 I NETWORK [conn4079139] end connection 10.0.27.237:34087 (1506 connections now open) 2016-03-13T08:59:12.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34100 #4079151 (1506 connections now open) 2016-03-13T08:59:12.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34101 #4079152 (1507 connections now open) 2016-03-13T08:59:14.956+0000 I NETWORK [conn4079141] end connection 10.0.7.113:57801 (1506 connections now open) 2016-03-13T08:59:14.956+0000 I NETWORK [conn4079142] end connection 10.0.7.113:57802 (1506 connections now open) 2016-03-13T08:59:14.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57813 #4079153 (1506 connections now open) 2016-03-13T08:59:14.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57814 #4079154 (1507 connections now open) 2016-03-13T08:59:15.034+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5633ffc6584ad10f212db43d')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ffc6584ad10f212db43d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563400d2ce48b40ea18f188b') }, shard: "shmydomain1", lastmod: Timestamp 30575000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:59:15.189+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30575|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5633ffc6584ad10f212db43d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563400d2ce48b40ea18f188b') }) shmydomain1 -> shmydomain3 2016-03-13T08:59:16.109+0000 I NETWORK [conn4079144] end connection 10.0.216.90:55829 (1506 connections now open) 2016-03-13T08:59:16.109+0000 I NETWORK [conn4079143] end connection 10.0.216.90:55828 (1506 connections now open) 2016-03-13T08:59:16.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55845 #4079155 (1506 connections now open) 2016-03-13T08:59:16.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55846 #4079156 (1507 connections now open) 2016-03-13T08:59:16.118+0000 I NETWORK [conn4074211] end connection 10.0.179.209:33852 (1506 connections now open) 2016-03-13T08:59:16.118+0000 I NETWORK [conn4067858] end connection 10.0.216.90:43757 (1506 connections now open) 2016-03-13T08:59:16.118+0000 I NETWORK [conn4076310] end connection 10.0.20.53:46126 (1506 connections now open) 2016-03-13T08:59:16.118+0000 I NETWORK [conn4071310] end connection 10.0.27.237:54362 (1503 connections now open) 2016-03-13T08:59:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55848 #4079157 (1504 connections now open) 2016-03-13T08:59:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38871 #4079158 (1505 connections now open) 2016-03-13T08:59:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49033 #4079159 (1506 connections now open) 2016-03-13T08:59:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34108 #4079160 (1507 connections now open) 2016-03-13T08:59:17.035+0000 I NETWORK [conn4079146] end connection 10.0.20.53:49026 (1506 connections now open) 2016-03-13T08:59:17.035+0000 I NETWORK [conn4079145] end connection 10.0.20.53:49025 (1506 connections now open) 2016-03-13T08:59:17.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49035 #4079161 (1506 connections now open) 2016-03-13T08:59:17.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49034 #4079162 (1507 connections now open) 2016-03-13T08:59:19.321+0000 I NETWORK [conn4079148] end connection 10.0.179.209:38865 (1506 connections now open) 2016-03-13T08:59:19.321+0000 I NETWORK [conn4079147] end connection 10.0.179.209:38866 (1506 connections now open) 2016-03-13T08:59:19.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38877 #4079163 (1506 connections now open) 2016-03-13T08:59:19.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38878 #4079164 (1507 connections now open) 2016-03-13T08:59:19.961+0000 I NETWORK [conn4079150] end connection 10.0.26.69:40845 (1506 connections now open) 2016-03-13T08:59:19.961+0000 I NETWORK [conn4079149] end connection 10.0.26.69:40844 (1506 connections now open) 2016-03-13T08:59:19.962+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40862 #4079165 (1507 connections now open) 2016-03-13T08:59:19.962+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40861 #4079166 (1507 connections now open) 2016-03-13T08:59:22.865+0000 I NETWORK [conn4079151] end connection 10.0.27.237:34100 (1506 connections now open) 2016-03-13T08:59:22.865+0000 I NETWORK [conn4079152] end connection 10.0.27.237:34101 (1506 connections now open) 2016-03-13T08:59:22.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34121 #4079167 (1506 connections now open) 2016-03-13T08:59:22.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34120 #4079168 (1507 connections now open) 2016-03-13T08:59:24.970+0000 I NETWORK [conn4079154] end connection 10.0.7.113:57814 (1506 connections now open) 2016-03-13T08:59:24.970+0000 I NETWORK [conn4079153] end connection 10.0.7.113:57813 (1505 connections now open) 2016-03-13T08:59:24.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57825 #4079169 (1506 connections now open) 2016-03-13T08:59:24.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57826 #4079170 (1507 connections now open) 2016-03-13T08:59:25.041+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16751 version: 30576|1||563ba74d869758a4542b9075 based on: 30575|1||563ba74d869758a4542b9075 2016-03-13T08:59:25.312+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:59:25.312+0000-56e52bed18fa188537623ce4", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859565312), what: "balancer.round", ns: "", details: { executionTimeMillis: 13089, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:59:25.459+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:59:26.122+0000 I NETWORK [conn4079156] end connection 10.0.216.90:55846 (1506 connections now open) 2016-03-13T08:59:26.122+0000 I NETWORK [conn4079155] end connection 10.0.216.90:55845 (1506 connections now open) 2016-03-13T08:59:26.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55854 #4079171 (1506 connections now open) 2016-03-13T08:59:26.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55853 #4079172 (1507 connections now open) 2016-03-13T08:59:26.771+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52bee18fa188537623ce5 2016-03-13T08:59:27.048+0000 I NETWORK [conn4079162] end connection 10.0.20.53:49034 (1506 connections now open) 2016-03-13T08:59:27.048+0000 I NETWORK [conn4079161] end connection 10.0.20.53:49035 (1505 connections now open) 2016-03-13T08:59:27.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49047 #4079173 (1506 connections now open) 2016-03-13T08:59:27.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49048 #4079174 (1507 connections now open) 2016-03-13T08:59:28.114+0000 I NETWORK [conn4068873] end connection 10.0.26.69:58089 (1506 connections now open) 2016-03-13T08:59:28.116+0000 I NETWORK [conn4071963] end connection 10.0.27.237:55058 (1505 connections now open) 2016-03-13T08:59:28.117+0000 I NETWORK [conn4072370] end connection 10.0.7.113:50582 (1504 connections now open) 2016-03-13T08:59:28.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34129 #4079175 (1505 connections now open) 2016-03-13T08:59:28.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57832 #4079176 (1506 connections now open) 2016-03-13T08:59:28.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40879 #4079177 (1507 connections now open) 2016-03-13T08:59:29.334+0000 I NETWORK [conn4079163] end connection 10.0.179.209:38877 (1506 connections now open) 2016-03-13T08:59:29.334+0000 I NETWORK [conn4079164] end connection 10.0.179.209:38878 (1506 connections now open) 2016-03-13T08:59:29.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38889 #4079178 (1506 connections now open) 2016-03-13T08:59:29.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38890 #4079179 (1507 connections now open) 2016-03-13T08:59:29.397+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:59:29.117+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T08:59:29.519+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563400d2ce48b40ea18f188b')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563400d2ce48b40ea18f188b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563401d799bdd00ea51b14e2') }, shard: "shmydomain1", lastmod: Timestamp 30576000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:59:29.672+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30576|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563400d2ce48b40ea18f188b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563401d799bdd00ea51b14e2') }) shmydomain1 -> shmydomain3 2016-03-13T08:59:29.975+0000 I NETWORK [conn4079165] end connection 10.0.26.69:40862 (1506 connections now open) 2016-03-13T08:59:29.975+0000 I NETWORK [conn4079166] end connection 10.0.26.69:40861 (1506 connections now open) 2016-03-13T08:59:29.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40881 #4079180 (1507 connections now open) 2016-03-13T08:59:29.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40880 #4079181 (1507 connections now open) 2016-03-13T08:59:32.886+0000 I NETWORK [conn4079167] end connection 10.0.27.237:34121 (1506 connections now open) 2016-03-13T08:59:32.886+0000 I NETWORK [conn4079168] end connection 10.0.27.237:34120 (1506 connections now open) 2016-03-13T08:59:32.886+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34131 #4079182 (1506 connections now open) 2016-03-13T08:59:32.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34130 #4079183 (1507 connections now open) 2016-03-13T08:59:34.983+0000 I NETWORK [conn4079169] end connection 10.0.7.113:57825 (1506 connections now open) 2016-03-13T08:59:34.983+0000 I NETWORK [conn4079170] end connection 10.0.7.113:57826 (1505 connections now open) 2016-03-13T08:59:34.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57835 #4079184 (1507 connections now open) 2016-03-13T08:59:34.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57836 #4079185 (1507 connections now open) 2016-03-13T08:59:36.143+0000 I NETWORK [conn4079171] end connection 10.0.216.90:55854 (1506 connections now open) 2016-03-13T08:59:36.143+0000 I NETWORK [conn4079172] end connection 10.0.216.90:55853 (1506 connections now open) 2016-03-13T08:59:36.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55872 #4079186 (1506 connections now open) 2016-03-13T08:59:36.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55871 #4079187 (1507 connections now open) 2016-03-13T08:59:37.060+0000 I NETWORK [conn4079174] end connection 10.0.20.53:49048 (1506 connections now open) 2016-03-13T08:59:37.060+0000 I NETWORK [conn4079173] end connection 10.0.20.53:49047 (1506 connections now open) 2016-03-13T08:59:37.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49072 #4079188 (1506 connections now open) 2016-03-13T08:59:37.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49071 #4079189 (1507 connections now open) 2016-03-13T08:59:38.562+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 676ms sequenceNumber: 16752 version: 30577|1||563ba74d869758a4542b9075 based on: 30576|1||563ba74d869758a4542b9075 2016-03-13T08:59:38.750+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:59:38.750+0000-56e52bfa18fa188537623ce6", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859578750), what: "balancer.round", ns: "", details: { executionTimeMillis: 12242, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:59:38.945+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:59:39.347+0000 I NETWORK [conn4079178] end connection 10.0.179.209:38889 (1506 connections now open) 2016-03-13T08:59:39.347+0000 I NETWORK [conn4079179] end connection 10.0.179.209:38890 (1506 connections now open) 2016-03-13T08:59:39.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38906 #4079190 (1506 connections now open) 2016-03-13T08:59:39.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38907 #4079191 (1507 connections now open) 2016-03-13T08:59:39.988+0000 I NETWORK [conn4079181] end connection 10.0.26.69:40880 (1506 connections now open) 2016-03-13T08:59:39.988+0000 I NETWORK [conn4079180] end connection 10.0.26.69:40881 (1506 connections now open) 2016-03-13T08:59:39.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40888 #4079192 (1507 connections now open) 2016-03-13T08:59:39.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40889 #4079193 (1507 connections now open) 2016-03-13T08:59:40.311+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52bfc18fa188537623ce7 2016-03-13T08:59:42.866+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563401d799bdd00ea51b14e2')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563401d799bdd00ea51b14e2') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563402d87bc3500e9459b0eb') }, shard: "shmydomain1", lastmod: Timestamp 30577000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:59:42.900+0000 I NETWORK [conn4079182] end connection 10.0.27.237:34131 (1506 connections now open) 2016-03-13T08:59:42.900+0000 I NETWORK [conn4079183] end connection 10.0.27.237:34130 (1506 connections now open) 2016-03-13T08:59:42.900+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34143 #4079194 (1506 connections now open) 2016-03-13T08:59:42.900+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34144 #4079195 (1507 connections now open) 2016-03-13T08:59:43.024+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30577|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563401d799bdd00ea51b14e2') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563402d87bc3500e9459b0eb') }) shmydomain1 -> shmydomain3 2016-03-13T08:59:44.997+0000 I NETWORK [conn4079184] end connection 10.0.7.113:57835 (1506 connections now open) 2016-03-13T08:59:44.997+0000 I NETWORK [conn4079185] end connection 10.0.7.113:57836 (1506 connections now open) 2016-03-13T08:59:44.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57852 #4079196 (1507 connections now open) 2016-03-13T08:59:44.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57853 #4079197 (1507 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [conn4068943] end connection 10.0.179.209:56786 (1506 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [conn4073205] end connection 10.0.20.53:42828 (1506 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [conn4075394] end connection 10.0.27.237:58577 (1504 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [conn4068749] end connection 10.0.7.113:46700 (1503 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38912 #4079198 (1504 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49080 #4079199 (1505 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34150 #4079200 (1506 connections now open) 2016-03-13T08:59:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57855 #4079201 (1507 connections now open) 2016-03-13T08:59:46.156+0000 I NETWORK [conn4079187] end connection 10.0.216.90:55871 (1506 connections now open) 2016-03-13T08:59:46.156+0000 I NETWORK [conn4079186] end connection 10.0.216.90:55872 (1506 connections now open) 2016-03-13T08:59:46.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55892 #4079202 (1506 connections now open) 2016-03-13T08:59:46.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55891 #4079203 (1507 connections now open) 2016-03-13T08:59:47.074+0000 I NETWORK [conn4079188] end connection 10.0.20.53:49072 (1506 connections now open) 2016-03-13T08:59:47.074+0000 I NETWORK [conn4079189] end connection 10.0.20.53:49071 (1506 connections now open) 2016-03-13T08:59:47.074+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49082 #4079204 (1506 connections now open) 2016-03-13T08:59:47.074+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49081 #4079205 (1507 connections now open) 2016-03-13T08:59:49.361+0000 I NETWORK [conn4079190] end connection 10.0.179.209:38906 (1506 connections now open) 2016-03-13T08:59:49.361+0000 I NETWORK [conn4079191] end connection 10.0.179.209:38907 (1506 connections now open) 2016-03-13T08:59:49.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38915 #4079206 (1506 connections now open) 2016-03-13T08:59:49.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38916 #4079207 (1507 connections now open) 2016-03-13T08:59:50.001+0000 I NETWORK [conn4079193] end connection 10.0.26.69:40889 (1506 connections now open) 2016-03-13T08:59:50.001+0000 I NETWORK [conn4079192] end connection 10.0.26.69:40888 (1506 connections now open) 2016-03-13T08:59:50.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40903 #4079208 (1507 connections now open) 2016-03-13T08:59:50.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40904 #4079209 (1507 connections now open) 2016-03-13T08:59:51.883+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16753 version: 30578|1||563ba74d869758a4542b9075 based on: 30577|1||563ba74d869758a4542b9075 2016-03-13T08:59:52.156+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T08:59:52.156+0000-56e52c0818fa188537623ce8", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859592156), what: "balancer.round", ns: "", details: { executionTimeMillis: 12176, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T08:59:52.305+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T08:59:52.913+0000 I NETWORK [conn4079194] end connection 10.0.27.237:34143 (1506 connections now open) 2016-03-13T08:59:52.913+0000 I NETWORK [conn4079195] end connection 10.0.27.237:34144 (1506 connections now open) 2016-03-13T08:59:52.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34162 #4079210 (1506 connections now open) 2016-03-13T08:59:52.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34163 #4079211 (1507 connections now open) 2016-03-13T08:59:53.654+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c0918fa188537623ce9 2016-03-13T08:59:55.010+0000 I NETWORK [conn4079196] end connection 10.0.7.113:57852 (1506 connections now open) 2016-03-13T08:59:55.010+0000 I NETWORK [conn4079197] end connection 10.0.7.113:57853 (1505 connections now open) 2016-03-13T08:59:55.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57860 #4079212 (1507 connections now open) 2016-03-13T08:59:55.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57861 #4079213 (1507 connections now open) 2016-03-13T08:59:56.169+0000 I NETWORK [conn4079203] end connection 10.0.216.90:55891 (1506 connections now open) 2016-03-13T08:59:56.169+0000 I NETWORK [conn4079202] end connection 10.0.216.90:55892 (1505 connections now open) 2016-03-13T08:59:56.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55897 #4079214 (1506 connections now open) 2016-03-13T08:59:56.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55898 #4079215 (1507 connections now open) 2016-03-13T08:59:56.179+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563402d87bc3500e9459b0eb')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563402d87bc3500e9459b0eb') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563403dc584ad10f212e10bd') }, shard: "shmydomain1", lastmod: Timestamp 30578000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T08:59:56.331+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30578|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563402d87bc3500e9459b0eb') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563403dc584ad10f212e10bd') }) shmydomain1 -> shmydomain3 2016-03-13T08:59:57.087+0000 I NETWORK [conn4079204] end connection 10.0.20.53:49082 (1506 connections now open) 2016-03-13T08:59:57.087+0000 I NETWORK [conn4079205] end connection 10.0.20.53:49081 (1506 connections now open) 2016-03-13T08:59:57.088+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49095 #4079216 (1506 connections now open) 2016-03-13T08:59:57.088+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49094 #4079217 (1507 connections now open) 2016-03-13T08:59:58.113+0000 I NETWORK [conn4071523] end connection 10.0.216.90:47677 (1506 connections now open) 2016-03-13T08:59:58.113+0000 I NETWORK [conn4072637] end connection 10.0.26.69:33903 (1506 connections now open) 2016-03-13T08:59:58.113+0000 I NETWORK [conn4071355] end connection 10.0.7.113:49499 (1504 connections now open) 2016-03-13T08:59:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40921 #4079218 (1505 connections now open) 2016-03-13T08:59:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55905 #4079219 (1506 connections now open) 2016-03-13T08:59:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57871 #4079220 (1507 connections now open) 2016-03-13T08:59:59.374+0000 I NETWORK [conn4079206] end connection 10.0.179.209:38915 (1506 connections now open) 2016-03-13T08:59:59.374+0000 I NETWORK [conn4079207] end connection 10.0.179.209:38916 (1506 connections now open) 2016-03-13T08:59:59.384+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38929 #4079221 (1506 connections now open) 2016-03-13T08:59:59.384+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38928 #4079222 (1507 connections now open) 2016-03-13T08:59:59.668+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T08:59:59.397+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:00:00.015+0000 I NETWORK [conn4079208] end connection 10.0.26.69:40903 (1506 connections now open) 2016-03-13T09:00:00.015+0000 I NETWORK [conn4079209] end connection 10.0.26.69:40904 (1506 connections now open) 2016-03-13T09:00:00.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40924 #4079223 (1506 connections now open) 2016-03-13T09:00:00.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40923 #4079224 (1507 connections now open) 2016-03-13T09:00:02.926+0000 I NETWORK [conn4079211] end connection 10.0.27.237:34163 (1506 connections now open) 2016-03-13T09:00:02.926+0000 I NETWORK [conn4079210] end connection 10.0.27.237:34162 (1506 connections now open) 2016-03-13T09:00:02.926+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34175 #4079225 (1506 connections now open) 2016-03-13T09:00:02.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34174 #4079226 (1507 connections now open) 2016-03-13T09:00:05.023+0000 I NETWORK [conn4079213] end connection 10.0.7.113:57861 (1506 connections now open) 2016-03-13T09:00:05.024+0000 I NETWORK [conn4079212] end connection 10.0.7.113:57860 (1505 connections now open) 2016-03-13T09:00:05.024+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57877 #4079227 (1506 connections now open) 2016-03-13T09:00:05.024+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57878 #4079228 (1507 connections now open) 2016-03-13T09:00:06.182+0000 I NETWORK [conn4079214] end connection 10.0.216.90:55897 (1506 connections now open) 2016-03-13T09:00:06.182+0000 I NETWORK [conn4079215] end connection 10.0.216.90:55898 (1506 connections now open) 2016-03-13T09:00:06.182+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55916 #4079229 (1506 connections now open) 2016-03-13T09:00:06.182+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55915 #4079230 (1507 connections now open) 2016-03-13T09:00:06.306+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16754 version: 30579|1||563ba74d869758a4542b9075 based on: 30578|1||563ba74d869758a4542b9075 2016-03-13T09:00:06.580+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:00:06.580+0000-56e52c1618fa188537623cea", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859606580), what: "balancer.round", ns: "", details: { executionTimeMillis: 13238, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:00:06.800+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:00:07.100+0000 I NETWORK [conn4079216] end connection 10.0.20.53:49095 (1506 connections now open) 2016-03-13T09:00:07.100+0000 I NETWORK [conn4079217] end connection 10.0.20.53:49094 (1506 connections now open) 2016-03-13T09:00:07.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49119 #4079231 (1506 connections now open) 2016-03-13T09:00:07.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49118 #4079232 (1507 connections now open) 2016-03-13T09:00:08.250+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c1718fa188537623ceb 2016-03-13T09:00:09.396+0000 I NETWORK [conn4079222] end connection 10.0.179.209:38928 (1506 connections now open) 2016-03-13T09:00:09.396+0000 I NETWORK [conn4079221] end connection 10.0.179.209:38929 (1506 connections now open) 2016-03-13T09:00:09.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38949 #4079233 (1506 connections now open) 2016-03-13T09:00:09.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38948 #4079234 (1507 connections now open) 2016-03-13T09:00:10.028+0000 I NETWORK [conn4079224] end connection 10.0.26.69:40923 (1506 connections now open) 2016-03-13T09:00:10.028+0000 I NETWORK [conn4079223] end connection 10.0.26.69:40924 (1506 connections now open) 2016-03-13T09:00:10.029+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40934 #4079235 (1506 connections now open) 2016-03-13T09:00:10.029+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40933 #4079236 (1507 connections now open) 2016-03-13T09:00:11.169+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563403dc584ad10f212e10bd')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563403dc584ad10f212e10bd') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563404e3ce48b40ea369afed') }, shard: "shmydomain1", lastmod: Timestamp 30579000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:00:11.322+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30579|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563403dc584ad10f212e10bd') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563404e3ce48b40ea369afed') }) shmydomain1 -> shmydomain3 2016-03-13T09:00:12.939+0000 I NETWORK [conn4079226] end connection 10.0.27.237:34174 (1506 connections now open) 2016-03-13T09:00:12.939+0000 I NETWORK [conn4079225] end connection 10.0.27.237:34175 (1506 connections now open) 2016-03-13T09:00:12.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34192 #4079237 (1506 connections now open) 2016-03-13T09:00:12.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34191 #4079238 (1507 connections now open) 2016-03-13T09:00:15.037+0000 I NETWORK [conn4079228] end connection 10.0.7.113:57878 (1506 connections now open) 2016-03-13T09:00:15.037+0000 I NETWORK [conn4079227] end connection 10.0.7.113:57877 (1505 connections now open) 2016-03-13T09:00:15.046+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57898 #4079239 (1506 connections now open) 2016-03-13T09:00:15.046+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57897 #4079240 (1507 connections now open) 2016-03-13T09:00:16.118+0000 I NETWORK [conn4071608] end connection 10.0.20.53:41164 (1506 connections now open) 2016-03-13T09:00:16.118+0000 I NETWORK [conn4070460] end connection 10.0.26.69:59805 (1505 connections now open) 2016-03-13T09:00:16.118+0000 I NETWORK [conn4068834] end connection 10.0.7.113:46789 (1504 connections now open) 2016-03-13T09:00:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49127 #4079241 (1505 connections now open) 2016-03-13T09:00:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57901 #4079242 (1506 connections now open) 2016-03-13T09:00:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40948 #4079243 (1507 connections now open) 2016-03-13T09:00:16.195+0000 I NETWORK [conn4079230] end connection 10.0.216.90:55915 (1506 connections now open) 2016-03-13T09:00:16.195+0000 I NETWORK [conn4079229] end connection 10.0.216.90:55916 (1506 connections now open) 2016-03-13T09:00:16.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55935 #4079244 (1506 connections now open) 2016-03-13T09:00:16.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55936 #4079245 (1507 connections now open) 2016-03-13T09:00:17.123+0000 I NETWORK [conn4079232] end connection 10.0.20.53:49118 (1506 connections now open) 2016-03-13T09:00:17.123+0000 I NETWORK [conn4079231] end connection 10.0.20.53:49119 (1506 connections now open) 2016-03-13T09:00:17.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49128 #4079246 (1506 connections now open) 2016-03-13T09:00:17.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49129 #4079247 (1507 connections now open) 2016-03-13T09:00:19.410+0000 I NETWORK [conn4079234] end connection 10.0.179.209:38948 (1506 connections now open) 2016-03-13T09:00:19.410+0000 I NETWORK [conn4079233] end connection 10.0.179.209:38949 (1505 connections now open) 2016-03-13T09:00:19.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38961 #4079248 (1506 connections now open) 2016-03-13T09:00:19.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38962 #4079249 (1507 connections now open) 2016-03-13T09:00:20.041+0000 I NETWORK [conn4079236] end connection 10.0.26.69:40933 (1506 connections now open) 2016-03-13T09:00:20.041+0000 I NETWORK [conn4079235] end connection 10.0.26.69:40934 (1506 connections now open) 2016-03-13T09:00:20.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40951 #4079250 (1506 connections now open) 2016-03-13T09:00:20.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40950 #4079251 (1507 connections now open) 2016-03-13T09:00:22.953+0000 I NETWORK [conn4079238] end connection 10.0.27.237:34191 (1506 connections now open) 2016-03-13T09:00:22.953+0000 I NETWORK [conn4079237] end connection 10.0.27.237:34192 (1506 connections now open) 2016-03-13T09:00:22.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34213 #4079252 (1506 connections now open) 2016-03-13T09:00:22.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34212 #4079253 (1507 connections now open) 2016-03-13T09:00:23.304+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16755 version: 30580|1||563ba74d869758a4542b9075 based on: 30579|1||563ba74d869758a4542b9075 2016-03-13T09:00:23.574+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:00:23.574+0000-56e52c2718fa188537623cec", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859623574), what: "balancer.round", ns: "", details: { executionTimeMillis: 15739, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:00:23.775+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:00:25.059+0000 I NETWORK [conn4079240] end connection 10.0.7.113:57897 (1506 connections now open) 2016-03-13T09:00:25.059+0000 I NETWORK [conn4079239] end connection 10.0.7.113:57898 (1506 connections now open) 2016-03-13T09:00:25.059+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57906 #4079254 (1506 connections now open) 2016-03-13T09:00:25.059+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57907 #4079255 (1507 connections now open) 2016-03-13T09:00:26.208+0000 I NETWORK [conn4079244] end connection 10.0.216.90:55935 (1506 connections now open) 2016-03-13T09:00:26.208+0000 I NETWORK [conn4079245] end connection 10.0.216.90:55936 (1506 connections now open) 2016-03-13T09:00:26.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55941 #4079256 (1506 connections now open) 2016-03-13T09:00:26.218+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55940 #4079257 (1507 connections now open) 2016-03-13T09:00:27.136+0000 I NETWORK [conn4079247] end connection 10.0.20.53:49129 (1506 connections now open) 2016-03-13T09:00:27.136+0000 I NETWORK [conn4079246] end connection 10.0.20.53:49128 (1506 connections now open) 2016-03-13T09:00:27.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49141 #4079258 (1506 connections now open) 2016-03-13T09:00:27.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49142 #4079259 (1507 connections now open) 2016-03-13T09:00:28.113+0000 I NETWORK [conn4069762] end connection 10.0.179.209:57611 (1506 connections now open) 2016-03-13T09:00:28.113+0000 I NETWORK [conn4078756] end connection 10.0.26.69:40456 (1505 connections now open) 2016-03-13T09:00:28.113+0000 I NETWORK [conn4070376] end connection 10.0.7.113:48469 (1504 connections now open) 2016-03-13T09:00:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38972 #4079260 (1505 connections now open) 2016-03-13T09:00:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57917 #4079261 (1506 connections now open) 2016-03-13T09:00:28.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40968 #4079262 (1507 connections now open) 2016-03-13T09:00:29.423+0000 I NETWORK [conn4079249] end connection 10.0.179.209:38962 (1506 connections now open) 2016-03-13T09:00:29.423+0000 I NETWORK [conn4079248] end connection 10.0.179.209:38961 (1506 connections now open) 2016-03-13T09:00:29.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38974 #4079263 (1506 connections now open) 2016-03-13T09:00:29.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38975 #4079264 (1507 connections now open) 2016-03-13T09:00:29.818+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:00:29.668+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:00:29.818+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:00:29.886+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52c2818fa188537623ced 2016-03-13T09:00:30.055+0000 I NETWORK [conn4079251] end connection 10.0.26.69:40950 (1506 connections now open) 2016-03-13T09:00:30.055+0000 I NETWORK [conn4079250] end connection 10.0.26.69:40951 (1506 connections now open) 2016-03-13T09:00:30.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40970 #4079265 (1506 connections now open) 2016-03-13T09:00:30.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40969 #4079266 (1507 connections now open) 2016-03-13T09:00:32.974+0000 I NETWORK [conn4079253] end connection 10.0.27.237:34212 (1506 connections now open) 2016-03-13T09:00:32.974+0000 I NETWORK [conn4079252] end connection 10.0.27.237:34213 (1506 connections now open) 2016-03-13T09:00:32.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34223 #4079267 (1506 connections now open) 2016-03-13T09:00:32.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34222 #4079268 (1507 connections now open) 2016-03-13T09:00:35.072+0000 I NETWORK [conn4079254] end connection 10.0.7.113:57906 (1506 connections now open) 2016-03-13T09:00:35.072+0000 I NETWORK [conn4079255] end connection 10.0.7.113:57907 (1506 connections now open) 2016-03-13T09:00:35.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57920 #4079269 (1506 connections now open) 2016-03-13T09:00:35.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57921 #4079270 (1507 connections now open) 2016-03-13T09:00:35.855+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16756 version: 30581|1||563ba74d869758a4542b9075 based on: 30580|1||563ba74d869758a4542b9075 2016-03-13T09:00:36.230+0000 I NETWORK [conn4079256] end connection 10.0.216.90:55941 (1506 connections now open) 2016-03-13T09:00:36.230+0000 I NETWORK [conn4079257] end connection 10.0.216.90:55940 (1506 connections now open) 2016-03-13T09:00:36.231+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55959 #4079271 (1506 connections now open) 2016-03-13T09:00:36.231+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55958 #4079272 (1507 connections now open) 2016-03-13T09:00:37.149+0000 I NETWORK [conn4079258] end connection 10.0.20.53:49141 (1506 connections now open) 2016-03-13T09:00:37.149+0000 I NETWORK [conn4079259] end connection 10.0.20.53:49142 (1506 connections now open) 2016-03-13T09:00:37.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49167 #4079273 (1507 connections now open) 2016-03-13T09:00:37.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49166 #4079274 (1507 connections now open) 2016-03-13T09:00:39.436+0000 I NETWORK [conn4079264] end connection 10.0.179.209:38975 (1506 connections now open) 2016-03-13T09:00:39.436+0000 I NETWORK [conn4079263] end connection 10.0.179.209:38974 (1506 connections now open) 2016-03-13T09:00:39.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38991 #4079275 (1506 connections now open) 2016-03-13T09:00:39.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38992 #4079276 (1507 connections now open) 2016-03-13T09:00:40.068+0000 I NETWORK [conn4079266] end connection 10.0.26.69:40969 (1506 connections now open) 2016-03-13T09:00:40.068+0000 I NETWORK [conn4079265] end connection 10.0.26.69:40970 (1506 connections now open) 2016-03-13T09:00:40.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40976 #4079277 (1506 connections now open) 2016-03-13T09:00:40.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40977 #4079278 (1507 connections now open) 2016-03-13T09:00:42.987+0000 I NETWORK [conn4079268] end connection 10.0.27.237:34222 (1506 connections now open) 2016-03-13T09:00:42.987+0000 I NETWORK [conn4079267] end connection 10.0.27.237:34223 (1506 connections now open) 2016-03-13T09:00:42.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34234 #4079279 (1506 connections now open) 2016-03-13T09:00:42.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34235 #4079280 (1507 connections now open) 2016-03-13T09:00:45.085+0000 I NETWORK [conn4079269] end connection 10.0.7.113:57920 (1506 connections now open) 2016-03-13T09:00:45.085+0000 I NETWORK [conn4079270] end connection 10.0.7.113:57921 (1506 connections now open) 2016-03-13T09:00:45.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57936 #4079281 (1506 connections now open) 2016-03-13T09:00:45.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57937 #4079282 (1507 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [conn4067650] end connection 10.0.20.53:37026 (1506 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [conn4073725] end connection 10.0.216.90:50038 (1506 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [conn4071141] end connection 10.0.26.69:60537 (1506 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [conn4074669] end connection 10.0.179.209:34343 (1504 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [conn4069432] end connection 10.0.7.113:47428 (1502 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49174 #4079283 (1503 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55975 #4079284 (1504 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40989 #4079285 (1505 connections now open) 2016-03-13T09:00:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:38996 #4079286 (1506 connections now open) 2016-03-13T09:00:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57941 #4079287 (1507 connections now open) 2016-03-13T09:00:46.243+0000 I NETWORK [conn4079272] end connection 10.0.216.90:55958 (1506 connections now open) 2016-03-13T09:00:46.243+0000 I NETWORK [conn4079271] end connection 10.0.216.90:55959 (1506 connections now open) 2016-03-13T09:00:46.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55976 #4079288 (1506 connections now open) 2016-03-13T09:00:46.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55977 #4079289 (1507 connections now open) 2016-03-13T09:00:47.162+0000 I NETWORK [conn4079274] end connection 10.0.20.53:49166 (1506 connections now open) 2016-03-13T09:00:47.163+0000 I NETWORK [conn4079273] end connection 10.0.20.53:49167 (1506 connections now open) 2016-03-13T09:00:47.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49176 #4079290 (1506 connections now open) 2016-03-13T09:00:47.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49175 #4079291 (1507 connections now open) 2016-03-13T09:00:49.449+0000 I NETWORK [conn4079275] end connection 10.0.179.209:38991 (1506 connections now open) 2016-03-13T09:00:49.449+0000 I NETWORK [conn4079276] end connection 10.0.179.209:38992 (1505 connections now open) 2016-03-13T09:00:49.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39000 #4079292 (1506 connections now open) 2016-03-13T09:00:49.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39001 #4079293 (1507 connections now open) 2016-03-13T09:00:50.082+0000 I NETWORK [conn4079277] end connection 10.0.26.69:40976 (1506 connections now open) 2016-03-13T09:00:50.082+0000 I NETWORK [conn4079278] end connection 10.0.26.69:40977 (1506 connections now open) 2016-03-13T09:00:50.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40993 #4079294 (1506 connections now open) 2016-03-13T09:00:50.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:40992 #4079295 (1507 connections now open) 2016-03-13T09:00:50.181+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16757 version: 30582|1||563ba74d869758a4542b9075 based on: 30581|1||563ba74d869758a4542b9075 2016-03-13T09:00:53.000+0000 I NETWORK [conn4079279] end connection 10.0.27.237:34234 (1506 connections now open) 2016-03-13T09:00:53.000+0000 I NETWORK [conn4079280] end connection 10.0.27.237:34235 (1506 connections now open) 2016-03-13T09:00:53.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34254 #4079296 (1506 connections now open) 2016-03-13T09:00:53.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34255 #4079297 (1507 connections now open) 2016-03-13T09:00:55.099+0000 I NETWORK [conn4079282] end connection 10.0.7.113:57937 (1506 connections now open) 2016-03-13T09:00:55.099+0000 I NETWORK [conn4079281] end connection 10.0.7.113:57936 (1506 connections now open) 2016-03-13T09:00:55.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57946 #4079298 (1507 connections now open) 2016-03-13T09:00:55.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57945 #4079299 (1507 connections now open) 2016-03-13T09:00:56.257+0000 I NETWORK [conn4079289] end connection 10.0.216.90:55977 (1506 connections now open) 2016-03-13T09:00:56.257+0000 I NETWORK [conn4079288] end connection 10.0.216.90:55976 (1506 connections now open) 2016-03-13T09:00:56.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55983 #4079300 (1506 connections now open) 2016-03-13T09:00:56.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55984 #4079301 (1507 connections now open) 2016-03-13T09:00:57.176+0000 I NETWORK [conn4079290] end connection 10.0.20.53:49176 (1506 connections now open) 2016-03-13T09:00:57.176+0000 I NETWORK [conn4079291] end connection 10.0.20.53:49175 (1506 connections now open) 2016-03-13T09:00:57.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49187 #4079302 (1506 connections now open) 2016-03-13T09:00:57.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49188 #4079303 (1507 connections now open) 2016-03-13T09:00:58.113+0000 I NETWORK [conn4072552] end connection 10.0.179.209:60383 (1506 connections now open) 2016-03-13T09:00:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39011 #4079304 (1507 connections now open) 2016-03-13T09:00:58.115+0000 I NETWORK [conn4069154] end connection 10.0.216.90:45146 (1506 connections now open) 2016-03-13T09:00:58.115+0000 I NETWORK [conn4078080] end connection 10.0.7.113:56685 (1505 connections now open) 2016-03-13T09:00:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55990 #4079305 (1506 connections now open) 2016-03-13T09:00:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57957 #4079306 (1507 connections now open) 2016-03-13T09:00:59.461+0000 I NETWORK [conn4079292] end connection 10.0.179.209:39000 (1506 connections now open) 2016-03-13T09:00:59.462+0000 I NETWORK [conn4079293] end connection 10.0.179.209:39001 (1505 connections now open) 2016-03-13T09:00:59.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39013 #4079307 (1506 connections now open) 2016-03-13T09:00:59.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39012 #4079308 (1507 connections now open) 2016-03-13T09:01:00.095+0000 I NETWORK [conn4079294] end connection 10.0.26.69:40993 (1506 connections now open) 2016-03-13T09:01:00.095+0000 I NETWORK [conn4079295] end connection 10.0.26.69:40992 (1506 connections now open) 2016-03-13T09:01:00.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41011 #4079309 (1506 connections now open) 2016-03-13T09:01:00.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41012 #4079310 (1507 connections now open) 2016-03-13T09:01:00.164+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:00:59.886+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:01:00.164+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:01:00.300+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52c3218fa188537623cee 2016-03-13T09:01:00.437+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52c3d18fa188537623cef 2016-03-13T09:01:00.573+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52c4718fa188537623cf0 2016-03-13T09:01:03.013+0000 I NETWORK [conn4079296] end connection 10.0.27.237:34254 (1506 connections now open) 2016-03-13T09:01:03.013+0000 I NETWORK [conn4079297] end connection 10.0.27.237:34255 (1505 connections now open) 2016-03-13T09:01:03.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34266 #4079311 (1506 connections now open) 2016-03-13T09:01:03.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34267 #4079312 (1507 connections now open) 2016-03-13T09:01:04.019+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16758 version: 30583|1||563ba74d869758a4542b9075 based on: 30582|1||563ba74d869758a4542b9075 2016-03-13T09:01:05.111+0000 I NETWORK [conn4079299] end connection 10.0.7.113:57945 (1506 connections now open) 2016-03-13T09:01:05.111+0000 I NETWORK [conn4079298] end connection 10.0.7.113:57946 (1506 connections now open) 2016-03-13T09:01:05.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57962 #4079313 (1507 connections now open) 2016-03-13T09:01:05.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57963 #4079314 (1507 connections now open) 2016-03-13T09:01:06.270+0000 I NETWORK [conn4079300] end connection 10.0.216.90:55983 (1506 connections now open) 2016-03-13T09:01:06.270+0000 I NETWORK [conn4079301] end connection 10.0.216.90:55984 (1506 connections now open) 2016-03-13T09:01:06.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55999 #4079315 (1506 connections now open) 2016-03-13T09:01:06.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:55998 #4079316 (1507 connections now open) 2016-03-13T09:01:06.589+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c5118fa188537623cf1 2016-03-13T09:01:07.190+0000 I NETWORK [conn4079302] end connection 10.0.20.53:49187 (1506 connections now open) 2016-03-13T09:01:07.190+0000 I NETWORK [conn4079303] end connection 10.0.20.53:49188 (1506 connections now open) 2016-03-13T09:01:07.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49208 #4079317 (1506 connections now open) 2016-03-13T09:01:07.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49209 #4079318 (1507 connections now open) 2016-03-13T09:01:09.329+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563407eea2a4330ea28ebf60')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563407eea2a4330ea28ebf60') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563408f6a2a4330ea41aac59') }, shard: "shmydomain1", lastmod: Timestamp 30583000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:01:09.481+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30583|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563407eea2a4330ea28ebf60') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563408f6a2a4330ea41aac59') }) shmydomain1 -> shmydomain3 2016-03-13T09:01:09.541+0000 I NETWORK [conn4079307] end connection 10.0.179.209:39013 (1506 connections now open) 2016-03-13T09:01:09.541+0000 I NETWORK [conn4079308] end connection 10.0.179.209:39012 (1506 connections now open) 2016-03-13T09:01:09.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39032 #4079319 (1506 connections now open) 2016-03-13T09:01:09.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39031 #4079320 (1507 connections now open) 2016-03-13T09:01:10.109+0000 I NETWORK [conn4079309] end connection 10.0.26.69:41011 (1506 connections now open) 2016-03-13T09:01:10.109+0000 I NETWORK [conn4079310] end connection 10.0.26.69:41012 (1505 connections now open) 2016-03-13T09:01:10.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41020 #4079321 (1507 connections now open) 2016-03-13T09:01:10.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41021 #4079322 (1507 connections now open) 2016-03-13T09:01:13.026+0000 I NETWORK [conn4079311] end connection 10.0.27.237:34266 (1506 connections now open) 2016-03-13T09:01:13.026+0000 I NETWORK [conn4079312] end connection 10.0.27.237:34267 (1506 connections now open) 2016-03-13T09:01:13.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34279 #4079323 (1506 connections now open) 2016-03-13T09:01:13.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34280 #4079324 (1507 connections now open) 2016-03-13T09:01:15.124+0000 I NETWORK [conn4079314] end connection 10.0.7.113:57963 (1506 connections now open) 2016-03-13T09:01:15.124+0000 I NETWORK [conn4079313] end connection 10.0.7.113:57962 (1506 connections now open) 2016-03-13T09:01:15.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57982 #4079325 (1506 connections now open) 2016-03-13T09:01:15.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57983 #4079326 (1507 connections now open) 2016-03-13T09:01:16.118+0000 I NETWORK [conn4073666] end connection 10.0.179.209:33281 (1506 connections now open) 2016-03-13T09:01:16.118+0000 I NETWORK [conn4071695] end connection 10.0.20.53:41255 (1506 connections now open) 2016-03-13T09:01:16.118+0000 I NETWORK [conn4071183] end connection 10.0.26.69:60579 (1504 connections now open) 2016-03-13T09:01:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49218 #4079327 (1505 connections now open) 2016-03-13T09:01:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39036 #4079328 (1506 connections now open) 2016-03-13T09:01:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41035 #4079329 (1507 connections now open) 2016-03-13T09:01:16.282+0000 I NETWORK [conn4079315] end connection 10.0.216.90:55999 (1506 connections now open) 2016-03-13T09:01:16.283+0000 I NETWORK [conn4079316] end connection 10.0.216.90:55998 (1505 connections now open) 2016-03-13T09:01:16.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56016 #4079330 (1506 connections now open) 2016-03-13T09:01:16.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56017 #4079331 (1507 connections now open) 2016-03-13T09:01:17.212+0000 I NETWORK [conn4079318] end connection 10.0.20.53:49209 (1506 connections now open) 2016-03-13T09:01:17.212+0000 I NETWORK [conn4079317] end connection 10.0.20.53:49208 (1505 connections now open) 2016-03-13T09:01:17.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49220 #4079332 (1506 connections now open) 2016-03-13T09:01:17.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49219 #4079333 (1507 connections now open) 2016-03-13T09:01:18.528+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16759 version: 30584|1||563ba74d869758a4542b9075 based on: 30583|1||563ba74d869758a4542b9075 2016-03-13T09:01:18.803+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:01:18.803+0000-56e52c5e18fa188537623cf2", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859678803), what: "balancer.round", ns: "", details: { executionTimeMillis: 13700, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:01:18.920+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:01:19.564+0000 I NETWORK [conn4079320] end connection 10.0.179.209:39031 (1506 connections now open) 2016-03-13T09:01:19.564+0000 I NETWORK [conn4079319] end connection 10.0.179.209:39032 (1506 connections now open) 2016-03-13T09:01:19.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39044 #4079334 (1506 connections now open) 2016-03-13T09:01:19.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39043 #4079335 (1507 connections now open) 2016-03-13T09:01:20.115+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c5f18fa188537623cf3 2016-03-13T09:01:20.121+0000 I NETWORK [conn4079321] end connection 10.0.26.69:41020 (1506 connections now open) 2016-03-13T09:01:20.121+0000 I NETWORK [conn4079322] end connection 10.0.26.69:41021 (1506 connections now open) 2016-03-13T09:01:20.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41037 #4079336 (1506 connections now open) 2016-03-13T09:01:20.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41038 #4079337 (1507 connections now open) 2016-03-13T09:01:22.825+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563408f6a2a4330ea41aac59')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563408f6a2a4330ea41aac59') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563409ffce48b40ea36a33c7') }, shard: "shmydomain1", lastmod: Timestamp 30584000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:01:22.980+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30584|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563408f6a2a4330ea41aac59') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563409ffce48b40ea36a33c7') }) shmydomain1 -> shmydomain3 2016-03-13T09:01:23.038+0000 I NETWORK [conn4079323] end connection 10.0.27.237:34279 (1506 connections now open) 2016-03-13T09:01:23.038+0000 I NETWORK [conn4079324] end connection 10.0.27.237:34280 (1506 connections now open) 2016-03-13T09:01:23.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34298 #4079338 (1506 connections now open) 2016-03-13T09:01:23.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34299 #4079339 (1507 connections now open) 2016-03-13T09:01:25.145+0000 I NETWORK [conn4079326] end connection 10.0.7.113:57983 (1506 connections now open) 2016-03-13T09:01:25.145+0000 I NETWORK [conn4079325] end connection 10.0.7.113:57982 (1506 connections now open) 2016-03-13T09:01:25.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57990 #4079340 (1506 connections now open) 2016-03-13T09:01:25.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:57991 #4079341 (1507 connections now open) 2016-03-13T09:01:26.295+0000 I NETWORK [conn4079331] end connection 10.0.216.90:56017 (1506 connections now open) 2016-03-13T09:01:26.295+0000 I NETWORK [conn4079330] end connection 10.0.216.90:56016 (1506 connections now open) 2016-03-13T09:01:26.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56023 #4079342 (1506 connections now open) 2016-03-13T09:01:26.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56024 #4079343 (1507 connections now open) 2016-03-13T09:01:27.226+0000 I NETWORK [conn4079332] end connection 10.0.20.53:49220 (1506 connections now open) 2016-03-13T09:01:27.226+0000 I NETWORK [conn4079333] end connection 10.0.20.53:49219 (1506 connections now open) 2016-03-13T09:01:27.226+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49232 #4079344 (1506 connections now open) 2016-03-13T09:01:27.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49233 #4079345 (1507 connections now open) 2016-03-13T09:01:28.113+0000 I NETWORK [conn4075073] end connection 10.0.216.90:51487 (1506 connections now open) 2016-03-13T09:01:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56032 #4079346 (1507 connections now open) 2016-03-13T09:01:29.577+0000 I NETWORK [conn4079334] end connection 10.0.179.209:39044 (1506 connections now open) 2016-03-13T09:01:29.577+0000 I NETWORK [conn4079335] end connection 10.0.179.209:39043 (1506 connections now open) 2016-03-13T09:01:29.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39056 #4079347 (1506 connections now open) 2016-03-13T09:01:29.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39055 #4079348 (1507 connections now open) 2016-03-13T09:01:30.134+0000 I NETWORK [conn4079337] end connection 10.0.26.69:41038 (1506 connections now open) 2016-03-13T09:01:30.135+0000 I NETWORK [conn4079336] end connection 10.0.26.69:41037 (1505 connections now open) 2016-03-13T09:01:30.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41055 #4079349 (1506 connections now open) 2016-03-13T09:01:30.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41056 #4079350 (1507 connections now open) 2016-03-13T09:01:30.703+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:01:30.574+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:01:31.797+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16760 version: 30585|1||563ba74d869758a4542b9075 based on: 30584|1||563ba74d869758a4542b9075 2016-03-13T09:01:32.068+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:01:32.068+0000-56e52c6c18fa188537623cf4", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859692068), what: "balancer.round", ns: "", details: { executionTimeMillis: 12114, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:01:32.233+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:01:33.052+0000 I NETWORK [conn4079338] end connection 10.0.27.237:34298 (1506 connections now open) 2016-03-13T09:01:33.052+0000 I NETWORK [conn4079339] end connection 10.0.27.237:34299 (1506 connections now open) 2016-03-13T09:01:33.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34309 #4079351 (1506 connections now open) 2016-03-13T09:01:33.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34310 #4079352 (1507 connections now open) 2016-03-13T09:01:33.599+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c6d18fa188537623cf5 2016-03-13T09:01:35.159+0000 I NETWORK [conn4079340] end connection 10.0.7.113:57990 (1506 connections now open) 2016-03-13T09:01:35.159+0000 I NETWORK [conn4079341] end connection 10.0.7.113:57991 (1506 connections now open) 2016-03-13T09:01:35.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58006 #4079353 (1506 connections now open) 2016-03-13T09:01:35.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58005 #4079354 (1507 connections now open) 2016-03-13T09:01:36.087+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563409ffce48b40ea36a33c7')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563409ffce48b40ea36a33c7') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340b0fa2a4330ea28f0aed') }, shard: "shmydomain1", lastmod: Timestamp 30585000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:01:36.242+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30585|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563409ffce48b40ea36a33c7') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340b0fa2a4330ea28f0aed') }) shmydomain1 -> shmydomain3 2016-03-13T09:01:36.317+0000 I NETWORK [conn4079342] end connection 10.0.216.90:56023 (1506 connections now open) 2016-03-13T09:01:36.317+0000 I NETWORK [conn4079343] end connection 10.0.216.90:56024 (1506 connections now open) 2016-03-13T09:01:36.317+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56040 #4079355 (1506 connections now open) 2016-03-13T09:01:36.317+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56041 #4079356 (1507 connections now open) 2016-03-13T09:01:37.239+0000 I NETWORK [conn4079345] end connection 10.0.20.53:49233 (1506 connections now open) 2016-03-13T09:01:37.239+0000 I NETWORK [conn4079344] end connection 10.0.20.53:49232 (1506 connections now open) 2016-03-13T09:01:37.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49256 #4079357 (1506 connections now open) 2016-03-13T09:01:37.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49255 #4079358 (1507 connections now open) 2016-03-13T09:01:39.590+0000 I NETWORK [conn4079348] end connection 10.0.179.209:39055 (1506 connections now open) 2016-03-13T09:01:39.590+0000 I NETWORK [conn4079347] end connection 10.0.179.209:39056 (1506 connections now open) 2016-03-13T09:01:39.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39072 #4079359 (1506 connections now open) 2016-03-13T09:01:39.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39073 #4079360 (1507 connections now open) 2016-03-13T09:01:40.156+0000 I NETWORK [conn4079350] end connection 10.0.26.69:41056 (1506 connections now open) 2016-03-13T09:01:40.156+0000 I NETWORK [conn4079349] end connection 10.0.26.69:41055 (1506 connections now open) 2016-03-13T09:01:40.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41062 #4079361 (1506 connections now open) 2016-03-13T09:01:40.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41063 #4079362 (1507 connections now open) 2016-03-13T09:01:43.066+0000 I NETWORK [conn4079352] end connection 10.0.27.237:34310 (1506 connections now open) 2016-03-13T09:01:43.066+0000 I NETWORK [conn4079351] end connection 10.0.27.237:34309 (1506 connections now open) 2016-03-13T09:01:43.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34320 #4079363 (1506 connections now open) 2016-03-13T09:01:43.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34319 #4079364 (1507 connections now open) 2016-03-13T09:01:44.975+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 679ms sequenceNumber: 16761 version: 30586|1||563ba74d869758a4542b9075 based on: 30585|1||563ba74d869758a4542b9075 2016-03-13T09:01:45.172+0000 I NETWORK [conn4079353] end connection 10.0.7.113:58006 (1506 connections now open) 2016-03-13T09:01:45.172+0000 I NETWORK [conn4079354] end connection 10.0.7.113:58005 (1506 connections now open) 2016-03-13T09:01:45.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58024 #4079365 (1506 connections now open) 2016-03-13T09:01:45.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58023 #4079366 (1507 connections now open) 2016-03-13T09:01:45.248+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:01:45.248+0000-56e52c7918fa188537623cf6", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859705248), what: "balancer.round", ns: "", details: { executionTimeMillis: 11971, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:01:45.353+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:01:46.118+0000 I NETWORK [conn4073583] end connection 10.0.20.53:43229 (1506 connections now open) 2016-03-13T09:01:46.118+0000 I NETWORK [conn4073582] end connection 10.0.27.237:56779 (1506 connections now open) 2016-03-13T09:01:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49265 #4079367 (1506 connections now open) 2016-03-13T09:01:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34327 #4079368 (1507 connections now open) 2016-03-13T09:01:46.330+0000 I NETWORK [conn4079355] end connection 10.0.216.90:56040 (1506 connections now open) 2016-03-13T09:01:46.330+0000 I NETWORK [conn4079356] end connection 10.0.216.90:56041 (1506 connections now open) 2016-03-13T09:01:46.330+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56058 #4079369 (1506 connections now open) 2016-03-13T09:01:46.330+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56059 #4079370 (1507 connections now open) 2016-03-13T09:01:46.615+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c7a18fa188537623cf7 2016-03-13T09:01:47.252+0000 I NETWORK [conn4079358] end connection 10.0.20.53:49255 (1506 connections now open) 2016-03-13T09:01:47.252+0000 I NETWORK [conn4079357] end connection 10.0.20.53:49256 (1506 connections now open) 2016-03-13T09:01:47.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49267 #4079371 (1506 connections now open) 2016-03-13T09:01:47.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49266 #4079372 (1507 connections now open) 2016-03-13T09:01:49.205+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56340b0fa2a4330ea28f0aed')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340b0fa2a4330ea28f0aed') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340c1a584ad10f1f91c16a') }, shard: "shmydomain1", lastmod: Timestamp 30586000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:01:49.358+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30586|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340b0fa2a4330ea28f0aed') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340c1a584ad10f1f91c16a') }) shmydomain1 -> shmydomain3 2016-03-13T09:01:49.603+0000 I NETWORK [conn4079360] end connection 10.0.179.209:39073 (1506 connections now open) 2016-03-13T09:01:49.603+0000 I NETWORK [conn4079359] end connection 10.0.179.209:39072 (1506 connections now open) 2016-03-13T09:01:49.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39082 #4079373 (1506 connections now open) 2016-03-13T09:01:49.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39083 #4079374 (1507 connections now open) 2016-03-13T09:01:50.169+0000 I NETWORK [conn4079362] end connection 10.0.26.69:41063 (1506 connections now open) 2016-03-13T09:01:50.169+0000 I NETWORK [conn4079361] end connection 10.0.26.69:41062 (1506 connections now open) 2016-03-13T09:01:50.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41078 #4079375 (1506 connections now open) 2016-03-13T09:01:50.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41077 #4079376 (1507 connections now open) 2016-03-13T09:01:53.079+0000 I NETWORK [conn4079363] end connection 10.0.27.237:34320 (1506 connections now open) 2016-03-13T09:01:53.079+0000 I NETWORK [conn4079364] end connection 10.0.27.237:34319 (1506 connections now open) 2016-03-13T09:01:53.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34341 #4079377 (1506 connections now open) 2016-03-13T09:01:53.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34340 #4079378 (1507 connections now open) 2016-03-13T09:01:55.187+0000 I NETWORK [conn4079365] end connection 10.0.7.113:58024 (1506 connections now open) 2016-03-13T09:01:55.187+0000 I NETWORK [conn4079366] end connection 10.0.7.113:58023 (1506 connections now open) 2016-03-13T09:01:55.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58033 #4079379 (1506 connections now open) 2016-03-13T09:01:55.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58032 #4079380 (1507 connections now open) 2016-03-13T09:01:56.343+0000 I NETWORK [conn4079369] end connection 10.0.216.90:56058 (1506 connections now open) 2016-03-13T09:01:56.343+0000 I NETWORK [conn4079370] end connection 10.0.216.90:56059 (1506 connections now open) 2016-03-13T09:01:56.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56066 #4079381 (1506 connections now open) 2016-03-13T09:01:56.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56065 #4079382 (1507 connections now open) 2016-03-13T09:01:57.266+0000 I NETWORK [conn4079372] end connection 10.0.20.53:49266 (1506 connections now open) 2016-03-13T09:01:57.266+0000 I NETWORK [conn4079371] end connection 10.0.20.53:49267 (1506 connections now open) 2016-03-13T09:01:57.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49279 #4079383 (1506 connections now open) 2016-03-13T09:01:57.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49278 #4079384 (1507 connections now open) 2016-03-13T09:01:58.113+0000 I NETWORK [conn4075490] end connection 10.0.20.53:45242 (1506 connections now open) 2016-03-13T09:01:58.113+0000 I NETWORK [conn4072679] end connection 10.0.179.209:60511 (1506 connections now open) 2016-03-13T09:01:58.113+0000 I NETWORK [conn4071373] end connection 10.0.27.237:54418 (1504 connections now open) 2016-03-13T09:01:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39093 #4079385 (1506 connections now open) 2016-03-13T09:01:58.113+0000 I NETWORK [conn4071229] end connection 10.0.7.113:49371 (1504 connections now open) 2016-03-13T09:01:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49282 #4079386 (1505 connections now open) 2016-03-13T09:01:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34349 #4079387 (1506 connections now open) 2016-03-13T09:01:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58042 #4079388 (1507 connections now open) 2016-03-13T09:01:59.196+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16762 version: 30587|1||563ba74d869758a4542b9075 based on: 30586|1||563ba74d869758a4542b9075 2016-03-13T09:01:59.467+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:01:59.467+0000-56e52c8718fa188537623cf8", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859719467), what: "balancer.round", ns: "", details: { executionTimeMillis: 13080, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:01:59.613+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:01:59.617+0000 I NETWORK [conn4079373] end connection 10.0.179.209:39082 (1506 connections now open) 2016-03-13T09:01:59.617+0000 I NETWORK [conn4079374] end connection 10.0.179.209:39083 (1506 connections now open) 2016-03-13T09:01:59.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39094 #4079389 (1506 connections now open) 2016-03-13T09:01:59.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39095 #4079390 (1507 connections now open) 2016-03-13T09:02:00.183+0000 I NETWORK [conn4079375] end connection 10.0.26.69:41078 (1506 connections now open) 2016-03-13T09:02:00.183+0000 I NETWORK [conn4079376] end connection 10.0.26.69:41077 (1506 connections now open) 2016-03-13T09:02:00.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41097 #4079391 (1506 connections now open) 2016-03-13T09:02:00.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41096 #4079392 (1507 connections now open) 2016-03-13T09:02:00.842+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:02:00.703+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:02:00.842+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c8818fa188537623cf9 2016-03-13T09:02:03.092+0000 I NETWORK [conn4079378] end connection 10.0.27.237:34340 (1506 connections now open) 2016-03-13T09:02:03.092+0000 I NETWORK [conn4079377] end connection 10.0.27.237:34341 (1505 connections now open) 2016-03-13T09:02:03.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34353 #4079393 (1506 connections now open) 2016-03-13T09:02:03.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34354 #4079394 (1507 connections now open) 2016-03-13T09:02:03.866+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56340c1a584ad10f1f91c16a')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340c1a584ad10f1f91c16a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340d2a7bc3500e945ac59c') }, shard: "shmydomain1", lastmod: Timestamp 30587000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:02:04.019+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30587|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340c1a584ad10f1f91c16a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340d2a7bc3500e945ac59c') }) shmydomain1 -> shmydomain3 2016-03-13T09:02:05.201+0000 I NETWORK [conn4079380] end connection 10.0.7.113:58032 (1506 connections now open) 2016-03-13T09:02:05.201+0000 I NETWORK [conn4079379] end connection 10.0.7.113:58033 (1506 connections now open) 2016-03-13T09:02:05.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58048 #4079395 (1506 connections now open) 2016-03-13T09:02:05.202+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58047 #4079396 (1507 connections now open) 2016-03-13T09:02:06.356+0000 I NETWORK [conn4079381] end connection 10.0.216.90:56066 (1506 connections now open) 2016-03-13T09:02:06.356+0000 I NETWORK [conn4079382] end connection 10.0.216.90:56065 (1506 connections now open) 2016-03-13T09:02:06.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56080 #4079397 (1506 connections now open) 2016-03-13T09:02:06.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56079 #4079398 (1507 connections now open) 2016-03-13T09:02:07.280+0000 I NETWORK [conn4079383] end connection 10.0.20.53:49279 (1506 connections now open) 2016-03-13T09:02:07.280+0000 I NETWORK [conn4079384] end connection 10.0.20.53:49278 (1506 connections now open) 2016-03-13T09:02:07.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49300 #4079399 (1506 connections now open) 2016-03-13T09:02:07.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49301 #4079400 (1507 connections now open) 2016-03-13T09:02:09.630+0000 I NETWORK [conn4079389] end connection 10.0.179.209:39094 (1506 connections now open) 2016-03-13T09:02:09.630+0000 I NETWORK [conn4079390] end connection 10.0.179.209:39095 (1505 connections now open) 2016-03-13T09:02:09.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39114 #4079401 (1506 connections now open) 2016-03-13T09:02:09.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39115 #4079402 (1507 connections now open) 2016-03-13T09:02:10.197+0000 I NETWORK [conn4079392] end connection 10.0.26.69:41096 (1506 connections now open) 2016-03-13T09:02:10.197+0000 I NETWORK [conn4079391] end connection 10.0.26.69:41097 (1506 connections now open) 2016-03-13T09:02:10.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41105 #4079403 (1506 connections now open) 2016-03-13T09:02:10.198+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41106 #4079404 (1507 connections now open) 2016-03-13T09:02:13.105+0000 I NETWORK [conn4079393] end connection 10.0.27.237:34353 (1506 connections now open) 2016-03-13T09:02:13.105+0000 I NETWORK [conn4079394] end connection 10.0.27.237:34354 (1505 connections now open) 2016-03-13T09:02:13.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34369 #4079405 (1506 connections now open) 2016-03-13T09:02:13.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34368 #4079406 (1507 connections now open) 2016-03-13T09:02:14.092+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 678ms sequenceNumber: 16763 version: 30588|1||563ba74d869758a4542b9075 based on: 30587|1||563ba74d869758a4542b9075 2016-03-13T09:02:14.280+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:02:14.280+0000-56e52c9618fa188537623cfa", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859734280), what: "balancer.round", ns: "", details: { executionTimeMillis: 13633, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:02:14.487+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:02:15.214+0000 I NETWORK [conn4079396] end connection 10.0.7.113:58047 (1506 connections now open) 2016-03-13T09:02:15.214+0000 I NETWORK [conn4079395] end connection 10.0.7.113:58048 (1506 connections now open) 2016-03-13T09:02:15.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58068 #4079407 (1506 connections now open) 2016-03-13T09:02:15.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58069 #4079408 (1507 connections now open) 2016-03-13T09:02:15.929+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52c9718fa188537623cfb 2016-03-13T09:02:16.117+0000 I NETWORK [conn4069409] end connection 10.0.179.209:57260 (1506 connections now open) 2016-03-13T09:02:16.117+0000 I NETWORK [conn4068704] end connection 10.0.20.53:38131 (1505 connections now open) 2016-03-13T09:02:16.118+0000 I NETWORK [conn4068830] end connection 10.0.216.90:44791 (1505 connections now open) 2016-03-13T09:02:16.118+0000 I NETWORK [conn4071055] end connection 10.0.26.69:60443 (1503 connections now open) 2016-03-13T09:02:16.118+0000 I NETWORK [conn4073753] end connection 10.0.27.237:56936 (1502 connections now open) 2016-03-13T09:02:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39119 #4079409 (1503 connections now open) 2016-03-13T09:02:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56098 #4079410 (1504 connections now open) 2016-03-13T09:02:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41119 #4079411 (1505 connections now open) 2016-03-13T09:02:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34375 #4079412 (1506 connections now open) 2016-03-13T09:02:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49309 #4079413 (1507 connections now open) 2016-03-13T09:02:16.369+0000 I NETWORK [conn4079398] end connection 10.0.216.90:56079 (1506 connections now open) 2016-03-13T09:02:16.369+0000 I NETWORK [conn4079397] end connection 10.0.216.90:56080 (1506 connections now open) 2016-03-13T09:02:16.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56099 #4079414 (1506 connections now open) 2016-03-13T09:02:16.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56100 #4079415 (1507 connections now open) 2016-03-13T09:02:17.294+0000 I NETWORK [conn4079400] end connection 10.0.20.53:49301 (1506 connections now open) 2016-03-13T09:02:17.294+0000 I NETWORK [conn4079399] end connection 10.0.20.53:49300 (1505 connections now open) 2016-03-13T09:02:17.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49310 #4079416 (1506 connections now open) 2016-03-13T09:02:17.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49311 #4079417 (1507 connections now open) 2016-03-13T09:02:18.448+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56340d2a7bc3500e945ac59c')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340d2a7bc3500e945ac59c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340e3da2a4330ea41b2a5d') }, shard: "shmydomain1", lastmod: Timestamp 30588000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:02:18.602+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30588|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340d2a7bc3500e945ac59c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340e3da2a4330ea41b2a5d') }) shmydomain1 -> shmydomain3 2016-03-13T09:02:19.652+0000 I NETWORK [conn4079401] end connection 10.0.179.209:39114 (1506 connections now open) 2016-03-13T09:02:19.652+0000 I NETWORK [conn4079402] end connection 10.0.179.209:39115 (1506 connections now open) 2016-03-13T09:02:19.652+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39126 #4079418 (1506 connections now open) 2016-03-13T09:02:19.652+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39127 #4079419 (1507 connections now open) 2016-03-13T09:02:20.210+0000 I NETWORK [conn4079403] end connection 10.0.26.69:41105 (1506 connections now open) 2016-03-13T09:02:20.210+0000 I NETWORK [conn4079404] end connection 10.0.26.69:41106 (1506 connections now open) 2016-03-13T09:02:20.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41121 #4079420 (1506 connections now open) 2016-03-13T09:02:20.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41122 #4079421 (1507 connections now open) 2016-03-13T09:02:23.127+0000 I NETWORK [conn4079406] end connection 10.0.27.237:34368 (1506 connections now open) 2016-03-13T09:02:23.127+0000 I NETWORK [conn4079405] end connection 10.0.27.237:34369 (1506 connections now open) 2016-03-13T09:02:23.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34389 #4079422 (1506 connections now open) 2016-03-13T09:02:23.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34388 #4079423 (1507 connections now open) 2016-03-13T09:02:25.236+0000 I NETWORK [conn4079407] end connection 10.0.7.113:58068 (1506 connections now open) 2016-03-13T09:02:25.236+0000 I NETWORK [conn4079408] end connection 10.0.7.113:58069 (1506 connections now open) 2016-03-13T09:02:25.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58079 #4079424 (1506 connections now open) 2016-03-13T09:02:25.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58078 #4079425 (1507 connections now open) 2016-03-13T09:02:26.382+0000 I NETWORK [conn4079415] end connection 10.0.216.90:56100 (1506 connections now open) 2016-03-13T09:02:26.382+0000 I NETWORK [conn4079414] end connection 10.0.216.90:56099 (1506 connections now open) 2016-03-13T09:02:26.390+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56106 #4079426 (1506 connections now open) 2016-03-13T09:02:26.390+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56105 #4079427 (1507 connections now open) 2016-03-13T09:02:27.307+0000 I NETWORK [conn4079417] end connection 10.0.20.53:49311 (1506 connections now open) 2016-03-13T09:02:27.307+0000 I NETWORK [conn4079416] end connection 10.0.20.53:49310 (1506 connections now open) 2016-03-13T09:02:27.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49323 #4079428 (1506 connections now open) 2016-03-13T09:02:27.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49322 #4079429 (1507 connections now open) 2016-03-13T09:02:28.114+0000 I NETWORK [conn4071029] end connection 10.0.179.209:58878 (1506 connections now open) 2016-03-13T09:02:28.114+0000 I NETWORK [conn4071500] end connection 10.0.27.237:54557 (1506 connections now open) 2016-03-13T09:02:28.114+0000 I NETWORK [conn4075617] end connection 10.0.26.69:37103 (1506 connections now open) 2016-03-13T09:02:28.114+0000 I NETWORK [conn4073138] end connection 10.0.20.53:42759 (1506 connections now open) 2016-03-13T09:02:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49326 #4079430 (1504 connections now open) 2016-03-13T09:02:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39138 #4079431 (1505 connections now open) 2016-03-13T09:02:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41137 #4079432 (1506 connections now open) 2016-03-13T09:02:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34397 #4079433 (1507 connections now open) 2016-03-13T09:02:28.484+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16764 version: 30589|1||563ba74d869758a4542b9075 based on: 30588|1||563ba74d869758a4542b9075 2016-03-13T09:02:28.759+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:02:28.759+0000-56e52ca418fa188537623cfc", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859748759), what: "balancer.round", ns: "", details: { executionTimeMillis: 13221, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:02:28.957+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:02:29.665+0000 I NETWORK [conn4079419] end connection 10.0.179.209:39127 (1506 connections now open) 2016-03-13T09:02:29.665+0000 I NETWORK [conn4079418] end connection 10.0.179.209:39126 (1505 connections now open) 2016-03-13T09:02:29.665+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39139 #4079434 (1506 connections now open) 2016-03-13T09:02:29.665+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39140 #4079435 (1507 connections now open) 2016-03-13T09:02:30.187+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52ca618fa188537623cfd 2016-03-13T09:02:30.224+0000 I NETWORK [conn4079421] end connection 10.0.26.69:41122 (1506 connections now open) 2016-03-13T09:02:30.224+0000 I NETWORK [conn4079420] end connection 10.0.26.69:41121 (1506 connections now open) 2016-03-13T09:02:30.225+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41140 #4079436 (1506 connections now open) 2016-03-13T09:02:30.225+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41139 #4079437 (1507 connections now open) 2016-03-13T09:02:31.107+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:02:30.842+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:02:32.721+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56340e3da2a4330ea41b2a5d')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340e3da2a4330ea41b2a5d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340f4bce48b40ea36ab4ba') }, shard: "shmydomain1", lastmod: Timestamp 30589000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:02:32.875+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30589|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340e3da2a4330ea41b2a5d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340f4bce48b40ea36ab4ba') }) shmydomain1 -> shmydomain3 2016-03-13T09:02:33.140+0000 I NETWORK [conn4079422] end connection 10.0.27.237:34389 (1506 connections now open) 2016-03-13T09:02:33.140+0000 I NETWORK [conn4079423] end connection 10.0.27.237:34388 (1506 connections now open) 2016-03-13T09:02:33.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34399 #4079438 (1506 connections now open) 2016-03-13T09:02:33.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34400 #4079439 (1507 connections now open) 2016-03-13T09:02:35.250+0000 I NETWORK [conn4079424] end connection 10.0.7.113:58079 (1506 connections now open) 2016-03-13T09:02:35.250+0000 I NETWORK [conn4079425] end connection 10.0.7.113:58078 (1506 connections now open) 2016-03-13T09:02:35.251+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58094 #4079440 (1506 connections now open) 2016-03-13T09:02:35.251+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58093 #4079441 (1507 connections now open) 2016-03-13T09:02:36.403+0000 I NETWORK [conn4079426] end connection 10.0.216.90:56106 (1506 connections now open) 2016-03-13T09:02:36.403+0000 I NETWORK [conn4079427] end connection 10.0.216.90:56105 (1506 connections now open) 2016-03-13T09:02:36.403+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56124 #4079442 (1506 connections now open) 2016-03-13T09:02:36.403+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56123 #4079443 (1507 connections now open) 2016-03-13T09:02:37.321+0000 I NETWORK [conn4079429] end connection 10.0.20.53:49322 (1506 connections now open) 2016-03-13T09:02:37.321+0000 I NETWORK [conn4079428] end connection 10.0.20.53:49323 (1506 connections now open) 2016-03-13T09:02:37.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49346 #4079444 (1506 connections now open) 2016-03-13T09:02:37.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49347 #4079445 (1507 connections now open) 2016-03-13T09:02:39.677+0000 I NETWORK [conn4079435] end connection 10.0.179.209:39140 (1506 connections now open) 2016-03-13T09:02:39.678+0000 I NETWORK [conn4079434] end connection 10.0.179.209:39139 (1505 connections now open) 2016-03-13T09:02:39.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39157 #4079446 (1506 connections now open) 2016-03-13T09:02:39.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39158 #4079447 (1507 connections now open) 2016-03-13T09:02:40.238+0000 I NETWORK [conn4079437] end connection 10.0.26.69:41139 (1506 connections now open) 2016-03-13T09:02:40.238+0000 I NETWORK [conn4079436] end connection 10.0.26.69:41140 (1506 connections now open) 2016-03-13T09:02:40.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41147 #4079448 (1507 connections now open) 2016-03-13T09:02:40.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41146 #4079449 (1507 connections now open) 2016-03-13T09:02:41.786+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16765 version: 30590|1||563ba74d869758a4542b9075 based on: 30589|1||563ba74d869758a4542b9075 2016-03-13T09:02:42.055+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:02:42.055+0000-56e52cb218fa188537623cfe", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859762055), what: "balancer.round", ns: "", details: { executionTimeMillis: 12064, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:02:42.241+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:02:43.153+0000 I NETWORK [conn4079438] end connection 10.0.27.237:34399 (1506 connections now open) 2016-03-13T09:02:43.153+0000 I NETWORK [conn4079439] end connection 10.0.27.237:34400 (1506 connections now open) 2016-03-13T09:02:43.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34412 #4079450 (1506 connections now open) 2016-03-13T09:02:43.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34411 #4079451 (1507 connections now open) 2016-03-13T09:02:43.572+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52cb318fa188537623cff 2016-03-13T09:02:43.613+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:02:45.264+0000 I NETWORK [conn4079440] end connection 10.0.7.113:58094 (1506 connections now open) 2016-03-13T09:02:45.264+0000 I NETWORK [conn4079441] end connection 10.0.7.113:58093 (1506 connections now open) 2016-03-13T09:02:45.265+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58109 #4079452 (1506 connections now open) 2016-03-13T09:02:45.265+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58110 #4079453 (1507 connections now open) 2016-03-13T09:02:46.117+0000 I NETWORK [conn4072704] end connection 10.0.179.209:60537 (1506 connections now open) 2016-03-13T09:02:46.117+0000 I NETWORK [conn4072388] end connection 10.0.216.90:48603 (1505 connections now open) 2016-03-13T09:02:46.118+0000 I NETWORK [conn4067444] end connection 10.0.7.113:45334 (1504 connections now open) 2016-03-13T09:02:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39162 #4079454 (1505 connections now open) 2016-03-13T09:02:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56141 #4079455 (1506 connections now open) 2016-03-13T09:02:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58113 #4079456 (1507 connections now open) 2016-03-13T09:02:46.152+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56340f4bce48b40ea36ab4ba')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340f4bce48b40ea36ab4ba') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341059ce48b40ea36ad002') }, shard: "shmydomain1", lastmod: Timestamp 30590000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:02:46.308+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30590|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56340f4bce48b40ea36ab4ba') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341059ce48b40ea36ad002') }) shmydomain1 -> shmydomain3 2016-03-13T09:02:46.416+0000 I NETWORK [conn4079442] end connection 10.0.216.90:56124 (1506 connections now open) 2016-03-13T09:02:46.416+0000 I NETWORK [conn4079443] end connection 10.0.216.90:56123 (1506 connections now open) 2016-03-13T09:02:46.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56143 #4079457 (1506 connections now open) 2016-03-13T09:02:46.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56142 #4079458 (1507 connections now open) 2016-03-13T09:02:47.334+0000 I NETWORK [conn4079445] end connection 10.0.20.53:49347 (1506 connections now open) 2016-03-13T09:02:47.334+0000 I NETWORK [conn4079444] end connection 10.0.20.53:49346 (1506 connections now open) 2016-03-13T09:02:47.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49356 #4079459 (1506 connections now open) 2016-03-13T09:02:47.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49357 #4079460 (1507 connections now open) 2016-03-13T09:02:49.690+0000 I NETWORK [conn4079447] end connection 10.0.179.209:39158 (1506 connections now open) 2016-03-13T09:02:49.690+0000 I NETWORK [conn4079446] end connection 10.0.179.209:39157 (1506 connections now open) 2016-03-13T09:02:49.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39166 #4079461 (1506 connections now open) 2016-03-13T09:02:49.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39167 #4079462 (1507 connections now open) 2016-03-13T09:02:50.252+0000 I NETWORK [conn4079449] end connection 10.0.26.69:41146 (1506 connections now open) 2016-03-13T09:02:50.252+0000 I NETWORK [conn4079448] end connection 10.0.26.69:41147 (1506 connections now open) 2016-03-13T09:02:50.252+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41161 #4079463 (1506 connections now open) 2016-03-13T09:02:50.252+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41162 #4079464 (1507 connections now open) 2016-03-13T09:02:53.165+0000 I NETWORK [conn4079451] end connection 10.0.27.237:34411 (1506 connections now open) 2016-03-13T09:02:53.165+0000 I NETWORK [conn4079450] end connection 10.0.27.237:34412 (1506 connections now open) 2016-03-13T09:02:53.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34432 #4079465 (1506 connections now open) 2016-03-13T09:02:53.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34431 #4079466 (1507 connections now open) 2016-03-13T09:02:55.277+0000 I NETWORK [conn4079453] end connection 10.0.7.113:58110 (1506 connections now open) 2016-03-13T09:02:55.277+0000 I NETWORK [conn4079452] end connection 10.0.7.113:58109 (1506 connections now open) 2016-03-13T09:02:55.278+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58118 #4079467 (1506 connections now open) 2016-03-13T09:02:55.278+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58119 #4079468 (1507 connections now open) 2016-03-13T09:02:56.105+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16766 version: 30591|1||563ba74d869758a4542b9075 based on: 30590|1||563ba74d869758a4542b9075 2016-03-13T09:02:56.377+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:02:56.377+0000-56e52cc018fa188537623d00", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859776377), what: "balancer.round", ns: "", details: { executionTimeMillis: 13100, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:02:56.429+0000 I NETWORK [conn4079458] end connection 10.0.216.90:56142 (1506 connections now open) 2016-03-13T09:02:56.429+0000 I NETWORK [conn4079457] end connection 10.0.216.90:56143 (1506 connections now open) 2016-03-13T09:02:56.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56149 #4079469 (1506 connections now open) 2016-03-13T09:02:56.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56150 #4079470 (1507 connections now open) 2016-03-13T09:02:56.492+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:02:57.348+0000 I NETWORK [conn4079460] end connection 10.0.20.53:49357 (1506 connections now open) 2016-03-13T09:02:57.348+0000 I NETWORK [conn4079459] end connection 10.0.20.53:49356 (1506 connections now open) 2016-03-13T09:02:57.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49368 #4079471 (1506 connections now open) 2016-03-13T09:02:57.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49369 #4079472 (1507 connections now open) 2016-03-13T09:02:57.757+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52cc118fa188537623d01 2016-03-13T09:02:58.113+0000 I NETWORK [conn4067359] end connection 10.0.179.209:55154 (1506 connections now open) 2016-03-13T09:02:58.114+0000 I NETWORK [conn4077615] end connection 10.0.216.90:54238 (1505 connections now open) 2016-03-13T09:02:58.114+0000 I NETWORK [conn4073516] end connection 10.0.26.69:34834 (1504 connections now open) 2016-03-13T09:02:58.114+0000 I NETWORK [conn4070334] end connection 10.0.7.113:48419 (1504 connections now open) 2016-03-13T09:02:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39173 #4079473 (1504 connections now open) 2016-03-13T09:02:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56157 #4079474 (1505 connections now open) 2016-03-13T09:02:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41180 #4079475 (1506 connections now open) 2016-03-13T09:02:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58130 #4079476 (1507 connections now open) 2016-03-13T09:02:59.703+0000 I NETWORK [conn4079461] end connection 10.0.179.209:39166 (1506 connections now open) 2016-03-13T09:02:59.703+0000 I NETWORK [conn4079462] end connection 10.0.179.209:39167 (1506 connections now open) 2016-03-13T09:02:59.704+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39178 #4079477 (1506 connections now open) 2016-03-13T09:02:59.704+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39179 #4079478 (1507 connections now open) 2016-03-13T09:03:00.265+0000 I NETWORK [conn4079463] end connection 10.0.26.69:41161 (1506 connections now open) 2016-03-13T09:03:00.265+0000 I NETWORK [conn4079464] end connection 10.0.26.69:41162 (1506 connections now open) 2016-03-13T09:03:00.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41181 #4079479 (1506 connections now open) 2016-03-13T09:03:00.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41182 #4079480 (1507 connections now open) 2016-03-13T09:03:00.303+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56341059ce48b40ea36ad002')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341059ce48b40ea36ad002') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634115b7bc3500e945b2eba') }, shard: "shmydomain1", lastmod: Timestamp 30591000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:03:00.460+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30591|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341059ce48b40ea36ad002') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634115b7bc3500e945b2eba') }) shmydomain1 -> shmydomain3 2016-03-13T09:03:01.374+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:03:01.107+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:03:01.576+0000 I NETWORK [conn4078747] end connection 10.0.91.115:53998 (1506 connections now open) 2016-03-13T09:03:01.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:54429 #4079481 (1507 connections now open) 2016-03-13T09:03:03.179+0000 I NETWORK [conn4079465] end connection 10.0.27.237:34432 (1506 connections now open) 2016-03-13T09:03:03.179+0000 I NETWORK [conn4079466] end connection 10.0.27.237:34431 (1506 connections now open) 2016-03-13T09:03:03.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34445 #4079482 (1506 connections now open) 2016-03-13T09:03:03.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34444 #4079483 (1507 connections now open) 2016-03-13T09:03:05.291+0000 I NETWORK [conn4079467] end connection 10.0.7.113:58118 (1506 connections now open) 2016-03-13T09:03:05.291+0000 I NETWORK [conn4079468] end connection 10.0.7.113:58119 (1506 connections now open) 2016-03-13T09:03:05.292+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58135 #4079484 (1506 connections now open) 2016-03-13T09:03:05.292+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58136 #4079485 (1507 connections now open) 2016-03-13T09:03:06.442+0000 I NETWORK [conn4079469] end connection 10.0.216.90:56149 (1506 connections now open) 2016-03-13T09:03:06.442+0000 I NETWORK [conn4079470] end connection 10.0.216.90:56150 (1506 connections now open) 2016-03-13T09:03:06.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56165 #4079486 (1506 connections now open) 2016-03-13T09:03:06.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56166 #4079487 (1507 connections now open) 2016-03-13T09:03:07.361+0000 I NETWORK [conn4079472] end connection 10.0.20.53:49369 (1506 connections now open) 2016-03-13T09:03:07.361+0000 I NETWORK [conn4079471] end connection 10.0.20.53:49368 (1506 connections now open) 2016-03-13T09:03:07.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49390 #4079488 (1506 connections now open) 2016-03-13T09:03:07.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49391 #4079489 (1507 connections now open) 2016-03-13T09:03:09.716+0000 I NETWORK [conn4079478] end connection 10.0.179.209:39179 (1506 connections now open) 2016-03-13T09:03:09.716+0000 I NETWORK [conn4079477] end connection 10.0.179.209:39178 (1506 connections now open) 2016-03-13T09:03:09.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39199 #4079490 (1506 connections now open) 2016-03-13T09:03:09.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39200 #4079491 (1507 connections now open) 2016-03-13T09:03:10.279+0000 I NETWORK [conn4079479] end connection 10.0.26.69:41181 (1506 connections now open) 2016-03-13T09:03:10.279+0000 I NETWORK [conn4079480] end connection 10.0.26.69:41182 (1506 connections now open) 2016-03-13T09:03:10.280+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41190 #4079492 (1506 connections now open) 2016-03-13T09:03:10.280+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41191 #4079493 (1507 connections now open) 2016-03-13T09:03:10.655+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 709ms sequenceNumber: 16767 version: 30592|1||563ba74d869758a4542b9075 based on: 30591|1||563ba74d869758a4542b9075 2016-03-13T09:03:10.843+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:03:10.843+0000-56e52cce18fa188537623d02", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859790843), what: "balancer.round", ns: "", details: { executionTimeMillis: 13301, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:03:11.005+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:03:12.232+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52cd018fa188537623d03 2016-03-13T09:03:13.192+0000 I NETWORK [conn4079482] end connection 10.0.27.237:34445 (1506 connections now open) 2016-03-13T09:03:13.192+0000 I NETWORK [conn4079483] end connection 10.0.27.237:34444 (1506 connections now open) 2016-03-13T09:03:13.202+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34459 #4079494 (1506 connections now open) 2016-03-13T09:03:13.202+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34460 #4079495 (1507 connections now open) 2016-03-13T09:03:14.831+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634115b7bc3500e945b2eba')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634115b7bc3500e945b2eba') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341262ce48b40ea190c99b') }, shard: "shmydomain1", lastmod: Timestamp 30592000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:03:14.987+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30592|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634115b7bc3500e945b2eba') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341262ce48b40ea190c99b') }) shmydomain1 -> shmydomain3 2016-03-13T09:03:15.305+0000 I NETWORK [conn4079484] end connection 10.0.7.113:58135 (1506 connections now open) 2016-03-13T09:03:15.305+0000 I NETWORK [conn4079485] end connection 10.0.7.113:58136 (1506 connections now open) 2016-03-13T09:03:15.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58154 #4079496 (1506 connections now open) 2016-03-13T09:03:15.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58155 #4079497 (1507 connections now open) 2016-03-13T09:03:16.117+0000 I NETWORK [conn4076353] end connection 10.0.20.53:46169 (1506 connections now open) 2016-03-13T09:03:16.117+0000 I NETWORK [conn4070775] end connection 10.0.216.90:46878 (1506 connections now open) 2016-03-13T09:03:16.117+0000 I NETWORK [conn4075556] end connection 10.0.26.69:37038 (1505 connections now open) 2016-03-13T09:03:16.118+0000 I NETWORK [conn4067216] end connection 10.0.27.237:50062 (1505 connections now open) 2016-03-13T09:03:16.118+0000 I NETWORK [conn4069556] end connection 10.0.7.113:47568 (1505 connections now open) 2016-03-13T09:03:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56183 #4079498 (1507 connections now open) 2016-03-13T09:03:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34467 #4079499 (1504 connections now open) 2016-03-13T09:03:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58159 #4079500 (1505 connections now open) 2016-03-13T09:03:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49399 #4079501 (1506 connections now open) 2016-03-13T09:03:16.183+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41205 #4079502 (1507 connections now open) 2016-03-13T09:03:16.455+0000 I NETWORK [conn4079486] end connection 10.0.216.90:56165 (1506 connections now open) 2016-03-13T09:03:16.455+0000 I NETWORK [conn4079487] end connection 10.0.216.90:56166 (1505 connections now open) 2016-03-13T09:03:16.455+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56184 #4079503 (1506 connections now open) 2016-03-13T09:03:16.455+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56185 #4079504 (1507 connections now open) 2016-03-13T09:03:17.375+0000 I NETWORK [conn4079489] end connection 10.0.20.53:49391 (1506 connections now open) 2016-03-13T09:03:17.375+0000 I NETWORK [conn4079488] end connection 10.0.20.53:49390 (1506 connections now open) 2016-03-13T09:03:17.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49400 #4079505 (1506 connections now open) 2016-03-13T09:03:17.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49401 #4079506 (1507 connections now open) 2016-03-13T09:03:19.739+0000 I NETWORK [conn4079490] end connection 10.0.179.209:39199 (1506 connections now open) 2016-03-13T09:03:19.739+0000 I NETWORK [conn4079491] end connection 10.0.179.209:39200 (1506 connections now open) 2016-03-13T09:03:19.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39212 #4079507 (1506 connections now open) 2016-03-13T09:03:19.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39213 #4079508 (1507 connections now open) 2016-03-13T09:03:20.292+0000 I NETWORK [conn4079492] end connection 10.0.26.69:41190 (1506 connections now open) 2016-03-13T09:03:20.292+0000 I NETWORK [conn4079493] end connection 10.0.26.69:41191 (1506 connections now open) 2016-03-13T09:03:20.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41208 #4079509 (1506 connections now open) 2016-03-13T09:03:20.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41209 #4079510 (1507 connections now open) 2016-03-13T09:03:23.214+0000 I NETWORK [conn4079494] end connection 10.0.27.237:34459 (1506 connections now open) 2016-03-13T09:03:23.214+0000 I NETWORK [conn4079495] end connection 10.0.27.237:34460 (1506 connections now open) 2016-03-13T09:03:23.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34481 #4079511 (1506 connections now open) 2016-03-13T09:03:23.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34480 #4079512 (1507 connections now open) 2016-03-13T09:03:25.327+0000 I NETWORK [conn4079497] end connection 10.0.7.113:58155 (1506 connections now open) 2016-03-13T09:03:25.327+0000 I NETWORK [conn4079496] end connection 10.0.7.113:58154 (1506 connections now open) 2016-03-13T09:03:25.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58164 #4079513 (1506 connections now open) 2016-03-13T09:03:25.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58165 #4079514 (1507 connections now open) 2016-03-13T09:03:26.468+0000 I NETWORK [conn4079503] end connection 10.0.216.90:56184 (1506 connections now open) 2016-03-13T09:03:26.468+0000 I NETWORK [conn4079504] end connection 10.0.216.90:56185 (1505 connections now open) 2016-03-13T09:03:26.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56191 #4079515 (1506 connections now open) 2016-03-13T09:03:26.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56192 #4079516 (1507 connections now open) 2016-03-13T09:03:27.388+0000 I NETWORK [conn4079506] end connection 10.0.20.53:49401 (1506 connections now open) 2016-03-13T09:03:27.388+0000 I NETWORK [conn4079505] end connection 10.0.20.53:49400 (1506 connections now open) 2016-03-13T09:03:27.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49413 #4079517 (1506 connections now open) 2016-03-13T09:03:27.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49412 #4079518 (1507 connections now open) 2016-03-13T09:03:28.092+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16768 version: 30593|1||563ba74d869758a4542b9075 based on: 30592|1||563ba74d869758a4542b9075 2016-03-13T09:03:28.113+0000 I NETWORK [conn4068560] end connection 10.0.216.90:44509 (1506 connections now open) 2016-03-13T09:03:28.113+0000 I NETWORK [conn4069847] end connection 10.0.20.53:39325 (1506 connections now open) 2016-03-13T09:03:28.113+0000 I NETWORK [conn4077616] end connection 10.0.26.69:39246 (1504 connections now open) 2016-03-13T09:03:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49415 #4079519 (1505 connections now open) 2016-03-13T09:03:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56195 #4079520 (1506 connections now open) 2016-03-13T09:03:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41225 #4079521 (1507 connections now open) 2016-03-13T09:03:28.366+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:03:28.366+0000-56e52ce018fa188537623d04", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859808366), what: "balancer.round", ns: "", details: { executionTimeMillis: 16329, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:03:28.575+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:03:29.752+0000 I NETWORK [conn4079507] end connection 10.0.179.209:39212 (1506 connections now open) 2016-03-13T09:03:29.752+0000 I NETWORK [conn4079508] end connection 10.0.179.209:39213 (1506 connections now open) 2016-03-13T09:03:29.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39226 #4079522 (1506 connections now open) 2016-03-13T09:03:29.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39225 #4079523 (1507 connections now open) 2016-03-13T09:03:30.306+0000 I NETWORK [conn4079510] end connection 10.0.26.69:41209 (1506 connections now open) 2016-03-13T09:03:30.306+0000 I NETWORK [conn4079509] end connection 10.0.26.69:41208 (1506 connections now open) 2016-03-13T09:03:30.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41226 #4079524 (1506 connections now open) 2016-03-13T09:03:30.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41227 #4079525 (1507 connections now open) 2016-03-13T09:03:31.706+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:03:31.374+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:03:31.706+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:03:31.843+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ce118fa188537623d05 2016-03-13T09:03:33.227+0000 I NETWORK [conn4079512] end connection 10.0.27.237:34480 (1506 connections now open) 2016-03-13T09:03:33.227+0000 I NETWORK [conn4079511] end connection 10.0.27.237:34481 (1506 connections now open) 2016-03-13T09:03:33.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34491 #4079526 (1506 connections now open) 2016-03-13T09:03:33.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34490 #4079527 (1507 connections now open) 2016-03-13T09:03:35.341+0000 I NETWORK [conn4079514] end connection 10.0.7.113:58165 (1506 connections now open) 2016-03-13T09:03:35.341+0000 I NETWORK [conn4079513] end connection 10.0.7.113:58164 (1506 connections now open) 2016-03-13T09:03:35.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58178 #4079528 (1506 connections now open) 2016-03-13T09:03:35.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58179 #4079529 (1507 connections now open) 2016-03-13T09:03:36.482+0000 I NETWORK [conn4079516] end connection 10.0.216.90:56192 (1506 connections now open) 2016-03-13T09:03:36.482+0000 I NETWORK [conn4079515] end connection 10.0.216.90:56191 (1506 connections now open) 2016-03-13T09:03:36.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56208 #4079530 (1506 connections now open) 2016-03-13T09:03:36.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56207 #4079531 (1507 connections now open) 2016-03-13T09:03:37.402+0000 I NETWORK [conn4079518] end connection 10.0.20.53:49412 (1506 connections now open) 2016-03-13T09:03:37.402+0000 I NETWORK [conn4079517] end connection 10.0.20.53:49413 (1506 connections now open) 2016-03-13T09:03:37.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49435 #4079532 (1506 connections now open) 2016-03-13T09:03:37.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49436 #4079533 (1507 connections now open) 2016-03-13T09:03:39.765+0000 I NETWORK [conn4079523] end connection 10.0.179.209:39225 (1506 connections now open) 2016-03-13T09:03:39.765+0000 I NETWORK [conn4079522] end connection 10.0.179.209:39226 (1506 connections now open) 2016-03-13T09:03:39.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39243 #4079534 (1506 connections now open) 2016-03-13T09:03:39.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39244 #4079535 (1507 connections now open) 2016-03-13T09:03:40.319+0000 I NETWORK [conn4079525] end connection 10.0.26.69:41227 (1506 connections now open) 2016-03-13T09:03:40.319+0000 I NETWORK [conn4079524] end connection 10.0.26.69:41226 (1506 connections now open) 2016-03-13T09:03:40.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41234 #4079536 (1506 connections now open) 2016-03-13T09:03:40.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41233 #4079537 (1507 connections now open) 2016-03-13T09:03:41.129+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16769 version: 30594|1||563ba74d869758a4542b9075 based on: 30593|1||563ba74d869758a4542b9075 2016-03-13T09:03:43.240+0000 I NETWORK [conn4079526] end connection 10.0.27.237:34491 (1506 connections now open) 2016-03-13T09:03:43.240+0000 I NETWORK [conn4079527] end connection 10.0.27.237:34490 (1506 connections now open) 2016-03-13T09:03:43.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34502 #4079538 (1506 connections now open) 2016-03-13T09:03:43.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34503 #4079539 (1507 connections now open) 2016-03-13T09:03:45.354+0000 I NETWORK [conn4079528] end connection 10.0.7.113:58178 (1506 connections now open) 2016-03-13T09:03:45.354+0000 I NETWORK [conn4079529] end connection 10.0.7.113:58179 (1506 connections now open) 2016-03-13T09:03:45.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58196 #4079540 (1506 connections now open) 2016-03-13T09:03:45.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58197 #4079541 (1507 connections now open) 2016-03-13T09:03:46.120+0000 I NETWORK [conn4071837] end connection 10.0.7.113:50023 (1506 connections now open) 2016-03-13T09:03:46.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58200 #4079542 (1507 connections now open) 2016-03-13T09:03:46.495+0000 I NETWORK [conn4079530] end connection 10.0.216.90:56208 (1506 connections now open) 2016-03-13T09:03:46.495+0000 I NETWORK [conn4079531] end connection 10.0.216.90:56207 (1506 connections now open) 2016-03-13T09:03:46.495+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56226 #4079543 (1506 connections now open) 2016-03-13T09:03:46.495+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56227 #4079544 (1507 connections now open) 2016-03-13T09:03:47.415+0000 I NETWORK [conn4079532] end connection 10.0.20.53:49435 (1506 connections now open) 2016-03-13T09:03:47.415+0000 I NETWORK [conn4079533] end connection 10.0.20.53:49436 (1506 connections now open) 2016-03-13T09:03:47.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49447 #4079545 (1506 connections now open) 2016-03-13T09:03:47.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49446 #4079546 (1507 connections now open) 2016-03-13T09:03:49.779+0000 I NETWORK [conn4079534] end connection 10.0.179.209:39243 (1506 connections now open) 2016-03-13T09:03:49.779+0000 I NETWORK [conn4079535] end connection 10.0.179.209:39244 (1506 connections now open) 2016-03-13T09:03:49.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39253 #4079547 (1506 connections now open) 2016-03-13T09:03:49.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39254 #4079548 (1507 connections now open) 2016-03-13T09:03:50.333+0000 I NETWORK [conn4079537] end connection 10.0.26.69:41233 (1506 connections now open) 2016-03-13T09:03:50.333+0000 I NETWORK [conn4079536] end connection 10.0.26.69:41234 (1506 connections now open) 2016-03-13T09:03:50.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41248 #4079549 (1506 connections now open) 2016-03-13T09:03:50.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41247 #4079550 (1507 connections now open) 2016-03-13T09:03:53.253+0000 I NETWORK [conn4079538] end connection 10.0.27.237:34502 (1506 connections now open) 2016-03-13T09:03:53.253+0000 I NETWORK [conn4079539] end connection 10.0.27.237:34503 (1506 connections now open) 2016-03-13T09:03:53.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34523 #4079551 (1506 connections now open) 2016-03-13T09:03:53.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34522 #4079552 (1507 connections now open) 2016-03-13T09:03:55.368+0000 I NETWORK [conn4079540] end connection 10.0.7.113:58196 (1506 connections now open) 2016-03-13T09:03:55.368+0000 I NETWORK [conn4079541] end connection 10.0.7.113:58197 (1506 connections now open) 2016-03-13T09:03:55.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58205 #4079553 (1506 connections now open) 2016-03-13T09:03:55.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58204 #4079554 (1507 connections now open) 2016-03-13T09:03:55.978+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16770 version: 30595|1||563ba74d869758a4542b9075 based on: 30594|1||563ba74d869758a4542b9075 2016-03-13T09:03:56.508+0000 I NETWORK [conn4079544] end connection 10.0.216.90:56227 (1506 connections now open) 2016-03-13T09:03:56.508+0000 I NETWORK [conn4079543] end connection 10.0.216.90:56226 (1506 connections now open) 2016-03-13T09:03:56.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56231 #4079555 (1506 connections now open) 2016-03-13T09:03:56.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56232 #4079556 (1507 connections now open) 2016-03-13T09:03:57.428+0000 I NETWORK [conn4079546] end connection 10.0.20.53:49446 (1506 connections now open) 2016-03-13T09:03:57.428+0000 I NETWORK [conn4079545] end connection 10.0.20.53:49447 (1506 connections now open) 2016-03-13T09:03:57.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49459 #4079557 (1506 connections now open) 2016-03-13T09:03:57.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49460 #4079558 (1507 connections now open) 2016-03-13T09:03:58.113+0000 I NETWORK [conn4075997] end connection 10.0.20.53:45778 (1506 connections now open) 2016-03-13T09:03:58.113+0000 I NETWORK [conn4067146] end connection 10.0.179.209:54928 (1506 connections now open) 2016-03-13T09:03:58.114+0000 I NETWORK [conn4069240] end connection 10.0.7.113:47235 (1504 connections now open) 2016-03-13T09:03:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39260 #4079559 (1505 connections now open) 2016-03-13T09:03:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49463 #4079560 (1506 connections now open) 2016-03-13T09:03:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58214 #4079561 (1507 connections now open) 2016-03-13T09:03:59.792+0000 I NETWORK [conn4079547] end connection 10.0.179.209:39253 (1506 connections now open) 2016-03-13T09:03:59.792+0000 I NETWORK [conn4079548] end connection 10.0.179.209:39254 (1506 connections now open) 2016-03-13T09:03:59.793+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39266 #4079562 (1506 connections now open) 2016-03-13T09:03:59.793+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39267 #4079563 (1507 connections now open) 2016-03-13T09:04:00.346+0000 I NETWORK [conn4079549] end connection 10.0.26.69:41248 (1506 connections now open) 2016-03-13T09:04:00.346+0000 I NETWORK [conn4079550] end connection 10.0.26.69:41247 (1506 connections now open) 2016-03-13T09:04:00.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41268 #4079564 (1506 connections now open) 2016-03-13T09:04:00.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41267 #4079565 (1507 connections now open) 2016-03-13T09:04:02.097+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:04:01.843+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:04:02.097+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:04:02.234+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ceb18fa188537623d06 2016-03-13T09:04:02.370+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52cf518fa188537623d07 2016-03-13T09:04:02.507+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52cff18fa188537623d08 2016-03-13T09:04:03.266+0000 I NETWORK [conn4079551] end connection 10.0.27.237:34523 (1506 connections now open) 2016-03-13T09:04:03.266+0000 I NETWORK [conn4079552] end connection 10.0.27.237:34522 (1506 connections now open) 2016-03-13T09:04:03.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34535 #4079566 (1506 connections now open) 2016-03-13T09:04:03.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34534 #4079567 (1507 connections now open) 2016-03-13T09:04:05.381+0000 I NETWORK [conn4079554] end connection 10.0.7.113:58204 (1506 connections now open) 2016-03-13T09:04:05.381+0000 I NETWORK [conn4079553] end connection 10.0.7.113:58205 (1506 connections now open) 2016-03-13T09:04:05.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58220 #4079568 (1506 connections now open) 2016-03-13T09:04:05.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58219 #4079569 (1507 connections now open) 2016-03-13T09:04:06.521+0000 I NETWORK [conn4079556] end connection 10.0.216.90:56232 (1506 connections now open) 2016-03-13T09:04:06.521+0000 I NETWORK [conn4079555] end connection 10.0.216.90:56231 (1506 connections now open) 2016-03-13T09:04:06.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56249 #4079570 (1506 connections now open) 2016-03-13T09:04:06.522+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56248 #4079571 (1507 connections now open) 2016-03-13T09:04:07.442+0000 I NETWORK [conn4079558] end connection 10.0.20.53:49460 (1506 connections now open) 2016-03-13T09:04:07.442+0000 I NETWORK [conn4079557] end connection 10.0.20.53:49459 (1506 connections now open) 2016-03-13T09:04:07.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49481 #4079572 (1506 connections now open) 2016-03-13T09:04:07.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49480 #4079573 (1507 connections now open) 2016-03-13T09:04:09.806+0000 I NETWORK [conn4079563] end connection 10.0.179.209:39267 (1506 connections now open) 2016-03-13T09:04:09.806+0000 I NETWORK [conn4079562] end connection 10.0.179.209:39266 (1506 connections now open) 2016-03-13T09:04:09.815+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39285 #4079574 (1506 connections now open) 2016-03-13T09:04:09.815+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39286 #4079575 (1507 connections now open) 2016-03-13T09:04:10.359+0000 I NETWORK [conn4079564] end connection 10.0.26.69:41268 (1506 connections now open) 2016-03-13T09:04:10.359+0000 I NETWORK [conn4079565] end connection 10.0.26.69:41267 (1506 connections now open) 2016-03-13T09:04:10.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41276 #4079576 (1506 connections now open) 2016-03-13T09:04:10.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41277 #4079577 (1507 connections now open) 2016-03-13T09:04:10.631+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 702ms sequenceNumber: 16771 version: 30596|1||563ba74d869758a4542b9075 based on: 30595|1||563ba74d869758a4542b9075 2016-03-13T09:04:13.280+0000 I NETWORK [conn4079566] end connection 10.0.27.237:34535 (1506 connections now open) 2016-03-13T09:04:13.280+0000 I NETWORK [conn4079567] end connection 10.0.27.237:34534 (1506 connections now open) 2016-03-13T09:04:13.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34549 #4079578 (1506 connections now open) 2016-03-13T09:04:13.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34550 #4079579 (1507 connections now open) 2016-03-13T09:04:15.394+0000 I NETWORK [conn4079569] end connection 10.0.7.113:58219 (1506 connections now open) 2016-03-13T09:04:15.394+0000 I NETWORK [conn4079568] end connection 10.0.7.113:58220 (1506 connections now open) 2016-03-13T09:04:15.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58239 #4079580 (1506 connections now open) 2016-03-13T09:04:15.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58240 #4079581 (1507 connections now open) 2016-03-13T09:04:16.118+0000 I NETWORK [conn4071538] end connection 10.0.216.90:47699 (1506 connections now open) 2016-03-13T09:04:16.119+0000 I NETWORK [conn4067087] end connection 10.0.26.69:56191 (1505 connections now open) 2016-03-13T09:04:16.119+0000 I NETWORK [conn4067607] end connection 10.0.7.113:45494 (1505 connections now open) 2016-03-13T09:04:16.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41291 #4079582 (1505 connections now open) 2016-03-13T09:04:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56266 #4079583 (1506 connections now open) 2016-03-13T09:04:16.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58243 #4079584 (1507 connections now open) 2016-03-13T09:04:16.533+0000 I NETWORK [conn4079570] end connection 10.0.216.90:56249 (1506 connections now open) 2016-03-13T09:04:16.534+0000 I NETWORK [conn4079571] end connection 10.0.216.90:56248 (1505 connections now open) 2016-03-13T09:04:16.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56267 #4079585 (1506 connections now open) 2016-03-13T09:04:16.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56268 #4079586 (1507 connections now open) 2016-03-13T09:04:17.455+0000 I NETWORK [conn4079573] end connection 10.0.20.53:49480 (1506 connections now open) 2016-03-13T09:04:17.455+0000 I NETWORK [conn4079572] end connection 10.0.20.53:49481 (1506 connections now open) 2016-03-13T09:04:17.465+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49491 #4079587 (1506 connections now open) 2016-03-13T09:04:17.465+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49490 #4079588 (1507 connections now open) 2016-03-13T09:04:19.828+0000 I NETWORK [conn4079574] end connection 10.0.179.209:39285 (1506 connections now open) 2016-03-13T09:04:19.828+0000 I NETWORK [conn4079575] end connection 10.0.179.209:39286 (1506 connections now open) 2016-03-13T09:04:19.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39298 #4079589 (1506 connections now open) 2016-03-13T09:04:19.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39299 #4079590 (1507 connections now open) 2016-03-13T09:04:20.373+0000 I NETWORK [conn4079576] end connection 10.0.26.69:41276 (1506 connections now open) 2016-03-13T09:04:20.373+0000 I NETWORK [conn4079577] end connection 10.0.26.69:41277 (1506 connections now open) 2016-03-13T09:04:20.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41294 #4079591 (1506 connections now open) 2016-03-13T09:04:20.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41293 #4079592 (1507 connections now open) 2016-03-13T09:04:23.301+0000 I NETWORK [conn4079579] end connection 10.0.27.237:34550 (1506 connections now open) 2016-03-13T09:04:23.301+0000 I NETWORK [conn4079578] end connection 10.0.27.237:34549 (1506 connections now open) 2016-03-13T09:04:23.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34570 #4079593 (1506 connections now open) 2016-03-13T09:04:23.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34571 #4079594 (1507 connections now open) 2016-03-13T09:04:25.215+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16772 version: 30597|1||563ba74d869758a4542b9075 based on: 30596|1||563ba74d869758a4542b9075 2016-03-13T09:04:25.408+0000 I NETWORK [conn4079580] end connection 10.0.7.113:58239 (1506 connections now open) 2016-03-13T09:04:25.408+0000 I NETWORK [conn4079581] end connection 10.0.7.113:58240 (1506 connections now open) 2016-03-13T09:04:25.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58248 #4079595 (1506 connections now open) 2016-03-13T09:04:25.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58247 #4079596 (1507 connections now open) 2016-03-13T09:04:26.546+0000 I NETWORK [conn4079585] end connection 10.0.216.90:56267 (1506 connections now open) 2016-03-13T09:04:26.546+0000 I NETWORK [conn4079586] end connection 10.0.216.90:56268 (1506 connections now open) 2016-03-13T09:04:26.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56272 #4079597 (1506 connections now open) 2016-03-13T09:04:26.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56273 #4079598 (1507 connections now open) 2016-03-13T09:04:27.478+0000 I NETWORK [conn4079588] end connection 10.0.20.53:49490 (1506 connections now open) 2016-03-13T09:04:27.478+0000 I NETWORK [conn4079587] end connection 10.0.20.53:49491 (1506 connections now open) 2016-03-13T09:04:27.479+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49503 #4079599 (1506 connections now open) 2016-03-13T09:04:27.479+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49504 #4079600 (1507 connections now open) 2016-03-13T09:04:28.113+0000 I NETWORK [conn4076159] end connection 10.0.26.69:37692 (1506 connections now open) 2016-03-13T09:04:28.113+0000 I NETWORK [conn4071920] end connection 10.0.27.237:55015 (1505 connections now open) 2016-03-13T09:04:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41310 #4079601 (1506 connections now open) 2016-03-13T09:04:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34576 #4079602 (1507 connections now open) 2016-03-13T09:04:29.841+0000 I NETWORK [conn4079589] end connection 10.0.179.209:39298 (1506 connections now open) 2016-03-13T09:04:29.841+0000 I NETWORK [conn4079590] end connection 10.0.179.209:39299 (1506 connections now open) 2016-03-13T09:04:29.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39312 #4079603 (1506 connections now open) 2016-03-13T09:04:29.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39313 #4079604 (1507 connections now open) 2016-03-13T09:04:30.395+0000 I NETWORK [conn4079592] end connection 10.0.26.69:41293 (1506 connections now open) 2016-03-13T09:04:30.395+0000 I NETWORK [conn4079591] end connection 10.0.26.69:41294 (1506 connections now open) 2016-03-13T09:04:30.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41313 #4079605 (1506 connections now open) 2016-03-13T09:04:30.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41312 #4079606 (1507 connections now open) 2016-03-13T09:04:32.697+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:04:32.507+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:04:32.697+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:04:32.766+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52d0918fa188537623d09 2016-03-13T09:04:32.834+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52d1418fa188537623d0a 2016-03-13T09:04:32.902+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52d1f18fa188537623d0b 2016-03-13T09:04:33.314+0000 I NETWORK [conn4079594] end connection 10.0.27.237:34571 (1506 connections now open) 2016-03-13T09:04:33.314+0000 I NETWORK [conn4079593] end connection 10.0.27.237:34570 (1506 connections now open) 2016-03-13T09:04:33.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34579 #4079607 (1506 connections now open) 2016-03-13T09:04:33.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34580 #4079608 (1507 connections now open) 2016-03-13T09:04:35.422+0000 I NETWORK [conn4079595] end connection 10.0.7.113:58248 (1506 connections now open) 2016-03-13T09:04:35.422+0000 I NETWORK [conn4079596] end connection 10.0.7.113:58247 (1506 connections now open) 2016-03-13T09:04:35.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58263 #4079609 (1506 connections now open) 2016-03-13T09:04:35.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58264 #4079610 (1507 connections now open) 2016-03-13T09:04:36.559+0000 I NETWORK [conn4079598] end connection 10.0.216.90:56273 (1506 connections now open) 2016-03-13T09:04:36.559+0000 I NETWORK [conn4079597] end connection 10.0.216.90:56272 (1505 connections now open) 2016-03-13T09:04:36.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56290 #4079611 (1506 connections now open) 2016-03-13T09:04:36.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56291 #4079612 (1507 connections now open) 2016-03-13T09:04:37.491+0000 I NETWORK [conn4079600] end connection 10.0.20.53:49504 (1506 connections now open) 2016-03-13T09:04:37.491+0000 I NETWORK [conn4079599] end connection 10.0.20.53:49503 (1506 connections now open) 2016-03-13T09:04:37.492+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49526 #4079613 (1506 connections now open) 2016-03-13T09:04:37.492+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49527 #4079614 (1507 connections now open) 2016-03-13T09:04:39.854+0000 I NETWORK [conn4079603] end connection 10.0.179.209:39312 (1506 connections now open) 2016-03-13T09:04:39.854+0000 I NETWORK [conn4079604] end connection 10.0.179.209:39313 (1506 connections now open) 2016-03-13T09:04:39.855+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39329 #4079615 (1506 connections now open) 2016-03-13T09:04:39.855+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39330 #4079616 (1507 connections now open) 2016-03-13T09:04:40.409+0000 I NETWORK [conn4079605] end connection 10.0.26.69:41313 (1506 connections now open) 2016-03-13T09:04:40.409+0000 I NETWORK [conn4079606] end connection 10.0.26.69:41312 (1506 connections now open) 2016-03-13T09:04:40.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41319 #4079617 (1506 connections now open) 2016-03-13T09:04:40.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41320 #4079618 (1507 connections now open) 2016-03-13T09:04:41.159+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16773 version: 30598|1||563ba74d869758a4542b9075 based on: 30597|1||563ba74d869758a4542b9075 2016-03-13T09:04:42.096+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d2918fa188537623d0c 2016-03-13T09:04:43.327+0000 I NETWORK [conn4079608] end connection 10.0.27.237:34580 (1506 connections now open) 2016-03-13T09:04:43.327+0000 I NETWORK [conn4079607] end connection 10.0.27.237:34579 (1506 connections now open) 2016-03-13T09:04:43.327+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34588 #4079619 (1506 connections now open) 2016-03-13T09:04:43.327+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34589 #4079620 (1507 connections now open) 2016-03-13T09:04:44.963+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56341799ce48b40ea36b7657')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341799ce48b40ea36b7657') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563418a4584ad10f21302d09') }, shard: "shmydomain1", lastmod: Timestamp 30598000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:04:45.120+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30598|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341799ce48b40ea36b7657') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563418a4584ad10f21302d09') }) shmydomain1 -> shmydomain3 2016-03-13T09:04:45.436+0000 I NETWORK [conn4079610] end connection 10.0.7.113:58264 (1506 connections now open) 2016-03-13T09:04:45.436+0000 I NETWORK [conn4079609] end connection 10.0.7.113:58263 (1506 connections now open) 2016-03-13T09:04:45.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58279 #4079621 (1506 connections now open) 2016-03-13T09:04:45.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58280 #4079622 (1507 connections now open) 2016-03-13T09:04:46.118+0000 I NETWORK [conn4065950] end connection 10.0.179.209:53664 (1506 connections now open) 2016-03-13T09:04:46.118+0000 I NETWORK [conn4070290] end connection 10.0.26.69:59621 (1506 connections now open) 2016-03-13T09:04:46.118+0000 I NETWORK [conn4069129] end connection 10.0.216.90:45112 (1504 connections now open) 2016-03-13T09:04:46.118+0000 I NETWORK [conn4075644] end connection 10.0.27.237:58830 (1503 connections now open) 2016-03-13T09:04:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39334 #4079623 (1504 connections now open) 2016-03-13T09:04:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56306 #4079624 (1505 connections now open) 2016-03-13T09:04:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34596 #4079625 (1506 connections now open) 2016-03-13T09:04:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41331 #4079626 (1507 connections now open) 2016-03-13T09:04:46.572+0000 I NETWORK [conn4079612] end connection 10.0.216.90:56291 (1506 connections now open) 2016-03-13T09:04:46.572+0000 I NETWORK [conn4079611] end connection 10.0.216.90:56290 (1506 connections now open) 2016-03-13T09:04:46.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56308 #4079627 (1506 connections now open) 2016-03-13T09:04:46.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56307 #4079628 (1507 connections now open) 2016-03-13T09:04:47.504+0000 I NETWORK [conn4079614] end connection 10.0.20.53:49527 (1506 connections now open) 2016-03-13T09:04:47.504+0000 I NETWORK [conn4079613] end connection 10.0.20.53:49526 (1505 connections now open) 2016-03-13T09:04:47.505+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49536 #4079629 (1506 connections now open) 2016-03-13T09:04:47.505+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49537 #4079630 (1507 connections now open) 2016-03-13T09:04:49.867+0000 I NETWORK [conn4079616] end connection 10.0.179.209:39330 (1506 connections now open) 2016-03-13T09:04:49.867+0000 I NETWORK [conn4079615] end connection 10.0.179.209:39329 (1506 connections now open) 2016-03-13T09:04:49.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39338 #4079631 (1506 connections now open) 2016-03-13T09:04:49.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39339 #4079632 (1507 connections now open) 2016-03-13T09:04:50.422+0000 I NETWORK [conn4079617] end connection 10.0.26.69:41319 (1506 connections now open) 2016-03-13T09:04:50.422+0000 I NETWORK [conn4079618] end connection 10.0.26.69:41320 (1506 connections now open) 2016-03-13T09:04:50.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41334 #4079633 (1506 connections now open) 2016-03-13T09:04:50.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41333 #4079634 (1507 connections now open) 2016-03-13T09:04:53.340+0000 I NETWORK [conn4079620] end connection 10.0.27.237:34589 (1506 connections now open) 2016-03-13T09:04:53.340+0000 I NETWORK [conn4079619] end connection 10.0.27.237:34588 (1506 connections now open) 2016-03-13T09:04:53.340+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34610 #4079635 (1506 connections now open) 2016-03-13T09:04:53.340+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34611 #4079636 (1507 connections now open) 2016-03-13T09:04:53.882+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16774 version: 30599|1||563ba74d869758a4542b9075 based on: 30598|1||563ba74d869758a4542b9075 2016-03-13T09:04:54.153+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:04:54.153+0000-56e52d3618fa188537623d0d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859894153), what: "balancer.round", ns: "", details: { executionTimeMillis: 12340, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:04:54.332+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:04:55.449+0000 I NETWORK [conn4079621] end connection 10.0.7.113:58279 (1506 connections now open) 2016-03-13T09:04:55.449+0000 I NETWORK [conn4079622] end connection 10.0.7.113:58280 (1506 connections now open) 2016-03-13T09:04:55.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58289 #4079637 (1506 connections now open) 2016-03-13T09:04:55.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58288 #4079638 (1507 connections now open) 2016-03-13T09:04:55.667+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d3718fa188537623d0e 2016-03-13T09:04:56.585+0000 I NETWORK [conn4079628] end connection 10.0.216.90:56307 (1506 connections now open) 2016-03-13T09:04:56.585+0000 I NETWORK [conn4079627] end connection 10.0.216.90:56308 (1506 connections now open) 2016-03-13T09:04:56.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56315 #4079639 (1506 connections now open) 2016-03-13T09:04:56.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56314 #4079640 (1507 connections now open) 2016-03-13T09:04:57.517+0000 I NETWORK [conn4079629] end connection 10.0.20.53:49536 (1506 connections now open) 2016-03-13T09:04:57.517+0000 I NETWORK [conn4079630] end connection 10.0.20.53:49537 (1505 connections now open) 2016-03-13T09:04:57.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49548 #4079641 (1506 connections now open) 2016-03-13T09:04:57.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49549 #4079642 (1507 connections now open) 2016-03-13T09:04:58.113+0000 I NETWORK [conn4073010] end connection 10.0.179.209:60856 (1506 connections now open) 2016-03-13T09:04:58.113+0000 I NETWORK [conn4070562] end connection 10.0.20.53:40071 (1505 connections now open) 2016-03-13T09:04:58.113+0000 I NETWORK [conn4071863] end connection 10.0.216.90:48042 (1505 connections now open) 2016-03-13T09:04:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39346 #4079643 (1505 connections now open) 2016-03-13T09:04:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56318 #4079644 (1506 connections now open) 2016-03-13T09:04:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49551 #4079645 (1507 connections now open) 2016-03-13T09:04:58.446+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563418a4584ad10f21302d09')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563418a4584ad10f21302d09') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563419aea2a4330ea41c4bde') }, shard: "shmydomain1", lastmod: Timestamp 30599000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:04:58.600+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30599|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563418a4584ad10f21302d09') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563419aea2a4330ea41c4bde') }) shmydomain1 -> shmydomain3 2016-03-13T09:04:59.880+0000 I NETWORK [conn4079632] end connection 10.0.179.209:39339 (1506 connections now open) 2016-03-13T09:04:59.880+0000 I NETWORK [conn4079631] end connection 10.0.179.209:39338 (1506 connections now open) 2016-03-13T09:04:59.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39352 #4079646 (1506 connections now open) 2016-03-13T09:04:59.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39353 #4079647 (1507 connections now open) 2016-03-13T09:05:00.436+0000 I NETWORK [conn4079634] end connection 10.0.26.69:41333 (1506 connections now open) 2016-03-13T09:05:00.436+0000 I NETWORK [conn4079633] end connection 10.0.26.69:41334 (1506 connections now open) 2016-03-13T09:05:00.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41353 #4079648 (1506 connections now open) 2016-03-13T09:05:00.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41354 #4079649 (1507 connections now open) 2016-03-13T09:05:03.156+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:05:02.903+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:05:03.353+0000 I NETWORK [conn4079636] end connection 10.0.27.237:34611 (1506 connections now open) 2016-03-13T09:05:03.353+0000 I NETWORK [conn4079635] end connection 10.0.27.237:34610 (1506 connections now open) 2016-03-13T09:05:03.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34624 #4079650 (1506 connections now open) 2016-03-13T09:05:03.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34625 #4079651 (1507 connections now open) 2016-03-13T09:05:05.463+0000 I NETWORK [conn4079637] end connection 10.0.7.113:58289 (1506 connections now open) 2016-03-13T09:05:05.463+0000 I NETWORK [conn4079638] end connection 10.0.7.113:58288 (1506 connections now open) 2016-03-13T09:05:05.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58306 #4079652 (1506 connections now open) 2016-03-13T09:05:05.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58307 #4079653 (1507 connections now open) 2016-03-13T09:05:06.598+0000 I NETWORK [conn4079640] end connection 10.0.216.90:56314 (1506 connections now open) 2016-03-13T09:05:06.598+0000 I NETWORK [conn4079639] end connection 10.0.216.90:56315 (1506 connections now open) 2016-03-13T09:05:06.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56332 #4079654 (1506 connections now open) 2016-03-13T09:05:06.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56331 #4079655 (1507 connections now open) 2016-03-13T09:05:07.531+0000 I NETWORK [conn4079642] end connection 10.0.20.53:49549 (1506 connections now open) 2016-03-13T09:05:07.531+0000 I NETWORK [conn4079641] end connection 10.0.20.53:49548 (1506 connections now open) 2016-03-13T09:05:07.531+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49570 #4079656 (1506 connections now open) 2016-03-13T09:05:07.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49569 #4079657 (1507 connections now open) 2016-03-13T09:05:08.576+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16775 version: 30600|1||563ba74d869758a4542b9075 based on: 30599|1||563ba74d869758a4542b9075 2016-03-13T09:05:08.846+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:05:08.846+0000-56e52d4418fa188537623d0f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859908846), what: "balancer.round", ns: "", details: { executionTimeMillis: 13475, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:05:08.995+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:05:09.893+0000 I NETWORK [conn4079646] end connection 10.0.179.209:39352 (1506 connections now open) 2016-03-13T09:05:09.893+0000 I NETWORK [conn4079647] end connection 10.0.179.209:39353 (1506 connections now open) 2016-03-13T09:05:09.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39373 #4079658 (1506 connections now open) 2016-03-13T09:05:09.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39374 #4079659 (1507 connections now open) 2016-03-13T09:05:10.227+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d4618fa188537623d10 2016-03-13T09:05:10.450+0000 I NETWORK [conn4079648] end connection 10.0.26.69:41353 (1506 connections now open) 2016-03-13T09:05:10.450+0000 I NETWORK [conn4079649] end connection 10.0.26.69:41354 (1506 connections now open) 2016-03-13T09:05:10.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41365 #4079660 (1506 connections now open) 2016-03-13T09:05:10.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41364 #4079661 (1507 connections now open) 2016-03-13T09:05:12.820+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563419aea2a4330ea41c4bde')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563419aea2a4330ea41c4bde') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ab8584ad10f2130622e') }, shard: "shmydomain1", lastmod: Timestamp 30600000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:05:12.975+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30600|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563419aea2a4330ea41c4bde') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ab8584ad10f2130622e') }) shmydomain1 -> shmydomain3 2016-03-13T09:05:13.366+0000 I NETWORK [conn4079651] end connection 10.0.27.237:34625 (1506 connections now open) 2016-03-13T09:05:13.366+0000 I NETWORK [conn4079650] end connection 10.0.27.237:34624 (1506 connections now open) 2016-03-13T09:05:13.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34638 #4079662 (1506 connections now open) 2016-03-13T09:05:13.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34639 #4079663 (1507 connections now open) 2016-03-13T09:05:15.476+0000 I NETWORK [conn4079652] end connection 10.0.7.113:58306 (1506 connections now open) 2016-03-13T09:05:15.476+0000 I NETWORK [conn4079653] end connection 10.0.7.113:58307 (1506 connections now open) 2016-03-13T09:05:15.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58326 #4079664 (1506 connections now open) 2016-03-13T09:05:15.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58325 #4079665 (1507 connections now open) 2016-03-13T09:05:16.118+0000 I NETWORK [conn4075181] end connection 10.0.20.53:44908 (1506 connections now open) 2016-03-13T09:05:16.118+0000 I NETWORK [conn4079624] end connection 10.0.216.90:56306 (1506 connections now open) 2016-03-13T09:05:16.118+0000 I NETWORK [conn4074930] end connection 10.0.179.209:34600 (1506 connections now open) 2016-03-13T09:05:16.118+0000 I NETWORK [conn4070224] end connection 10.0.7.113:48301 (1503 connections now open) 2016-03-13T09:05:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39379 #4079666 (1504 connections now open) 2016-03-13T09:05:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49579 #4079667 (1505 connections now open) 2016-03-13T09:05:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58330 #4079668 (1506 connections now open) 2016-03-13T09:05:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56349 #4079669 (1507 connections now open) 2016-03-13T09:05:16.611+0000 I NETWORK [conn4079654] end connection 10.0.216.90:56332 (1506 connections now open) 2016-03-13T09:05:16.611+0000 I NETWORK [conn4079655] end connection 10.0.216.90:56331 (1506 connections now open) 2016-03-13T09:05:16.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56351 #4079670 (1506 connections now open) 2016-03-13T09:05:16.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56350 #4079671 (1507 connections now open) 2016-03-13T09:05:17.544+0000 I NETWORK [conn4079657] end connection 10.0.20.53:49569 (1506 connections now open) 2016-03-13T09:05:17.544+0000 I NETWORK [conn4079656] end connection 10.0.20.53:49570 (1505 connections now open) 2016-03-13T09:05:17.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49581 #4079672 (1506 connections now open) 2016-03-13T09:05:17.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49582 #4079673 (1507 connections now open) 2016-03-13T09:05:19.915+0000 I NETWORK [conn4079659] end connection 10.0.179.209:39374 (1506 connections now open) 2016-03-13T09:05:19.915+0000 I NETWORK [conn4079658] end connection 10.0.179.209:39373 (1506 connections now open) 2016-03-13T09:05:19.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39386 #4079674 (1506 connections now open) 2016-03-13T09:05:19.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39387 #4079675 (1507 connections now open) 2016-03-13T09:05:20.463+0000 I NETWORK [conn4079661] end connection 10.0.26.69:41364 (1506 connections now open) 2016-03-13T09:05:20.464+0000 I NETWORK [conn4079660] end connection 10.0.26.69:41365 (1505 connections now open) 2016-03-13T09:05:20.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41382 #4079676 (1506 connections now open) 2016-03-13T09:05:20.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41383 #4079677 (1507 connections now open) 2016-03-13T09:05:22.978+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16776 version: 30601|1||563ba74d869758a4542b9075 based on: 30600|1||563ba74d869758a4542b9075 2016-03-13T09:05:23.165+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:05:23.165+0000-56e52d5318fa188537623d11", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859923165), what: "balancer.round", ns: "", details: { executionTimeMillis: 13137, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:05:23.387+0000 I NETWORK [conn4079662] end connection 10.0.27.237:34638 (1506 connections now open) 2016-03-13T09:05:23.387+0000 I NETWORK [conn4079663] end connection 10.0.27.237:34639 (1506 connections now open) 2016-03-13T09:05:23.388+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34659 #4079678 (1506 connections now open) 2016-03-13T09:05:23.388+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34658 #4079679 (1507 connections now open) 2016-03-13T09:05:25.490+0000 I NETWORK [conn4079664] end connection 10.0.7.113:58326 (1506 connections now open) 2016-03-13T09:05:25.490+0000 I NETWORK [conn4079665] end connection 10.0.7.113:58325 (1506 connections now open) 2016-03-13T09:05:25.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58337 #4079680 (1506 connections now open) 2016-03-13T09:05:25.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58336 #4079681 (1507 connections now open) 2016-03-13T09:05:25.750+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:05:26.624+0000 I NETWORK [conn4079671] end connection 10.0.216.90:56350 (1506 connections now open) 2016-03-13T09:05:26.624+0000 I NETWORK [conn4079670] end connection 10.0.216.90:56351 (1506 connections now open) 2016-03-13T09:05:26.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56356 #4079682 (1506 connections now open) 2016-03-13T09:05:26.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56357 #4079683 (1507 connections now open) 2016-03-13T09:05:27.565+0000 I NETWORK [conn4079672] end connection 10.0.20.53:49581 (1506 connections now open) 2016-03-13T09:05:27.565+0000 I NETWORK [conn4079673] end connection 10.0.20.53:49582 (1506 connections now open) 2016-03-13T09:05:27.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49593 #4079684 (1506 connections now open) 2016-03-13T09:05:27.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49594 #4079685 (1507 connections now open) 2016-03-13T09:05:28.113+0000 I NETWORK [conn4071835] end connection 10.0.20.53:41397 (1506 connections now open) 2016-03-13T09:05:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49597 #4079686 (1507 connections now open) 2016-03-13T09:05:28.349+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d5818fa188537623d12 2016-03-13T09:05:29.928+0000 I NETWORK [conn4079674] end connection 10.0.179.209:39386 (1506 connections now open) 2016-03-13T09:05:29.928+0000 I NETWORK [conn4079675] end connection 10.0.179.209:39387 (1506 connections now open) 2016-03-13T09:05:29.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39399 #4079687 (1506 connections now open) 2016-03-13T09:05:29.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39400 #4079688 (1507 connections now open) 2016-03-13T09:05:30.486+0000 I NETWORK [conn4079676] end connection 10.0.26.69:41382 (1506 connections now open) 2016-03-13T09:05:30.486+0000 I NETWORK [conn4079677] end connection 10.0.26.69:41383 (1506 connections now open) 2016-03-13T09:05:30.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41402 #4079689 (1506 connections now open) 2016-03-13T09:05:30.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41403 #4079690 (1507 connections now open) 2016-03-13T09:05:31.115+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56341ab8584ad10f2130622e')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ab8584ad10f2130622e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341bca584ad10f1f93550e') }, shard: "shmydomain1", lastmod: Timestamp 30601000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:05:31.270+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30601|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ab8584ad10f2130622e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341bca584ad10f1f93550e') }) shmydomain1 -> shmydomain3 2016-03-13T09:05:33.400+0000 I NETWORK [conn4079678] end connection 10.0.27.237:34659 (1506 connections now open) 2016-03-13T09:05:33.400+0000 I NETWORK [conn4079679] end connection 10.0.27.237:34658 (1506 connections now open) 2016-03-13T09:05:33.401+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34668 #4079691 (1506 connections now open) 2016-03-13T09:05:33.401+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34669 #4079692 (1507 connections now open) 2016-03-13T09:05:33.417+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:05:33.156+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:05:35.503+0000 I NETWORK [conn4079681] end connection 10.0.7.113:58336 (1506 connections now open) 2016-03-13T09:05:35.503+0000 I NETWORK [conn4079680] end connection 10.0.7.113:58337 (1505 connections now open) 2016-03-13T09:05:35.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58349 #4079693 (1506 connections now open) 2016-03-13T09:05:35.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58350 #4079694 (1507 connections now open) 2016-03-13T09:05:36.637+0000 I NETWORK [conn4079683] end connection 10.0.216.90:56357 (1506 connections now open) 2016-03-13T09:05:36.637+0000 I NETWORK [conn4079682] end connection 10.0.216.90:56356 (1505 connections now open) 2016-03-13T09:05:36.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56373 #4079695 (1506 connections now open) 2016-03-13T09:05:36.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56374 #4079696 (1507 connections now open) 2016-03-13T09:05:37.579+0000 I NETWORK [conn4079684] end connection 10.0.20.53:49593 (1506 connections now open) 2016-03-13T09:05:37.579+0000 I NETWORK [conn4079685] end connection 10.0.20.53:49594 (1506 connections now open) 2016-03-13T09:05:37.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49617 #4079697 (1506 connections now open) 2016-03-13T09:05:37.580+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49616 #4079698 (1507 connections now open) 2016-03-13T09:05:39.941+0000 I NETWORK [conn4079687] end connection 10.0.179.209:39399 (1506 connections now open) 2016-03-13T09:05:39.941+0000 I NETWORK [conn4079688] end connection 10.0.179.209:39400 (1506 connections now open) 2016-03-13T09:05:39.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39417 #4079699 (1506 connections now open) 2016-03-13T09:05:39.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39418 #4079700 (1507 connections now open) 2016-03-13T09:05:40.202+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16777 version: 30602|1||563ba74d869758a4542b9075 based on: 30601|1||563ba74d869758a4542b9075 2016-03-13T09:05:40.397+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:05:40.396+0000-56e52d6418fa188537623d13", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859940396), what: "balancer.round", ns: "", details: { executionTimeMillis: 12227, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:05:40.500+0000 I NETWORK [conn4079689] end connection 10.0.26.69:41402 (1506 connections now open) 2016-03-13T09:05:40.500+0000 I NETWORK [conn4079690] end connection 10.0.26.69:41403 (1506 connections now open) 2016-03-13T09:05:40.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41409 #4079701 (1506 connections now open) 2016-03-13T09:05:40.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41410 #4079702 (1507 connections now open) 2016-03-13T09:05:40.595+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:05:41.924+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d6518fa188537623d14 2016-03-13T09:05:43.413+0000 I NETWORK [conn4079692] end connection 10.0.27.237:34669 (1506 connections now open) 2016-03-13T09:05:43.413+0000 I NETWORK [conn4079691] end connection 10.0.27.237:34668 (1506 connections now open) 2016-03-13T09:05:43.414+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34679 #4079703 (1506 connections now open) 2016-03-13T09:05:43.414+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34680 #4079704 (1507 connections now open) 2016-03-13T09:05:44.793+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56341bca584ad10f1f93550e')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341bca584ad10f1f93550e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ce5584ad10f2130994c') }, shard: "shmydomain1", lastmod: Timestamp 30602000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:05:44.946+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30602|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341bca584ad10f1f93550e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ce5584ad10f2130994c') }) shmydomain1 -> shmydomain3 2016-03-13T09:05:45.515+0000 I NETWORK [conn4079693] end connection 10.0.7.113:58349 (1506 connections now open) 2016-03-13T09:05:45.516+0000 I NETWORK [conn4079694] end connection 10.0.7.113:58350 (1505 connections now open) 2016-03-13T09:05:45.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58365 #4079705 (1506 connections now open) 2016-03-13T09:05:45.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58366 #4079706 (1507 connections now open) 2016-03-13T09:05:46.117+0000 I NETWORK [conn4071715] end connection 10.0.216.90:47883 (1506 connections now open) 2016-03-13T09:05:46.117+0000 I NETWORK [conn4071902] end connection 10.0.27.237:55001 (1505 connections now open) 2016-03-13T09:05:46.118+0000 I NETWORK [conn4072073] end connection 10.0.26.69:33309 (1505 connections now open) 2016-03-13T09:05:46.118+0000 I NETWORK [conn4069849] end connection 10.0.7.113:47885 (1503 connections now open) 2016-03-13T09:05:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34687 #4079707 (1504 connections now open) 2016-03-13T09:05:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56390 #4079708 (1505 connections now open) 2016-03-13T09:05:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41420 #4079709 (1506 connections now open) 2016-03-13T09:05:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58368 #4079710 (1507 connections now open) 2016-03-13T09:05:46.650+0000 I NETWORK [conn4079696] end connection 10.0.216.90:56374 (1506 connections now open) 2016-03-13T09:05:46.650+0000 I NETWORK [conn4079695] end connection 10.0.216.90:56373 (1506 connections now open) 2016-03-13T09:05:46.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56391 #4079711 (1506 connections now open) 2016-03-13T09:05:46.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56392 #4079712 (1507 connections now open) 2016-03-13T09:05:47.592+0000 I NETWORK [conn4079698] end connection 10.0.20.53:49616 (1506 connections now open) 2016-03-13T09:05:47.592+0000 I NETWORK [conn4079697] end connection 10.0.20.53:49617 (1506 connections now open) 2016-03-13T09:05:47.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49627 #4079713 (1506 connections now open) 2016-03-13T09:05:47.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49628 #4079714 (1507 connections now open) 2016-03-13T09:05:49.954+0000 I NETWORK [conn4079699] end connection 10.0.179.209:39417 (1506 connections now open) 2016-03-13T09:05:49.954+0000 I NETWORK [conn4079700] end connection 10.0.179.209:39418 (1506 connections now open) 2016-03-13T09:05:49.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39427 #4079715 (1506 connections now open) 2016-03-13T09:05:49.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39428 #4079716 (1507 connections now open) 2016-03-13T09:05:50.513+0000 I NETWORK [conn4079702] end connection 10.0.26.69:41410 (1506 connections now open) 2016-03-13T09:05:50.513+0000 I NETWORK [conn4079701] end connection 10.0.26.69:41409 (1506 connections now open) 2016-03-13T09:05:50.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41424 #4079717 (1506 connections now open) 2016-03-13T09:05:50.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41423 #4079718 (1507 connections now open) 2016-03-13T09:05:53.426+0000 I NETWORK [conn4079703] end connection 10.0.27.237:34679 (1506 connections now open) 2016-03-13T09:05:53.426+0000 I NETWORK [conn4079704] end connection 10.0.27.237:34680 (1506 connections now open) 2016-03-13T09:05:53.427+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34702 #4079719 (1506 connections now open) 2016-03-13T09:05:53.427+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34701 #4079720 (1507 connections now open) 2016-03-13T09:05:55.528+0000 I NETWORK [conn4079705] end connection 10.0.7.113:58365 (1506 connections now open) 2016-03-13T09:05:55.529+0000 I NETWORK [conn4079706] end connection 10.0.7.113:58366 (1505 connections now open) 2016-03-13T09:05:55.529+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58372 #4079721 (1506 connections now open) 2016-03-13T09:05:55.529+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58373 #4079722 (1507 connections now open) 2016-03-13T09:05:55.861+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16778 version: 30603|1||563ba74d869758a4542b9075 based on: 30602|1||563ba74d869758a4542b9075 2016-03-13T09:05:56.048+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:05:56.048+0000-56e52d7418fa188537623d15", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859956048), what: "balancer.round", ns: "", details: { executionTimeMillis: 14422, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:05:56.144+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:05:56.663+0000 I NETWORK [conn4079712] end connection 10.0.216.90:56392 (1506 connections now open) 2016-03-13T09:05:56.663+0000 I NETWORK [conn4079711] end connection 10.0.216.90:56391 (1506 connections now open) 2016-03-13T09:05:56.663+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56399 #4079723 (1506 connections now open) 2016-03-13T09:05:56.663+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56398 #4079724 (1507 connections now open) 2016-03-13T09:05:57.339+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d7518fa188537623d16 2016-03-13T09:05:57.605+0000 I NETWORK [conn4079714] end connection 10.0.20.53:49628 (1506 connections now open) 2016-03-13T09:05:57.605+0000 I NETWORK [conn4079713] end connection 10.0.20.53:49627 (1506 connections now open) 2016-03-13T09:05:57.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49637 #4079725 (1506 connections now open) 2016-03-13T09:05:57.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49638 #4079726 (1507 connections now open) 2016-03-13T09:05:58.114+0000 I NETWORK [conn4072175] end connection 10.0.27.237:55284 (1506 connections now open) 2016-03-13T09:05:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34709 #4079727 (1507 connections now open) 2016-03-13T09:05:59.967+0000 I NETWORK [conn4079715] end connection 10.0.179.209:39427 (1506 connections now open) 2016-03-13T09:05:59.967+0000 I NETWORK [conn4079716] end connection 10.0.179.209:39428 (1506 connections now open) 2016-03-13T09:05:59.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39440 #4079728 (1506 connections now open) 2016-03-13T09:05:59.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39441 #4079729 (1507 connections now open) 2016-03-13T09:06:00.092+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56341ce5584ad10f2130994c')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ce5584ad10f2130994c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341dffa2a4330ea290e0bd') }, shard: "shmydomain1", lastmod: Timestamp 30603000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:06:00.247+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30603|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341ce5584ad10f2130994c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341dffa2a4330ea290e0bd') }) shmydomain1 -> shmydomain3 2016-03-13T09:06:00.526+0000 I NETWORK [conn4079717] end connection 10.0.26.69:41424 (1506 connections now open) 2016-03-13T09:06:00.526+0000 I NETWORK [conn4079718] end connection 10.0.26.69:41423 (1506 connections now open) 2016-03-13T09:06:00.527+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41443 #4079730 (1506 connections now open) 2016-03-13T09:06:00.527+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41444 #4079731 (1507 connections now open) 2016-03-13T09:06:03.439+0000 I NETWORK [conn4079720] end connection 10.0.27.237:34701 (1506 connections now open) 2016-03-13T09:06:03.439+0000 I NETWORK [conn4079719] end connection 10.0.27.237:34702 (1506 connections now open) 2016-03-13T09:06:03.440+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34715 #4079732 (1506 connections now open) 2016-03-13T09:06:03.440+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34714 #4079733 (1507 connections now open) 2016-03-13T09:06:03.551+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:06:03.417+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:06:05.542+0000 I NETWORK [conn4079721] end connection 10.0.7.113:58372 (1506 connections now open) 2016-03-13T09:06:05.542+0000 I NETWORK [conn4079722] end connection 10.0.7.113:58373 (1506 connections now open) 2016-03-13T09:06:05.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58387 #4079734 (1506 connections now open) 2016-03-13T09:06:05.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58388 #4079735 (1507 connections now open) 2016-03-13T09:06:06.676+0000 I NETWORK [conn4079723] end connection 10.0.216.90:56399 (1506 connections now open) 2016-03-13T09:06:06.676+0000 I NETWORK [conn4079724] end connection 10.0.216.90:56398 (1506 connections now open) 2016-03-13T09:06:06.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56416 #4079736 (1506 connections now open) 2016-03-13T09:06:06.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56415 #4079737 (1507 connections now open) 2016-03-13T09:06:07.618+0000 I NETWORK [conn4079725] end connection 10.0.20.53:49637 (1506 connections now open) 2016-03-13T09:06:07.618+0000 I NETWORK [conn4079726] end connection 10.0.20.53:49638 (1506 connections now open) 2016-03-13T09:06:07.618+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49659 #4079738 (1506 connections now open) 2016-03-13T09:06:07.618+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49660 #4079739 (1507 connections now open) 2016-03-13T09:06:09.981+0000 I NETWORK [conn4079729] end connection 10.0.179.209:39441 (1506 connections now open) 2016-03-13T09:06:09.981+0000 I NETWORK [conn4079728] end connection 10.0.179.209:39440 (1506 connections now open) 2016-03-13T09:06:09.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39460 #4079740 (1506 connections now open) 2016-03-13T09:06:09.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39461 #4079741 (1507 connections now open) 2016-03-13T09:06:09.996+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16779 version: 30604|1||563ba74d869758a4542b9075 based on: 30603|1||563ba74d869758a4542b9075 2016-03-13T09:06:10.279+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:06:10.279+0000-56e52d8218fa188537623d17", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859970279), what: "balancer.round", ns: "", details: { executionTimeMillis: 13101, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:06:10.416+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:06:10.540+0000 I NETWORK [conn4079730] end connection 10.0.26.69:41443 (1506 connections now open) 2016-03-13T09:06:10.540+0000 I NETWORK [conn4079731] end connection 10.0.26.69:41444 (1506 connections now open) 2016-03-13T09:06:10.540+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41452 #4079742 (1506 connections now open) 2016-03-13T09:06:10.540+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41451 #4079743 (1507 connections now open) 2016-03-13T09:06:11.645+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d8318fa188537623d18 2016-03-13T09:06:13.453+0000 I NETWORK [conn4079732] end connection 10.0.27.237:34715 (1506 connections now open) 2016-03-13T09:06:13.453+0000 I NETWORK [conn4079733] end connection 10.0.27.237:34714 (1506 connections now open) 2016-03-13T09:06:13.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34729 #4079744 (1506 connections now open) 2016-03-13T09:06:13.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34728 #4079745 (1507 connections now open) 2016-03-13T09:06:14.186+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56341dffa2a4330ea290e0bd')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341dffa2a4330ea290e0bd') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341f19a2a4330ea290fb6b') }, shard: "shmydomain1", lastmod: Timestamp 30604000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:06:14.343+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30604|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341dffa2a4330ea290e0bd') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341f19a2a4330ea290fb6b') }) shmydomain1 -> shmydomain3 2016-03-13T09:06:15.555+0000 I NETWORK [conn4079735] end connection 10.0.7.113:58388 (1506 connections now open) 2016-03-13T09:06:15.555+0000 I NETWORK [conn4079734] end connection 10.0.7.113:58387 (1506 connections now open) 2016-03-13T09:06:15.556+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58408 #4079746 (1506 connections now open) 2016-03-13T09:06:15.556+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58409 #4079747 (1507 connections now open) 2016-03-13T09:06:16.118+0000 I NETWORK [conn4073752] end connection 10.0.20.53:43408 (1506 connections now open) 2016-03-13T09:06:16.118+0000 I NETWORK [conn4065864] end connection 10.0.179.209:53575 (1506 connections now open) 2016-03-13T09:06:16.118+0000 I NETWORK [conn4071074] end connection 10.0.216.90:47196 (1506 connections now open) 2016-03-13T09:06:16.118+0000 I NETWORK [conn4071564] end connection 10.0.27.237:54633 (1503 connections now open) 2016-03-13T09:06:16.118+0000 I NETWORK [conn4071332] end connection 10.0.7.113:49484 (1502 connections now open) 2016-03-13T09:06:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39466 #4079748 (1503 connections now open) 2016-03-13T09:06:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49668 #4079749 (1504 connections now open) 2016-03-13T09:06:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34736 #4079750 (1505 connections now open) 2016-03-13T09:06:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58412 #4079751 (1506 connections now open) 2016-03-13T09:06:16.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56433 #4079752 (1507 connections now open) 2016-03-13T09:06:16.689+0000 I NETWORK [conn4079736] end connection 10.0.216.90:56416 (1506 connections now open) 2016-03-13T09:06:16.689+0000 I NETWORK [conn4079737] end connection 10.0.216.90:56415 (1506 connections now open) 2016-03-13T09:06:16.689+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56434 #4079753 (1506 connections now open) 2016-03-13T09:06:16.689+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56435 #4079754 (1507 connections now open) 2016-03-13T09:06:17.631+0000 I NETWORK [conn4079738] end connection 10.0.20.53:49659 (1506 connections now open) 2016-03-13T09:06:17.631+0000 I NETWORK [conn4079739] end connection 10.0.20.53:49660 (1506 connections now open) 2016-03-13T09:06:17.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49670 #4079755 (1506 connections now open) 2016-03-13T09:06:17.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49671 #4079756 (1507 connections now open) 2016-03-13T09:06:20.003+0000 I NETWORK [conn4079741] end connection 10.0.179.209:39461 (1506 connections now open) 2016-03-13T09:06:20.003+0000 I NETWORK [conn4079740] end connection 10.0.179.209:39460 (1505 connections now open) 2016-03-13T09:06:20.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39473 #4079757 (1506 connections now open) 2016-03-13T09:06:20.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39474 #4079758 (1507 connections now open) 2016-03-13T09:06:20.552+0000 I NETWORK [conn4079742] end connection 10.0.26.69:41452 (1506 connections now open) 2016-03-13T09:06:20.552+0000 I NETWORK [conn4079743] end connection 10.0.26.69:41451 (1506 connections now open) 2016-03-13T09:06:20.561+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41468 #4079759 (1506 connections now open) 2016-03-13T09:06:20.561+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41469 #4079760 (1507 connections now open) 2016-03-13T09:06:23.475+0000 I NETWORK [conn4079745] end connection 10.0.27.237:34728 (1506 connections now open) 2016-03-13T09:06:23.475+0000 I NETWORK [conn4079744] end connection 10.0.27.237:34729 (1505 connections now open) 2016-03-13T09:06:23.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34750 #4079761 (1506 connections now open) 2016-03-13T09:06:23.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34751 #4079762 (1507 connections now open) 2016-03-13T09:06:25.290+0000 I NETWORK [conn4054546] end connection 10.0.53.235:55987 (1506 connections now open) 2016-03-13T09:06:25.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.53.235:42494 #4079763 (1507 connections now open) 2016-03-13T09:06:25.568+0000 I NETWORK [conn4079747] end connection 10.0.7.113:58409 (1506 connections now open) 2016-03-13T09:06:25.568+0000 I NETWORK [conn4079746] end connection 10.0.7.113:58408 (1506 connections now open) 2016-03-13T09:06:25.569+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58418 #4079764 (1507 connections now open) 2016-03-13T09:06:25.569+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58417 #4079765 (1507 connections now open) 2016-03-13T09:06:26.132+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16780 version: 30605|1||563ba74d869758a4542b9075 based on: 30604|1||563ba74d869758a4542b9075 2016-03-13T09:06:26.409+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:06:26.409+0000-56e52d9218fa188537623d19", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457859986409), what: "balancer.round", ns: "", details: { executionTimeMillis: 14959, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:06:26.524+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:06:26.702+0000 I NETWORK [conn4079753] end connection 10.0.216.90:56434 (1506 connections now open) 2016-03-13T09:06:26.702+0000 I NETWORK [conn4079754] end connection 10.0.216.90:56435 (1506 connections now open) 2016-03-13T09:06:26.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56440 #4079766 (1506 connections now open) 2016-03-13T09:06:26.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56441 #4079767 (1507 connections now open) 2016-03-13T09:06:27.653+0000 I NETWORK [conn4079756] end connection 10.0.20.53:49671 (1506 connections now open) 2016-03-13T09:06:27.653+0000 I NETWORK [conn4079755] end connection 10.0.20.53:49670 (1506 connections now open) 2016-03-13T09:06:27.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49679 #4079768 (1506 connections now open) 2016-03-13T09:06:27.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49680 #4079769 (1507 connections now open) 2016-03-13T09:06:28.113+0000 I NETWORK [conn4070855] end connection 10.0.20.53:40377 (1506 connections now open) 2016-03-13T09:06:28.113+0000 I NETWORK [conn4069702] end connection 10.0.216.90:45732 (1506 connections now open) 2016-03-13T09:06:28.113+0000 I NETWORK [conn4071583] end connection 10.0.27.237:54648 (1504 connections now open) 2016-03-13T09:06:28.113+0000 I NETWORK [conn4077057] end connection 10.0.26.69:38652 (1504 connections now open) 2016-03-13T09:06:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56445 #4079770 (1504 connections now open) 2016-03-13T09:06:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49684 #4079771 (1505 connections now open) 2016-03-13T09:06:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41486 #4079772 (1506 connections now open) 2016-03-13T09:06:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34757 #4079773 (1507 connections now open) 2016-03-13T09:06:30.016+0000 I NETWORK [conn4079757] end connection 10.0.179.209:39473 (1506 connections now open) 2016-03-13T09:06:30.016+0000 I NETWORK [conn4079758] end connection 10.0.179.209:39474 (1505 connections now open) 2016-03-13T09:06:30.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39486 #4079774 (1506 connections now open) 2016-03-13T09:06:30.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39487 #4079775 (1507 connections now open) 2016-03-13T09:06:30.573+0000 I NETWORK [conn4079760] end connection 10.0.26.69:41469 (1506 connections now open) 2016-03-13T09:06:30.573+0000 I NETWORK [conn4079759] end connection 10.0.26.69:41468 (1506 connections now open) 2016-03-13T09:06:30.574+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41489 #4079776 (1507 connections now open) 2016-03-13T09:06:30.574+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41488 #4079777 (1507 connections now open) 2016-03-13T09:06:31.635+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52d9418fa188537623d1a 2016-03-13T09:06:33.488+0000 I NETWORK [conn4079762] end connection 10.0.27.237:34751 (1506 connections now open) 2016-03-13T09:06:33.488+0000 I NETWORK [conn4079761] end connection 10.0.27.237:34750 (1506 connections now open) 2016-03-13T09:06:33.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34760 #4079778 (1506 connections now open) 2016-03-13T09:06:33.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34761 #4079779 (1507 connections now open) 2016-03-13T09:06:33.692+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:06:33.552+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:06:34.168+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56341f19a2a4330ea290fb6b')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341f19a2a4330ea290fb6b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634202c7bc3500e945c991e') }, shard: "shmydomain1", lastmod: Timestamp 30605000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:06:34.335+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30605|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56341f19a2a4330ea290fb6b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634202c7bc3500e945c991e') }) shmydomain1 -> shmydomain3 2016-03-13T09:06:35.582+0000 I NETWORK [conn4079765] end connection 10.0.7.113:58417 (1506 connections now open) 2016-03-13T09:06:35.582+0000 I NETWORK [conn4079764] end connection 10.0.7.113:58418 (1506 connections now open) 2016-03-13T09:06:35.583+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58432 #4079780 (1507 connections now open) 2016-03-13T09:06:35.583+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58433 #4079781 (1507 connections now open) 2016-03-13T09:06:36.714+0000 I NETWORK [conn4079767] end connection 10.0.216.90:56441 (1506 connections now open) 2016-03-13T09:06:36.715+0000 I NETWORK [conn4079766] end connection 10.0.216.90:56440 (1505 connections now open) 2016-03-13T09:06:36.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56457 #4079782 (1506 connections now open) 2016-03-13T09:06:36.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56458 #4079783 (1507 connections now open) 2016-03-13T09:06:37.667+0000 I NETWORK [conn4079768] end connection 10.0.20.53:49679 (1506 connections now open) 2016-03-13T09:06:37.667+0000 I NETWORK [conn4079769] end connection 10.0.20.53:49680 (1506 connections now open) 2016-03-13T09:06:37.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49704 #4079784 (1506 connections now open) 2016-03-13T09:06:37.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49705 #4079785 (1507 connections now open) 2016-03-13T09:06:40.028+0000 I NETWORK [conn4079774] end connection 10.0.179.209:39486 (1506 connections now open) 2016-03-13T09:06:40.029+0000 I NETWORK [conn4079775] end connection 10.0.179.209:39487 (1505 connections now open) 2016-03-13T09:06:40.029+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39503 #4079786 (1506 connections now open) 2016-03-13T09:06:40.029+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39504 #4079787 (1507 connections now open) 2016-03-13T09:06:40.587+0000 I NETWORK [conn4079777] end connection 10.0.26.69:41488 (1506 connections now open) 2016-03-13T09:06:40.587+0000 I NETWORK [conn4079776] end connection 10.0.26.69:41489 (1506 connections now open) 2016-03-13T09:06:40.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41495 #4079788 (1508 connections now open) 2016-03-13T09:06:40.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41494 #4079789 (1507 connections now open) 2016-03-13T09:06:43.501+0000 I NETWORK [conn4079779] end connection 10.0.27.237:34761 (1506 connections now open) 2016-03-13T09:06:43.501+0000 I NETWORK [conn4079778] end connection 10.0.27.237:34760 (1506 connections now open) 2016-03-13T09:06:43.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34770 #4079790 (1506 connections now open) 2016-03-13T09:06:43.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34771 #4079791 (1507 connections now open) 2016-03-13T09:06:44.211+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16781 version: 30606|1||563ba74d869758a4542b9075 based on: 30605|1||563ba74d869758a4542b9075 2016-03-13T09:06:44.481+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:06:44.481+0000-56e52da418fa188537623d1b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860004481), what: "balancer.round", ns: "", details: { executionTimeMillis: 16924, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:06:44.588+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:06:45.595+0000 I NETWORK [conn4079781] end connection 10.0.7.113:58433 (1506 connections now open) 2016-03-13T09:06:45.595+0000 I NETWORK [conn4079780] end connection 10.0.7.113:58432 (1506 connections now open) 2016-03-13T09:06:45.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58448 #4079792 (1507 connections now open) 2016-03-13T09:06:45.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58447 #4079793 (1507 connections now open) 2016-03-13T09:06:45.956+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52da518fa188537623d1c 2016-03-13T09:06:46.119+0000 I NETWORK [conn4071011] end connection 10.0.20.53:40539 (1506 connections now open) 2016-03-13T09:06:46.119+0000 I NETWORK [conn4073335] end connection 10.0.7.113:51612 (1506 connections now open) 2016-03-13T09:06:46.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49712 #4079794 (1506 connections now open) 2016-03-13T09:06:46.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58451 #4079795 (1507 connections now open) 2016-03-13T09:06:46.727+0000 I NETWORK [conn4079782] end connection 10.0.216.90:56457 (1506 connections now open) 2016-03-13T09:06:46.727+0000 I NETWORK [conn4079783] end connection 10.0.216.90:56458 (1505 connections now open) 2016-03-13T09:06:46.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56475 #4079796 (1506 connections now open) 2016-03-13T09:06:46.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56476 #4079797 (1507 connections now open) 2016-03-13T09:06:47.680+0000 I NETWORK [conn4079784] end connection 10.0.20.53:49704 (1506 connections now open) 2016-03-13T09:06:47.680+0000 I NETWORK [conn4079785] end connection 10.0.20.53:49705 (1506 connections now open) 2016-03-13T09:06:47.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49714 #4079798 (1506 connections now open) 2016-03-13T09:06:47.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49715 #4079799 (1507 connections now open) 2016-03-13T09:06:48.536+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634202c7bc3500e945c991e')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634202c7bc3500e945c991e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634214a7bc3500e961ed0f3') }, shard: "shmydomain1", lastmod: Timestamp 30606000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:06:48.690+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30606|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634202c7bc3500e945c991e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634214a7bc3500e961ed0f3') }) shmydomain1 -> shmydomain3 2016-03-13T09:06:50.042+0000 I NETWORK [conn4079787] end connection 10.0.179.209:39504 (1506 connections now open) 2016-03-13T09:06:50.042+0000 I NETWORK [conn4079786] end connection 10.0.179.209:39503 (1506 connections now open) 2016-03-13T09:06:50.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39512 #4079800 (1506 connections now open) 2016-03-13T09:06:50.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39513 #4079801 (1507 connections now open) 2016-03-13T09:06:50.601+0000 I NETWORK [conn4079789] end connection 10.0.26.69:41494 (1506 connections now open) 2016-03-13T09:06:50.601+0000 I NETWORK [conn4079788] end connection 10.0.26.69:41495 (1505 connections now open) 2016-03-13T09:06:50.602+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41509 #4079802 (1507 connections now open) 2016-03-13T09:06:50.602+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41510 #4079803 (1507 connections now open) 2016-03-13T09:06:53.514+0000 I NETWORK [conn4079791] end connection 10.0.27.237:34771 (1506 connections now open) 2016-03-13T09:06:53.514+0000 I NETWORK [conn4079790] end connection 10.0.27.237:34770 (1506 connections now open) 2016-03-13T09:06:53.515+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34791 #4079804 (1506 connections now open) 2016-03-13T09:06:53.515+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34790 #4079805 (1507 connections now open) 2016-03-13T09:06:55.609+0000 I NETWORK [conn4079793] end connection 10.0.7.113:58447 (1506 connections now open) 2016-03-13T09:06:55.609+0000 I NETWORK [conn4079792] end connection 10.0.7.113:58448 (1506 connections now open) 2016-03-13T09:06:55.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58456 #4079806 (1507 connections now open) 2016-03-13T09:06:55.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58457 #4079807 (1507 connections now open) 2016-03-13T09:06:56.740+0000 I NETWORK [conn4079796] end connection 10.0.216.90:56475 (1506 connections now open) 2016-03-13T09:06:56.740+0000 I NETWORK [conn4079797] end connection 10.0.216.90:56476 (1506 connections now open) 2016-03-13T09:06:56.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56482 #4079808 (1506 connections now open) 2016-03-13T09:06:56.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56481 #4079809 (1507 connections now open) 2016-03-13T09:06:57.694+0000 I NETWORK [conn4079798] end connection 10.0.20.53:49714 (1506 connections now open) 2016-03-13T09:06:57.694+0000 I NETWORK [conn4079799] end connection 10.0.20.53:49715 (1506 connections now open) 2016-03-13T09:06:57.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49726 #4079810 (1506 connections now open) 2016-03-13T09:06:57.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49725 #4079811 (1507 connections now open) 2016-03-13T09:06:58.113+0000 I NETWORK [conn4076372] end connection 10.0.20.53:46183 (1506 connections now open) 2016-03-13T09:06:58.113+0000 I NETWORK [conn4073266] end connection 10.0.26.69:34565 (1505 connections now open) 2016-03-13T09:06:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49728 #4079812 (1506 connections now open) 2016-03-13T09:06:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41526 #4079813 (1507 connections now open) 2016-03-13T09:06:58.489+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16782 version: 30607|1||563ba74d869758a4542b9075 based on: 30606|1||563ba74d869758a4542b9075 2016-03-13T09:06:58.680+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:06:58.680+0000-56e52db218fa188537623d1d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860018680), what: "balancer.round", ns: "", details: { executionTimeMillis: 13058, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:06:58.804+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:07:00.055+0000 I NETWORK [conn4079801] end connection 10.0.179.209:39513 (1506 connections now open) 2016-03-13T09:07:00.055+0000 I NETWORK [conn4079800] end connection 10.0.179.209:39512 (1506 connections now open) 2016-03-13T09:07:00.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39525 #4079814 (1506 connections now open) 2016-03-13T09:07:00.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39526 #4079815 (1507 connections now open) 2016-03-13T09:07:00.137+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52db318fa188537623d1e 2016-03-13T09:07:00.614+0000 I NETWORK [conn4079802] end connection 10.0.26.69:41509 (1506 connections now open) 2016-03-13T09:07:00.614+0000 I NETWORK [conn4079803] end connection 10.0.26.69:41510 (1505 connections now open) 2016-03-13T09:07:00.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41528 #4079816 (1507 connections now open) 2016-03-13T09:07:00.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41529 #4079817 (1507 connections now open) 2016-03-13T09:07:02.685+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634214a7bc3500e961ed0f3')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634214a7bc3500e961ed0f3') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634225d584ad10f21312491') }, shard: "shmydomain1", lastmod: Timestamp 30607000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:07:02.840+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30607|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634214a7bc3500e961ed0f3') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634225d584ad10f21312491') }) shmydomain1 -> shmydomain3 2016-03-13T09:07:03.527+0000 I NETWORK [conn4079804] end connection 10.0.27.237:34791 (1506 connections now open) 2016-03-13T09:07:03.527+0000 I NETWORK [conn4079805] end connection 10.0.27.237:34790 (1506 connections now open) 2016-03-13T09:07:03.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34803 #4079818 (1506 connections now open) 2016-03-13T09:07:03.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34804 #4079819 (1507 connections now open) 2016-03-13T09:07:03.961+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:07:03.692+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:07:05.621+0000 I NETWORK [conn4079807] end connection 10.0.7.113:58457 (1506 connections now open) 2016-03-13T09:07:05.621+0000 I NETWORK [conn4079806] end connection 10.0.7.113:58456 (1506 connections now open) 2016-03-13T09:07:05.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58473 #4079820 (1506 connections now open) 2016-03-13T09:07:05.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58474 #4079821 (1507 connections now open) 2016-03-13T09:07:06.753+0000 I NETWORK [conn4079808] end connection 10.0.216.90:56482 (1506 connections now open) 2016-03-13T09:07:06.753+0000 I NETWORK [conn4079809] end connection 10.0.216.90:56481 (1506 connections now open) 2016-03-13T09:07:06.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56499 #4079822 (1506 connections now open) 2016-03-13T09:07:06.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56498 #4079823 (1507 connections now open) 2016-03-13T09:07:07.708+0000 I NETWORK [conn4079811] end connection 10.0.20.53:49725 (1506 connections now open) 2016-03-13T09:07:07.708+0000 I NETWORK [conn4079810] end connection 10.0.20.53:49726 (1506 connections now open) 2016-03-13T09:07:07.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49747 #4079824 (1506 connections now open) 2016-03-13T09:07:07.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49746 #4079825 (1507 connections now open) 2016-03-13T09:07:10.069+0000 I NETWORK [conn4079814] end connection 10.0.179.209:39525 (1506 connections now open) 2016-03-13T09:07:10.069+0000 I NETWORK [conn4079815] end connection 10.0.179.209:39526 (1505 connections now open) 2016-03-13T09:07:10.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39539 #4079826 (1506 connections now open) 2016-03-13T09:07:10.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39540 #4079827 (1507 connections now open) 2016-03-13T09:07:10.628+0000 I NETWORK [conn4079817] end connection 10.0.26.69:41529 (1506 connections now open) 2016-03-13T09:07:10.628+0000 I NETWORK [conn4079816] end connection 10.0.26.69:41528 (1506 connections now open) 2016-03-13T09:07:10.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41538 #4079828 (1507 connections now open) 2016-03-13T09:07:10.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41537 #4079829 (1507 connections now open) 2016-03-13T09:07:12.691+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16783 version: 30608|1||563ba74d869758a4542b9075 based on: 30607|1||563ba74d869758a4542b9075 2016-03-13T09:07:12.962+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:07:12.962+0000-56e52dc018fa188537623d1f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860032962), what: "balancer.round", ns: "", details: { executionTimeMillis: 13123, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:07:13.146+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:07:13.540+0000 I NETWORK [conn4079819] end connection 10.0.27.237:34804 (1506 connections now open) 2016-03-13T09:07:13.540+0000 I NETWORK [conn4079818] end connection 10.0.27.237:34803 (1506 connections now open) 2016-03-13T09:07:13.549+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34817 #4079830 (1506 connections now open) 2016-03-13T09:07:13.549+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34818 #4079831 (1507 connections now open) 2016-03-13T09:07:15.634+0000 I NETWORK [conn4079821] end connection 10.0.7.113:58474 (1506 connections now open) 2016-03-13T09:07:15.634+0000 I NETWORK [conn4079820] end connection 10.0.7.113:58473 (1506 connections now open) 2016-03-13T09:07:15.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58495 #4079832 (1506 connections now open) 2016-03-13T09:07:15.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58494 #4079833 (1507 connections now open) 2016-03-13T09:07:16.117+0000 I NETWORK [conn4075054] end connection 10.0.179.209:34731 (1506 connections now open) 2016-03-13T09:07:16.118+0000 I NETWORK [conn4071202] end connection 10.0.216.90:47333 (1505 connections now open) 2016-03-13T09:07:16.118+0000 I NETWORK [conn4071697] end connection 10.0.26.69:32890 (1504 connections now open) 2016-03-13T09:07:16.118+0000 I NETWORK [conn4066093] end connection 10.0.7.113:43894 (1503 connections now open) 2016-03-13T09:07:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39549 #4079834 (1504 connections now open) 2016-03-13T09:07:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41551 #4079835 (1505 connections now open) 2016-03-13T09:07:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56516 #4079836 (1506 connections now open) 2016-03-13T09:07:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58498 #4079837 (1507 connections now open) 2016-03-13T09:07:16.766+0000 I NETWORK [conn4079822] end connection 10.0.216.90:56499 (1506 connections now open) 2016-03-13T09:07:16.766+0000 I NETWORK [conn4079823] end connection 10.0.216.90:56498 (1506 connections now open) 2016-03-13T09:07:16.775+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56518 #4079838 (1506 connections now open) 2016-03-13T09:07:16.775+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56519 #4079839 (1507 connections now open) 2016-03-13T09:07:17.722+0000 I NETWORK [conn4079824] end connection 10.0.20.53:49747 (1506 connections now open) 2016-03-13T09:07:17.722+0000 I NETWORK [conn4079825] end connection 10.0.20.53:49746 (1506 connections now open) 2016-03-13T09:07:17.732+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49758 #4079840 (1506 connections now open) 2016-03-13T09:07:17.732+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49757 #4079841 (1507 connections now open) 2016-03-13T09:07:20.090+0000 I NETWORK [conn4079826] end connection 10.0.179.209:39539 (1506 connections now open) 2016-03-13T09:07:20.090+0000 I NETWORK [conn4079827] end connection 10.0.179.209:39540 (1506 connections now open) 2016-03-13T09:07:20.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39557 #4079842 (1506 connections now open) 2016-03-13T09:07:20.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39556 #4079843 (1507 connections now open) 2016-03-13T09:07:20.641+0000 I NETWORK [conn4079828] end connection 10.0.26.69:41538 (1506 connections now open) 2016-03-13T09:07:20.641+0000 I NETWORK [conn4079829] end connection 10.0.26.69:41537 (1506 connections now open) 2016-03-13T09:07:20.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41555 #4079844 (1506 connections now open) 2016-03-13T09:07:20.706+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41554 #4079845 (1507 connections now open) 2016-03-13T09:07:23.561+0000 I NETWORK [conn4079831] end connection 10.0.27.237:34818 (1506 connections now open) 2016-03-13T09:07:23.561+0000 I NETWORK [conn4079830] end connection 10.0.27.237:34817 (1506 connections now open) 2016-03-13T09:07:23.562+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34837 #4079846 (1506 connections now open) 2016-03-13T09:07:23.562+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34838 #4079847 (1507 connections now open) 2016-03-13T09:07:25.648+0000 I NETWORK [conn4079832] end connection 10.0.7.113:58495 (1506 connections now open) 2016-03-13T09:07:25.648+0000 I NETWORK [conn4079833] end connection 10.0.7.113:58494 (1506 connections now open) 2016-03-13T09:07:25.657+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58504 #4079848 (1506 connections now open) 2016-03-13T09:07:25.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58503 #4079849 (1507 connections now open) 2016-03-13T09:07:26.787+0000 I NETWORK [conn4079839] end connection 10.0.216.90:56519 (1506 connections now open) 2016-03-13T09:07:26.787+0000 I NETWORK [conn4079838] end connection 10.0.216.90:56518 (1505 connections now open) 2016-03-13T09:07:26.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56524 #4079850 (1506 connections now open) 2016-03-13T09:07:26.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56525 #4079851 (1507 connections now open) 2016-03-13T09:07:26.912+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16784 version: 30609|1||563ba74d869758a4542b9075 based on: 30608|1||563ba74d869758a4542b9075 2016-03-13T09:07:27.744+0000 I NETWORK [conn4079840] end connection 10.0.20.53:49758 (1506 connections now open) 2016-03-13T09:07:27.744+0000 I NETWORK [conn4079841] end connection 10.0.20.53:49757 (1506 connections now open) 2016-03-13T09:07:27.745+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49769 #4079852 (1506 connections now open) 2016-03-13T09:07:27.745+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49768 #4079853 (1507 connections now open) 2016-03-13T09:07:28.113+0000 I NETWORK [conn4071653] end connection 10.0.216.90:47814 (1506 connections now open) 2016-03-13T09:07:28.113+0000 I NETWORK [conn4073012] end connection 10.0.26.69:34300 (1506 connections now open) 2016-03-13T09:07:28.113+0000 I NETWORK [conn4072723] end connection 10.0.27.237:55872 (1504 connections now open) 2016-03-13T09:07:28.113+0000 I NETWORK [conn4070755] end connection 10.0.7.113:48869 (1503 connections now open) 2016-03-13T09:07:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56528 #4079854 (1504 connections now open) 2016-03-13T09:07:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41572 #4079855 (1505 connections now open) 2016-03-13T09:07:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34844 #4079856 (1506 connections now open) 2016-03-13T09:07:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58511 #4079857 (1507 connections now open) 2016-03-13T09:07:30.103+0000 I NETWORK [conn4079842] end connection 10.0.179.209:39557 (1506 connections now open) 2016-03-13T09:07:30.103+0000 I NETWORK [conn4079843] end connection 10.0.179.209:39556 (1506 connections now open) 2016-03-13T09:07:30.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39566 #4079858 (1506 connections now open) 2016-03-13T09:07:30.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39567 #4079859 (1507 connections now open) 2016-03-13T09:07:30.718+0000 I NETWORK [conn4079844] end connection 10.0.26.69:41555 (1506 connections now open) 2016-03-13T09:07:30.718+0000 I NETWORK [conn4079845] end connection 10.0.26.69:41554 (1506 connections now open) 2016-03-13T09:07:30.719+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41574 #4079860 (1506 connections now open) 2016-03-13T09:07:30.719+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41573 #4079861 (1507 connections now open) 2016-03-13T09:07:33.574+0000 I NETWORK [conn4079846] end connection 10.0.27.237:34837 (1506 connections now open) 2016-03-13T09:07:33.574+0000 I NETWORK [conn4079847] end connection 10.0.27.237:34838 (1506 connections now open) 2016-03-13T09:07:33.575+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34847 #4079862 (1506 connections now open) 2016-03-13T09:07:33.575+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34848 #4079863 (1507 connections now open) 2016-03-13T09:07:35.670+0000 I NETWORK [conn4079849] end connection 10.0.7.113:58503 (1506 connections now open) 2016-03-13T09:07:35.670+0000 I NETWORK [conn4079848] end connection 10.0.7.113:58504 (1506 connections now open) 2016-03-13T09:07:35.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58518 #4079864 (1506 connections now open) 2016-03-13T09:07:35.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58519 #4079865 (1507 connections now open) 2016-03-13T09:07:36.018+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:07:33.961+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:07:36.018+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:07:36.085+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52dc218fa188537623d20 2016-03-13T09:07:36.153+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52dcc18fa188537623d21 2016-03-13T09:07:36.221+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52dd618fa188537623d22 2016-03-13T09:07:36.800+0000 I NETWORK [conn4079850] end connection 10.0.216.90:56524 (1506 connections now open) 2016-03-13T09:07:36.800+0000 I NETWORK [conn4079851] end connection 10.0.216.90:56525 (1506 connections now open) 2016-03-13T09:07:36.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56543 #4079866 (1506 connections now open) 2016-03-13T09:07:36.801+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56542 #4079867 (1507 connections now open) 2016-03-13T09:07:37.758+0000 I NETWORK [conn4079853] end connection 10.0.20.53:49768 (1506 connections now open) 2016-03-13T09:07:37.758+0000 I NETWORK [conn4079852] end connection 10.0.20.53:49769 (1506 connections now open) 2016-03-13T09:07:37.758+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49793 #4079868 (1506 connections now open) 2016-03-13T09:07:37.758+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49794 #4079869 (1507 connections now open) 2016-03-13T09:07:40.116+0000 I NETWORK [conn4079858] end connection 10.0.179.209:39566 (1506 connections now open) 2016-03-13T09:07:40.116+0000 I NETWORK [conn4079859] end connection 10.0.179.209:39567 (1506 connections now open) 2016-03-13T09:07:40.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39578 #4079870 (1506 connections now open) 2016-03-13T09:07:40.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39579 #4079871 (1507 connections now open) 2016-03-13T09:07:40.732+0000 I NETWORK [conn4079861] end connection 10.0.26.69:41573 (1506 connections now open) 2016-03-13T09:07:40.732+0000 I NETWORK [conn4079860] end connection 10.0.26.69:41574 (1506 connections now open) 2016-03-13T09:07:40.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41579 #4079872 (1506 connections now open) 2016-03-13T09:07:40.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41580 #4079873 (1507 connections now open) 2016-03-13T09:07:43.587+0000 I NETWORK [conn4079862] end connection 10.0.27.237:34847 (1506 connections now open) 2016-03-13T09:07:43.587+0000 I NETWORK [conn4079863] end connection 10.0.27.237:34848 (1506 connections now open) 2016-03-13T09:07:43.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34859 #4079874 (1506 connections now open) 2016-03-13T09:07:43.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34860 #4079875 (1507 connections now open) 2016-03-13T09:07:44.412+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16785 version: 30610|1||563ba74d869758a4542b9075 based on: 30609|1||563ba74d869758a4542b9075 2016-03-13T09:07:45.683+0000 I NETWORK [conn4079865] end connection 10.0.7.113:58519 (1506 connections now open) 2016-03-13T09:07:45.683+0000 I NETWORK [conn4079864] end connection 10.0.7.113:58518 (1506 connections now open) 2016-03-13T09:07:45.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58536 #4079876 (1506 connections now open) 2016-03-13T09:07:45.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58535 #4079877 (1507 connections now open) 2016-03-13T09:07:46.118+0000 I NETWORK [conn4071946] end connection 10.0.20.53:41506 (1506 connections now open) 2016-03-13T09:07:46.118+0000 I NETWORK [conn4071836] end connection 10.0.216.90:48019 (1506 connections now open) 2016-03-13T09:07:46.118+0000 I NETWORK [conn4068604] end connection 10.0.26.69:57793 (1504 connections now open) 2016-03-13T09:07:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56560 #4079878 (1505 connections now open) 2016-03-13T09:07:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49800 #4079879 (1506 connections now open) 2016-03-13T09:07:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41589 #4079880 (1507 connections now open) 2016-03-13T09:07:46.813+0000 I NETWORK [conn4079867] end connection 10.0.216.90:56542 (1506 connections now open) 2016-03-13T09:07:46.813+0000 I NETWORK [conn4079866] end connection 10.0.216.90:56543 (1506 connections now open) 2016-03-13T09:07:46.814+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56561 #4079881 (1506 connections now open) 2016-03-13T09:07:46.814+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56562 #4079882 (1507 connections now open) 2016-03-13T09:07:47.771+0000 I NETWORK [conn4079869] end connection 10.0.20.53:49794 (1506 connections now open) 2016-03-13T09:07:47.771+0000 I NETWORK [conn4079868] end connection 10.0.20.53:49793 (1506 connections now open) 2016-03-13T09:07:47.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49802 #4079883 (1506 connections now open) 2016-03-13T09:07:47.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49803 #4079884 (1507 connections now open) 2016-03-13T09:07:50.129+0000 I NETWORK [conn4079871] end connection 10.0.179.209:39579 (1506 connections now open) 2016-03-13T09:07:50.129+0000 I NETWORK [conn4079870] end connection 10.0.179.209:39578 (1506 connections now open) 2016-03-13T09:07:50.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39598 #4079885 (1506 connections now open) 2016-03-13T09:07:50.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39599 #4079886 (1507 connections now open) 2016-03-13T09:07:50.746+0000 I NETWORK [conn4079872] end connection 10.0.26.69:41579 (1506 connections now open) 2016-03-13T09:07:50.746+0000 I NETWORK [conn4079873] end connection 10.0.26.69:41580 (1506 connections now open) 2016-03-13T09:07:50.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41592 #4079887 (1506 connections now open) 2016-03-13T09:07:50.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41593 #4079888 (1507 connections now open) 2016-03-13T09:07:53.600+0000 I NETWORK [conn4079874] end connection 10.0.27.237:34859 (1506 connections now open) 2016-03-13T09:07:53.600+0000 I NETWORK [conn4079875] end connection 10.0.27.237:34860 (1505 connections now open) 2016-03-13T09:07:53.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34879 #4079889 (1506 connections now open) 2016-03-13T09:07:53.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34880 #4079890 (1507 connections now open) 2016-03-13T09:07:55.697+0000 I NETWORK [conn4079876] end connection 10.0.7.113:58536 (1506 connections now open) 2016-03-13T09:07:55.697+0000 I NETWORK [conn4079877] end connection 10.0.7.113:58535 (1506 connections now open) 2016-03-13T09:07:55.697+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58543 #4079891 (1506 connections now open) 2016-03-13T09:07:55.697+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58544 #4079892 (1507 connections now open) 2016-03-13T09:07:56.826+0000 I NETWORK [conn4079882] end connection 10.0.216.90:56562 (1506 connections now open) 2016-03-13T09:07:56.826+0000 I NETWORK [conn4079881] end connection 10.0.216.90:56561 (1506 connections now open) 2016-03-13T09:07:56.827+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56567 #4079893 (1506 connections now open) 2016-03-13T09:07:56.827+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56568 #4079894 (1507 connections now open) 2016-03-13T09:07:57.784+0000 I NETWORK [conn4079883] end connection 10.0.20.53:49802 (1506 connections now open) 2016-03-13T09:07:57.784+0000 I NETWORK [conn4079884] end connection 10.0.20.53:49803 (1506 connections now open) 2016-03-13T09:07:57.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49813 #4079895 (1506 connections now open) 2016-03-13T09:07:57.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49812 #4079896 (1507 connections now open) 2016-03-13T09:07:58.113+0000 I NETWORK [conn4070394] end connection 10.0.20.53:39895 (1506 connections now open) 2016-03-13T09:07:58.113+0000 I NETWORK [conn4069828] end connection 10.0.216.90:45867 (1505 connections now open) 2016-03-13T09:07:58.113+0000 I NETWORK [conn4069555] end connection 10.0.27.237:52484 (1504 connections now open) 2016-03-13T09:07:58.113+0000 I NETWORK [conn4077953] end connection 10.0.7.113:56554 (1503 connections now open) 2016-03-13T09:07:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56571 #4079897 (1504 connections now open) 2016-03-13T09:07:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49816 #4079898 (1505 connections now open) 2016-03-13T09:07:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34888 #4079899 (1506 connections now open) 2016-03-13T09:07:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58550 #4079900 (1507 connections now open) 2016-03-13T09:07:59.165+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16786 version: 30611|1||563ba74d869758a4542b9075 based on: 30610|1||563ba74d869758a4542b9075 2016-03-13T09:08:00.142+0000 I NETWORK [conn4079885] end connection 10.0.179.209:39598 (1506 connections now open) 2016-03-13T09:08:00.142+0000 I NETWORK [conn4079886] end connection 10.0.179.209:39599 (1506 connections now open) 2016-03-13T09:08:00.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39609 #4079901 (1506 connections now open) 2016-03-13T09:08:00.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39608 #4079902 (1507 connections now open) 2016-03-13T09:08:00.760+0000 I NETWORK [conn4079887] end connection 10.0.26.69:41592 (1506 connections now open) 2016-03-13T09:08:00.760+0000 I NETWORK [conn4079888] end connection 10.0.26.69:41593 (1506 connections now open) 2016-03-13T09:08:00.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41611 #4079903 (1507 connections now open) 2016-03-13T09:08:00.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41612 #4079904 (1507 connections now open) 2016-03-13T09:08:03.613+0000 I NETWORK [conn4079890] end connection 10.0.27.237:34880 (1506 connections now open) 2016-03-13T09:08:03.613+0000 I NETWORK [conn4079889] end connection 10.0.27.237:34879 (1506 connections now open) 2016-03-13T09:08:03.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34893 #4079905 (1506 connections now open) 2016-03-13T09:08:03.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34894 #4079906 (1507 connections now open) 2016-03-13T09:08:05.710+0000 I NETWORK [conn4079892] end connection 10.0.7.113:58544 (1506 connections now open) 2016-03-13T09:08:05.710+0000 I NETWORK [conn4079891] end connection 10.0.7.113:58543 (1506 connections now open) 2016-03-13T09:08:05.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58560 #4079907 (1506 connections now open) 2016-03-13T09:08:05.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58559 #4079908 (1507 connections now open) 2016-03-13T09:08:06.492+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:08:06.222+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:08:06.492+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:08:06.628+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52de218fa188537623d23 2016-03-13T09:08:06.765+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52dec18fa188537623d24 2016-03-13T09:08:06.839+0000 I NETWORK [conn4079894] end connection 10.0.216.90:56568 (1506 connections now open) 2016-03-13T09:08:06.839+0000 I NETWORK [conn4079893] end connection 10.0.216.90:56567 (1506 connections now open) 2016-03-13T09:08:06.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56585 #4079909 (1506 connections now open) 2016-03-13T09:08:06.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56584 #4079910 (1507 connections now open) 2016-03-13T09:08:06.901+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52df618fa188537623d25 2016-03-13T09:08:07.797+0000 I NETWORK [conn4079896] end connection 10.0.20.53:49812 (1506 connections now open) 2016-03-13T09:08:07.797+0000 I NETWORK [conn4079895] end connection 10.0.20.53:49813 (1506 connections now open) 2016-03-13T09:08:07.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49836 #4079911 (1506 connections now open) 2016-03-13T09:08:07.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49835 #4079912 (1507 connections now open) 2016-03-13T09:08:10.155+0000 I NETWORK [conn4079902] end connection 10.0.179.209:39608 (1506 connections now open) 2016-03-13T09:08:10.155+0000 I NETWORK [conn4079901] end connection 10.0.179.209:39609 (1505 connections now open) 2016-03-13T09:08:10.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39622 #4079913 (1506 connections now open) 2016-03-13T09:08:10.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39623 #4079914 (1507 connections now open) 2016-03-13T09:08:10.773+0000 I NETWORK [conn4079904] end connection 10.0.26.69:41612 (1506 connections now open) 2016-03-13T09:08:10.773+0000 I NETWORK [conn4079903] end connection 10.0.26.69:41611 (1506 connections now open) 2016-03-13T09:08:10.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41620 #4079915 (1506 connections now open) 2016-03-13T09:08:10.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41621 #4079916 (1507 connections now open) 2016-03-13T09:08:13.626+0000 I NETWORK [conn4079906] end connection 10.0.27.237:34894 (1506 connections now open) 2016-03-13T09:08:13.626+0000 I NETWORK [conn4079905] end connection 10.0.27.237:34893 (1506 connections now open) 2016-03-13T09:08:13.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34908 #4079917 (1506 connections now open) 2016-03-13T09:08:13.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34907 #4079918 (1507 connections now open) 2016-03-13T09:08:15.723+0000 I NETWORK [conn4079907] end connection 10.0.7.113:58560 (1506 connections now open) 2016-03-13T09:08:15.723+0000 I NETWORK [conn4079908] end connection 10.0.7.113:58559 (1505 connections now open) 2016-03-13T09:08:15.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58580 #4079919 (1506 connections now open) 2016-03-13T09:08:15.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58581 #4079920 (1507 connections now open) 2016-03-13T09:08:15.943+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16787 version: 30612|1||563ba74d869758a4542b9075 based on: 30611|1||563ba74d869758a4542b9075 2016-03-13T09:08:16.117+0000 I NETWORK [conn4067774] end connection 10.0.20.53:37152 (1506 connections now open) 2016-03-13T09:08:16.117+0000 I NETWORK [conn4071353] end connection 10.0.27.237:54404 (1506 connections now open) 2016-03-13T09:08:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49842 #4079921 (1506 connections now open) 2016-03-13T09:08:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34914 #4079922 (1507 connections now open) 2016-03-13T09:08:16.852+0000 I NETWORK [conn4079909] end connection 10.0.216.90:56585 (1506 connections now open) 2016-03-13T09:08:16.852+0000 I NETWORK [conn4079910] end connection 10.0.216.90:56584 (1506 connections now open) 2016-03-13T09:08:16.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56604 #4079923 (1506 connections now open) 2016-03-13T09:08:16.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56605 #4079924 (1507 connections now open) 2016-03-13T09:08:17.811+0000 I NETWORK [conn4079911] end connection 10.0.20.53:49836 (1506 connections now open) 2016-03-13T09:08:17.811+0000 I NETWORK [conn4079912] end connection 10.0.20.53:49835 (1506 connections now open) 2016-03-13T09:08:17.820+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49843 #4079925 (1506 connections now open) 2016-03-13T09:08:17.820+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49844 #4079926 (1507 connections now open) 2016-03-13T09:08:20.176+0000 I NETWORK [conn4079913] end connection 10.0.179.209:39622 (1506 connections now open) 2016-03-13T09:08:20.176+0000 I NETWORK [conn4079914] end connection 10.0.179.209:39623 (1506 connections now open) 2016-03-13T09:08:20.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39645 #4079927 (1506 connections now open) 2016-03-13T09:08:20.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39646 #4079928 (1507 connections now open) 2016-03-13T09:08:20.787+0000 I NETWORK [conn4079915] end connection 10.0.26.69:41620 (1506 connections now open) 2016-03-13T09:08:20.787+0000 I NETWORK [conn4079916] end connection 10.0.26.69:41621 (1506 connections now open) 2016-03-13T09:08:20.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41638 #4079929 (1506 connections now open) 2016-03-13T09:08:20.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41639 #4079930 (1507 connections now open) 2016-03-13T09:08:23.648+0000 I NETWORK [conn4079917] end connection 10.0.27.237:34908 (1506 connections now open) 2016-03-13T09:08:23.648+0000 I NETWORK [conn4079918] end connection 10.0.27.237:34907 (1506 connections now open) 2016-03-13T09:08:23.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34927 #4079931 (1506 connections now open) 2016-03-13T09:08:23.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34928 #4079932 (1507 connections now open) 2016-03-13T09:08:25.737+0000 I NETWORK [conn4079919] end connection 10.0.7.113:58580 (1506 connections now open) 2016-03-13T09:08:25.737+0000 I NETWORK [conn4079920] end connection 10.0.7.113:58581 (1506 connections now open) 2016-03-13T09:08:25.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58590 #4079933 (1506 connections now open) 2016-03-13T09:08:25.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58589 #4079934 (1507 connections now open) 2016-03-13T09:08:26.874+0000 I NETWORK [conn4079924] end connection 10.0.216.90:56605 (1506 connections now open) 2016-03-13T09:08:26.874+0000 I NETWORK [conn4079923] end connection 10.0.216.90:56604 (1506 connections now open) 2016-03-13T09:08:26.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56612 #4079935 (1506 connections now open) 2016-03-13T09:08:26.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56611 #4079936 (1507 connections now open) 2016-03-13T09:08:27.833+0000 I NETWORK [conn4079926] end connection 10.0.20.53:49844 (1506 connections now open) 2016-03-13T09:08:27.833+0000 I NETWORK [conn4079925] end connection 10.0.20.53:49843 (1506 connections now open) 2016-03-13T09:08:27.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49857 #4079937 (1506 connections now open) 2016-03-13T09:08:27.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49856 #4079938 (1507 connections now open) 2016-03-13T09:08:28.113+0000 I NETWORK [conn4070056] end connection 10.0.26.69:59361 (1506 connections now open) 2016-03-13T09:08:28.113+0000 I NETWORK [conn4068578] end connection 10.0.27.237:51455 (1505 connections now open) 2016-03-13T09:08:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41655 #4079939 (1506 connections now open) 2016-03-13T09:08:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34935 #4079940 (1507 connections now open) 2016-03-13T09:08:29.882+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16788 version: 30613|1||563ba74d869758a4542b9075 based on: 30612|1||563ba74d869758a4542b9075 2016-03-13T09:08:30.190+0000 I NETWORK [conn4079928] end connection 10.0.179.209:39646 (1506 connections now open) 2016-03-13T09:08:30.190+0000 I NETWORK [conn4079927] end connection 10.0.179.209:39645 (1506 connections now open) 2016-03-13T09:08:30.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39653 #4079941 (1506 connections now open) 2016-03-13T09:08:30.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39654 #4079942 (1507 connections now open) 2016-03-13T09:08:30.808+0000 I NETWORK [conn4079929] end connection 10.0.26.69:41638 (1506 connections now open) 2016-03-13T09:08:30.808+0000 I NETWORK [conn4079930] end connection 10.0.26.69:41639 (1506 connections now open) 2016-03-13T09:08:30.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41657 #4079943 (1506 connections now open) 2016-03-13T09:08:30.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41658 #4079944 (1507 connections now open) 2016-03-13T09:08:33.660+0000 I NETWORK [conn4079932] end connection 10.0.27.237:34928 (1506 connections now open) 2016-03-13T09:08:33.661+0000 I NETWORK [conn4079931] end connection 10.0.27.237:34927 (1506 connections now open) 2016-03-13T09:08:33.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34937 #4079945 (1506 connections now open) 2016-03-13T09:08:33.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34938 #4079946 (1507 connections now open) 2016-03-13T09:08:35.760+0000 I NETWORK [conn4079934] end connection 10.0.7.113:58589 (1506 connections now open) 2016-03-13T09:08:35.760+0000 I NETWORK [conn4079933] end connection 10.0.7.113:58590 (1506 connections now open) 2016-03-13T09:08:35.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58604 #4079947 (1506 connections now open) 2016-03-13T09:08:35.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58605 #4079948 (1507 connections now open) 2016-03-13T09:08:36.887+0000 I NETWORK [conn4079936] end connection 10.0.216.90:56611 (1506 connections now open) 2016-03-13T09:08:36.887+0000 I NETWORK [conn4079935] end connection 10.0.216.90:56612 (1506 connections now open) 2016-03-13T09:08:36.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56630 #4079949 (1506 connections now open) 2016-03-13T09:08:36.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56629 #4079950 (1507 connections now open) 2016-03-13T09:08:37.846+0000 I NETWORK [conn4079937] end connection 10.0.20.53:49857 (1506 connections now open) 2016-03-13T09:08:37.846+0000 I NETWORK [conn4079938] end connection 10.0.20.53:49856 (1506 connections now open) 2016-03-13T09:08:37.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49882 #4079951 (1506 connections now open) 2016-03-13T09:08:37.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49881 #4079952 (1507 connections now open) 2016-03-13T09:08:38.833+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:08:36.901+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:08:38.833+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:08:40.203+0000 I NETWORK [conn4079941] end connection 10.0.179.209:39653 (1506 connections now open) 2016-03-13T09:08:40.203+0000 I NETWORK [conn4079942] end connection 10.0.179.209:39654 (1506 connections now open) 2016-03-13T09:08:40.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39665 #4079953 (1506 connections now open) 2016-03-13T09:08:40.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39666 #4079954 (1507 connections now open) 2016-03-13T09:08:40.256+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52e0018fa188537623d26 2016-03-13T09:08:40.393+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52e0a18fa188537623d27 2016-03-13T09:08:40.529+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52e1518fa188537623d28 2016-03-13T09:08:40.822+0000 I NETWORK [conn4079944] end connection 10.0.26.69:41658 (1506 connections now open) 2016-03-13T09:08:40.822+0000 I NETWORK [conn4079943] end connection 10.0.26.69:41657 (1506 connections now open) 2016-03-13T09:08:40.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41663 #4079955 (1506 connections now open) 2016-03-13T09:08:40.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41664 #4079956 (1507 connections now open) 2016-03-13T09:08:43.674+0000 I NETWORK [conn4079945] end connection 10.0.27.237:34937 (1506 connections now open) 2016-03-13T09:08:43.674+0000 I NETWORK [conn4079946] end connection 10.0.27.237:34938 (1506 connections now open) 2016-03-13T09:08:43.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34949 #4079957 (1506 connections now open) 2016-03-13T09:08:43.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34950 #4079958 (1507 connections now open) 2016-03-13T09:08:44.754+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16789 version: 30614|1||563ba74d869758a4542b9075 based on: 30613|1||563ba74d869758a4542b9075 2016-03-13T09:08:45.773+0000 I NETWORK [conn4079947] end connection 10.0.7.113:58604 (1506 connections now open) 2016-03-13T09:08:45.773+0000 I NETWORK [conn4079948] end connection 10.0.7.113:58605 (1506 connections now open) 2016-03-13T09:08:45.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58621 #4079959 (1506 connections now open) 2016-03-13T09:08:45.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58622 #4079960 (1507 connections now open) 2016-03-13T09:08:46.117+0000 I NETWORK [conn4067857] end connection 10.0.20.53:37241 (1506 connections now open) 2016-03-13T09:08:46.117+0000 I NETWORK [conn4070310] end connection 10.0.7.113:48391 (1505 connections now open) 2016-03-13T09:08:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49890 #4079961 (1506 connections now open) 2016-03-13T09:08:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58624 #4079962 (1507 connections now open) 2016-03-13T09:08:46.900+0000 I NETWORK [conn4079949] end connection 10.0.216.90:56630 (1506 connections now open) 2016-03-13T09:08:46.900+0000 I NETWORK [conn4079950] end connection 10.0.216.90:56629 (1506 connections now open) 2016-03-13T09:08:46.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56648 #4079963 (1506 connections now open) 2016-03-13T09:08:46.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56649 #4079964 (1507 connections now open) 2016-03-13T09:08:47.860+0000 I NETWORK [conn4079952] end connection 10.0.20.53:49881 (1506 connections now open) 2016-03-13T09:08:47.860+0000 I NETWORK [conn4079951] end connection 10.0.20.53:49882 (1506 connections now open) 2016-03-13T09:08:47.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49892 #4079965 (1506 connections now open) 2016-03-13T09:08:47.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49891 #4079966 (1507 connections now open) 2016-03-13T09:08:50.216+0000 I NETWORK [conn4079954] end connection 10.0.179.209:39666 (1506 connections now open) 2016-03-13T09:08:50.216+0000 I NETWORK [conn4079953] end connection 10.0.179.209:39665 (1506 connections now open) 2016-03-13T09:08:50.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39685 #4079967 (1506 connections now open) 2016-03-13T09:08:50.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39686 #4079968 (1507 connections now open) 2016-03-13T09:08:50.836+0000 I NETWORK [conn4079956] end connection 10.0.26.69:41664 (1506 connections now open) 2016-03-13T09:08:50.836+0000 I NETWORK [conn4079955] end connection 10.0.26.69:41663 (1506 connections now open) 2016-03-13T09:08:50.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41677 #4079969 (1506 connections now open) 2016-03-13T09:08:50.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41676 #4079970 (1507 connections now open) 2016-03-13T09:08:53.686+0000 I NETWORK [conn4079958] end connection 10.0.27.237:34950 (1506 connections now open) 2016-03-13T09:08:53.686+0000 I NETWORK [conn4079957] end connection 10.0.27.237:34949 (1505 connections now open) 2016-03-13T09:08:53.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34970 #4079971 (1506 connections now open) 2016-03-13T09:08:53.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34969 #4079972 (1507 connections now open) 2016-03-13T09:08:55.786+0000 I NETWORK [conn4079960] end connection 10.0.7.113:58622 (1506 connections now open) 2016-03-13T09:08:55.786+0000 I NETWORK [conn4079959] end connection 10.0.7.113:58621 (1506 connections now open) 2016-03-13T09:08:55.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58629 #4079973 (1506 connections now open) 2016-03-13T09:08:55.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58630 #4079974 (1507 connections now open) 2016-03-13T09:08:56.913+0000 I NETWORK [conn4079964] end connection 10.0.216.90:56649 (1506 connections now open) 2016-03-13T09:08:56.913+0000 I NETWORK [conn4079963] end connection 10.0.216.90:56648 (1506 connections now open) 2016-03-13T09:08:56.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56656 #4079975 (1506 connections now open) 2016-03-13T09:08:56.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56655 #4079976 (1507 connections now open) 2016-03-13T09:08:57.873+0000 I NETWORK [conn4079965] end connection 10.0.20.53:49892 (1506 connections now open) 2016-03-13T09:08:57.873+0000 I NETWORK [conn4079966] end connection 10.0.20.53:49891 (1506 connections now open) 2016-03-13T09:08:57.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49904 #4079977 (1506 connections now open) 2016-03-13T09:08:57.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49903 #4079978 (1507 connections now open) 2016-03-13T09:08:58.113+0000 I NETWORK [conn4067942] end connection 10.0.179.209:55783 (1506 connections now open) 2016-03-13T09:08:58.113+0000 I NETWORK [conn4077745] end connection 10.0.216.90:54361 (1506 connections now open) 2016-03-13T09:08:58.113+0000 I NETWORK [conn4070141] end connection 10.0.27.237:53120 (1504 connections now open) 2016-03-13T09:08:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56660 #4079979 (1505 connections now open) 2016-03-13T09:08:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39693 #4079980 (1506 connections now open) 2016-03-13T09:08:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34978 #4079981 (1507 connections now open) 2016-03-13T09:08:58.240+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16790 version: 30615|1||563ba74d869758a4542b9075 based on: 30614|1||563ba74d869758a4542b9075 2016-03-13T09:08:59.114+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e2a18fa188537623d2a 2016-03-13T09:09:00.230+0000 I NETWORK [conn4079967] end connection 10.0.179.209:39685 (1506 connections now open) 2016-03-13T09:09:00.230+0000 I NETWORK [conn4079968] end connection 10.0.179.209:39686 (1506 connections now open) 2016-03-13T09:09:00.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39694 #4079982 (1506 connections now open) 2016-03-13T09:09:00.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39695 #4079983 (1507 connections now open) 2016-03-13T09:09:00.849+0000 I NETWORK [conn4079969] end connection 10.0.26.69:41677 (1506 connections now open) 2016-03-13T09:09:00.849+0000 I NETWORK [conn4079970] end connection 10.0.26.69:41676 (1506 connections now open) 2016-03-13T09:09:00.850+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41695 #4079984 (1507 connections now open) 2016-03-13T09:09:00.850+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41696 #4079985 (1507 connections now open) 2016-03-13T09:09:02.494+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563429ffa2a4330ea291fecc')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563429ffa2a4330ea291fecc') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342b257bc3500e945d9d20') }, shard: "shmydomain1", lastmod: Timestamp 30615000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:09:02.650+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30615|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563429ffa2a4330ea291fecc') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342b257bc3500e945d9d20') }) shmydomain1 -> shmydomain3 2016-03-13T09:09:03.700+0000 I NETWORK [conn4079971] end connection 10.0.27.237:34970 (1506 connections now open) 2016-03-13T09:09:03.700+0000 I NETWORK [conn4079972] end connection 10.0.27.237:34969 (1506 connections now open) 2016-03-13T09:09:03.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34983 #4079986 (1506 connections now open) 2016-03-13T09:09:03.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34982 #4079987 (1507 connections now open) 2016-03-13T09:09:05.799+0000 I NETWORK [conn4079974] end connection 10.0.7.113:58630 (1506 connections now open) 2016-03-13T09:09:05.799+0000 I NETWORK [conn4079973] end connection 10.0.7.113:58629 (1506 connections now open) 2016-03-13T09:09:05.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58644 #4079988 (1506 connections now open) 2016-03-13T09:09:05.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58645 #4079989 (1507 connections now open) 2016-03-13T09:09:06.927+0000 I NETWORK [conn4079976] end connection 10.0.216.90:56655 (1506 connections now open) 2016-03-13T09:09:06.927+0000 I NETWORK [conn4079975] end connection 10.0.216.90:56656 (1506 connections now open) 2016-03-13T09:09:06.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56672 #4079990 (1506 connections now open) 2016-03-13T09:09:06.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56671 #4079991 (1507 connections now open) 2016-03-13T09:09:07.887+0000 I NETWORK [conn4079978] end connection 10.0.20.53:49903 (1506 connections now open) 2016-03-13T09:09:07.887+0000 I NETWORK [conn4079977] end connection 10.0.20.53:49904 (1506 connections now open) 2016-03-13T09:09:07.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49926 #4079992 (1506 connections now open) 2016-03-13T09:09:07.888+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49925 #4079993 (1507 connections now open) 2016-03-13T09:09:10.243+0000 I NETWORK [conn4079982] end connection 10.0.179.209:39694 (1506 connections now open) 2016-03-13T09:09:10.243+0000 I NETWORK [conn4079983] end connection 10.0.179.209:39695 (1506 connections now open) 2016-03-13T09:09:10.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39709 #4079994 (1506 connections now open) 2016-03-13T09:09:10.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39708 #4079995 (1507 connections now open) 2016-03-13T09:09:10.788+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:09:10.530+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:09:10.788+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:09:10.863+0000 I NETWORK [conn4079984] end connection 10.0.26.69:41695 (1506 connections now open) 2016-03-13T09:09:10.863+0000 I NETWORK [conn4079985] end connection 10.0.26.69:41696 (1506 connections now open) 2016-03-13T09:09:10.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41704 #4079996 (1506 connections now open) 2016-03-13T09:09:10.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41705 #4079997 (1507 connections now open) 2016-03-13T09:09:10.923+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52e2018fa188537623d29 2016-03-13T09:09:11.563+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16791 version: 30616|1||563ba74d869758a4542b9075 based on: 30615|1||563ba74d869758a4542b9075 2016-03-13T09:09:11.835+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:09:11.835+0000-56e52e3718fa188537623d2b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860151835), what: "balancer.round", ns: "", details: { executionTimeMillis: 13014, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:09:11.985+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:09:13.283+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e3918fa188537623d2c 2016-03-13T09:09:13.713+0000 I NETWORK [conn4079986] end connection 10.0.27.237:34983 (1506 connections now open) 2016-03-13T09:09:13.713+0000 I NETWORK [conn4079987] end connection 10.0.27.237:34982 (1505 connections now open) 2016-03-13T09:09:13.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34998 #4079998 (1506 connections now open) 2016-03-13T09:09:13.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:34997 #4079999 (1507 connections now open) 2016-03-13T09:09:15.798+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56342b257bc3500e945d9d20')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342b257bc3500e945d9d20') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342c4d584ad10f1f94f292') }, shard: "shmydomain1", lastmod: Timestamp 30616000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:09:15.813+0000 I NETWORK [conn4079988] end connection 10.0.7.113:58644 (1506 connections now open) 2016-03-13T09:09:15.813+0000 I NETWORK [conn4079989] end connection 10.0.7.113:58645 (1506 connections now open) 2016-03-13T09:09:15.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58664 #4080000 (1506 connections now open) 2016-03-13T09:09:15.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58665 #4080001 (1507 connections now open) 2016-03-13T09:09:15.952+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30616|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342b257bc3500e945d9d20') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342c4d584ad10f1f94f292') }) shmydomain1 -> shmydomain3 2016-03-13T09:09:16.118+0000 I NETWORK [conn4068494] end connection 10.0.216.90:44436 (1506 connections now open) 2016-03-13T09:09:16.118+0000 I NETWORK [conn4071051] end connection 10.0.27.237:54091 (1505 connections now open) 2016-03-13T09:09:16.118+0000 I NETWORK [conn4071781] end connection 10.0.26.69:32983 (1504 connections now open) 2016-03-13T09:09:16.118+0000 I NETWORK [conn4067732] end connection 10.0.7.113:45623 (1503 connections now open) 2016-03-13T09:09:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35005 #4080002 (1504 connections now open) 2016-03-13T09:09:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41719 #4080003 (1505 connections now open) 2016-03-13T09:09:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58668 #4080004 (1506 connections now open) 2016-03-13T09:09:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56689 #4080005 (1507 connections now open) 2016-03-13T09:09:16.940+0000 I NETWORK [conn4079991] end connection 10.0.216.90:56671 (1506 connections now open) 2016-03-13T09:09:16.940+0000 I NETWORK [conn4079990] end connection 10.0.216.90:56672 (1506 connections now open) 2016-03-13T09:09:16.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56691 #4080006 (1506 connections now open) 2016-03-13T09:09:16.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56690 #4080007 (1507 connections now open) 2016-03-13T09:09:17.900+0000 I NETWORK [conn4079993] end connection 10.0.20.53:49925 (1506 connections now open) 2016-03-13T09:09:17.900+0000 I NETWORK [conn4079992] end connection 10.0.20.53:49926 (1506 connections now open) 2016-03-13T09:09:17.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49936 #4080008 (1506 connections now open) 2016-03-13T09:09:17.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49937 #4080009 (1507 connections now open) 2016-03-13T09:09:20.264+0000 I NETWORK [conn4079994] end connection 10.0.179.209:39709 (1506 connections now open) 2016-03-13T09:09:20.265+0000 I NETWORK [conn4079995] end connection 10.0.179.209:39708 (1505 connections now open) 2016-03-13T09:09:20.265+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39731 #4080010 (1506 connections now open) 2016-03-13T09:09:20.265+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39732 #4080011 (1507 connections now open) 2016-03-13T09:09:20.876+0000 I NETWORK [conn4079996] end connection 10.0.26.69:41704 (1506 connections now open) 2016-03-13T09:09:20.876+0000 I NETWORK [conn4079997] end connection 10.0.26.69:41705 (1506 connections now open) 2016-03-13T09:09:20.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41722 #4080012 (1506 connections now open) 2016-03-13T09:09:20.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41723 #4080013 (1507 connections now open) 2016-03-13T09:09:23.734+0000 I NETWORK [conn4079998] end connection 10.0.27.237:34998 (1506 connections now open) 2016-03-13T09:09:23.734+0000 I NETWORK [conn4079999] end connection 10.0.27.237:34997 (1506 connections now open) 2016-03-13T09:09:23.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35018 #4080014 (1506 connections now open) 2016-03-13T09:09:23.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35019 #4080015 (1507 connections now open) 2016-03-13T09:09:24.926+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16792 version: 30617|1||563ba74d869758a4542b9075 based on: 30616|1||563ba74d869758a4542b9075 2016-03-13T09:09:25.197+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:09:25.197+0000-56e52e4518fa188537623d2d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860165197), what: "balancer.round", ns: "", details: { executionTimeMillis: 12175, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:09:25.340+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:09:25.834+0000 I NETWORK [conn4080000] end connection 10.0.7.113:58664 (1506 connections now open) 2016-03-13T09:09:25.834+0000 I NETWORK [conn4080001] end connection 10.0.7.113:58665 (1506 connections now open) 2016-03-13T09:09:25.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58673 #4080016 (1506 connections now open) 2016-03-13T09:09:25.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58674 #4080017 (1507 connections now open) 2016-03-13T09:09:26.699+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e4618fa188537623d2e 2016-03-13T09:09:26.953+0000 I NETWORK [conn4080006] end connection 10.0.216.90:56691 (1506 connections now open) 2016-03-13T09:09:26.953+0000 I NETWORK [conn4080007] end connection 10.0.216.90:56690 (1506 connections now open) 2016-03-13T09:09:26.953+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56696 #4080018 (1506 connections now open) 2016-03-13T09:09:26.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56697 #4080019 (1507 connections now open) 2016-03-13T09:09:27.922+0000 I NETWORK [conn4080008] end connection 10.0.20.53:49936 (1506 connections now open) 2016-03-13T09:09:27.922+0000 I NETWORK [conn4080009] end connection 10.0.20.53:49937 (1506 connections now open) 2016-03-13T09:09:27.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49948 #4080020 (1506 connections now open) 2016-03-13T09:09:27.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49949 #4080021 (1507 connections now open) 2016-03-13T09:09:28.113+0000 I NETWORK [conn4071203] end connection 10.0.179.209:59048 (1506 connections now open) 2016-03-13T09:09:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39738 #4080022 (1507 connections now open) 2016-03-13T09:09:29.259+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56342c4d584ad10f1f94f292')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342c4d584ad10f1f94f292') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342d69584ad10f1f950b2b') }, shard: "shmydomain1", lastmod: Timestamp 30617000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:09:29.413+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30617|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342c4d584ad10f1f94f292') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342d69584ad10f1f950b2b') }) shmydomain1 -> shmydomain3 2016-03-13T09:09:30.278+0000 I NETWORK [conn4080011] end connection 10.0.179.209:39732 (1506 connections now open) 2016-03-13T09:09:30.278+0000 I NETWORK [conn4080010] end connection 10.0.179.209:39731 (1506 connections now open) 2016-03-13T09:09:30.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39739 #4080023 (1506 connections now open) 2016-03-13T09:09:30.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39740 #4080024 (1507 connections now open) 2016-03-13T09:09:30.898+0000 I NETWORK [conn4080013] end connection 10.0.26.69:41723 (1506 connections now open) 2016-03-13T09:09:30.898+0000 I NETWORK [conn4080012] end connection 10.0.26.69:41722 (1506 connections now open) 2016-03-13T09:09:30.899+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41741 #4080025 (1507 connections now open) 2016-03-13T09:09:30.899+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41740 #4080026 (1507 connections now open) 2016-03-13T09:09:33.747+0000 I NETWORK [conn4080014] end connection 10.0.27.237:35018 (1506 connections now open) 2016-03-13T09:09:33.747+0000 I NETWORK [conn4080015] end connection 10.0.27.237:35019 (1505 connections now open) 2016-03-13T09:09:33.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35028 #4080027 (1506 connections now open) 2016-03-13T09:09:33.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35029 #4080028 (1507 connections now open) 2016-03-13T09:09:35.856+0000 I NETWORK [conn4080017] end connection 10.0.7.113:58674 (1506 connections now open) 2016-03-13T09:09:35.856+0000 I NETWORK [conn4080016] end connection 10.0.7.113:58673 (1506 connections now open) 2016-03-13T09:09:35.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58687 #4080029 (1506 connections now open) 2016-03-13T09:09:35.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58686 #4080030 (1507 connections now open) 2016-03-13T09:09:36.966+0000 I NETWORK [conn4080019] end connection 10.0.216.90:56697 (1506 connections now open) 2016-03-13T09:09:36.966+0000 I NETWORK [conn4080018] end connection 10.0.216.90:56696 (1506 connections now open) 2016-03-13T09:09:36.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56713 #4080031 (1506 connections now open) 2016-03-13T09:09:36.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56714 #4080032 (1507 connections now open) 2016-03-13T09:09:37.935+0000 I NETWORK [conn4080020] end connection 10.0.20.53:49948 (1506 connections now open) 2016-03-13T09:09:37.935+0000 I NETWORK [conn4080021] end connection 10.0.20.53:49949 (1506 connections now open) 2016-03-13T09:09:37.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49971 #4080033 (1506 connections now open) 2016-03-13T09:09:37.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49972 #4080034 (1507 connections now open) 2016-03-13T09:09:38.352+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16793 version: 30618|1||563ba74d869758a4542b9075 based on: 30617|1||563ba74d869758a4542b9075 2016-03-13T09:09:38.623+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:09:38.623+0000-56e52e5218fa188537623d2f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860178623), what: "balancer.round", ns: "", details: { executionTimeMillis: 12226, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:09:38.804+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:09:40.172+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e5318fa188537623d30 2016-03-13T09:09:40.291+0000 I NETWORK [conn4080024] end connection 10.0.179.209:39740 (1506 connections now open) 2016-03-13T09:09:40.291+0000 I NETWORK [conn4080023] end connection 10.0.179.209:39739 (1506 connections now open) 2016-03-13T09:09:40.292+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39751 #4080035 (1506 connections now open) 2016-03-13T09:09:40.292+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39750 #4080036 (1507 connections now open) 2016-03-13T09:09:40.912+0000 I NETWORK [conn4080026] end connection 10.0.26.69:41740 (1506 connections now open) 2016-03-13T09:09:40.912+0000 I NETWORK [conn4080025] end connection 10.0.26.69:41741 (1506 connections now open) 2016-03-13T09:09:40.912+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41748 #4080037 (1506 connections now open) 2016-03-13T09:09:40.912+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41749 #4080038 (1507 connections now open) 2016-03-13T09:09:43.412+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:09:40.923+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:09:43.472+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56342d69584ad10f1f950b2b')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342d69584ad10f1f950b2b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342e90ce48b40ea36d8e59') }, shard: "shmydomain1", lastmod: Timestamp 30618000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:09:43.627+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30618|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342d69584ad10f1f950b2b') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342e90ce48b40ea36d8e59') }) shmydomain1 -> shmydomain3 2016-03-13T09:09:43.760+0000 I NETWORK [conn4080027] end connection 10.0.27.237:35028 (1506 connections now open) 2016-03-13T09:09:43.760+0000 I NETWORK [conn4080028] end connection 10.0.27.237:35029 (1506 connections now open) 2016-03-13T09:09:43.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35040 #4080039 (1506 connections now open) 2016-03-13T09:09:43.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35041 #4080040 (1507 connections now open) 2016-03-13T09:09:45.869+0000 I NETWORK [conn4080030] end connection 10.0.7.113:58686 (1506 connections now open) 2016-03-13T09:09:45.869+0000 I NETWORK [conn4080029] end connection 10.0.7.113:58687 (1506 connections now open) 2016-03-13T09:09:45.870+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58704 #4080041 (1506 connections now open) 2016-03-13T09:09:45.870+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58705 #4080042 (1507 connections now open) 2016-03-13T09:09:46.117+0000 I NETWORK [conn4066574] end connection 10.0.179.209:54327 (1506 connections now open) 2016-03-13T09:09:46.118+0000 I NETWORK [conn4071330] end connection 10.0.216.90:47472 (1505 connections now open) 2016-03-13T09:09:46.118+0000 I NETWORK [conn4072453] end connection 10.0.26.69:33709 (1504 connections now open) 2016-03-13T09:09:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39767 #4080043 (1505 connections now open) 2016-03-13T09:09:46.118+0000 I NETWORK [conn4077259] end connection 10.0.7.113:55800 (1504 connections now open) 2016-03-13T09:09:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56731 #4080044 (1505 connections now open) 2016-03-13T09:09:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41760 #4080045 (1506 connections now open) 2016-03-13T09:09:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58707 #4080046 (1507 connections now open) 2016-03-13T09:09:46.979+0000 I NETWORK [conn4080032] end connection 10.0.216.90:56714 (1506 connections now open) 2016-03-13T09:09:46.979+0000 I NETWORK [conn4080031] end connection 10.0.216.90:56713 (1506 connections now open) 2016-03-13T09:09:46.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56733 #4080047 (1506 connections now open) 2016-03-13T09:09:46.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56734 #4080048 (1507 connections now open) 2016-03-13T09:09:47.948+0000 I NETWORK [conn4080034] end connection 10.0.20.53:49972 (1506 connections now open) 2016-03-13T09:09:47.948+0000 I NETWORK [conn4080033] end connection 10.0.20.53:49971 (1506 connections now open) 2016-03-13T09:09:47.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49981 #4080049 (1506 connections now open) 2016-03-13T09:09:47.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49982 #4080050 (1507 connections now open) 2016-03-13T09:09:50.304+0000 I NETWORK [conn4080036] end connection 10.0.179.209:39750 (1506 connections now open) 2016-03-13T09:09:50.304+0000 I NETWORK [conn4080035] end connection 10.0.179.209:39751 (1506 connections now open) 2016-03-13T09:09:50.305+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39771 #4080051 (1506 connections now open) 2016-03-13T09:09:50.305+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39772 #4080052 (1507 connections now open) 2016-03-13T09:09:50.926+0000 I NETWORK [conn4080037] end connection 10.0.26.69:41748 (1506 connections now open) 2016-03-13T09:09:50.926+0000 I NETWORK [conn4080038] end connection 10.0.26.69:41749 (1506 connections now open) 2016-03-13T09:09:50.926+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41763 #4080053 (1506 connections now open) 2016-03-13T09:09:50.926+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41764 #4080054 (1507 connections now open) 2016-03-13T09:09:53.631+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16794 version: 30619|1||563ba74d869758a4542b9075 based on: 30618|1||563ba74d869758a4542b9075 2016-03-13T09:09:53.773+0000 I NETWORK [conn4080040] end connection 10.0.27.237:35041 (1506 connections now open) 2016-03-13T09:09:53.773+0000 I NETWORK [conn4080039] end connection 10.0.27.237:35040 (1506 connections now open) 2016-03-13T09:09:53.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35060 #4080055 (1506 connections now open) 2016-03-13T09:09:53.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35059 #4080056 (1507 connections now open) 2016-03-13T09:09:53.902+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:09:53.902+0000-56e52e6118fa188537623d31", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860193902), what: "balancer.round", ns: "", details: { executionTimeMillis: 14063, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:09:54.110+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:09:55.447+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e6318fa188537623d32 2016-03-13T09:09:55.883+0000 I NETWORK [conn4080041] end connection 10.0.7.113:58704 (1506 connections now open) 2016-03-13T09:09:55.883+0000 I NETWORK [conn4080042] end connection 10.0.7.113:58705 (1506 connections now open) 2016-03-13T09:09:55.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58712 #4080057 (1506 connections now open) 2016-03-13T09:09:55.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58711 #4080058 (1507 connections now open) 2016-03-13T09:09:56.992+0000 I NETWORK [conn4080047] end connection 10.0.216.90:56733 (1506 connections now open) 2016-03-13T09:09:56.992+0000 I NETWORK [conn4080048] end connection 10.0.216.90:56734 (1505 connections now open) 2016-03-13T09:09:56.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56739 #4080059 (1506 connections now open) 2016-03-13T09:09:56.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56740 #4080060 (1507 connections now open) 2016-03-13T09:09:57.962+0000 I NETWORK [conn4080050] end connection 10.0.20.53:49982 (1506 connections now open) 2016-03-13T09:09:57.962+0000 I NETWORK [conn4080049] end connection 10.0.20.53:49981 (1506 connections now open) 2016-03-13T09:09:57.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49994 #4080061 (1506 connections now open) 2016-03-13T09:09:57.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49995 #4080062 (1507 connections now open) 2016-03-13T09:09:58.113+0000 I NETWORK [conn4076329] end connection 10.0.179.209:36028 (1506 connections now open) 2016-03-13T09:09:58.113+0000 I NETWORK [conn4067465] end connection 10.0.20.53:36827 (1506 connections now open) 2016-03-13T09:09:58.113+0000 I NETWORK [conn4070797] end connection 10.0.216.90:46901 (1504 connections now open) 2016-03-13T09:09:58.113+0000 I NETWORK [conn4070267] end connection 10.0.26.69:59589 (1503 connections now open) 2016-03-13T09:09:58.113+0000 I NETWORK [conn4070879] end connection 10.0.7.113:49008 (1502 connections now open) 2016-03-13T09:09:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39779 #4080063 (1503 connections now open) 2016-03-13T09:09:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:49997 #4080064 (1504 connections now open) 2016-03-13T09:09:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41780 #4080065 (1505 connections now open) 2016-03-13T09:09:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56742 #4080066 (1506 connections now open) 2016-03-13T09:09:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58719 #4080067 (1507 connections now open) 2016-03-13T09:09:58.157+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56342e90ce48b40ea36d8e59')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342e90ce48b40ea36d8e59') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342fbc7bc3500e96202352') }, shard: "shmydomain1", lastmod: Timestamp 30619000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:09:58.311+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30619|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342e90ce48b40ea36d8e59') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342fbc7bc3500e96202352') }) shmydomain1 -> shmydomain3 2016-03-13T09:10:00.318+0000 I NETWORK [conn4080051] end connection 10.0.179.209:39771 (1506 connections now open) 2016-03-13T09:10:00.318+0000 I NETWORK [conn4080052] end connection 10.0.179.209:39772 (1506 connections now open) 2016-03-13T09:10:00.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39781 #4080068 (1506 connections now open) 2016-03-13T09:10:00.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39780 #4080069 (1507 connections now open) 2016-03-13T09:10:00.939+0000 I NETWORK [conn4080053] end connection 10.0.26.69:41763 (1506 connections now open) 2016-03-13T09:10:00.939+0000 I NETWORK [conn4080054] end connection 10.0.26.69:41764 (1506 connections now open) 2016-03-13T09:10:00.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41783 #4080070 (1507 connections now open) 2016-03-13T09:10:00.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41782 #4080071 (1507 connections now open) 2016-03-13T09:10:03.786+0000 I NETWORK [conn4080055] end connection 10.0.27.237:35060 (1506 connections now open) 2016-03-13T09:10:03.786+0000 I NETWORK [conn4080056] end connection 10.0.27.237:35059 (1506 connections now open) 2016-03-13T09:10:03.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35073 #4080072 (1506 connections now open) 2016-03-13T09:10:03.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35074 #4080073 (1507 connections now open) 2016-03-13T09:10:05.896+0000 I NETWORK [conn4080058] end connection 10.0.7.113:58711 (1506 connections now open) 2016-03-13T09:10:05.896+0000 I NETWORK [conn4080057] end connection 10.0.7.113:58712 (1506 connections now open) 2016-03-13T09:10:05.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58730 #4080074 (1506 connections now open) 2016-03-13T09:10:05.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58729 #4080075 (1507 connections now open) 2016-03-13T09:10:07.005+0000 I NETWORK [conn4080059] end connection 10.0.216.90:56739 (1506 connections now open) 2016-03-13T09:10:07.005+0000 I NETWORK [conn4080060] end connection 10.0.216.90:56740 (1506 connections now open) 2016-03-13T09:10:07.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56755 #4080076 (1507 connections now open) 2016-03-13T09:10:07.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56756 #4080077 (1507 connections now open) 2016-03-13T09:10:07.975+0000 I NETWORK [conn4080061] end connection 10.0.20.53:49994 (1506 connections now open) 2016-03-13T09:10:07.975+0000 I NETWORK [conn4080062] end connection 10.0.20.53:49995 (1506 connections now open) 2016-03-13T09:10:07.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50016 #4080078 (1506 connections now open) 2016-03-13T09:10:07.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50017 #4080079 (1507 connections now open) 2016-03-13T09:10:08.257+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16795 version: 30620|1||563ba74d869758a4542b9075 based on: 30619|1||563ba74d869758a4542b9075 2016-03-13T09:10:08.527+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:10:08.527+0000-56e52e7018fa188537623d33", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860208527), what: "balancer.round", ns: "", details: { executionTimeMillis: 13383, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:10:08.678+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:10:10.011+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e7118fa188537623d34 2016-03-13T09:10:10.331+0000 I NETWORK [conn4080069] end connection 10.0.179.209:39780 (1506 connections now open) 2016-03-13T09:10:10.331+0000 I NETWORK [conn4080068] end connection 10.0.179.209:39781 (1506 connections now open) 2016-03-13T09:10:10.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39797 #4080080 (1506 connections now open) 2016-03-13T09:10:10.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39798 #4080081 (1507 connections now open) 2016-03-13T09:10:10.953+0000 I NETWORK [conn4080071] end connection 10.0.26.69:41782 (1506 connections now open) 2016-03-13T09:10:10.954+0000 I NETWORK [conn4080070] end connection 10.0.26.69:41783 (1505 connections now open) 2016-03-13T09:10:10.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41791 #4080082 (1506 connections now open) 2016-03-13T09:10:10.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41792 #4080083 (1507 connections now open) 2016-03-13T09:10:12.903+0000 I NETWORK [conn4078439] end connection 10.0.91.115:53823 (1506 connections now open) 2016-03-13T09:10:12.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:54791 #4080084 (1507 connections now open) 2016-03-13T09:10:12.996+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56342fbc7bc3500e96202352')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342fbc7bc3500e96202352') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563430eea2a4330ea41e7d4d') }, shard: "shmydomain1", lastmod: Timestamp 30620000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:10:13.149+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30620|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56342fbc7bc3500e96202352') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563430eea2a4330ea41e7d4d') }) shmydomain1 -> shmydomain2 2016-03-13T09:10:13.662+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:10:13.412+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:10:13.799+0000 I NETWORK [conn4080072] end connection 10.0.27.237:35073 (1506 connections now open) 2016-03-13T09:10:13.799+0000 I NETWORK [conn4080073] end connection 10.0.27.237:35074 (1506 connections now open) 2016-03-13T09:10:13.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35090 #4080085 (1506 connections now open) 2016-03-13T09:10:13.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35089 #4080086 (1507 connections now open) 2016-03-13T09:10:15.910+0000 I NETWORK [conn4080074] end connection 10.0.7.113:58730 (1506 connections now open) 2016-03-13T09:10:15.910+0000 I NETWORK [conn4080075] end connection 10.0.7.113:58729 (1506 connections now open) 2016-03-13T09:10:15.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58750 #4080087 (1506 connections now open) 2016-03-13T09:10:15.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58751 #4080088 (1507 connections now open) 2016-03-13T09:10:16.117+0000 I NETWORK [conn4080043] end connection 10.0.179.209:39767 (1506 connections now open) 2016-03-13T09:10:16.117+0000 I NETWORK [conn4071415] end connection 10.0.216.90:47564 (1505 connections now open) 2016-03-13T09:10:16.117+0000 I NETWORK [conn4071945] end connection 10.0.26.69:33168 (1504 connections now open) 2016-03-13T09:10:16.117+0000 I NETWORK [conn4072284] end connection 10.0.27.237:55401 (1504 connections now open) 2016-03-13T09:10:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39813 #4080089 (1504 connections now open) 2016-03-13T09:10:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35096 #4080090 (1505 connections now open) 2016-03-13T09:10:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41805 #4080091 (1506 connections now open) 2016-03-13T09:10:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56775 #4080092 (1507 connections now open) 2016-03-13T09:10:17.018+0000 I NETWORK [conn4080076] end connection 10.0.216.90:56755 (1506 connections now open) 2016-03-13T09:10:17.018+0000 I NETWORK [conn4080077] end connection 10.0.216.90:56756 (1506 connections now open) 2016-03-13T09:10:17.019+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56776 #4080093 (1506 connections now open) 2016-03-13T09:10:17.019+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56777 #4080094 (1507 connections now open) 2016-03-13T09:10:17.989+0000 I NETWORK [conn4080078] end connection 10.0.20.53:50016 (1506 connections now open) 2016-03-13T09:10:17.989+0000 I NETWORK [conn4080079] end connection 10.0.20.53:50017 (1506 connections now open) 2016-03-13T09:10:17.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50026 #4080095 (1506 connections now open) 2016-03-13T09:10:17.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50027 #4080096 (1507 connections now open) 2016-03-13T09:10:20.352+0000 I NETWORK [conn4080080] end connection 10.0.179.209:39797 (1506 connections now open) 2016-03-13T09:10:20.352+0000 I NETWORK [conn4080081] end connection 10.0.179.209:39798 (1506 connections now open) 2016-03-13T09:10:20.353+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39821 #4080097 (1506 connections now open) 2016-03-13T09:10:20.353+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39822 #4080098 (1507 connections now open) 2016-03-13T09:10:20.967+0000 I NETWORK [conn4080083] end connection 10.0.26.69:41792 (1506 connections now open) 2016-03-13T09:10:20.967+0000 I NETWORK [conn4080082] end connection 10.0.26.69:41791 (1506 connections now open) 2016-03-13T09:10:20.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41808 #4080099 (1506 connections now open) 2016-03-13T09:10:20.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41809 #4080100 (1507 connections now open) 2016-03-13T09:10:23.073+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16796 version: 30621|1||563ba74d869758a4542b9075 based on: 30620|1||563ba74d869758a4542b9075 2016-03-13T09:10:23.344+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:10:23.344+0000-56e52e7f18fa188537623d35", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860223344), what: "balancer.round", ns: "", details: { executionTimeMillis: 13624, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:10:23.506+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:10:23.821+0000 I NETWORK [conn4080085] end connection 10.0.27.237:35090 (1506 connections now open) 2016-03-13T09:10:23.821+0000 I NETWORK [conn4080086] end connection 10.0.27.237:35089 (1506 connections now open) 2016-03-13T09:10:23.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35109 #4080101 (1506 connections now open) 2016-03-13T09:10:23.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35110 #4080102 (1507 connections now open) 2016-03-13T09:10:24.838+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e8018fa188537623d36 2016-03-13T09:10:25.923+0000 I NETWORK [conn4080087] end connection 10.0.7.113:58750 (1506 connections now open) 2016-03-13T09:10:25.923+0000 I NETWORK [conn4080088] end connection 10.0.7.113:58751 (1506 connections now open) 2016-03-13T09:10:25.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58760 #4080103 (1506 connections now open) 2016-03-13T09:10:25.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58759 #4080104 (1507 connections now open) 2016-03-13T09:10:27.031+0000 I NETWORK [conn4080094] end connection 10.0.216.90:56777 (1506 connections now open) 2016-03-13T09:10:27.031+0000 I NETWORK [conn4080093] end connection 10.0.216.90:56776 (1506 connections now open) 2016-03-13T09:10:27.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56782 #4080105 (1506 connections now open) 2016-03-13T09:10:27.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56783 #4080106 (1507 connections now open) 2016-03-13T09:10:27.652+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563430eea2a4330ea41e7d4d')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563430eea2a4330ea41e7d4d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343221a2a4330ea41ea076') }, shard: "shmydomain1", lastmod: Timestamp 30621000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:10:27.809+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30621|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563430eea2a4330ea41e7d4d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343221a2a4330ea41ea076') }) shmydomain1 -> shmydomain3 2016-03-13T09:10:28.011+0000 I NETWORK [conn4080096] end connection 10.0.20.53:50027 (1506 connections now open) 2016-03-13T09:10:28.011+0000 I NETWORK [conn4080095] end connection 10.0.20.53:50026 (1506 connections now open) 2016-03-13T09:10:28.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50037 #4080107 (1506 connections now open) 2016-03-13T09:10:28.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50038 #4080108 (1507 connections now open) 2016-03-13T09:10:28.113+0000 I NETWORK [conn4069510] end connection 10.0.179.209:57353 (1506 connections now open) 2016-03-13T09:10:28.113+0000 I NETWORK [conn4070923] end connection 10.0.216.90:47038 (1505 connections now open) 2016-03-13T09:10:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39829 #4080109 (1506 connections now open) 2016-03-13T09:10:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56786 #4080110 (1507 connections now open) 2016-03-13T09:10:28.115+0000 I NETWORK [conn4070122] end connection 10.0.7.113:48195 (1506 connections now open) 2016-03-13T09:10:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58767 #4080111 (1507 connections now open) 2016-03-13T09:10:30.365+0000 I NETWORK [conn4080098] end connection 10.0.179.209:39822 (1506 connections now open) 2016-03-13T09:10:30.365+0000 I NETWORK [conn4080097] end connection 10.0.179.209:39821 (1506 connections now open) 2016-03-13T09:10:30.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39830 #4080112 (1506 connections now open) 2016-03-13T09:10:30.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39831 #4080113 (1507 connections now open) 2016-03-13T09:10:30.988+0000 I NETWORK [conn4080100] end connection 10.0.26.69:41809 (1506 connections now open) 2016-03-13T09:10:30.988+0000 I NETWORK [conn4080099] end connection 10.0.26.69:41808 (1506 connections now open) 2016-03-13T09:10:30.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41827 #4080114 (1506 connections now open) 2016-03-13T09:10:30.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41828 #4080115 (1507 connections now open) 2016-03-13T09:10:33.835+0000 I NETWORK [conn4080101] end connection 10.0.27.237:35109 (1506 connections now open) 2016-03-13T09:10:33.835+0000 I NETWORK [conn4080102] end connection 10.0.27.237:35110 (1506 connections now open) 2016-03-13T09:10:33.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35121 #4080116 (1506 connections now open) 2016-03-13T09:10:33.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35120 #4080117 (1507 connections now open) 2016-03-13T09:10:35.946+0000 I NETWORK [conn4080104] end connection 10.0.7.113:58759 (1506 connections now open) 2016-03-13T09:10:35.946+0000 I NETWORK [conn4080103] end connection 10.0.7.113:58760 (1506 connections now open) 2016-03-13T09:10:35.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58774 #4080118 (1507 connections now open) 2016-03-13T09:10:35.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58773 #4080119 (1507 connections now open) 2016-03-13T09:10:36.677+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16797 version: 30622|1||563ba74d869758a4542b9075 based on: 30621|1||563ba74d869758a4542b9075 2016-03-13T09:10:36.953+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:10:36.953+0000-56e52e8c18fa188537623d37", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860236953), what: "balancer.round", ns: "", details: { executionTimeMillis: 12413, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:10:37.044+0000 I NETWORK [conn4080106] end connection 10.0.216.90:56783 (1506 connections now open) 2016-03-13T09:10:37.044+0000 I NETWORK [conn4080105] end connection 10.0.216.90:56782 (1506 connections now open) 2016-03-13T09:10:37.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56800 #4080120 (1506 connections now open) 2016-03-13T09:10:37.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56799 #4080121 (1507 connections now open) 2016-03-13T09:10:37.114+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:10:38.024+0000 I NETWORK [conn4080107] end connection 10.0.20.53:50037 (1506 connections now open) 2016-03-13T09:10:38.024+0000 I NETWORK [conn4080108] end connection 10.0.20.53:50038 (1506 connections now open) 2016-03-13T09:10:38.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50061 #4080122 (1506 connections now open) 2016-03-13T09:10:38.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50062 #4080123 (1507 connections now open) 2016-03-13T09:10:38.448+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e8e18fa188537623d38 2016-03-13T09:10:40.378+0000 I NETWORK [conn4080112] end connection 10.0.179.209:39830 (1506 connections now open) 2016-03-13T09:10:40.378+0000 I NETWORK [conn4080113] end connection 10.0.179.209:39831 (1506 connections now open) 2016-03-13T09:10:40.379+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39843 #4080124 (1506 connections now open) 2016-03-13T09:10:40.379+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39842 #4080125 (1507 connections now open) 2016-03-13T09:10:40.984+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56343221a2a4330ea41ea076')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343221a2a4330ea41ea076') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343351a2a4330ea41ebb08') }, shard: "shmydomain1", lastmod: Timestamp 30622000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:10:41.002+0000 I NETWORK [conn4080115] end connection 10.0.26.69:41828 (1506 connections now open) 2016-03-13T09:10:41.002+0000 I NETWORK [conn4080114] end connection 10.0.26.69:41827 (1505 connections now open) 2016-03-13T09:10:41.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41834 #4080126 (1506 connections now open) 2016-03-13T09:10:41.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41835 #4080127 (1507 connections now open) 2016-03-13T09:10:41.139+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30622|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343221a2a4330ea41ea076') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343351a2a4330ea41ebb08') }) shmydomain1 -> shmydomain2 2016-03-13T09:10:43.847+0000 I NETWORK [conn4080117] end connection 10.0.27.237:35120 (1506 connections now open) 2016-03-13T09:10:43.847+0000 I NETWORK [conn4080116] end connection 10.0.27.237:35121 (1505 connections now open) 2016-03-13T09:10:43.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35131 #4080128 (1506 connections now open) 2016-03-13T09:10:43.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35132 #4080129 (1507 connections now open) 2016-03-13T09:10:43.943+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:10:43.662+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:10:45.959+0000 I NETWORK [conn4080118] end connection 10.0.7.113:58774 (1506 connections now open) 2016-03-13T09:10:45.959+0000 I NETWORK [conn4080119] end connection 10.0.7.113:58773 (1506 connections now open) 2016-03-13T09:10:45.960+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58790 #4080130 (1506 connections now open) 2016-03-13T09:10:45.960+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58791 #4080131 (1507 connections now open) 2016-03-13T09:10:46.118+0000 I NETWORK [conn4069381] end connection 10.0.216.90:45382 (1506 connections now open) 2016-03-13T09:10:46.118+0000 I NETWORK [conn4069745] end connection 10.0.27.237:52684 (1506 connections now open) 2016-03-13T09:10:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56817 #4080132 (1507 connections now open) 2016-03-13T09:10:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35136 #4080133 (1507 connections now open) 2016-03-13T09:10:47.057+0000 I NETWORK [conn4080120] end connection 10.0.216.90:56800 (1506 connections now open) 2016-03-13T09:10:47.057+0000 I NETWORK [conn4080121] end connection 10.0.216.90:56799 (1506 connections now open) 2016-03-13T09:10:47.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56819 #4080134 (1506 connections now open) 2016-03-13T09:10:47.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56818 #4080135 (1507 connections now open) 2016-03-13T09:10:48.038+0000 I NETWORK [conn4080122] end connection 10.0.20.53:50061 (1506 connections now open) 2016-03-13T09:10:48.038+0000 I NETWORK [conn4080123] end connection 10.0.20.53:50062 (1506 connections now open) 2016-03-13T09:10:48.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50072 #4080136 (1506 connections now open) 2016-03-13T09:10:48.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50073 #4080137 (1507 connections now open) 2016-03-13T09:10:50.391+0000 I NETWORK [conn4080125] end connection 10.0.179.209:39842 (1506 connections now open) 2016-03-13T09:10:50.391+0000 I NETWORK [conn4080124] end connection 10.0.179.209:39843 (1506 connections now open) 2016-03-13T09:10:50.392+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39863 #4080138 (1506 connections now open) 2016-03-13T09:10:50.392+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39864 #4080139 (1507 connections now open) 2016-03-13T09:10:51.015+0000 I NETWORK [conn4080126] end connection 10.0.26.69:41834 (1506 connections now open) 2016-03-13T09:10:51.015+0000 I NETWORK [conn4080127] end connection 10.0.26.69:41835 (1506 connections now open) 2016-03-13T09:10:51.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41847 #4080140 (1506 connections now open) 2016-03-13T09:10:51.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41848 #4080141 (1507 connections now open) 2016-03-13T09:10:51.051+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16798 version: 30623|1||563ba74d869758a4542b9075 based on: 30622|1||563ba74d869758a4542b9075 2016-03-13T09:10:51.322+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:10:51.322+0000-56e52e9b18fa188537623d39", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860251322), what: "balancer.round", ns: "", details: { executionTimeMillis: 13172, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:10:51.508+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:10:52.792+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52e9c18fa188537623d3a 2016-03-13T09:10:53.860+0000 I NETWORK [conn4080128] end connection 10.0.27.237:35131 (1506 connections now open) 2016-03-13T09:10:53.860+0000 I NETWORK [conn4080129] end connection 10.0.27.237:35132 (1505 connections now open) 2016-03-13T09:10:53.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35150 #4080142 (1506 connections now open) 2016-03-13T09:10:53.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35151 #4080143 (1507 connections now open) 2016-03-13T09:10:55.527+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56343351a2a4330ea41ebb08')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343351a2a4330ea41ebb08') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563434917bc3500e945e8073') }, shard: "shmydomain1", lastmod: Timestamp 30623000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:10:55.683+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30623|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56343351a2a4330ea41ebb08') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563434917bc3500e945e8073') }) shmydomain1 -> shmydomain3 2016-03-13T09:10:55.973+0000 I NETWORK [conn4080130] end connection 10.0.7.113:58790 (1506 connections now open) 2016-03-13T09:10:55.973+0000 I NETWORK [conn4080131] end connection 10.0.7.113:58791 (1506 connections now open) 2016-03-13T09:10:55.973+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58799 #4080144 (1506 connections now open) 2016-03-13T09:10:55.973+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58800 #4080145 (1507 connections now open) 2016-03-13T09:10:57.070+0000 I NETWORK [conn4080135] end connection 10.0.216.90:56818 (1506 connections now open) 2016-03-13T09:10:57.070+0000 I NETWORK [conn4080134] end connection 10.0.216.90:56819 (1506 connections now open) 2016-03-13T09:10:57.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56825 #4080146 (1507 connections now open) 2016-03-13T09:10:57.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56826 #4080147 (1507 connections now open) 2016-03-13T09:10:58.051+0000 I NETWORK [conn4080136] end connection 10.0.20.53:50072 (1506 connections now open) 2016-03-13T09:10:58.051+0000 I NETWORK [conn4080137] end connection 10.0.20.53:50073 (1505 connections now open) 2016-03-13T09:10:58.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50085 #4080148 (1506 connections now open) 2016-03-13T09:10:58.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50086 #4080149 (1507 connections now open) 2016-03-13T09:10:58.113+0000 I NETWORK [conn4072635] end connection 10.0.179.209:60465 (1506 connections now open) 2016-03-13T09:10:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39870 #4080150 (1507 connections now open) 2016-03-13T09:11:00.404+0000 I NETWORK [conn4080139] end connection 10.0.179.209:39864 (1506 connections now open) 2016-03-13T09:11:00.404+0000 I NETWORK [conn4080138] end connection 10.0.179.209:39863 (1506 connections now open) 2016-03-13T09:11:00.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39871 #4080151 (1506 connections now open) 2016-03-13T09:11:00.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39872 #4080152 (1507 connections now open) 2016-03-13T09:11:01.029+0000 I NETWORK [conn4080140] end connection 10.0.26.69:41847 (1506 connections now open) 2016-03-13T09:11:01.029+0000 I NETWORK [conn4080141] end connection 10.0.26.69:41848 (1506 connections now open) 2016-03-13T09:11:01.030+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41867 #4080153 (1506 connections now open) 2016-03-13T09:11:01.030+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41866 #4080154 (1507 connections now open) 2016-03-13T09:11:03.873+0000 I NETWORK [conn4080143] end connection 10.0.27.237:35151 (1506 connections now open) 2016-03-13T09:11:03.873+0000 I NETWORK [conn4080142] end connection 10.0.27.237:35150 (1506 connections now open) 2016-03-13T09:11:03.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35163 #4080155 (1506 connections now open) 2016-03-13T09:11:03.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35164 #4080156 (1507 connections now open) 2016-03-13T09:11:05.786+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16799 version: 30624|1||563ba74d869758a4542b9075 based on: 30623|1||563ba74d869758a4542b9075 2016-03-13T09:11:05.986+0000 I NETWORK [conn4080145] end connection 10.0.7.113:58800 (1506 connections now open) 2016-03-13T09:11:05.986+0000 I NETWORK [conn4080144] end connection 10.0.7.113:58799 (1506 connections now open) 2016-03-13T09:11:05.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58816 #4080157 (1506 connections now open) 2016-03-13T09:11:05.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58817 #4080158 (1507 connections now open) 2016-03-13T09:11:06.058+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:11:06.058+0000-56e52eaa18fa188537623d3b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860266058), what: "balancer.round", ns: "", details: { executionTimeMillis: 13516, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:11:06.248+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:11:07.084+0000 I NETWORK [conn4080146] end connection 10.0.216.90:56825 (1506 connections now open) 2016-03-13T09:11:07.084+0000 I NETWORK [conn4080147] end connection 10.0.216.90:56826 (1506 connections now open) 2016-03-13T09:11:07.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56841 #4080159 (1506 connections now open) 2016-03-13T09:11:07.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56840 #4080160 (1507 connections now open) 2016-03-13T09:11:07.581+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52eab18fa188537623d3c 2016-03-13T09:11:08.064+0000 I NETWORK [conn4080148] end connection 10.0.20.53:50085 (1506 connections now open) 2016-03-13T09:11:08.064+0000 I NETWORK [conn4080149] end connection 10.0.20.53:50086 (1506 connections now open) 2016-03-13T09:11:08.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50106 #4080161 (1506 connections now open) 2016-03-13T09:11:08.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50107 #4080162 (1507 connections now open) 2016-03-13T09:11:10.077+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563434917bc3500e945e8073')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563434917bc3500e945e8073') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563435d855418c0ec3f49c0d') }, shard: "shmydomain1", lastmod: Timestamp 30624000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:11:10.238+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30624|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563434917bc3500e945e8073') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563435d855418c0ec3f49c0d') }) shmydomain1 -> shmydomain2 2016-03-13T09:11:10.418+0000 I NETWORK [conn4080151] end connection 10.0.179.209:39871 (1506 connections now open) 2016-03-13T09:11:10.418+0000 I NETWORK [conn4080152] end connection 10.0.179.209:39872 (1506 connections now open) 2016-03-13T09:11:10.427+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39886 #4080163 (1506 connections now open) 2016-03-13T09:11:10.427+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39885 #4080164 (1507 connections now open) 2016-03-13T09:11:11.042+0000 I NETWORK [conn4080154] end connection 10.0.26.69:41866 (1506 connections now open) 2016-03-13T09:11:11.042+0000 I NETWORK [conn4080153] end connection 10.0.26.69:41867 (1506 connections now open) 2016-03-13T09:11:11.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41879 #4080165 (1506 connections now open) 2016-03-13T09:11:11.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41880 #4080166 (1507 connections now open) 2016-03-13T09:11:13.886+0000 I NETWORK [conn4080156] end connection 10.0.27.237:35164 (1506 connections now open) 2016-03-13T09:11:13.886+0000 I NETWORK [conn4080155] end connection 10.0.27.237:35163 (1506 connections now open) 2016-03-13T09:11:13.896+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35179 #4080167 (1506 connections now open) 2016-03-13T09:11:13.896+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35178 #4080168 (1507 connections now open) 2016-03-13T09:11:14.138+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:11:13.943+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:11:15.999+0000 I NETWORK [conn4080158] end connection 10.0.7.113:58817 (1506 connections now open) 2016-03-13T09:11:15.999+0000 I NETWORK [conn4080157] end connection 10.0.7.113:58816 (1506 connections now open) 2016-03-13T09:11:16.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58836 #4080169 (1506 connections now open) 2016-03-13T09:11:16.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58837 #4080170 (1507 connections now open) 2016-03-13T09:11:16.117+0000 I NETWORK [conn4070671] end connection 10.0.20.53:40189 (1506 connections now open) 2016-03-13T09:11:16.118+0000 I NETWORK [conn4080003] end connection 10.0.26.69:41719 (1505 connections now open) 2016-03-13T09:11:16.118+0000 I NETWORK [conn4074462] end connection 10.0.27.237:57650 (1504 connections now open) 2016-03-13T09:11:16.118+0000 I NETWORK [conn4070479] end connection 10.0.7.113:48578 (1504 connections now open) 2016-03-13T09:11:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50115 #4080171 (1504 connections now open) 2016-03-13T09:11:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41885 #4080172 (1505 connections now open) 2016-03-13T09:11:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58840 #4080173 (1506 connections now open) 2016-03-13T09:11:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35183 #4080174 (1507 connections now open) 2016-03-13T09:11:17.104+0000 I NETWORK [conn4080160] end connection 10.0.216.90:56840 (1506 connections now open) 2016-03-13T09:11:17.104+0000 I NETWORK [conn4080159] end connection 10.0.216.90:56841 (1506 connections now open) 2016-03-13T09:11:17.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56861 #4080175 (1506 connections now open) 2016-03-13T09:11:17.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56860 #4080176 (1507 connections now open) 2016-03-13T09:11:18.086+0000 I NETWORK [conn4080162] end connection 10.0.20.53:50107 (1506 connections now open) 2016-03-13T09:11:18.086+0000 I NETWORK [conn4080161] end connection 10.0.20.53:50106 (1506 connections now open) 2016-03-13T09:11:18.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50116 #4080177 (1506 connections now open) 2016-03-13T09:11:18.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50117 #4080178 (1507 connections now open) 2016-03-13T09:11:20.143+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16800 version: 30625|1||563ba74d869758a4542b9075 based on: 30624|1||563ba74d869758a4542b9075 2016-03-13T09:11:20.418+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:11:20.418+0000-56e52eb818fa188537623d3d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860280418), what: "balancer.round", ns: "", details: { executionTimeMillis: 13136, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:11:20.439+0000 I NETWORK [conn4080164] end connection 10.0.179.209:39885 (1506 connections now open) 2016-03-13T09:11:20.439+0000 I NETWORK [conn4080163] end connection 10.0.179.209:39886 (1506 connections now open) 2016-03-13T09:11:20.440+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39909 #4080179 (1506 connections now open) 2016-03-13T09:11:20.440+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39908 #4080180 (1507 connections now open) 2016-03-13T09:11:20.580+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:11:21.064+0000 I NETWORK [conn4080166] end connection 10.0.26.69:41880 (1506 connections now open) 2016-03-13T09:11:21.064+0000 I NETWORK [conn4080165] end connection 10.0.26.69:41879 (1506 connections now open) 2016-03-13T09:11:21.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41891 #4080181 (1506 connections now open) 2016-03-13T09:11:21.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41892 #4080182 (1507 connections now open) 2016-03-13T09:11:23.908+0000 I NETWORK [conn4080168] end connection 10.0.27.237:35178 (1506 connections now open) 2016-03-13T09:11:23.908+0000 I NETWORK [conn4080167] end connection 10.0.27.237:35179 (1506 connections now open) 2016-03-13T09:11:23.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35197 #4080183 (1506 connections now open) 2016-03-13T09:11:23.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35198 #4080184 (1507 connections now open) 2016-03-13T09:11:26.012+0000 I NETWORK [conn4080169] end connection 10.0.7.113:58836 (1506 connections now open) 2016-03-13T09:11:26.013+0000 I NETWORK [conn4080170] end connection 10.0.7.113:58837 (1505 connections now open) 2016-03-13T09:11:26.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58846 #4080185 (1506 connections now open) 2016-03-13T09:11:26.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58845 #4080186 (1507 connections now open) 2016-03-13T09:11:27.118+0000 I NETWORK [conn4080175] end connection 10.0.216.90:56861 (1506 connections now open) 2016-03-13T09:11:27.118+0000 I NETWORK [conn4080176] end connection 10.0.216.90:56860 (1506 connections now open) 2016-03-13T09:11:27.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56868 #4080187 (1506 connections now open) 2016-03-13T09:11:27.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56867 #4080188 (1507 connections now open) 2016-03-13T09:11:28.107+0000 I NETWORK [conn4080177] end connection 10.0.20.53:50116 (1506 connections now open) 2016-03-13T09:11:28.107+0000 I NETWORK [conn4080178] end connection 10.0.20.53:50117 (1506 connections now open) 2016-03-13T09:11:28.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50128 #4080189 (1506 connections now open) 2016-03-13T09:11:28.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50129 #4080190 (1507 connections now open) 2016-03-13T09:11:28.114+0000 I NETWORK [conn4076760] end connection 10.0.20.53:46586 (1506 connections now open) 2016-03-13T09:11:28.114+0000 I NETWORK [conn4073432] end connection 10.0.27.237:56624 (1505 connections now open) 2016-03-13T09:11:28.114+0000 I NETWORK [conn4075113] end connection 10.0.26.69:36560 (1505 connections now open) 2016-03-13T09:11:28.114+0000 I NETWORK [conn4068009] end connection 10.0.7.113:45921 (1504 connections now open) 2016-03-13T09:11:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50131 #4080191 (1504 connections now open) 2016-03-13T09:11:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41910 #4080192 (1505 connections now open) 2016-03-13T09:11:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35204 #4080193 (1506 connections now open) 2016-03-13T09:11:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58851 #4080194 (1507 connections now open) 2016-03-13T09:11:30.452+0000 I NETWORK [conn4080180] end connection 10.0.179.209:39908 (1506 connections now open) 2016-03-13T09:11:30.452+0000 I NETWORK [conn4080179] end connection 10.0.179.209:39909 (1506 connections now open) 2016-03-13T09:11:30.453+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39918 #4080195 (1506 connections now open) 2016-03-13T09:11:30.453+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39917 #4080196 (1507 connections now open) 2016-03-13T09:11:31.085+0000 I NETWORK [conn4080181] end connection 10.0.26.69:41891 (1506 connections now open) 2016-03-13T09:11:31.085+0000 I NETWORK [conn4080182] end connection 10.0.26.69:41892 (1505 connections now open) 2016-03-13T09:11:31.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41911 #4080197 (1506 connections now open) 2016-03-13T09:11:31.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41912 #4080198 (1507 connections now open) 2016-03-13T09:11:32.838+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16801 version: 30626|1||563ba74d869758a4542b9075 based on: 30625|1||563ba74d869758a4542b9075 2016-03-13T09:11:33.921+0000 I NETWORK [conn4080184] end connection 10.0.27.237:35198 (1506 connections now open) 2016-03-13T09:11:33.921+0000 I NETWORK [conn4080183] end connection 10.0.27.237:35197 (1506 connections now open) 2016-03-13T09:11:33.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35208 #4080199 (1506 connections now open) 2016-03-13T09:11:33.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35207 #4080200 (1507 connections now open) 2016-03-13T09:11:36.034+0000 I NETWORK [conn4080186] end connection 10.0.7.113:58845 (1506 connections now open) 2016-03-13T09:11:36.034+0000 I NETWORK [conn4080185] end connection 10.0.7.113:58846 (1505 connections now open) 2016-03-13T09:11:36.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58859 #4080201 (1506 connections now open) 2016-03-13T09:11:36.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58860 #4080202 (1507 connections now open) 2016-03-13T09:11:37.131+0000 I NETWORK [conn4080188] end connection 10.0.216.90:56867 (1506 connections now open) 2016-03-13T09:11:37.131+0000 I NETWORK [conn4080187] end connection 10.0.216.90:56868 (1505 connections now open) 2016-03-13T09:11:37.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56885 #4080203 (1506 connections now open) 2016-03-13T09:11:37.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56886 #4080204 (1507 connections now open) 2016-03-13T09:11:38.120+0000 I NETWORK [conn4080189] end connection 10.0.20.53:50128 (1506 connections now open) 2016-03-13T09:11:38.120+0000 I NETWORK [conn4080190] end connection 10.0.20.53:50129 (1506 connections now open) 2016-03-13T09:11:38.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50151 #4080205 (1506 connections now open) 2016-03-13T09:11:38.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50152 #4080206 (1507 connections now open) 2016-03-13T09:11:40.466+0000 I NETWORK [conn4080196] end connection 10.0.179.209:39917 (1506 connections now open) 2016-03-13T09:11:40.466+0000 I NETWORK [conn4080195] end connection 10.0.179.209:39918 (1506 connections now open) 2016-03-13T09:11:40.467+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39929 #4080207 (1506 connections now open) 2016-03-13T09:11:40.467+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39930 #4080208 (1507 connections now open) 2016-03-13T09:11:41.099+0000 I NETWORK [conn4080198] end connection 10.0.26.69:41912 (1506 connections now open) 2016-03-13T09:11:41.099+0000 I NETWORK [conn4080197] end connection 10.0.26.69:41911 (1506 connections now open) 2016-03-13T09:11:41.100+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41919 #4080209 (1506 connections now open) 2016-03-13T09:11:41.100+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41918 #4080210 (1507 connections now open) 2016-03-13T09:11:43.934+0000 I NETWORK [conn4080200] end connection 10.0.27.237:35207 (1506 connections now open) 2016-03-13T09:11:43.934+0000 I NETWORK [conn4080199] end connection 10.0.27.237:35208 (1506 connections now open) 2016-03-13T09:11:43.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35219 #4080211 (1506 connections now open) 2016-03-13T09:11:43.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35218 #4080212 (1507 connections now open) 2016-03-13T09:11:44.266+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:11:44.138+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:11:44.266+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:11:44.334+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52eb918fa188537623d3e 2016-03-13T09:11:44.402+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ec318fa188537623d3f 2016-03-13T09:11:44.470+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ecd18fa188537623d40 2016-03-13T09:11:46.047+0000 I NETWORK [conn4080202] end connection 10.0.7.113:58860 (1506 connections now open) 2016-03-13T09:11:46.047+0000 I NETWORK [conn4080201] end connection 10.0.7.113:58859 (1506 connections now open) 2016-03-13T09:11:46.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58876 #4080213 (1506 connections now open) 2016-03-13T09:11:46.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58877 #4080214 (1507 connections now open) 2016-03-13T09:11:46.117+0000 I NETWORK [conn4071651] end connection 10.0.179.209:59493 (1506 connections now open) 2016-03-13T09:11:46.117+0000 I NETWORK [conn4067755] end connection 10.0.26.69:56898 (1505 connections now open) 2016-03-13T09:11:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41926 #4080215 (1506 connections now open) 2016-03-13T09:11:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39944 #4080216 (1507 connections now open) 2016-03-13T09:11:47.145+0000 I NETWORK [conn4080204] end connection 10.0.216.90:56886 (1506 connections now open) 2016-03-13T09:11:47.145+0000 I NETWORK [conn4080203] end connection 10.0.216.90:56885 (1506 connections now open) 2016-03-13T09:11:47.145+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56904 #4080217 (1506 connections now open) 2016-03-13T09:11:47.145+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56905 #4080218 (1507 connections now open) 2016-03-13T09:11:47.398+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16802 version: 30627|1||563ba74d869758a4542b9075 based on: 30626|1||563ba74d869758a4542b9075 2016-03-13T09:11:48.133+0000 I NETWORK [conn4080206] end connection 10.0.20.53:50152 (1506 connections now open) 2016-03-13T09:11:48.134+0000 I NETWORK [conn4080205] end connection 10.0.20.53:50151 (1505 connections now open) 2016-03-13T09:11:48.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50161 #4080219 (1506 connections now open) 2016-03-13T09:11:48.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50162 #4080220 (1507 connections now open) 2016-03-13T09:11:50.480+0000 I NETWORK [conn4080208] end connection 10.0.179.209:39930 (1506 connections now open) 2016-03-13T09:11:50.480+0000 I NETWORK [conn4080207] end connection 10.0.179.209:39929 (1506 connections now open) 2016-03-13T09:11:50.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39949 #4080221 (1506 connections now open) 2016-03-13T09:11:50.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39950 #4080222 (1507 connections now open) 2016-03-13T09:11:51.112+0000 I NETWORK [conn4080210] end connection 10.0.26.69:41918 (1506 connections now open) 2016-03-13T09:11:51.112+0000 I NETWORK [conn4080209] end connection 10.0.26.69:41919 (1506 connections now open) 2016-03-13T09:11:51.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41934 #4080223 (1506 connections now open) 2016-03-13T09:11:51.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41933 #4080224 (1507 connections now open) 2016-03-13T09:11:53.947+0000 I NETWORK [conn4080211] end connection 10.0.27.237:35219 (1506 connections now open) 2016-03-13T09:11:53.947+0000 I NETWORK [conn4080212] end connection 10.0.27.237:35218 (1506 connections now open) 2016-03-13T09:11:53.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35238 #4080225 (1506 connections now open) 2016-03-13T09:11:53.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35239 #4080226 (1507 connections now open) 2016-03-13T09:11:56.061+0000 I NETWORK [conn4080213] end connection 10.0.7.113:58876 (1506 connections now open) 2016-03-13T09:11:56.061+0000 I NETWORK [conn4080214] end connection 10.0.7.113:58877 (1506 connections now open) 2016-03-13T09:11:56.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58884 #4080227 (1507 connections now open) 2016-03-13T09:11:56.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58883 #4080228 (1507 connections now open) 2016-03-13T09:11:57.157+0000 I NETWORK [conn4080218] end connection 10.0.216.90:56905 (1506 connections now open) 2016-03-13T09:11:57.157+0000 I NETWORK [conn4080217] end connection 10.0.216.90:56904 (1505 connections now open) 2016-03-13T09:11:57.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56911 #4080229 (1506 connections now open) 2016-03-13T09:11:57.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56912 #4080230 (1507 connections now open) 2016-03-13T09:11:58.113+0000 I NETWORK [conn4079560] end connection 10.0.20.53:49463 (1506 connections now open) 2016-03-13T09:11:58.113+0000 I NETWORK [conn4069194] end connection 10.0.179.209:57047 (1506 connections now open) 2016-03-13T09:11:58.113+0000 I NETWORK [conn4068495] end connection 10.0.27.237:51368 (1504 connections now open) 2016-03-13T09:11:58.113+0000 I NETWORK [conn4070520] end connection 10.0.26.69:59866 (1504 connections now open) 2016-03-13T09:11:58.113+0000 I NETWORK [conn4067424] end connection 10.0.7.113:45309 (1502 connections now open) 2016-03-13T09:11:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39956 #4080231 (1504 connections now open) 2016-03-13T09:11:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50174 #4080232 (1504 connections now open) 2016-03-13T09:11:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35245 #4080233 (1505 connections now open) 2016-03-13T09:11:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41951 #4080234 (1506 connections now open) 2016-03-13T09:11:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58891 #4080235 (1507 connections now open) 2016-03-13T09:11:58.147+0000 I NETWORK [conn4080220] end connection 10.0.20.53:50162 (1506 connections now open) 2016-03-13T09:11:58.147+0000 I NETWORK [conn4080219] end connection 10.0.20.53:50161 (1505 connections now open) 2016-03-13T09:11:58.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50175 #4080236 (1506 connections now open) 2016-03-13T09:11:58.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50176 #4080237 (1507 connections now open) 2016-03-13T09:12:00.493+0000 I NETWORK [conn4080222] end connection 10.0.179.209:39950 (1506 connections now open) 2016-03-13T09:12:00.493+0000 I NETWORK [conn4080221] end connection 10.0.179.209:39949 (1505 connections now open) 2016-03-13T09:12:00.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39957 #4080238 (1506 connections now open) 2016-03-13T09:12:00.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39958 #4080239 (1507 connections now open) 2016-03-13T09:12:01.126+0000 I NETWORK [conn4080223] end connection 10.0.26.69:41934 (1506 connections now open) 2016-03-13T09:12:01.126+0000 I NETWORK [conn4080224] end connection 10.0.26.69:41933 (1506 connections now open) 2016-03-13T09:12:01.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41954 #4080240 (1506 connections now open) 2016-03-13T09:12:01.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41953 #4080241 (1507 connections now open) 2016-03-13T09:12:03.960+0000 I NETWORK [conn4080226] end connection 10.0.27.237:35239 (1506 connections now open) 2016-03-13T09:12:03.960+0000 I NETWORK [conn4080225] end connection 10.0.27.237:35238 (1506 connections now open) 2016-03-13T09:12:03.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35250 #4080242 (1506 connections now open) 2016-03-13T09:12:03.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35251 #4080243 (1507 connections now open) 2016-03-13T09:12:05.978+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16803 version: 30628|1||563ba74d869758a4542b9075 based on: 30627|1||563ba74d869758a4542b9075 2016-03-13T09:12:06.074+0000 I NETWORK [conn4080227] end connection 10.0.7.113:58884 (1506 connections now open) 2016-03-13T09:12:06.074+0000 I NETWORK [conn4080228] end connection 10.0.7.113:58883 (1505 connections now open) 2016-03-13T09:12:06.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58900 #4080244 (1506 connections now open) 2016-03-13T09:12:06.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58901 #4080245 (1507 connections now open) 2016-03-13T09:12:07.170+0000 I NETWORK [conn4080230] end connection 10.0.216.90:56912 (1506 connections now open) 2016-03-13T09:12:07.170+0000 I NETWORK [conn4080229] end connection 10.0.216.90:56911 (1505 connections now open) 2016-03-13T09:12:07.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56927 #4080246 (1506 connections now open) 2016-03-13T09:12:07.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56928 #4080247 (1507 connections now open) 2016-03-13T09:12:08.160+0000 I NETWORK [conn4080236] end connection 10.0.20.53:50175 (1506 connections now open) 2016-03-13T09:12:08.160+0000 I NETWORK [conn4080237] end connection 10.0.20.53:50176 (1506 connections now open) 2016-03-13T09:12:08.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50193 #4080248 (1506 connections now open) 2016-03-13T09:12:08.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50194 #4080249 (1507 connections now open) 2016-03-13T09:12:10.507+0000 I NETWORK [conn4080238] end connection 10.0.179.209:39957 (1506 connections now open) 2016-03-13T09:12:10.507+0000 I NETWORK [conn4080239] end connection 10.0.179.209:39958 (1506 connections now open) 2016-03-13T09:12:10.515+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39970 #4080250 (1506 connections now open) 2016-03-13T09:12:10.515+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39971 #4080251 (1507 connections now open) 2016-03-13T09:12:11.140+0000 I NETWORK [conn4080240] end connection 10.0.26.69:41954 (1506 connections now open) 2016-03-13T09:12:11.140+0000 I NETWORK [conn4080241] end connection 10.0.26.69:41953 (1506 connections now open) 2016-03-13T09:12:11.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41965 #4080252 (1506 connections now open) 2016-03-13T09:12:11.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41964 #4080253 (1507 connections now open) 2016-03-13T09:12:13.972+0000 I NETWORK [conn4080243] end connection 10.0.27.237:35251 (1506 connections now open) 2016-03-13T09:12:13.972+0000 I NETWORK [conn4080242] end connection 10.0.27.237:35250 (1506 connections now open) 2016-03-13T09:12:13.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35265 #4080254 (1506 connections now open) 2016-03-13T09:12:13.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35264 #4080255 (1507 connections now open) 2016-03-13T09:12:14.727+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:12:14.471+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:12:14.727+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:12:14.863+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ed918fa188537623d41 2016-03-13T09:12:15.000+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ee318fa188537623d42 2016-03-13T09:12:15.136+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52eed18fa188537623d43 2016-03-13T09:12:16.088+0000 I NETWORK [conn4080245] end connection 10.0.7.113:58901 (1506 connections now open) 2016-03-13T09:12:16.088+0000 I NETWORK [conn4080244] end connection 10.0.7.113:58900 (1506 connections now open) 2016-03-13T09:12:16.088+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58920 #4080256 (1506 connections now open) 2016-03-13T09:12:16.088+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58921 #4080257 (1507 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [conn4074212] end connection 10.0.20.53:43893 (1506 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [conn4068748] end connection 10.0.216.90:44702 (1505 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [conn4072412] end connection 10.0.27.237:55542 (1504 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [conn4068646] end connection 10.0.26.69:57835 (1504 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [conn4066390] end connection 10.0.7.113:44209 (1502 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50201 #4080258 (1503 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35270 #4080259 (1504 connections now open) 2016-03-13T09:12:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41971 #4080260 (1505 connections now open) 2016-03-13T09:12:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58924 #4080261 (1506 connections now open) 2016-03-13T09:12:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56944 #4080262 (1507 connections now open) 2016-03-13T09:12:17.183+0000 I NETWORK [conn4080246] end connection 10.0.216.90:56927 (1506 connections now open) 2016-03-13T09:12:17.183+0000 I NETWORK [conn4080247] end connection 10.0.216.90:56928 (1505 connections now open) 2016-03-13T09:12:17.183+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56946 #4080263 (1506 connections now open) 2016-03-13T09:12:17.183+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56947 #4080264 (1507 connections now open) 2016-03-13T09:12:18.172+0000 I NETWORK [conn4080248] end connection 10.0.20.53:50193 (1506 connections now open) 2016-03-13T09:12:18.172+0000 I NETWORK [conn4080249] end connection 10.0.20.53:50194 (1505 connections now open) 2016-03-13T09:12:18.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50204 #4080265 (1506 connections now open) 2016-03-13T09:12:18.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50203 #4080266 (1507 connections now open) 2016-03-13T09:12:20.528+0000 I NETWORK [conn4080250] end connection 10.0.179.209:39970 (1506 connections now open) 2016-03-13T09:12:20.528+0000 I NETWORK [conn4080251] end connection 10.0.179.209:39971 (1506 connections now open) 2016-03-13T09:12:20.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39993 #4080267 (1506 connections now open) 2016-03-13T09:12:20.529+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:39994 #4080268 (1507 connections now open) 2016-03-13T09:12:20.679+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 674ms sequenceNumber: 16804 version: 30629|1||563ba74d869758a4542b9075 based on: 30628|1||563ba74d869758a4542b9075 2016-03-13T09:12:21.153+0000 I NETWORK [conn4080252] end connection 10.0.26.69:41965 (1506 connections now open) 2016-03-13T09:12:21.153+0000 I NETWORK [conn4080253] end connection 10.0.26.69:41964 (1506 connections now open) 2016-03-13T09:12:21.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41978 #4080269 (1506 connections now open) 2016-03-13T09:12:21.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41977 #4080270 (1507 connections now open) 2016-03-13T09:12:23.994+0000 I NETWORK [conn4080255] end connection 10.0.27.237:35264 (1506 connections now open) 2016-03-13T09:12:23.995+0000 I NETWORK [conn4080254] end connection 10.0.27.237:35265 (1505 connections now open) 2016-03-13T09:12:23.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35274 #4080271 (1506 connections now open) 2016-03-13T09:12:23.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35275 #4080272 (1507 connections now open) 2016-03-13T09:12:26.101+0000 I NETWORK [conn4080257] end connection 10.0.7.113:58921 (1506 connections now open) 2016-03-13T09:12:26.101+0000 I NETWORK [conn4080256] end connection 10.0.7.113:58920 (1506 connections now open) 2016-03-13T09:12:26.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58930 #4080273 (1506 connections now open) 2016-03-13T09:12:26.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58929 #4080274 (1507 connections now open) 2016-03-13T09:12:27.195+0000 I NETWORK [conn4080263] end connection 10.0.216.90:56946 (1506 connections now open) 2016-03-13T09:12:27.196+0000 I NETWORK [conn4080264] end connection 10.0.216.90:56947 (1505 connections now open) 2016-03-13T09:12:27.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56951 #4080275 (1506 connections now open) 2016-03-13T09:12:27.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56952 #4080276 (1507 connections now open) 2016-03-13T09:12:28.113+0000 I NETWORK [conn4070944] end connection 10.0.179.209:58796 (1506 connections now open) 2016-03-13T09:12:28.113+0000 I NETWORK [conn4076591] end connection 10.0.20.53:46405 (1505 connections now open) 2016-03-13T09:12:28.113+0000 I NETWORK [conn4079644] end connection 10.0.216.90:56318 (1504 connections now open) 2016-03-13T09:12:28.113+0000 I NETWORK [conn4069931] end connection 10.0.27.237:52887 (1504 connections now open) 2016-03-13T09:12:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40001 #4080277 (1504 connections now open) 2016-03-13T09:12:28.113+0000 I NETWORK [conn4072116] end connection 10.0.7.113:50314 (1502 connections now open) 2016-03-13T09:12:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50216 #4080278 (1504 connections now open) 2016-03-13T09:12:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35290 #4080279 (1505 connections now open) 2016-03-13T09:12:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56954 #4080280 (1506 connections now open) 2016-03-13T09:12:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58937 #4080281 (1507 connections now open) 2016-03-13T09:12:28.193+0000 I NETWORK [conn4080266] end connection 10.0.20.53:50203 (1506 connections now open) 2016-03-13T09:12:28.193+0000 I NETWORK [conn4080265] end connection 10.0.20.53:50204 (1506 connections now open) 2016-03-13T09:12:28.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50218 #4080282 (1506 connections now open) 2016-03-13T09:12:28.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50217 #4080283 (1507 connections now open) 2016-03-13T09:12:30.541+0000 I NETWORK [conn4080268] end connection 10.0.179.209:39994 (1506 connections now open) 2016-03-13T09:12:30.541+0000 I NETWORK [conn4080267] end connection 10.0.179.209:39993 (1506 connections now open) 2016-03-13T09:12:30.542+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40002 #4080284 (1506 connections now open) 2016-03-13T09:12:30.542+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40003 #4080285 (1507 connections now open) 2016-03-13T09:12:31.175+0000 I NETWORK [conn4080269] end connection 10.0.26.69:41978 (1506 connections now open) 2016-03-13T09:12:31.175+0000 I NETWORK [conn4080270] end connection 10.0.26.69:41977 (1506 connections now open) 2016-03-13T09:12:31.176+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41996 #4080286 (1507 connections now open) 2016-03-13T09:12:31.176+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:41997 #4080287 (1507 connections now open) 2016-03-13T09:12:34.007+0000 I NETWORK [conn4080272] end connection 10.0.27.237:35275 (1506 connections now open) 2016-03-13T09:12:34.008+0000 I NETWORK [conn4080271] end connection 10.0.27.237:35274 (1505 connections now open) 2016-03-13T09:12:34.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35293 #4080288 (1506 connections now open) 2016-03-13T09:12:34.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35294 #4080289 (1507 connections now open) 2016-03-13T09:12:35.436+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16805 version: 30630|1||563ba74d869758a4542b9075 based on: 30629|1||563ba74d869758a4542b9075 2016-03-13T09:12:36.122+0000 I NETWORK [conn4080273] end connection 10.0.7.113:58930 (1506 connections now open) 2016-03-13T09:12:36.122+0000 I NETWORK [conn4080274] end connection 10.0.7.113:58929 (1505 connections now open) 2016-03-13T09:12:36.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58944 #4080290 (1506 connections now open) 2016-03-13T09:12:36.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58945 #4080291 (1507 connections now open) 2016-03-13T09:12:37.208+0000 I NETWORK [conn4080275] end connection 10.0.216.90:56951 (1506 connections now open) 2016-03-13T09:12:37.208+0000 I NETWORK [conn4080276] end connection 10.0.216.90:56952 (1506 connections now open) 2016-03-13T09:12:37.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56968 #4080292 (1506 connections now open) 2016-03-13T09:12:37.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56967 #4080293 (1507 connections now open) 2016-03-13T09:12:38.206+0000 I NETWORK [conn4080282] end connection 10.0.20.53:50218 (1506 connections now open) 2016-03-13T09:12:38.206+0000 I NETWORK [conn4080283] end connection 10.0.20.53:50217 (1506 connections now open) 2016-03-13T09:12:38.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50239 #4080294 (1506 connections now open) 2016-03-13T09:12:38.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50240 #4080295 (1507 connections now open) 2016-03-13T09:12:40.554+0000 I NETWORK [conn4080284] end connection 10.0.179.209:40002 (1506 connections now open) 2016-03-13T09:12:40.554+0000 I NETWORK [conn4080285] end connection 10.0.179.209:40003 (1506 connections now open) 2016-03-13T09:12:40.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40013 #4080296 (1506 connections now open) 2016-03-13T09:12:40.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40014 #4080297 (1507 connections now open) 2016-03-13T09:12:41.189+0000 I NETWORK [conn4080286] end connection 10.0.26.69:41996 (1506 connections now open) 2016-03-13T09:12:41.189+0000 I NETWORK [conn4080287] end connection 10.0.26.69:41997 (1506 connections now open) 2016-03-13T09:12:41.189+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42003 #4080298 (1506 connections now open) 2016-03-13T09:12:41.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42004 #4080299 (1507 connections now open) 2016-03-13T09:12:44.021+0000 I NETWORK [conn4080288] end connection 10.0.27.237:35293 (1506 connections now open) 2016-03-13T09:12:44.021+0000 I NETWORK [conn4080289] end connection 10.0.27.237:35294 (1506 connections now open) 2016-03-13T09:12:44.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35302 #4080300 (1506 connections now open) 2016-03-13T09:12:44.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35301 #4080301 (1507 connections now open) 2016-03-13T09:12:45.401+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:12:45.136+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:12:45.401+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:12:45.538+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52ef718fa188537623d44 2016-03-13T09:12:45.674+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f0118fa188537623d45 2016-03-13T09:12:45.811+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f0b18fa188537623d46 2016-03-13T09:12:46.118+0000 I NETWORK [conn4076312] end connection 10.0.179.209:36005 (1506 connections now open) 2016-03-13T09:12:46.118+0000 I NETWORK [conn4077430] end connection 10.0.26.69:39052 (1505 connections now open) 2016-03-13T09:12:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40028 #4080302 (1506 connections now open) 2016-03-13T09:12:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42009 #4080303 (1507 connections now open) 2016-03-13T09:12:46.135+0000 I NETWORK [conn4080291] end connection 10.0.7.113:58945 (1506 connections now open) 2016-03-13T09:12:46.135+0000 I NETWORK [conn4080290] end connection 10.0.7.113:58944 (1506 connections now open) 2016-03-13T09:12:46.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58964 #4080304 (1506 connections now open) 2016-03-13T09:12:46.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58963 #4080305 (1507 connections now open) 2016-03-13T09:12:47.221+0000 I NETWORK [conn4080292] end connection 10.0.216.90:56968 (1506 connections now open) 2016-03-13T09:12:47.221+0000 I NETWORK [conn4080293] end connection 10.0.216.90:56967 (1506 connections now open) 2016-03-13T09:12:47.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56986 #4080306 (1506 connections now open) 2016-03-13T09:12:47.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56987 #4080307 (1507 connections now open) 2016-03-13T09:12:48.219+0000 I NETWORK [conn4080295] end connection 10.0.20.53:50240 (1506 connections now open) 2016-03-13T09:12:48.219+0000 I NETWORK [conn4080294] end connection 10.0.20.53:50239 (1506 connections now open) 2016-03-13T09:12:48.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50249 #4080308 (1506 connections now open) 2016-03-13T09:12:48.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50250 #4080309 (1507 connections now open) 2016-03-13T09:12:50.567+0000 I NETWORK [conn4080297] end connection 10.0.179.209:40014 (1506 connections now open) 2016-03-13T09:12:50.567+0000 I NETWORK [conn4080296] end connection 10.0.179.209:40013 (1506 connections now open) 2016-03-13T09:12:50.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40032 #4080310 (1506 connections now open) 2016-03-13T09:12:50.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40033 #4080311 (1507 connections now open) 2016-03-13T09:12:51.203+0000 I NETWORK [conn4080299] end connection 10.0.26.69:42004 (1506 connections now open) 2016-03-13T09:12:51.203+0000 I NETWORK [conn4080298] end connection 10.0.26.69:42003 (1506 connections now open) 2016-03-13T09:12:51.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42016 #4080312 (1506 connections now open) 2016-03-13T09:12:51.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42015 #4080313 (1507 connections now open) 2016-03-13T09:12:51.517+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 675ms sequenceNumber: 16806 version: 30631|1||563ba74d869758a4542b9075 based on: 30630|1||563ba74d869758a4542b9075 2016-03-13T09:12:54.034+0000 I NETWORK [conn4080300] end connection 10.0.27.237:35302 (1506 connections now open) 2016-03-13T09:12:54.034+0000 I NETWORK [conn4080301] end connection 10.0.27.237:35301 (1506 connections now open) 2016-03-13T09:12:54.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35316 #4080314 (1506 connections now open) 2016-03-13T09:12:54.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35317 #4080315 (1507 connections now open) 2016-03-13T09:12:56.149+0000 I NETWORK [conn4080305] end connection 10.0.7.113:58963 (1506 connections now open) 2016-03-13T09:12:56.149+0000 I NETWORK [conn4080304] end connection 10.0.7.113:58964 (1506 connections now open) 2016-03-13T09:12:56.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58969 #4080316 (1506 connections now open) 2016-03-13T09:12:56.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58970 #4080317 (1507 connections now open) 2016-03-13T09:12:57.234+0000 I NETWORK [conn4080307] end connection 10.0.216.90:56987 (1506 connections now open) 2016-03-13T09:12:57.234+0000 I NETWORK [conn4080306] end connection 10.0.216.90:56986 (1506 connections now open) 2016-03-13T09:12:57.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56994 #4080318 (1506 connections now open) 2016-03-13T09:12:57.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56993 #4080319 (1507 connections now open) 2016-03-13T09:12:58.113+0000 I NETWORK [conn4071670] end connection 10.0.20.53:41220 (1506 connections now open) 2016-03-13T09:12:58.113+0000 I NETWORK [conn4072494] end connection 10.0.216.90:48723 (1506 connections now open) 2016-03-13T09:12:58.113+0000 I NETWORK [conn4072007] end connection 10.0.27.237:55107 (1504 connections now open) 2016-03-13T09:12:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:56996 #4080320 (1505 connections now open) 2016-03-13T09:12:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50262 #4080321 (1506 connections now open) 2016-03-13T09:12:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35334 #4080322 (1507 connections now open) 2016-03-13T09:12:58.232+0000 I NETWORK [conn4080308] end connection 10.0.20.53:50249 (1506 connections now open) 2016-03-13T09:12:58.232+0000 I NETWORK [conn4080309] end connection 10.0.20.53:50250 (1506 connections now open) 2016-03-13T09:12:58.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50264 #4080323 (1506 connections now open) 2016-03-13T09:12:58.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50263 #4080324 (1507 connections now open) 2016-03-13T09:13:00.581+0000 I NETWORK [conn4080311] end connection 10.0.179.209:40033 (1506 connections now open) 2016-03-13T09:13:00.581+0000 I NETWORK [conn4080310] end connection 10.0.179.209:40032 (1506 connections now open) 2016-03-13T09:13:00.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40042 #4080325 (1506 connections now open) 2016-03-13T09:13:00.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40043 #4080326 (1507 connections now open) 2016-03-13T09:13:01.216+0000 I NETWORK [conn4080313] end connection 10.0.26.69:42015 (1506 connections now open) 2016-03-13T09:13:01.216+0000 I NETWORK [conn4080312] end connection 10.0.26.69:42016 (1506 connections now open) 2016-03-13T09:13:01.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42035 #4080327 (1507 connections now open) 2016-03-13T09:13:01.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42036 #4080328 (1507 connections now open) 2016-03-13T09:13:04.047+0000 I NETWORK [conn4080315] end connection 10.0.27.237:35317 (1506 connections now open) 2016-03-13T09:13:04.047+0000 I NETWORK [conn4080314] end connection 10.0.27.237:35316 (1506 connections now open) 2016-03-13T09:13:04.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35340 #4080329 (1506 connections now open) 2016-03-13T09:13:04.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35341 #4080330 (1507 connections now open) 2016-03-13T09:13:06.163+0000 I NETWORK [conn4080317] end connection 10.0.7.113:58970 (1506 connections now open) 2016-03-13T09:13:06.163+0000 I NETWORK [conn4080316] end connection 10.0.7.113:58969 (1506 connections now open) 2016-03-13T09:13:06.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58986 #4080331 (1506 connections now open) 2016-03-13T09:13:06.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:58987 #4080332 (1507 connections now open) 2016-03-13T09:13:06.963+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16807 version: 30632|1||563ba74d869758a4542b9075 based on: 30631|1||563ba74d869758a4542b9075 2016-03-13T09:13:07.247+0000 I NETWORK [conn4080319] end connection 10.0.216.90:56993 (1506 connections now open) 2016-03-13T09:13:07.247+0000 I NETWORK [conn4080318] end connection 10.0.216.90:56994 (1506 connections now open) 2016-03-13T09:13:07.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57007 #4080333 (1506 connections now open) 2016-03-13T09:13:07.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57008 #4080334 (1507 connections now open) 2016-03-13T09:13:08.245+0000 I NETWORK [conn4080324] end connection 10.0.20.53:50263 (1506 connections now open) 2016-03-13T09:13:08.245+0000 I NETWORK [conn4080323] end connection 10.0.20.53:50264 (1506 connections now open) 2016-03-13T09:13:08.246+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50282 #4080335 (1506 connections now open) 2016-03-13T09:13:08.246+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50283 #4080336 (1507 connections now open) 2016-03-13T09:13:10.594+0000 I NETWORK [conn4080325] end connection 10.0.179.209:40042 (1506 connections now open) 2016-03-13T09:13:10.594+0000 I NETWORK [conn4080326] end connection 10.0.179.209:40043 (1506 connections now open) 2016-03-13T09:13:10.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40055 #4080337 (1506 connections now open) 2016-03-13T09:13:10.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40056 #4080338 (1507 connections now open) 2016-03-13T09:13:11.229+0000 I NETWORK [conn4080328] end connection 10.0.26.69:42036 (1506 connections now open) 2016-03-13T09:13:11.229+0000 I NETWORK [conn4080327] end connection 10.0.26.69:42035 (1505 connections now open) 2016-03-13T09:13:11.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42046 #4080339 (1506 connections now open) 2016-03-13T09:13:11.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42047 #4080340 (1507 connections now open) 2016-03-13T09:13:14.061+0000 I NETWORK [conn4080329] end connection 10.0.27.237:35340 (1506 connections now open) 2016-03-13T09:13:14.061+0000 I NETWORK [conn4080330] end connection 10.0.27.237:35341 (1506 connections now open) 2016-03-13T09:13:14.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35350 #4080341 (1506 connections now open) 2016-03-13T09:13:14.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35349 #4080342 (1507 connections now open) 2016-03-13T09:13:16.019+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:13:15.811+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:13:16.019+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:13:16.118+0000 I NETWORK [conn4075012] end connection 10.0.179.209:34687 (1506 connections now open) 2016-03-13T09:13:16.118+0000 I NETWORK [conn4072132] end connection 10.0.216.90:48337 (1506 connections now open) 2016-03-13T09:13:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40071 #4080343 (1506 connections now open) 2016-03-13T09:13:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57026 #4080344 (1507 connections now open) 2016-03-13T09:13:16.155+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f1618fa188537623d47 2016-03-13T09:13:16.177+0000 I NETWORK [conn4080332] end connection 10.0.7.113:58987 (1506 connections now open) 2016-03-13T09:13:16.177+0000 I NETWORK [conn4080331] end connection 10.0.7.113:58986 (1506 connections now open) 2016-03-13T09:13:16.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59009 #4080345 (1506 connections now open) 2016-03-13T09:13:16.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59010 #4080346 (1507 connections now open) 2016-03-13T09:13:16.258+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f2118fa188537623d48 2016-03-13T09:13:16.326+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f2b18fa188537623d49 2016-03-13T09:13:17.260+0000 I NETWORK [conn4080333] end connection 10.0.216.90:57007 (1506 connections now open) 2016-03-13T09:13:17.260+0000 I NETWORK [conn4080334] end connection 10.0.216.90:57008 (1506 connections now open) 2016-03-13T09:13:17.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57027 #4080347 (1506 connections now open) 2016-03-13T09:13:17.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57028 #4080348 (1507 connections now open) 2016-03-13T09:13:18.258+0000 I NETWORK [conn4080335] end connection 10.0.20.53:50282 (1506 connections now open) 2016-03-13T09:13:18.258+0000 I NETWORK [conn4080336] end connection 10.0.20.53:50283 (1506 connections now open) 2016-03-13T09:13:18.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50292 #4080349 (1506 connections now open) 2016-03-13T09:13:18.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50291 #4080350 (1507 connections now open) 2016-03-13T09:13:20.616+0000 I NETWORK [conn4080338] end connection 10.0.179.209:40056 (1506 connections now open) 2016-03-13T09:13:20.616+0000 I NETWORK [conn4080337] end connection 10.0.179.209:40055 (1506 connections now open) 2016-03-13T09:13:20.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40078 #4080351 (1506 connections now open) 2016-03-13T09:13:20.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40079 #4080352 (1507 connections now open) 2016-03-13T09:13:21.243+0000 I NETWORK [conn4080339] end connection 10.0.26.69:42046 (1506 connections now open) 2016-03-13T09:13:21.243+0000 I NETWORK [conn4080340] end connection 10.0.26.69:42047 (1506 connections now open) 2016-03-13T09:13:21.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42060 #4080353 (1506 connections now open) 2016-03-13T09:13:21.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42061 #4080354 (1507 connections now open) 2016-03-13T09:13:22.530+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16808 version: 30633|1||563ba74d869758a4542b9075 based on: 30632|1||563ba74d869758a4542b9075 2016-03-13T09:13:24.082+0000 I NETWORK [conn4080341] end connection 10.0.27.237:35350 (1506 connections now open) 2016-03-13T09:13:24.082+0000 I NETWORK [conn4080342] end connection 10.0.27.237:35349 (1506 connections now open) 2016-03-13T09:13:24.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35365 #4080355 (1506 connections now open) 2016-03-13T09:13:24.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35366 #4080356 (1507 connections now open) 2016-03-13T09:13:26.190+0000 I NETWORK [conn4080346] end connection 10.0.7.113:59010 (1506 connections now open) 2016-03-13T09:13:26.190+0000 I NETWORK [conn4080345] end connection 10.0.7.113:59009 (1506 connections now open) 2016-03-13T09:13:26.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59015 #4080357 (1506 connections now open) 2016-03-13T09:13:26.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59016 #4080358 (1507 connections now open) 2016-03-13T09:13:27.273+0000 I NETWORK [conn4080348] end connection 10.0.216.90:57028 (1506 connections now open) 2016-03-13T09:13:27.273+0000 I NETWORK [conn4080347] end connection 10.0.216.90:57027 (1506 connections now open) 2016-03-13T09:13:27.273+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57034 #4080359 (1506 connections now open) 2016-03-13T09:13:27.273+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57035 #4080360 (1507 connections now open) 2016-03-13T09:13:28.113+0000 I NETWORK [conn4071498] end connection 10.0.179.209:59352 (1506 connections now open) 2016-03-13T09:13:28.113+0000 I NETWORK [conn4072536] end connection 10.0.216.90:48768 (1505 connections now open) 2016-03-13T09:13:28.113+0000 I NETWORK [conn4073223] end connection 10.0.26.69:34516 (1504 connections now open) 2016-03-13T09:13:28.114+0000 I NETWORK [conn4077830] end connection 10.0.27.237:32770 (1503 connections now open) 2016-03-13T09:13:28.114+0000 I NETWORK [conn4068563] end connection 10.0.7.113:46497 (1502 connections now open) 2016-03-13T09:13:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40085 #4080361 (1503 connections now open) 2016-03-13T09:13:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57037 #4080362 (1504 connections now open) 2016-03-13T09:13:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42078 #4080363 (1505 connections now open) 2016-03-13T09:13:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35383 #4080364 (1506 connections now open) 2016-03-13T09:13:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59022 #4080365 (1507 connections now open) 2016-03-13T09:13:28.281+0000 I NETWORK [conn4080349] end connection 10.0.20.53:50292 (1506 connections now open) 2016-03-13T09:13:28.281+0000 I NETWORK [conn4080350] end connection 10.0.20.53:50291 (1506 connections now open) 2016-03-13T09:13:28.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50306 #4080366 (1506 connections now open) 2016-03-13T09:13:28.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50307 #4080367 (1507 connections now open) 2016-03-13T09:13:30.629+0000 I NETWORK [conn4080351] end connection 10.0.179.209:40078 (1506 connections now open) 2016-03-13T09:13:30.629+0000 I NETWORK [conn4080352] end connection 10.0.179.209:40079 (1506 connections now open) 2016-03-13T09:13:30.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40088 #4080368 (1506 connections now open) 2016-03-13T09:13:30.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40087 #4080369 (1507 connections now open) 2016-03-13T09:13:31.257+0000 I NETWORK [conn4080354] end connection 10.0.26.69:42061 (1506 connections now open) 2016-03-13T09:13:31.257+0000 I NETWORK [conn4080353] end connection 10.0.26.69:42060 (1506 connections now open) 2016-03-13T09:13:31.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42080 #4080370 (1507 connections now open) 2016-03-13T09:13:31.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42081 #4080371 (1507 connections now open) 2016-03-13T09:13:34.094+0000 I NETWORK [conn4080356] end connection 10.0.27.237:35366 (1506 connections now open) 2016-03-13T09:13:34.095+0000 I NETWORK [conn4080355] end connection 10.0.27.237:35365 (1505 connections now open) 2016-03-13T09:13:34.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35386 #4080372 (1506 connections now open) 2016-03-13T09:13:34.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35387 #4080373 (1507 connections now open) 2016-03-13T09:13:36.212+0000 I NETWORK [conn4080358] end connection 10.0.7.113:59016 (1506 connections now open) 2016-03-13T09:13:36.212+0000 I NETWORK [conn4080357] end connection 10.0.7.113:59015 (1506 connections now open) 2016-03-13T09:13:36.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59030 #4080374 (1506 connections now open) 2016-03-13T09:13:36.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59031 #4080375 (1507 connections now open) 2016-03-13T09:13:37.286+0000 I NETWORK [conn4080360] end connection 10.0.216.90:57035 (1506 connections now open) 2016-03-13T09:13:37.286+0000 I NETWORK [conn4080359] end connection 10.0.216.90:57034 (1505 connections now open) 2016-03-13T09:13:37.286+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57050 #4080376 (1506 connections now open) 2016-03-13T09:13:37.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57051 #4080377 (1507 connections now open) 2016-03-13T09:13:37.290+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 670ms sequenceNumber: 16809 version: 30634|1||563ba74d869758a4542b9075 based on: 30633|1||563ba74d869758a4542b9075 2016-03-13T09:13:38.294+0000 I NETWORK [conn4080367] end connection 10.0.20.53:50307 (1506 connections now open) 2016-03-13T09:13:38.294+0000 I NETWORK [conn4080366] end connection 10.0.20.53:50306 (1506 connections now open) 2016-03-13T09:13:38.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50328 #4080378 (1506 connections now open) 2016-03-13T09:13:38.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50329 #4080379 (1507 connections now open) 2016-03-13T09:13:40.642+0000 I NETWORK [conn4080368] end connection 10.0.179.209:40088 (1506 connections now open) 2016-03-13T09:13:40.642+0000 I NETWORK [conn4080369] end connection 10.0.179.209:40087 (1506 connections now open) 2016-03-13T09:13:40.643+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40099 #4080380 (1506 connections now open) 2016-03-13T09:13:40.643+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40100 #4080381 (1507 connections now open) 2016-03-13T09:13:41.271+0000 I NETWORK [conn4080371] end connection 10.0.26.69:42081 (1506 connections now open) 2016-03-13T09:13:41.271+0000 I NETWORK [conn4080370] end connection 10.0.26.69:42080 (1506 connections now open) 2016-03-13T09:13:41.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42088 #4080382 (1506 connections now open) 2016-03-13T09:13:41.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42089 #4080383 (1507 connections now open) 2016-03-13T09:13:44.107+0000 I NETWORK [conn4080373] end connection 10.0.27.237:35387 (1506 connections now open) 2016-03-13T09:13:44.107+0000 I NETWORK [conn4080372] end connection 10.0.27.237:35386 (1505 connections now open) 2016-03-13T09:13:44.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35393 #4080384 (1506 connections now open) 2016-03-13T09:13:44.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35394 #4080385 (1507 connections now open) 2016-03-13T09:13:46.117+0000 I NETWORK [conn4078191] end connection 10.0.179.209:37880 (1506 connections now open) 2016-03-13T09:13:46.117+0000 I NETWORK [conn4072452] end connection 10.0.20.53:42018 (1506 connections now open) 2016-03-13T09:13:46.117+0000 I NETWORK [conn4077814] end connection 10.0.27.237:60995 (1506 connections now open) 2016-03-13T09:13:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40115 #4080386 (1505 connections now open) 2016-03-13T09:13:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50337 #4080387 (1506 connections now open) 2016-03-13T09:13:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35404 #4080388 (1507 connections now open) 2016-03-13T09:13:46.225+0000 I NETWORK [conn4080375] end connection 10.0.7.113:59031 (1506 connections now open) 2016-03-13T09:13:46.225+0000 I NETWORK [conn4080374] end connection 10.0.7.113:59030 (1506 connections now open) 2016-03-13T09:13:46.226+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59048 #4080389 (1506 connections now open) 2016-03-13T09:13:46.226+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59049 #4080390 (1507 connections now open) 2016-03-13T09:13:46.588+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:13:46.326+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:13:46.588+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:13:46.724+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f3518fa188537623d4a 2016-03-13T09:13:46.895+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f3f18fa188537623d4b 2016-03-13T09:13:47.032+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f4918fa188537623d4c 2016-03-13T09:13:47.299+0000 I NETWORK [conn4080376] end connection 10.0.216.90:57050 (1506 connections now open) 2016-03-13T09:13:47.299+0000 I NETWORK [conn4080377] end connection 10.0.216.90:57051 (1506 connections now open) 2016-03-13T09:13:47.299+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57068 #4080391 (1506 connections now open) 2016-03-13T09:13:47.299+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57067 #4080392 (1507 connections now open) 2016-03-13T09:13:48.307+0000 I NETWORK [conn4080379] end connection 10.0.20.53:50329 (1506 connections now open) 2016-03-13T09:13:48.307+0000 I NETWORK [conn4080378] end connection 10.0.20.53:50328 (1506 connections now open) 2016-03-13T09:13:48.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50339 #4080393 (1506 connections now open) 2016-03-13T09:13:48.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50340 #4080394 (1507 connections now open) 2016-03-13T09:13:50.655+0000 I NETWORK [conn4080380] end connection 10.0.179.209:40099 (1506 connections now open) 2016-03-13T09:13:50.655+0000 I NETWORK [conn4080381] end connection 10.0.179.209:40100 (1506 connections now open) 2016-03-13T09:13:50.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40119 #4080395 (1506 connections now open) 2016-03-13T09:13:50.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40120 #4080396 (1507 connections now open) 2016-03-13T09:13:51.285+0000 I NETWORK [conn4080383] end connection 10.0.26.69:42089 (1506 connections now open) 2016-03-13T09:13:51.285+0000 I NETWORK [conn4080382] end connection 10.0.26.69:42088 (1506 connections now open) 2016-03-13T09:13:51.286+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42102 #4080397 (1506 connections now open) 2016-03-13T09:13:51.286+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42103 #4080398 (1507 connections now open) 2016-03-13T09:13:52.929+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16810 version: 30635|1||563ba74d869758a4542b9075 based on: 30634|1||563ba74d869758a4542b9075 2016-03-13T09:13:54.120+0000 I NETWORK [conn4080385] end connection 10.0.27.237:35394 (1506 connections now open) 2016-03-13T09:13:54.120+0000 I NETWORK [conn4080384] end connection 10.0.27.237:35393 (1506 connections now open) 2016-03-13T09:13:54.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35408 #4080399 (1506 connections now open) 2016-03-13T09:13:54.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35409 #4080400 (1507 connections now open) 2016-03-13T09:13:56.238+0000 I NETWORK [conn4080390] end connection 10.0.7.113:59049 (1506 connections now open) 2016-03-13T09:13:56.238+0000 I NETWORK [conn4080389] end connection 10.0.7.113:59048 (1506 connections now open) 2016-03-13T09:13:56.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59054 #4080401 (1507 connections now open) 2016-03-13T09:13:56.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59055 #4080402 (1507 connections now open) 2016-03-13T09:13:57.312+0000 I NETWORK [conn4080391] end connection 10.0.216.90:57068 (1506 connections now open) 2016-03-13T09:13:57.312+0000 I NETWORK [conn4080392] end connection 10.0.216.90:57067 (1505 connections now open) 2016-03-13T09:13:57.312+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57076 #4080403 (1506 connections now open) 2016-03-13T09:13:57.312+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57075 #4080404 (1507 connections now open) 2016-03-13T09:13:58.113+0000 I NETWORK [conn4068791] end connection 10.0.27.237:51680 (1506 connections now open) 2016-03-13T09:13:58.113+0000 I NETWORK [conn4069495] end connection 10.0.7.113:47509 (1505 connections now open) 2016-03-13T09:13:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35424 #4080405 (1506 connections now open) 2016-03-13T09:13:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59060 #4080406 (1507 connections now open) 2016-03-13T09:13:58.321+0000 I NETWORK [conn4080394] end connection 10.0.20.53:50340 (1506 connections now open) 2016-03-13T09:13:58.321+0000 I NETWORK [conn4080393] end connection 10.0.20.53:50339 (1506 connections now open) 2016-03-13T09:13:58.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50354 #4080407 (1506 connections now open) 2016-03-13T09:13:58.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50353 #4080408 (1507 connections now open) 2016-03-13T09:14:00.667+0000 I NETWORK [conn4080396] end connection 10.0.179.209:40120 (1506 connections now open) 2016-03-13T09:14:00.667+0000 I NETWORK [conn4080395] end connection 10.0.179.209:40119 (1506 connections now open) 2016-03-13T09:14:00.668+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40129 #4080409 (1506 connections now open) 2016-03-13T09:14:00.668+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40128 #4080410 (1507 connections now open) 2016-03-13T09:14:01.299+0000 I NETWORK [conn4080397] end connection 10.0.26.69:42102 (1506 connections now open) 2016-03-13T09:14:01.299+0000 I NETWORK [conn4080398] end connection 10.0.26.69:42103 (1506 connections now open) 2016-03-13T09:14:01.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42121 #4080411 (1506 connections now open) 2016-03-13T09:14:01.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42122 #4080412 (1507 connections now open) 2016-03-13T09:14:04.133+0000 I NETWORK [conn4080399] end connection 10.0.27.237:35408 (1506 connections now open) 2016-03-13T09:14:04.133+0000 I NETWORK [conn4080400] end connection 10.0.27.237:35409 (1506 connections now open) 2016-03-13T09:14:04.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35429 #4080413 (1506 connections now open) 2016-03-13T09:14:04.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35430 #4080414 (1507 connections now open) 2016-03-13T09:14:06.252+0000 I NETWORK [conn4080402] end connection 10.0.7.113:59055 (1506 connections now open) 2016-03-13T09:14:06.252+0000 I NETWORK [conn4080401] end connection 10.0.7.113:59054 (1506 connections now open) 2016-03-13T09:14:06.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59070 #4080415 (1506 connections now open) 2016-03-13T09:14:06.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59069 #4080416 (1507 connections now open) 2016-03-13T09:14:07.325+0000 I NETWORK [conn4080404] end connection 10.0.216.90:57075 (1506 connections now open) 2016-03-13T09:14:07.325+0000 I NETWORK [conn4080403] end connection 10.0.216.90:57076 (1506 connections now open) 2016-03-13T09:14:07.325+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57092 #4080417 (1506 connections now open) 2016-03-13T09:14:07.325+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57091 #4080418 (1507 connections now open) 2016-03-13T09:14:08.334+0000 I NETWORK [conn4080408] end connection 10.0.20.53:50353 (1506 connections now open) 2016-03-13T09:14:08.334+0000 I NETWORK [conn4080407] end connection 10.0.20.53:50354 (1506 connections now open) 2016-03-13T09:14:08.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50374 #4080419 (1506 connections now open) 2016-03-13T09:14:08.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50375 #4080420 (1507 connections now open) 2016-03-13T09:14:10.454+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 708ms sequenceNumber: 16811 version: 30636|1||563ba74d869758a4542b9075 based on: 30635|1||563ba74d869758a4542b9075 2016-03-13T09:14:10.680+0000 I NETWORK [conn4080410] end connection 10.0.179.209:40128 (1506 connections now open) 2016-03-13T09:14:10.680+0000 I NETWORK [conn4080409] end connection 10.0.179.209:40129 (1506 connections now open) 2016-03-13T09:14:10.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40142 #4080421 (1506 connections now open) 2016-03-13T09:14:10.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40143 #4080422 (1507 connections now open) 2016-03-13T09:14:11.313+0000 I NETWORK [conn4080411] end connection 10.0.26.69:42121 (1506 connections now open) 2016-03-13T09:14:11.313+0000 I NETWORK [conn4080412] end connection 10.0.26.69:42122 (1506 connections now open) 2016-03-13T09:14:11.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42134 #4080423 (1506 connections now open) 2016-03-13T09:14:11.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42135 #4080424 (1507 connections now open) 2016-03-13T09:14:14.146+0000 I NETWORK [conn4080414] end connection 10.0.27.237:35430 (1506 connections now open) 2016-03-13T09:14:14.146+0000 I NETWORK [conn4080413] end connection 10.0.27.237:35429 (1505 connections now open) 2016-03-13T09:14:14.155+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35439 #4080425 (1506 connections now open) 2016-03-13T09:14:14.155+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35438 #4080426 (1507 connections now open) 2016-03-13T09:14:16.118+0000 I NETWORK [conn4072282] end connection 10.0.20.53:41854 (1506 connections now open) 2016-03-13T09:14:16.118+0000 I NETWORK [conn4074670] end connection 10.0.27.237:57868 (1506 connections now open) 2016-03-13T09:14:16.118+0000 I NETWORK [conn4072174] end connection 10.0.216.90:48383 (1506 connections now open) 2016-03-13T09:14:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50383 #4080427 (1505 connections now open) 2016-03-13T09:14:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35449 #4080428 (1506 connections now open) 2016-03-13T09:14:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57110 #4080429 (1507 connections now open) 2016-03-13T09:14:16.265+0000 I NETWORK [conn4080416] end connection 10.0.7.113:59069 (1506 connections now open) 2016-03-13T09:14:16.265+0000 I NETWORK [conn4080415] end connection 10.0.7.113:59070 (1506 connections now open) 2016-03-13T09:14:16.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59093 #4080430 (1506 connections now open) 2016-03-13T09:14:16.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59092 #4080431 (1507 connections now open) 2016-03-13T09:14:17.158+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:14:17.032+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:14:17.159+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:14:17.227+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f5318fa188537623d4d 2016-03-13T09:14:17.295+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f5e18fa188537623d4e 2016-03-13T09:14:17.338+0000 I NETWORK [conn4080418] end connection 10.0.216.90:57091 (1506 connections now open) 2016-03-13T09:14:17.338+0000 I NETWORK [conn4080417] end connection 10.0.216.90:57092 (1506 connections now open) 2016-03-13T09:14:17.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57112 #4080432 (1506 connections now open) 2016-03-13T09:14:17.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57111 #4080433 (1507 connections now open) 2016-03-13T09:14:17.363+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f6818fa188537623d4f 2016-03-13T09:14:18.347+0000 I NETWORK [conn4080419] end connection 10.0.20.53:50374 (1506 connections now open) 2016-03-13T09:14:18.347+0000 I NETWORK [conn4080420] end connection 10.0.20.53:50375 (1506 connections now open) 2016-03-13T09:14:18.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50386 #4080434 (1506 connections now open) 2016-03-13T09:14:18.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50385 #4080435 (1507 connections now open) 2016-03-13T09:14:20.702+0000 I NETWORK [conn4080422] end connection 10.0.179.209:40143 (1506 connections now open) 2016-03-13T09:14:20.702+0000 I NETWORK [conn4080421] end connection 10.0.179.209:40142 (1506 connections now open) 2016-03-13T09:14:20.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40165 #4080436 (1506 connections now open) 2016-03-13T09:14:20.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40166 #4080437 (1507 connections now open) 2016-03-13T09:14:21.327+0000 I NETWORK [conn4080424] end connection 10.0.26.69:42135 (1506 connections now open) 2016-03-13T09:14:21.327+0000 I NETWORK [conn4080423] end connection 10.0.26.69:42134 (1506 connections now open) 2016-03-13T09:14:21.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42148 #4080438 (1506 connections now open) 2016-03-13T09:14:21.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42149 #4080439 (1507 connections now open) 2016-03-13T09:14:24.167+0000 I NETWORK [conn4080425] end connection 10.0.27.237:35439 (1506 connections now open) 2016-03-13T09:14:24.167+0000 I NETWORK [conn4080426] end connection 10.0.27.237:35438 (1506 connections now open) 2016-03-13T09:14:24.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35453 #4080440 (1506 connections now open) 2016-03-13T09:14:24.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35452 #4080441 (1507 connections now open) 2016-03-13T09:14:25.554+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16812 version: 30637|1||563ba74d869758a4542b9075 based on: 30636|1||563ba74d869758a4542b9075 2016-03-13T09:14:26.278+0000 I NETWORK [conn4080431] end connection 10.0.7.113:59092 (1506 connections now open) 2016-03-13T09:14:26.278+0000 I NETWORK [conn4080430] end connection 10.0.7.113:59093 (1506 connections now open) 2016-03-13T09:14:26.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59101 #4080442 (1506 connections now open) 2016-03-13T09:14:26.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59100 #4080443 (1507 connections now open) 2016-03-13T09:14:26.478+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52f7218fa188537623d50 2016-03-13T09:14:27.351+0000 I NETWORK [conn4080433] end connection 10.0.216.90:57111 (1506 connections now open) 2016-03-13T09:14:27.351+0000 I NETWORK [conn4080432] end connection 10.0.216.90:57112 (1506 connections now open) 2016-03-13T09:14:27.352+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57119 #4080444 (1506 connections now open) 2016-03-13T09:14:27.352+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57120 #4080445 (1507 connections now open) 2016-03-13T09:14:28.113+0000 I NETWORK [conn4076759] end connection 10.0.179.209:36449 (1506 connections now open) 2016-03-13T09:14:28.113+0000 I NETWORK [conn4071918] end connection 10.0.20.53:41483 (1506 connections now open) 2016-03-13T09:14:28.113+0000 I NETWORK [conn4069848] end connection 10.0.26.69:59126 (1504 connections now open) 2016-03-13T09:14:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40172 #4080446 (1505 connections now open) 2016-03-13T09:14:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50398 #4080447 (1506 connections now open) 2016-03-13T09:14:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42165 #4080448 (1507 connections now open) 2016-03-13T09:14:28.368+0000 I NETWORK [conn4080434] end connection 10.0.20.53:50386 (1506 connections now open) 2016-03-13T09:14:28.368+0000 I NETWORK [conn4080435] end connection 10.0.20.53:50385 (1506 connections now open) 2016-03-13T09:14:28.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50400 #4080449 (1506 connections now open) 2016-03-13T09:14:28.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50399 #4080450 (1507 connections now open) 2016-03-13T09:14:29.177+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563446f855418c0ec3f67ffc')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563446f855418c0ec3f67ffc') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563448a3a2a4330ea420ec84') }, shard: "shmydomain1", lastmod: Timestamp 30637000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:14:29.331+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30637|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563446f855418c0ec3f67ffc') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563448a3a2a4330ea420ec84') }) shmydomain1 -> shmydomain3 2016-03-13T09:14:30.715+0000 I NETWORK [conn4080436] end connection 10.0.179.209:40165 (1506 connections now open) 2016-03-13T09:14:30.715+0000 I NETWORK [conn4080437] end connection 10.0.179.209:40166 (1506 connections now open) 2016-03-13T09:14:30.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40174 #4080451 (1506 connections now open) 2016-03-13T09:14:30.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40175 #4080452 (1507 connections now open) 2016-03-13T09:14:31.348+0000 I NETWORK [conn4080439] end connection 10.0.26.69:42149 (1506 connections now open) 2016-03-13T09:14:31.348+0000 I NETWORK [conn4080438] end connection 10.0.26.69:42148 (1506 connections now open) 2016-03-13T09:14:31.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42167 #4080453 (1506 connections now open) 2016-03-13T09:14:31.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42168 #4080454 (1507 connections now open) 2016-03-13T09:14:34.180+0000 I NETWORK [conn4080440] end connection 10.0.27.237:35453 (1506 connections now open) 2016-03-13T09:14:34.180+0000 I NETWORK [conn4080441] end connection 10.0.27.237:35452 (1506 connections now open) 2016-03-13T09:14:34.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35473 #4080455 (1506 connections now open) 2016-03-13T09:14:34.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35474 #4080456 (1507 connections now open) 2016-03-13T09:14:36.299+0000 I NETWORK [conn4080442] end connection 10.0.7.113:59101 (1506 connections now open) 2016-03-13T09:14:36.299+0000 I NETWORK [conn4080443] end connection 10.0.7.113:59100 (1506 connections now open) 2016-03-13T09:14:36.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59114 #4080457 (1506 connections now open) 2016-03-13T09:14:36.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59115 #4080458 (1507 connections now open) 2016-03-13T09:14:37.364+0000 I NETWORK [conn4080444] end connection 10.0.216.90:57119 (1506 connections now open) 2016-03-13T09:14:37.364+0000 I NETWORK [conn4080445] end connection 10.0.216.90:57120 (1505 connections now open) 2016-03-13T09:14:37.364+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57136 #4080459 (1506 connections now open) 2016-03-13T09:14:37.365+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57135 #4080460 (1507 connections now open) 2016-03-13T09:14:38.169+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16813 version: 30638|1||563ba74d869758a4542b9075 based on: 30637|1||563ba74d869758a4542b9075 2016-03-13T09:14:38.355+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:14:38.355+0000-56e52f7e18fa188537623d51", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860478355), what: "balancer.round", ns: "", details: { executionTimeMillis: 12216, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:14:38.381+0000 I NETWORK [conn4080450] end connection 10.0.20.53:50399 (1506 connections now open) 2016-03-13T09:14:38.381+0000 I NETWORK [conn4080449] end connection 10.0.20.53:50400 (1506 connections now open) 2016-03-13T09:14:38.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50422 #4080461 (1506 connections now open) 2016-03-13T09:14:38.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50421 #4080462 (1507 connections now open) 2016-03-13T09:14:38.521+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:14:39.861+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52f7f18fa188537623d52 2016-03-13T09:14:40.728+0000 I NETWORK [conn4080452] end connection 10.0.179.209:40175 (1506 connections now open) 2016-03-13T09:14:40.728+0000 I NETWORK [conn4080451] end connection 10.0.179.209:40174 (1506 connections now open) 2016-03-13T09:14:40.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40186 #4080463 (1506 connections now open) 2016-03-13T09:14:40.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40185 #4080464 (1507 connections now open) 2016-03-13T09:14:41.361+0000 I NETWORK [conn4080453] end connection 10.0.26.69:42167 (1506 connections now open) 2016-03-13T09:14:41.361+0000 I NETWORK [conn4080454] end connection 10.0.26.69:42168 (1506 connections now open) 2016-03-13T09:14:41.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42174 #4080465 (1506 connections now open) 2016-03-13T09:14:41.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42175 #4080466 (1507 connections now open) 2016-03-13T09:14:42.433+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563448a3a2a4330ea420ec84')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563448a3a2a4330ea420ec84') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56344a6a584ad10f2135352d') }, shard: "shmydomain1", lastmod: Timestamp 30638000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:14:42.589+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30638|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563448a3a2a4330ea420ec84') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56344a6a584ad10f2135352d') }) shmydomain1 -> shmydomain2 2016-03-13T09:14:44.194+0000 I NETWORK [conn4080456] end connection 10.0.27.237:35474 (1506 connections now open) 2016-03-13T09:14:44.194+0000 I NETWORK [conn4080455] end connection 10.0.27.237:35473 (1506 connections now open) 2016-03-13T09:14:44.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35481 #4080467 (1506 connections now open) 2016-03-13T09:14:44.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35482 #4080468 (1507 connections now open) 2016-03-13T09:14:46.118+0000 I NETWORK [conn4069953] end connection 10.0.20.53:39439 (1506 connections now open) 2016-03-13T09:14:46.118+0000 I NETWORK [conn4079668] end connection 10.0.7.113:58330 (1505 connections now open) 2016-03-13T09:14:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50430 #4080469 (1506 connections now open) 2016-03-13T09:14:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59132 #4080470 (1507 connections now open) 2016-03-13T09:14:46.313+0000 I NETWORK [conn4080458] end connection 10.0.7.113:59115 (1506 connections now open) 2016-03-13T09:14:46.313+0000 I NETWORK [conn4080457] end connection 10.0.7.113:59114 (1506 connections now open) 2016-03-13T09:14:46.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59133 #4080471 (1506 connections now open) 2016-03-13T09:14:46.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59134 #4080472 (1507 connections now open) 2016-03-13T09:14:47.377+0000 I NETWORK [conn4080460] end connection 10.0.216.90:57135 (1506 connections now open) 2016-03-13T09:14:47.377+0000 I NETWORK [conn4080459] end connection 10.0.216.90:57136 (1506 connections now open) 2016-03-13T09:14:47.378+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57154 #4080473 (1506 connections now open) 2016-03-13T09:14:47.378+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57155 #4080474 (1507 connections now open) 2016-03-13T09:14:47.625+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:14:47.363+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:14:48.395+0000 I NETWORK [conn4080462] end connection 10.0.20.53:50421 (1506 connections now open) 2016-03-13T09:14:48.395+0000 I NETWORK [conn4080461] end connection 10.0.20.53:50422 (1506 connections now open) 2016-03-13T09:14:48.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50432 #4080475 (1506 connections now open) 2016-03-13T09:14:48.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50433 #4080476 (1507 connections now open) 2016-03-13T09:14:50.741+0000 I NETWORK [conn4080464] end connection 10.0.179.209:40185 (1506 connections now open) 2016-03-13T09:14:50.741+0000 I NETWORK [conn4080463] end connection 10.0.179.209:40186 (1506 connections now open) 2016-03-13T09:14:50.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40206 #4080477 (1506 connections now open) 2016-03-13T09:14:50.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40205 #4080478 (1507 connections now open) 2016-03-13T09:14:51.374+0000 I NETWORK [conn4080465] end connection 10.0.26.69:42174 (1506 connections now open) 2016-03-13T09:14:51.374+0000 I NETWORK [conn4080466] end connection 10.0.26.69:42175 (1506 connections now open) 2016-03-13T09:14:51.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42188 #4080479 (1506 connections now open) 2016-03-13T09:14:51.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42189 #4080480 (1507 connections now open) 2016-03-13T09:14:52.539+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 674ms sequenceNumber: 16814 version: 30639|1||563ba74d869758a4542b9075 based on: 30638|1||563ba74d869758a4542b9075 2016-03-13T09:14:52.813+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:14:52.813+0000-56e52f8c18fa188537623d53", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860492813), what: "balancer.round", ns: "", details: { executionTimeMillis: 13258, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:14:52.986+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:14:54.207+0000 I NETWORK [conn4080468] end connection 10.0.27.237:35482 (1506 connections now open) 2016-03-13T09:14:54.207+0000 I NETWORK [conn4080467] end connection 10.0.27.237:35481 (1506 connections now open) 2016-03-13T09:14:54.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35494 #4080481 (1506 connections now open) 2016-03-13T09:14:54.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35495 #4080482 (1507 connections now open) 2016-03-13T09:14:54.319+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52f8e18fa188537623d54 2016-03-13T09:14:56.327+0000 I NETWORK [conn4080471] end connection 10.0.7.113:59133 (1506 connections now open) 2016-03-13T09:14:56.327+0000 I NETWORK [conn4080472] end connection 10.0.7.113:59134 (1506 connections now open) 2016-03-13T09:14:56.327+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59142 #4080483 (1506 connections now open) 2016-03-13T09:14:56.327+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59141 #4080484 (1507 connections now open) 2016-03-13T09:14:56.790+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56344a6a584ad10f2135352d')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56344a6a584ad10f2135352d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56344c42a2a4330ea29576e8') }, shard: "shmydomain1", lastmod: Timestamp 30639000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:14:56.944+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30639|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56344a6a584ad10f2135352d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56344c42a2a4330ea29576e8') }) shmydomain1 -> shmydomain3 2016-03-13T09:14:57.390+0000 I NETWORK [conn4080474] end connection 10.0.216.90:57155 (1506 connections now open) 2016-03-13T09:14:57.390+0000 I NETWORK [conn4080473] end connection 10.0.216.90:57154 (1506 connections now open) 2016-03-13T09:14:57.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57162 #4080485 (1506 connections now open) 2016-03-13T09:14:57.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57161 #4080486 (1507 connections now open) 2016-03-13T09:14:58.408+0000 I NETWORK [conn4080476] end connection 10.0.20.53:50433 (1506 connections now open) 2016-03-13T09:14:58.408+0000 I NETWORK [conn4080475] end connection 10.0.20.53:50432 (1506 connections now open) 2016-03-13T09:14:58.408+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50445 #4080487 (1506 connections now open) 2016-03-13T09:14:58.408+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50446 #4080488 (1507 connections now open) 2016-03-13T09:15:00.755+0000 I NETWORK [conn4080478] end connection 10.0.179.209:40205 (1506 connections now open) 2016-03-13T09:15:00.755+0000 I NETWORK [conn4080477] end connection 10.0.179.209:40206 (1506 connections now open) 2016-03-13T09:15:00.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40216 #4080489 (1506 connections now open) 2016-03-13T09:15:00.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40215 #4080490 (1507 connections now open) 2016-03-13T09:15:01.387+0000 I NETWORK [conn4080480] end connection 10.0.26.69:42189 (1506 connections now open) 2016-03-13T09:15:01.387+0000 I NETWORK [conn4080479] end connection 10.0.26.69:42188 (1506 connections now open) 2016-03-13T09:15:01.388+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42207 #4080491 (1506 connections now open) 2016-03-13T09:15:01.388+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42208 #4080492 (1507 connections now open) 2016-03-13T09:15:04.219+0000 I NETWORK [conn4080482] end connection 10.0.27.237:35495 (1506 connections now open) 2016-03-13T09:15:04.219+0000 I NETWORK [conn4080481] end connection 10.0.27.237:35494 (1505 connections now open) 2016-03-13T09:15:04.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35520 #4080493 (1506 connections now open) 2016-03-13T09:15:04.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35519 #4080494 (1507 connections now open) 2016-03-13T09:15:06.340+0000 I NETWORK [conn4080484] end connection 10.0.7.113:59141 (1506 connections now open) 2016-03-13T09:15:06.340+0000 I NETWORK [conn4080483] end connection 10.0.7.113:59142 (1506 connections now open) 2016-03-13T09:15:06.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59161 #4080495 (1506 connections now open) 2016-03-13T09:15:06.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59160 #4080496 (1507 connections now open) 2016-03-13T09:15:06.879+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16815 version: 30640|1||563ba74d869758a4542b9075 based on: 30639|1||563ba74d869758a4542b9075 2016-03-13T09:15:07.066+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:15:07.066+0000-56e52f9b18fa188537623d55", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860507066), what: "balancer.round", ns: "", details: { executionTimeMillis: 13050, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:15:07.275+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:15:07.403+0000 I NETWORK [conn4080486] end connection 10.0.216.90:57161 (1506 connections now open) 2016-03-13T09:15:07.403+0000 I NETWORK [conn4080485] end connection 10.0.216.90:57162 (1506 connections now open) 2016-03-13T09:15:07.404+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57178 #4080497 (1506 connections now open) 2016-03-13T09:15:07.404+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57177 #4080498 (1507 connections now open) 2016-03-13T09:15:08.421+0000 I NETWORK [conn4080487] end connection 10.0.20.53:50445 (1506 connections now open) 2016-03-13T09:15:08.421+0000 I NETWORK [conn4080488] end connection 10.0.20.53:50446 (1506 connections now open) 2016-03-13T09:15:08.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50467 #4080499 (1506 connections now open) 2016-03-13T09:15:08.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50466 #4080500 (1507 connections now open) 2016-03-13T09:15:10.768+0000 I NETWORK [conn4080490] end connection 10.0.179.209:40215 (1506 connections now open) 2016-03-13T09:15:10.768+0000 I NETWORK [conn4080489] end connection 10.0.179.209:40216 (1506 connections now open) 2016-03-13T09:15:10.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40231 #4080501 (1506 connections now open) 2016-03-13T09:15:10.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40230 #4080502 (1507 connections now open) 2016-03-13T09:15:11.401+0000 I NETWORK [conn4080492] end connection 10.0.26.69:42208 (1506 connections now open) 2016-03-13T09:15:11.401+0000 I NETWORK [conn4080491] end connection 10.0.26.69:42207 (1506 connections now open) 2016-03-13T09:15:11.401+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42221 #4080503 (1507 connections now open) 2016-03-13T09:15:11.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42222 #4080504 (1507 connections now open) 2016-03-13T09:15:14.232+0000 I NETWORK [conn4080494] end connection 10.0.27.237:35519 (1506 connections now open) 2016-03-13T09:15:14.232+0000 I NETWORK [conn4080493] end connection 10.0.27.237:35520 (1506 connections now open) 2016-03-13T09:15:14.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35530 #4080505 (1506 connections now open) 2016-03-13T09:15:14.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35531 #4080506 (1507 connections now open) 2016-03-13T09:15:16.118+0000 I NETWORK [conn4079836] end connection 10.0.216.90:56516 (1506 connections now open) 2016-03-13T09:15:16.118+0000 I NETWORK [conn4070098] end connection 10.0.7.113:48161 (1505 connections now open) 2016-03-13T09:15:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59181 #4080507 (1506 connections now open) 2016-03-13T09:15:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57196 #4080508 (1507 connections now open) 2016-03-13T09:15:16.354+0000 I NETWORK [conn4080495] end connection 10.0.7.113:59161 (1506 connections now open) 2016-03-13T09:15:16.354+0000 I NETWORK [conn4080496] end connection 10.0.7.113:59160 (1506 connections now open) 2016-03-13T09:15:16.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59182 #4080509 (1506 connections now open) 2016-03-13T09:15:16.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59183 #4080510 (1507 connections now open) 2016-03-13T09:15:17.416+0000 I NETWORK [conn4080497] end connection 10.0.216.90:57178 (1506 connections now open) 2016-03-13T09:15:17.416+0000 I NETWORK [conn4080498] end connection 10.0.216.90:57177 (1506 connections now open) 2016-03-13T09:15:17.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57197 #4080511 (1506 connections now open) 2016-03-13T09:15:17.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57198 #4080512 (1507 connections now open) 2016-03-13T09:15:17.904+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:15:17.625+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:15:17.904+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:15:18.041+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52f9c18fa188537623d56 2016-03-13T09:15:18.434+0000 I NETWORK [conn4080500] end connection 10.0.20.53:50466 (1506 connections now open) 2016-03-13T09:15:18.434+0000 I NETWORK [conn4080499] end connection 10.0.20.53:50467 (1506 connections now open) 2016-03-13T09:15:18.444+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50477 #4080513 (1506 connections now open) 2016-03-13T09:15:18.444+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50478 #4080514 (1507 connections now open) 2016-03-13T09:15:20.391+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16816 version: 30641|1||563ba74d869758a4542b9075 based on: 30640|1||563ba74d869758a4542b9075 2016-03-13T09:15:20.789+0000 I NETWORK [conn4080502] end connection 10.0.179.209:40230 (1506 connections now open) 2016-03-13T09:15:20.789+0000 I NETWORK [conn4080501] end connection 10.0.179.209:40231 (1506 connections now open) 2016-03-13T09:15:20.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40254 #4080515 (1506 connections now open) 2016-03-13T09:15:20.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40255 #4080516 (1507 connections now open) 2016-03-13T09:15:21.414+0000 I NETWORK [conn4080503] end connection 10.0.26.69:42221 (1506 connections now open) 2016-03-13T09:15:21.414+0000 I NETWORK [conn4080504] end connection 10.0.26.69:42222 (1506 connections now open) 2016-03-13T09:15:21.424+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42235 #4080517 (1506 connections now open) 2016-03-13T09:15:21.424+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42234 #4080518 (1507 connections now open) 2016-03-13T09:15:24.254+0000 I NETWORK [conn4080505] end connection 10.0.27.237:35530 (1506 connections now open) 2016-03-13T09:15:24.254+0000 I NETWORK [conn4080506] end connection 10.0.27.237:35531 (1506 connections now open) 2016-03-13T09:15:24.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35544 #4080519 (1506 connections now open) 2016-03-13T09:15:24.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35545 #4080520 (1507 connections now open) 2016-03-13T09:15:26.368+0000 I NETWORK [conn4080510] end connection 10.0.7.113:59183 (1506 connections now open) 2016-03-13T09:15:26.368+0000 I NETWORK [conn4080509] end connection 10.0.7.113:59182 (1506 connections now open) 2016-03-13T09:15:26.378+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59189 #4080521 (1506 connections now open) 2016-03-13T09:15:26.378+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59188 #4080522 (1507 connections now open) 2016-03-13T09:15:27.429+0000 I NETWORK [conn4080512] end connection 10.0.216.90:57198 (1506 connections now open) 2016-03-13T09:15:27.429+0000 I NETWORK [conn4080511] end connection 10.0.216.90:57197 (1506 connections now open) 2016-03-13T09:15:27.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57205 #4080523 (1506 connections now open) 2016-03-13T09:15:27.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57204 #4080524 (1507 connections now open) 2016-03-13T09:15:28.113+0000 I NETWORK [conn4068364] end connection 10.0.179.209:56211 (1506 connections now open) 2016-03-13T09:15:28.113+0000 I NETWORK [conn4072133] end connection 10.0.27.237:55242 (1505 connections now open) 2016-03-13T09:15:28.113+0000 I NETWORK [conn4072158] end connection 10.0.7.113:50363 (1504 connections now open) 2016-03-13T09:15:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40262 #4080525 (1505 connections now open) 2016-03-13T09:15:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35562 #4080526 (1506 connections now open) 2016-03-13T09:15:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59194 #4080527 (1507 connections now open) 2016-03-13T09:15:28.457+0000 I NETWORK [conn4080514] end connection 10.0.20.53:50478 (1506 connections now open) 2016-03-13T09:15:28.457+0000 I NETWORK [conn4080513] end connection 10.0.20.53:50477 (1506 connections now open) 2016-03-13T09:15:28.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50492 #4080528 (1506 connections now open) 2016-03-13T09:15:28.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50491 #4080529 (1507 connections now open) 2016-03-13T09:15:30.802+0000 I NETWORK [conn4080516] end connection 10.0.179.209:40255 (1506 connections now open) 2016-03-13T09:15:30.802+0000 I NETWORK [conn4080515] end connection 10.0.179.209:40254 (1505 connections now open) 2016-03-13T09:15:30.802+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40264 #4080530 (1506 connections now open) 2016-03-13T09:15:30.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40265 #4080531 (1507 connections now open) 2016-03-13T09:15:31.436+0000 I NETWORK [conn4080517] end connection 10.0.26.69:42235 (1506 connections now open) 2016-03-13T09:15:31.436+0000 I NETWORK [conn4080518] end connection 10.0.26.69:42234 (1506 connections now open) 2016-03-13T09:15:31.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42254 #4080532 (1506 connections now open) 2016-03-13T09:15:31.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42253 #4080533 (1507 connections now open) 2016-03-13T09:15:34.267+0000 I NETWORK [conn4080519] end connection 10.0.27.237:35544 (1506 connections now open) 2016-03-13T09:15:34.267+0000 I NETWORK [conn4080520] end connection 10.0.27.237:35545 (1506 connections now open) 2016-03-13T09:15:34.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35565 #4080534 (1506 connections now open) 2016-03-13T09:15:34.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35566 #4080535 (1507 connections now open) 2016-03-13T09:15:35.296+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16817 version: 30642|1||563ba74d869758a4542b9075 based on: 30641|1||563ba74d869758a4542b9075 2016-03-13T09:15:36.391+0000 I NETWORK [conn4080522] end connection 10.0.7.113:59188 (1506 connections now open) 2016-03-13T09:15:36.391+0000 I NETWORK [conn4080521] end connection 10.0.7.113:59189 (1506 connections now open) 2016-03-13T09:15:36.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59202 #4080536 (1506 connections now open) 2016-03-13T09:15:36.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59203 #4080537 (1507 connections now open) 2016-03-13T09:15:37.442+0000 I NETWORK [conn4080523] end connection 10.0.216.90:57205 (1506 connections now open) 2016-03-13T09:15:37.442+0000 I NETWORK [conn4080524] end connection 10.0.216.90:57204 (1506 connections now open) 2016-03-13T09:15:37.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57220 #4080538 (1506 connections now open) 2016-03-13T09:15:37.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57221 #4080539 (1507 connections now open) 2016-03-13T09:15:38.470+0000 I NETWORK [conn4080528] end connection 10.0.20.53:50492 (1506 connections now open) 2016-03-13T09:15:38.470+0000 I NETWORK [conn4080529] end connection 10.0.20.53:50491 (1506 connections now open) 2016-03-13T09:15:38.471+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50515 #4080540 (1506 connections now open) 2016-03-13T09:15:38.471+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50514 #4080541 (1507 connections now open) 2016-03-13T09:15:40.815+0000 I NETWORK [conn4080530] end connection 10.0.179.209:40264 (1506 connections now open) 2016-03-13T09:15:40.815+0000 I NETWORK [conn4080531] end connection 10.0.179.209:40265 (1506 connections now open) 2016-03-13T09:15:40.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40275 #4080542 (1506 connections now open) 2016-03-13T09:15:40.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40276 #4080543 (1507 connections now open) 2016-03-13T09:15:41.450+0000 I NETWORK [conn4080533] end connection 10.0.26.69:42253 (1506 connections now open) 2016-03-13T09:15:41.450+0000 I NETWORK [conn4080532] end connection 10.0.26.69:42254 (1506 connections now open) 2016-03-13T09:15:41.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42259 #4080544 (1506 connections now open) 2016-03-13T09:15:41.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42260 #4080545 (1507 connections now open) 2016-03-13T09:15:44.280+0000 I NETWORK [conn4080534] end connection 10.0.27.237:35565 (1506 connections now open) 2016-03-13T09:15:44.280+0000 I NETWORK [conn4080535] end connection 10.0.27.237:35566 (1506 connections now open) 2016-03-13T09:15:44.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35575 #4080546 (1506 connections now open) 2016-03-13T09:15:44.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35576 #4080547 (1507 connections now open) 2016-03-13T09:15:46.118+0000 I NETWORK [conn4069910] end connection 10.0.179.209:57766 (1506 connections now open) 2016-03-13T09:15:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40293 #4080548 (1507 connections now open) 2016-03-13T09:15:46.404+0000 I NETWORK [conn4080537] end connection 10.0.7.113:59203 (1506 connections now open) 2016-03-13T09:15:46.404+0000 I NETWORK [conn4080536] end connection 10.0.7.113:59202 (1506 connections now open) 2016-03-13T09:15:46.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59226 #4080549 (1506 connections now open) 2016-03-13T09:15:46.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59225 #4080550 (1507 connections now open) 2016-03-13T09:15:47.455+0000 I NETWORK [conn4080539] end connection 10.0.216.90:57221 (1506 connections now open) 2016-03-13T09:15:47.455+0000 I NETWORK [conn4080538] end connection 10.0.216.90:57220 (1506 connections now open) 2016-03-13T09:15:47.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57245 #4080551 (1506 connections now open) 2016-03-13T09:15:47.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57244 #4080552 (1507 connections now open) 2016-03-13T09:15:48.349+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:15:48.041+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:15:48.349+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:15:48.452+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52fa618fa188537623d57 2016-03-13T09:15:48.483+0000 I NETWORK [conn4080540] end connection 10.0.20.53:50515 (1506 connections now open) 2016-03-13T09:15:48.483+0000 I NETWORK [conn4080541] end connection 10.0.20.53:50514 (1506 connections now open) 2016-03-13T09:15:48.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50529 #4080553 (1506 connections now open) 2016-03-13T09:15:48.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50528 #4080554 (1507 connections now open) 2016-03-13T09:15:48.591+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52fb018fa188537623d58 2016-03-13T09:15:48.745+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52fba18fa188537623d59 2016-03-13T09:15:50.719+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16818 version: 30643|1||563ba74d869758a4542b9075 based on: 30642|1||563ba74d869758a4542b9075 2016-03-13T09:15:50.828+0000 I NETWORK [conn4080543] end connection 10.0.179.209:40276 (1506 connections now open) 2016-03-13T09:15:50.828+0000 I NETWORK [conn4080542] end connection 10.0.179.209:40275 (1506 connections now open) 2016-03-13T09:15:50.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40298 #4080555 (1506 connections now open) 2016-03-13T09:15:50.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40299 #4080556 (1507 connections now open) 2016-03-13T09:15:51.463+0000 I NETWORK [conn4080545] end connection 10.0.26.69:42260 (1506 connections now open) 2016-03-13T09:15:51.463+0000 I NETWORK [conn4080544] end connection 10.0.26.69:42259 (1506 connections now open) 2016-03-13T09:15:51.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42276 #4080557 (1507 connections now open) 2016-03-13T09:15:51.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42277 #4080558 (1507 connections now open) 2016-03-13T09:15:54.293+0000 I NETWORK [conn4080547] end connection 10.0.27.237:35576 (1506 connections now open) 2016-03-13T09:15:54.293+0000 I NETWORK [conn4080546] end connection 10.0.27.237:35575 (1506 connections now open) 2016-03-13T09:15:54.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35589 #4080559 (1506 connections now open) 2016-03-13T09:15:54.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35590 #4080560 (1507 connections now open) 2016-03-13T09:15:56.417+0000 I NETWORK [conn4080549] end connection 10.0.7.113:59226 (1506 connections now open) 2016-03-13T09:15:56.417+0000 I NETWORK [conn4080550] end connection 10.0.7.113:59225 (1506 connections now open) 2016-03-13T09:15:56.418+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59231 #4080561 (1506 connections now open) 2016-03-13T09:15:56.418+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59232 #4080562 (1507 connections now open) 2016-03-13T09:15:57.468+0000 I NETWORK [conn4080552] end connection 10.0.216.90:57244 (1506 connections now open) 2016-03-13T09:15:57.468+0000 I NETWORK [conn4080551] end connection 10.0.216.90:57245 (1506 connections now open) 2016-03-13T09:15:57.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57252 #4080563 (1506 connections now open) 2016-03-13T09:15:57.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57251 #4080564 (1507 connections now open) 2016-03-13T09:15:58.113+0000 I NETWORK [conn4074272] end connection 10.0.179.209:33910 (1506 connections now open) 2016-03-13T09:15:58.113+0000 I NETWORK [conn4070222] end connection 10.0.20.53:39722 (1505 connections now open) 2016-03-13T09:15:58.113+0000 I NETWORK [conn4070629] end connection 10.0.216.90:46716 (1504 connections now open) 2016-03-13T09:15:58.113+0000 I NETWORK [conn4070522] end connection 10.0.27.237:53517 (1503 connections now open) 2016-03-13T09:15:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40305 #4080565 (1505 connections now open) 2016-03-13T09:15:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50542 #4080566 (1505 connections now open) 2016-03-13T09:15:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57255 #4080567 (1506 connections now open) 2016-03-13T09:15:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35606 #4080568 (1507 connections now open) 2016-03-13T09:15:58.497+0000 I NETWORK [conn4080554] end connection 10.0.20.53:50528 (1506 connections now open) 2016-03-13T09:15:58.497+0000 I NETWORK [conn4080553] end connection 10.0.20.53:50529 (1506 connections now open) 2016-03-13T09:15:58.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50544 #4080569 (1506 connections now open) 2016-03-13T09:15:58.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50543 #4080570 (1507 connections now open) 2016-03-13T09:16:00.841+0000 I NETWORK [conn4080555] end connection 10.0.179.209:40298 (1506 connections now open) 2016-03-13T09:16:00.841+0000 I NETWORK [conn4080556] end connection 10.0.179.209:40299 (1506 connections now open) 2016-03-13T09:16:00.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40307 #4080571 (1506 connections now open) 2016-03-13T09:16:00.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40308 #4080572 (1507 connections now open) 2016-03-13T09:16:01.476+0000 I NETWORK [conn4080558] end connection 10.0.26.69:42277 (1506 connections now open) 2016-03-13T09:16:01.476+0000 I NETWORK [conn4080557] end connection 10.0.26.69:42276 (1506 connections now open) 2016-03-13T09:16:01.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42295 #4080573 (1506 connections now open) 2016-03-13T09:16:01.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42296 #4080574 (1507 connections now open) 2016-03-13T09:16:04.306+0000 I NETWORK [conn4080560] end connection 10.0.27.237:35590 (1506 connections now open) 2016-03-13T09:16:04.306+0000 I NETWORK [conn4080559] end connection 10.0.27.237:35589 (1506 connections now open) 2016-03-13T09:16:04.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35611 #4080575 (1506 connections now open) 2016-03-13T09:16:04.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35612 #4080576 (1507 connections now open) 2016-03-13T09:16:05.655+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 679ms sequenceNumber: 16819 version: 30644|1||563ba74d869758a4542b9075 based on: 30643|1||563ba74d869758a4542b9075 2016-03-13T09:16:06.431+0000 I NETWORK [conn4080562] end connection 10.0.7.113:59232 (1506 connections now open) 2016-03-13T09:16:06.431+0000 I NETWORK [conn4080561] end connection 10.0.7.113:59231 (1506 connections now open) 2016-03-13T09:16:06.432+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59248 #4080577 (1506 connections now open) 2016-03-13T09:16:06.432+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59249 #4080578 (1507 connections now open) 2016-03-13T09:16:07.481+0000 I NETWORK [conn4080563] end connection 10.0.216.90:57252 (1506 connections now open) 2016-03-13T09:16:07.481+0000 I NETWORK [conn4080564] end connection 10.0.216.90:57251 (1506 connections now open) 2016-03-13T09:16:07.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57268 #4080579 (1506 connections now open) 2016-03-13T09:16:07.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57267 #4080580 (1507 connections now open) 2016-03-13T09:16:08.510+0000 I NETWORK [conn4080569] end connection 10.0.20.53:50544 (1506 connections now open) 2016-03-13T09:16:08.510+0000 I NETWORK [conn4080570] end connection 10.0.20.53:50543 (1506 connections now open) 2016-03-13T09:16:08.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50563 #4080581 (1506 connections now open) 2016-03-13T09:16:08.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50564 #4080582 (1507 connections now open) 2016-03-13T09:16:09.583+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52fd918fa188537623d5c 2016-03-13T09:16:10.854+0000 I NETWORK [conn4080572] end connection 10.0.179.209:40308 (1506 connections now open) 2016-03-13T09:16:10.854+0000 I NETWORK [conn4080571] end connection 10.0.179.209:40307 (1506 connections now open) 2016-03-13T09:16:10.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40320 #4080583 (1506 connections now open) 2016-03-13T09:16:10.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40321 #4080584 (1507 connections now open) 2016-03-13T09:16:11.490+0000 I NETWORK [conn4080574] end connection 10.0.26.69:42296 (1506 connections now open) 2016-03-13T09:16:11.490+0000 I NETWORK [conn4080573] end connection 10.0.26.69:42295 (1506 connections now open) 2016-03-13T09:16:11.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42308 #4080585 (1506 connections now open) 2016-03-13T09:16:11.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42307 #4080586 (1507 connections now open) 2016-03-13T09:16:12.447+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56345421584ad10f1f990b05')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345421584ad10f1f990b05') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634564b55418c0ec3f816f0') }, shard: "shmydomain1", lastmod: Timestamp 30644000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:16:12.604+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30644|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345421584ad10f1f990b05') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634564b55418c0ec3f816f0') }) shmydomain1 -> shmydomain2 2016-03-13T09:16:14.319+0000 I NETWORK [conn4080575] end connection 10.0.27.237:35611 (1506 connections now open) 2016-03-13T09:16:14.319+0000 I NETWORK [conn4080576] end connection 10.0.27.237:35612 (1506 connections now open) 2016-03-13T09:16:14.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35623 #4080587 (1506 connections now open) 2016-03-13T09:16:14.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35622 #4080588 (1507 connections now open) 2016-03-13T09:16:16.117+0000 I NETWORK [conn4068687] end connection 10.0.179.209:56532 (1506 connections now open) 2016-03-13T09:16:16.117+0000 I NETWORK [conn4068689] end connection 10.0.26.69:57885 (1505 connections now open) 2016-03-13T09:16:16.117+0000 I NETWORK [conn4070776] end connection 10.0.7.113:48893 (1504 connections now open) 2016-03-13T09:16:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40335 #4080589 (1505 connections now open) 2016-03-13T09:16:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42313 #4080590 (1506 connections now open) 2016-03-13T09:16:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59270 #4080591 (1507 connections now open) 2016-03-13T09:16:16.444+0000 I NETWORK [conn4080577] end connection 10.0.7.113:59248 (1506 connections now open) 2016-03-13T09:16:16.444+0000 I NETWORK [conn4080578] end connection 10.0.7.113:59249 (1506 connections now open) 2016-03-13T09:16:16.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59271 #4080592 (1506 connections now open) 2016-03-13T09:16:16.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59272 #4080593 (1507 connections now open) 2016-03-13T09:16:17.494+0000 I NETWORK [conn4080579] end connection 10.0.216.90:57268 (1506 connections now open) 2016-03-13T09:16:17.494+0000 I NETWORK [conn4080580] end connection 10.0.216.90:57267 (1506 connections now open) 2016-03-13T09:16:17.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57286 #4080594 (1506 connections now open) 2016-03-13T09:16:17.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57287 #4080595 (1507 connections now open) 2016-03-13T09:16:18.524+0000 I NETWORK [conn4080581] end connection 10.0.20.53:50563 (1506 connections now open) 2016-03-13T09:16:18.524+0000 I NETWORK [conn4080582] end connection 10.0.20.53:50564 (1506 connections now open) 2016-03-13T09:16:18.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50574 #4080596 (1506 connections now open) 2016-03-13T09:16:18.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50573 #4080597 (1507 connections now open) 2016-03-13T09:16:19.032+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:16:18.745+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:16:19.032+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:16:19.138+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52fc418fa188537623d5a 2016-03-13T09:16:19.302+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e52fce18fa188537623d5b 2016-03-13T09:16:20.877+0000 I NETWORK [conn4080583] end connection 10.0.179.209:40320 (1506 connections now open) 2016-03-13T09:16:20.877+0000 I NETWORK [conn4080584] end connection 10.0.179.209:40321 (1506 connections now open) 2016-03-13T09:16:20.878+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40343 #4080598 (1506 connections now open) 2016-03-13T09:16:20.878+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40344 #4080599 (1507 connections now open) 2016-03-13T09:16:21.503+0000 I NETWORK [conn4080586] end connection 10.0.26.69:42307 (1506 connections now open) 2016-03-13T09:16:21.503+0000 I NETWORK [conn4080585] end connection 10.0.26.69:42308 (1506 connections now open) 2016-03-13T09:16:21.513+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42321 #4080600 (1506 connections now open) 2016-03-13T09:16:21.513+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42320 #4080601 (1507 connections now open) 2016-03-13T09:16:22.365+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16820 version: 30645|1||563ba74d869758a4542b9075 based on: 30644|1||563ba74d869758a4542b9075 2016-03-13T09:16:22.670+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:16:22.670+0000-56e52fe618fa188537623d5d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860582670), what: "balancer.round", ns: "", details: { executionTimeMillis: 13779, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:16:22.818+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:16:24.068+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52fe718fa188537623d5e 2016-03-13T09:16:24.341+0000 I NETWORK [conn4080588] end connection 10.0.27.237:35622 (1506 connections now open) 2016-03-13T09:16:24.341+0000 I NETWORK [conn4080587] end connection 10.0.27.237:35623 (1506 connections now open) 2016-03-13T09:16:24.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35636 #4080602 (1506 connections now open) 2016-03-13T09:16:24.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35637 #4080603 (1507 connections now open) 2016-03-13T09:16:26.457+0000 I NETWORK [conn4080593] end connection 10.0.7.113:59272 (1506 connections now open) 2016-03-13T09:16:26.457+0000 I NETWORK [conn4080592] end connection 10.0.7.113:59271 (1506 connections now open) 2016-03-13T09:16:26.467+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59277 #4080604 (1506 connections now open) 2016-03-13T09:16:26.467+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59278 #4080605 (1507 connections now open) 2016-03-13T09:16:26.800+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634564b55418c0ec3f816f0')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634564b55418c0ec3f816f0') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634589655418c0ec3f850a4') }, shard: "shmydomain1", lastmod: Timestamp 30645000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:16:26.958+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30645|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634564b55418c0ec3f816f0') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634589655418c0ec3f850a4') }) shmydomain1 -> shmydomain3 2016-03-13T09:16:27.265+0000 I NETWORK [conn4069321] end connection 10.0.20.53:38769 (1506 connections now open) 2016-03-13T09:16:27.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50585 #4080606 (1507 connections now open) 2016-03-13T09:16:27.269+0000 I NETWORK [conn4076442] end connection 10.0.20.53:46259 (1506 connections now open) 2016-03-13T09:16:27.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50586 #4080607 (1507 connections now open) 2016-03-13T09:16:27.276+0000 I NETWORK [conn4070500] end connection 10.0.20.53:40015 (1506 connections now open) 2016-03-13T09:16:27.277+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50587 #4080608 (1507 connections now open) 2016-03-13T09:16:27.283+0000 I NETWORK [conn4072618] end connection 10.0.20.53:42197 (1506 connections now open) 2016-03-13T09:16:27.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50588 #4080609 (1507 connections now open) 2016-03-13T09:16:27.515+0000 I NETWORK [conn4080594] end connection 10.0.216.90:57286 (1506 connections now open) 2016-03-13T09:16:27.515+0000 I NETWORK [conn4080595] end connection 10.0.216.90:57287 (1505 connections now open) 2016-03-13T09:16:27.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57293 #4080610 (1506 connections now open) 2016-03-13T09:16:27.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57294 #4080611 (1507 connections now open) 2016-03-13T09:16:28.113+0000 I NETWORK [conn4070713] end connection 10.0.216.90:46810 (1506 connections now open) 2016-03-13T09:16:28.113+0000 I NETWORK [conn4073642] end connection 10.0.27.237:56833 (1505 connections now open) 2016-03-13T09:16:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57296 #4080612 (1506 connections now open) 2016-03-13T09:16:28.114+0000 I NETWORK [conn4068902] end connection 10.0.7.113:46864 (1504 connections now open) 2016-03-13T09:16:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35655 #4080613 (1506 connections now open) 2016-03-13T09:16:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59282 #4080614 (1507 connections now open) 2016-03-13T09:16:28.410+0000 I NETWORK [conn4072537] end connection 10.0.27.237:55678 (1506 connections now open) 2016-03-13T09:16:28.411+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35657 #4080615 (1507 connections now open) 2016-03-13T09:16:28.443+0000 I NETWORK [conn4068644] end connection 10.0.27.237:51523 (1506 connections now open) 2016-03-13T09:16:28.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35659 #4080616 (1507 connections now open) 2016-03-13T09:16:28.465+0000 I NETWORK [conn4072746] end connection 10.0.27.237:55905 (1506 connections now open) 2016-03-13T09:16:28.465+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35660 #4080617 (1507 connections now open) 2016-03-13T09:16:28.466+0000 I NETWORK [conn4068817] end connection 10.0.27.237:51701 (1506 connections now open) 2016-03-13T09:16:28.467+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35661 #4080618 (1507 connections now open) 2016-03-13T09:16:28.487+0000 I NETWORK [conn4075182] end connection 10.0.27.237:58368 (1506 connections now open) 2016-03-13T09:16:28.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35663 #4080619 (1507 connections now open) 2016-03-13T09:16:28.546+0000 I NETWORK [conn4080596] end connection 10.0.20.53:50574 (1506 connections now open) 2016-03-13T09:16:28.546+0000 I NETWORK [conn4080597] end connection 10.0.20.53:50573 (1506 connections now open) 2016-03-13T09:16:28.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50594 #4080620 (1506 connections now open) 2016-03-13T09:16:28.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50595 #4080621 (1507 connections now open) 2016-03-13T09:16:30.890+0000 I NETWORK [conn4080598] end connection 10.0.179.209:40343 (1506 connections now open) 2016-03-13T09:16:30.890+0000 I NETWORK [conn4080599] end connection 10.0.179.209:40344 (1506 connections now open) 2016-03-13T09:16:30.891+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40353 #4080622 (1506 connections now open) 2016-03-13T09:16:30.891+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40354 #4080623 (1507 connections now open) 2016-03-13T09:16:31.525+0000 I NETWORK [conn4080600] end connection 10.0.26.69:42321 (1506 connections now open) 2016-03-13T09:16:31.525+0000 I NETWORK [conn4080601] end connection 10.0.26.69:42320 (1506 connections now open) 2016-03-13T09:16:31.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42341 #4080624 (1506 connections now open) 2016-03-13T09:16:31.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42340 #4080625 (1507 connections now open) 2016-03-13T09:16:34.354+0000 I NETWORK [conn4080602] end connection 10.0.27.237:35636 (1506 connections now open) 2016-03-13T09:16:34.354+0000 I NETWORK [conn4080603] end connection 10.0.27.237:35637 (1506 connections now open) 2016-03-13T09:16:34.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35666 #4080626 (1506 connections now open) 2016-03-13T09:16:34.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35667 #4080627 (1507 connections now open) 2016-03-13T09:16:36.480+0000 I NETWORK [conn4080605] end connection 10.0.7.113:59278 (1506 connections now open) 2016-03-13T09:16:36.480+0000 I NETWORK [conn4080604] end connection 10.0.7.113:59277 (1506 connections now open) 2016-03-13T09:16:36.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59290 #4080628 (1506 connections now open) 2016-03-13T09:16:36.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59289 #4080629 (1507 connections now open) 2016-03-13T09:16:36.761+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16821 version: 30646|1||563ba74d869758a4542b9075 based on: 30645|1||563ba74d869758a4542b9075 2016-03-13T09:16:37.030+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:16:37.030+0000-56e52ff518fa188537623d5f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860597030), what: "balancer.round", ns: "", details: { executionTimeMillis: 13177, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:16:37.212+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:16:37.528+0000 I NETWORK [conn4080610] end connection 10.0.216.90:57293 (1506 connections now open) 2016-03-13T09:16:37.528+0000 I NETWORK [conn4080611] end connection 10.0.216.90:57294 (1505 connections now open) 2016-03-13T09:16:37.528+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57308 #4080630 (1506 connections now open) 2016-03-13T09:16:37.529+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57309 #4080631 (1507 connections now open) 2016-03-13T09:16:38.554+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e52ff618fa188537623d60 2016-03-13T09:16:38.559+0000 I NETWORK [conn4080620] end connection 10.0.20.53:50594 (1506 connections now open) 2016-03-13T09:16:38.559+0000 I NETWORK [conn4080621] end connection 10.0.20.53:50595 (1506 connections now open) 2016-03-13T09:16:38.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50615 #4080632 (1506 connections now open) 2016-03-13T09:16:38.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50616 #4080633 (1507 connections now open) 2016-03-13T09:16:40.903+0000 I NETWORK [conn4080622] end connection 10.0.179.209:40353 (1506 connections now open) 2016-03-13T09:16:40.903+0000 I NETWORK [conn4080623] end connection 10.0.179.209:40354 (1506 connections now open) 2016-03-13T09:16:40.904+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40364 #4080634 (1506 connections now open) 2016-03-13T09:16:40.904+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40365 #4080635 (1507 connections now open) 2016-03-13T09:16:41.314+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634589655418c0ec3f850a4')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634589655418c0ec3f850a4') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345b0e55418c0ec5990cac') }, shard: "shmydomain1", lastmod: Timestamp 30646000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:16:41.469+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30646|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634589655418c0ec3f850a4') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345b0e55418c0ec5990cac') }) shmydomain1 -> shmydomain2 2016-03-13T09:16:41.539+0000 I NETWORK [conn4080625] end connection 10.0.26.69:42340 (1506 connections now open) 2016-03-13T09:16:41.539+0000 I NETWORK [conn4080624] end connection 10.0.26.69:42341 (1506 connections now open) 2016-03-13T09:16:41.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42347 #4080636 (1506 connections now open) 2016-03-13T09:16:41.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42348 #4080637 (1507 connections now open) 2016-03-13T09:16:44.367+0000 I NETWORK [conn4080627] end connection 10.0.27.237:35667 (1506 connections now open) 2016-03-13T09:16:44.367+0000 I NETWORK [conn4080626] end connection 10.0.27.237:35666 (1506 connections now open) 2016-03-13T09:16:44.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35674 #4080638 (1506 connections now open) 2016-03-13T09:16:44.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35675 #4080639 (1507 connections now open) 2016-03-13T09:16:46.117+0000 I NETWORK [conn4072577] end connection 10.0.26.69:33847 (1506 connections now open) 2016-03-13T09:16:46.117+0000 I NETWORK [conn4070818] end connection 10.0.7.113:48942 (1505 connections now open) 2016-03-13T09:16:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42354 #4080640 (1507 connections now open) 2016-03-13T09:16:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59308 #4080641 (1507 connections now open) 2016-03-13T09:16:46.494+0000 I NETWORK [conn4080628] end connection 10.0.7.113:59290 (1506 connections now open) 2016-03-13T09:16:46.494+0000 I NETWORK [conn4080629] end connection 10.0.7.113:59289 (1506 connections now open) 2016-03-13T09:16:46.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59309 #4080642 (1506 connections now open) 2016-03-13T09:16:46.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59310 #4080643 (1507 connections now open) 2016-03-13T09:16:47.540+0000 I NETWORK [conn4080631] end connection 10.0.216.90:57309 (1506 connections now open) 2016-03-13T09:16:47.541+0000 I NETWORK [conn4080630] end connection 10.0.216.90:57308 (1505 connections now open) 2016-03-13T09:16:47.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57327 #4080644 (1506 connections now open) 2016-03-13T09:16:47.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57328 #4080645 (1507 connections now open) 2016-03-13T09:16:48.572+0000 I NETWORK [conn4080632] end connection 10.0.20.53:50615 (1506 connections now open) 2016-03-13T09:16:48.572+0000 I NETWORK [conn4080633] end connection 10.0.20.53:50616 (1506 connections now open) 2016-03-13T09:16:48.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50627 #4080646 (1506 connections now open) 2016-03-13T09:16:48.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50626 #4080647 (1507 connections now open) 2016-03-13T09:16:49.577+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:16:49.302+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:16:50.916+0000 I NETWORK [conn4080635] end connection 10.0.179.209:40365 (1506 connections now open) 2016-03-13T09:16:50.916+0000 I NETWORK [conn4080634] end connection 10.0.179.209:40364 (1506 connections now open) 2016-03-13T09:16:50.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40385 #4080648 (1506 connections now open) 2016-03-13T09:16:50.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40384 #4080649 (1507 connections now open) 2016-03-13T09:16:51.481+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 672ms sequenceNumber: 16822 version: 30647|1||563ba74d869758a4542b9075 based on: 30646|1||563ba74d869758a4542b9075 2016-03-13T09:16:51.552+0000 I NETWORK [conn4080637] end connection 10.0.26.69:42348 (1506 connections now open) 2016-03-13T09:16:51.552+0000 I NETWORK [conn4080636] end connection 10.0.26.69:42347 (1506 connections now open) 2016-03-13T09:16:51.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42361 #4080650 (1506 connections now open) 2016-03-13T09:16:51.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42362 #4080651 (1507 connections now open) 2016-03-13T09:16:51.752+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:16:51.752+0000-56e5300318fa188537623d61", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860611752), what: "balancer.round", ns: "", details: { executionTimeMillis: 13506, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:16:51.919+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:16:53.270+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5300418fa188537623d62 2016-03-13T09:16:54.380+0000 I NETWORK [conn4080638] end connection 10.0.27.237:35674 (1506 connections now open) 2016-03-13T09:16:54.380+0000 I NETWORK [conn4080639] end connection 10.0.27.237:35675 (1506 connections now open) 2016-03-13T09:16:54.381+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35689 #4080652 (1506 connections now open) 2016-03-13T09:16:54.381+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35688 #4080653 (1507 connections now open) 2016-03-13T09:16:55.783+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56345b0e55418c0ec5990cac')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345b0e55418c0ec5990cac') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345d9e584ad10f1f9a10c6') }, shard: "shmydomain1", lastmod: Timestamp 30647000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:16:55.936+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30647|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345b0e55418c0ec5990cac') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345d9e584ad10f1f9a10c6') }) shmydomain1 -> shmydomain3 2016-03-13T09:16:56.507+0000 I NETWORK [conn4080642] end connection 10.0.7.113:59309 (1506 connections now open) 2016-03-13T09:16:56.507+0000 I NETWORK [conn4080643] end connection 10.0.7.113:59310 (1505 connections now open) 2016-03-13T09:16:56.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59315 #4080654 (1506 connections now open) 2016-03-13T09:16:56.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59316 #4080655 (1507 connections now open) 2016-03-13T09:16:57.553+0000 I NETWORK [conn4080644] end connection 10.0.216.90:57327 (1506 connections now open) 2016-03-13T09:16:57.553+0000 I NETWORK [conn4080645] end connection 10.0.216.90:57328 (1505 connections now open) 2016-03-13T09:16:57.554+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57333 #4080656 (1506 connections now open) 2016-03-13T09:16:57.554+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57334 #4080657 (1507 connections now open) 2016-03-13T09:16:58.113+0000 I NETWORK [conn4072003] end connection 10.0.20.53:41572 (1506 connections now open) 2016-03-13T09:16:58.113+0000 I NETWORK [conn4075831] end connection 10.0.26.69:37326 (1505 connections now open) 2016-03-13T09:16:58.113+0000 I NETWORK [conn4071780] end connection 10.0.27.237:54857 (1505 connections now open) 2016-03-13T09:16:58.114+0000 I NETWORK [conn4070798] end connection 10.0.7.113:48919 (1503 connections now open) 2016-03-13T09:16:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50637 #4080658 (1504 connections now open) 2016-03-13T09:16:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35706 #4080659 (1505 connections now open) 2016-03-13T09:16:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59322 #4080660 (1506 connections now open) 2016-03-13T09:16:58.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42368 #4080661 (1507 connections now open) 2016-03-13T09:16:58.585+0000 I NETWORK [conn4080646] end connection 10.0.20.53:50627 (1506 connections now open) 2016-03-13T09:16:58.585+0000 I NETWORK [conn4080647] end connection 10.0.20.53:50626 (1506 connections now open) 2016-03-13T09:16:58.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50638 #4080662 (1506 connections now open) 2016-03-13T09:16:58.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50639 #4080663 (1507 connections now open) 2016-03-13T09:17:00.930+0000 I NETWORK [conn4080649] end connection 10.0.179.209:40384 (1506 connections now open) 2016-03-13T09:17:00.930+0000 I NETWORK [conn4080648] end connection 10.0.179.209:40385 (1506 connections now open) 2016-03-13T09:17:00.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40394 #4080664 (1506 connections now open) 2016-03-13T09:17:00.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40395 #4080665 (1507 connections now open) 2016-03-13T09:17:01.565+0000 I NETWORK [conn4080650] end connection 10.0.26.69:42361 (1506 connections now open) 2016-03-13T09:17:01.565+0000 I NETWORK [conn4080651] end connection 10.0.26.69:42362 (1506 connections now open) 2016-03-13T09:17:01.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42379 #4080666 (1506 connections now open) 2016-03-13T09:17:01.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42380 #4080667 (1507 connections now open) 2016-03-13T09:17:02.590+0000 I NETWORK [conn4079481] end connection 10.0.91.115:54429 (1506 connections now open) 2016-03-13T09:17:02.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:55150 #4080668 (1507 connections now open) 2016-03-13T09:17:04.393+0000 I NETWORK [conn4080652] end connection 10.0.27.237:35689 (1506 connections now open) 2016-03-13T09:17:04.393+0000 I NETWORK [conn4080653] end connection 10.0.27.237:35688 (1506 connections now open) 2016-03-13T09:17:04.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35712 #4080669 (1506 connections now open) 2016-03-13T09:17:04.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35711 #4080670 (1507 connections now open) 2016-03-13T09:17:05.984+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 673ms sequenceNumber: 16823 version: 30648|1||563ba74d869758a4542b9075 based on: 30647|1||563ba74d869758a4542b9075 2016-03-13T09:17:06.171+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:17:06.171+0000-56e5301218fa188537623d63", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860626171), what: "balancer.round", ns: "", details: { executionTimeMillis: 13220, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:17:06.336+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:17:06.521+0000 I NETWORK [conn4080655] end connection 10.0.7.113:59316 (1506 connections now open) 2016-03-13T09:17:06.521+0000 I NETWORK [conn4080654] end connection 10.0.7.113:59315 (1506 connections now open) 2016-03-13T09:17:06.522+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59332 #4080671 (1506 connections now open) 2016-03-13T09:17:06.522+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59333 #4080672 (1507 connections now open) 2016-03-13T09:17:07.566+0000 I NETWORK [conn4080656] end connection 10.0.216.90:57333 (1506 connections now open) 2016-03-13T09:17:07.566+0000 I NETWORK [conn4080657] end connection 10.0.216.90:57334 (1506 connections now open) 2016-03-13T09:17:07.567+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57349 #4080673 (1506 connections now open) 2016-03-13T09:17:07.567+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57348 #4080674 (1507 connections now open) 2016-03-13T09:17:07.704+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5301318fa188537623d64 2016-03-13T09:17:08.598+0000 I NETWORK [conn4080663] end connection 10.0.20.53:50639 (1506 connections now open) 2016-03-13T09:17:08.598+0000 I NETWORK [conn4080662] end connection 10.0.20.53:50638 (1506 connections now open) 2016-03-13T09:17:08.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50658 #4080675 (1506 connections now open) 2016-03-13T09:17:08.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50659 #4080676 (1507 connections now open) 2016-03-13T09:17:10.262+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56345d9e584ad10f1f9a10c6')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345d9e584ad10f1f9a10c6') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634605ca2a4330ea423620f') }, shard: "shmydomain1", lastmod: Timestamp 30648000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:17:10.418+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30648|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56345d9e584ad10f1f9a10c6') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634605ca2a4330ea423620f') }) shmydomain1 -> shmydomain2 2016-03-13T09:17:10.944+0000 I NETWORK [conn4080665] end connection 10.0.179.209:40395 (1506 connections now open) 2016-03-13T09:17:10.944+0000 I NETWORK [conn4080664] end connection 10.0.179.209:40394 (1506 connections now open) 2016-03-13T09:17:10.952+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40408 #4080677 (1506 connections now open) 2016-03-13T09:17:10.952+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40407 #4080678 (1507 connections now open) 2016-03-13T09:17:11.578+0000 I NETWORK [conn4080667] end connection 10.0.26.69:42380 (1506 connections now open) 2016-03-13T09:17:11.578+0000 I NETWORK [conn4080666] end connection 10.0.26.69:42379 (1506 connections now open) 2016-03-13T09:17:11.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42390 #4080679 (1506 connections now open) 2016-03-13T09:17:11.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42391 #4080680 (1507 connections now open) 2016-03-13T09:17:14.407+0000 I NETWORK [conn4080670] end connection 10.0.27.237:35711 (1506 connections now open) 2016-03-13T09:17:14.407+0000 I NETWORK [conn4080669] end connection 10.0.27.237:35712 (1506 connections now open) 2016-03-13T09:17:14.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35723 #4080681 (1506 connections now open) 2016-03-13T09:17:14.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35722 #4080682 (1507 connections now open) 2016-03-13T09:17:16.118+0000 I NETWORK [conn4074627] end connection 10.0.20.53:44336 (1506 connections now open) 2016-03-13T09:17:16.118+0000 I NETWORK [conn4068816] end connection 10.0.179.209:56660 (1506 connections now open) 2016-03-13T09:17:16.118+0000 I NETWORK [conn4072663] end connection 10.0.26.69:33937 (1504 connections now open) 2016-03-13T09:17:16.118+0000 I NETWORK [conn4072301] end connection 10.0.7.113:50520 (1503 connections now open) 2016-03-13T09:17:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40422 #4080683 (1504 connections now open) 2016-03-13T09:17:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50667 #4080684 (1505 connections now open) 2016-03-13T09:17:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42397 #4080685 (1506 connections now open) 2016-03-13T09:17:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59353 #4080686 (1507 connections now open) 2016-03-13T09:17:16.534+0000 I NETWORK [conn4080672] end connection 10.0.7.113:59333 (1506 connections now open) 2016-03-13T09:17:16.534+0000 I NETWORK [conn4080671] end connection 10.0.7.113:59332 (1506 connections now open) 2016-03-13T09:17:16.535+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59354 #4080687 (1506 connections now open) 2016-03-13T09:17:16.535+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59355 #4080688 (1507 connections now open) 2016-03-13T09:17:17.579+0000 I NETWORK [conn4080673] end connection 10.0.216.90:57349 (1506 connections now open) 2016-03-13T09:17:17.579+0000 I NETWORK [conn4080674] end connection 10.0.216.90:57348 (1506 connections now open) 2016-03-13T09:17:17.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57368 #4080689 (1506 connections now open) 2016-03-13T09:17:17.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57369 #4080690 (1507 connections now open) 2016-03-13T09:17:18.612+0000 I NETWORK [conn4080675] end connection 10.0.20.53:50658 (1506 connections now open) 2016-03-13T09:17:18.612+0000 I NETWORK [conn4080676] end connection 10.0.20.53:50659 (1506 connections now open) 2016-03-13T09:17:18.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50669 #4080691 (1506 connections now open) 2016-03-13T09:17:18.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50668 #4080692 (1507 connections now open) 2016-03-13T09:17:19.774+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:17:19.577+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:17:20.964+0000 I NETWORK [conn4080677] end connection 10.0.179.209:40408 (1506 connections now open) 2016-03-13T09:17:20.965+0000 I NETWORK [conn4080678] end connection 10.0.179.209:40407 (1505 connections now open) 2016-03-13T09:17:20.965+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40430 #4080693 (1506 connections now open) 2016-03-13T09:17:20.965+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40431 #4080694 (1507 connections now open) 2016-03-13T09:17:21.592+0000 I NETWORK [conn4080680] end connection 10.0.26.69:42391 (1506 connections now open) 2016-03-13T09:17:21.592+0000 I NETWORK [conn4080679] end connection 10.0.26.69:42390 (1506 connections now open) 2016-03-13T09:17:21.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42403 #4080695 (1506 connections now open) 2016-03-13T09:17:21.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42402 #4080696 (1507 connections now open) 2016-03-13T09:17:22.866+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 671ms sequenceNumber: 16824 version: 30649|1||563ba74d869758a4542b9075 based on: 30648|1||563ba74d869758a4542b9075 2016-03-13T09:17:23.179+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:17:23.179+0000-56e5302318fa188537623d65", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860643179), what: "balancer.round", ns: "", details: { executionTimeMillis: 15806, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:17:23.384+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:17:24.429+0000 I NETWORK [conn4080682] end connection 10.0.27.237:35722 (1506 connections now open) 2016-03-13T09:17:24.429+0000 I NETWORK [conn4080681] end connection 10.0.27.237:35723 (1506 connections now open) 2016-03-13T09:17:24.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35737 #4080697 (1506 connections now open) 2016-03-13T09:17:24.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35738 #4080698 (1507 connections now open) 2016-03-13T09:17:26.548+0000 I NETWORK [conn4080687] end connection 10.0.7.113:59354 (1506 connections now open) 2016-03-13T09:17:26.548+0000 I NETWORK [conn4080688] end connection 10.0.7.113:59355 (1506 connections now open) 2016-03-13T09:17:26.557+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59360 #4080699 (1506 connections now open) 2016-03-13T09:17:26.558+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59361 #4080700 (1507 connections now open) 2016-03-13T09:17:27.601+0000 I NETWORK [conn4080689] end connection 10.0.216.90:57368 (1506 connections now open) 2016-03-13T09:17:27.601+0000 I NETWORK [conn4080690] end connection 10.0.216.90:57369 (1506 connections now open) 2016-03-13T09:17:27.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57376 #4080701 (1506 connections now open) 2016-03-13T09:17:27.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57377 #4080702 (1507 connections now open) 2016-03-13T09:17:28.113+0000 I NETWORK [conn4071714] end connection 10.0.179.209:59556 (1506 connections now open) 2016-03-13T09:17:28.113+0000 I NETWORK [conn4077574] end connection 10.0.20.53:47415 (1505 connections now open) 2016-03-13T09:17:28.113+0000 I NETWORK [conn4068688] end connection 10.0.216.90:44645 (1504 connections now open) 2016-03-13T09:17:28.114+0000 I NETWORK [conn4069996] end connection 10.0.27.237:52959 (1503 connections now open) 2016-03-13T09:17:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40437 #4080703 (1504 connections now open) 2016-03-13T09:17:28.114+0000 I NETWORK [conn4071740] end connection 10.0.7.113:49911 (1503 connections now open) 2016-03-13T09:17:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50680 #4080704 (1504 connections now open) 2016-03-13T09:17:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57379 #4080705 (1505 connections now open) 2016-03-13T09:17:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35753 #4080706 (1506 connections now open) 2016-03-13T09:17:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59367 #4080707 (1507 connections now open) 2016-03-13T09:17:28.633+0000 I NETWORK [conn4080691] end connection 10.0.20.53:50669 (1506 connections now open) 2016-03-13T09:17:28.633+0000 I NETWORK [conn4080692] end connection 10.0.20.53:50668 (1506 connections now open) 2016-03-13T09:17:28.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50681 #4080708 (1506 connections now open) 2016-03-13T09:17:28.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50682 #4080709 (1507 connections now open) 2016-03-13T09:17:30.977+0000 I NETWORK [conn4080693] end connection 10.0.179.209:40430 (1506 connections now open) 2016-03-13T09:17:30.977+0000 I NETWORK [conn4080694] end connection 10.0.179.209:40431 (1505 connections now open) 2016-03-13T09:17:30.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40439 #4080710 (1506 connections now open) 2016-03-13T09:17:30.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40440 #4080711 (1507 connections now open) 2016-03-13T09:17:31.580+0000 I NETWORK [conn4071540] end connection 10.0.7.113:49710 (1506 connections now open) 2016-03-13T09:17:31.580+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59370 #4080712 (1507 connections now open) 2016-03-13T09:17:31.588+0000 I NETWORK [conn4072341] end connection 10.0.216.90:48556 (1506 connections now open) 2016-03-13T09:17:31.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57387 #4080713 (1507 connections now open) 2016-03-13T09:17:31.606+0000 I NETWORK [conn4080695] end connection 10.0.26.69:42403 (1506 connections now open) 2016-03-13T09:17:31.606+0000 I NETWORK [conn4080696] end connection 10.0.26.69:42402 (1505 connections now open) 2016-03-13T09:17:31.610+0000 I NETWORK [conn4070119] end connection 10.0.179.209:57976 (1504 connections now open) 2016-03-13T09:17:31.610+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40444 #4080714 (1505 connections now open) 2016-03-13T09:17:31.618+0000 I NETWORK [conn4080683] end connection 10.0.179.209:40422 (1504 connections now open) 2016-03-13T09:17:31.619+0000 I NETWORK [conn4075015] end connection 10.0.7.113:53418 (1503 connections now open) 2016-03-13T09:17:31.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40445 #4080715 (1504 connections now open) 2016-03-13T09:17:31.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59373 #4080716 (1505 connections now open) 2016-03-13T09:17:31.621+0000 I NETWORK [conn4070163] end connection 10.0.179.209:58016 (1504 connections now open) 2016-03-13T09:17:31.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40446 #4080717 (1505 connections now open) 2016-03-13T09:17:31.623+0000 I NETWORK [conn4071286] end connection 10.0.216.90:47427 (1504 connections now open) 2016-03-13T09:17:31.623+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57391 #4080718 (1505 connections now open) 2016-03-13T09:17:31.623+0000 I NETWORK [conn4078405] end connection 10.0.7.113:57035 (1504 connections now open) 2016-03-13T09:17:31.624+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59375 #4080719 (1505 connections now open) 2016-03-13T09:17:31.625+0000 I NETWORK [conn4075768] end connection 10.0.179.209:35452 (1504 connections now open) 2016-03-13T09:17:31.626+0000 I NETWORK [conn4071075] end connection 10.0.7.113:49217 (1503 connections now open) 2016-03-13T09:17:31.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40448 #4080720 (1504 connections now open) 2016-03-13T09:17:31.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59376 #4080721 (1505 connections now open) 2016-03-13T09:17:31.629+0000 I NETWORK [conn4076441] end connection 10.0.216.90:52982 (1504 connections now open) 2016-03-13T09:17:31.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57393 #4080722 (1505 connections now open) 2016-03-13T09:17:31.630+0000 I NETWORK [conn4072006] end connection 10.0.7.113:50201 (1504 connections now open) 2016-03-13T09:17:31.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59377 #4080723 (1505 connections now open) 2016-03-13T09:17:31.631+0000 I NETWORK [conn4072046] end connection 10.0.216.90:48247 (1504 connections now open) 2016-03-13T09:17:31.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57394 #4080724 (1505 connections now open) 2016-03-13T09:17:31.632+0000 I NETWORK [conn4069298] end connection 10.0.7.113:47294 (1504 connections now open) 2016-03-13T09:17:31.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59378 #4080725 (1505 connections now open) 2016-03-13T09:17:31.633+0000 I NETWORK [conn4076956] end connection 10.0.216.90:53530 (1504 connections now open) 2016-03-13T09:17:31.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57395 #4080726 (1505 connections now open) 2016-03-13T09:17:31.639+0000 I NETWORK [conn4072593] end connection 10.0.216.90:48828 (1504 connections now open) 2016-03-13T09:17:31.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57398 #4080727 (1505 connections now open) 2016-03-13T09:17:31.640+0000 I NETWORK [conn4079328] end connection 10.0.179.209:39036 (1504 connections now open) 2016-03-13T09:17:31.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40453 #4080728 (1505 connections now open) 2016-03-13T09:17:31.640+0000 I NETWORK [conn4071458] end connection 10.0.216.90:47610 (1504 connections now open) 2016-03-13T09:17:31.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57399 #4080729 (1505 connections now open) 2016-03-13T09:17:31.642+0000 I NETWORK [conn4069784] end connection 10.0.179.209:57641 (1504 connections now open) 2016-03-13T09:17:31.642+0000 I NETWORK [conn4073100] end connection 10.0.7.113:51362 (1503 connections now open) 2016-03-13T09:17:31.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40454 #4080730 (1504 connections now open) 2016-03-13T09:17:31.642+0000 I NETWORK [conn4072681] end connection 10.0.216.90:48920 (1503 connections now open) 2016-03-13T09:17:31.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59383 #4080731 (1504 connections now open) 2016-03-13T09:17:31.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57400 #4080732 (1505 connections now open) 2016-03-13T09:17:31.643+0000 I NETWORK [conn4067753] end connection 10.0.179.209:55584 (1504 connections now open) 2016-03-13T09:17:31.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40455 #4080733 (1505 connections now open) 2016-03-13T09:17:31.644+0000 I NETWORK [conn4072843] end connection 10.0.216.90:49099 (1504 connections now open) 2016-03-13T09:17:31.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57401 #4080734 (1505 connections now open) 2016-03-13T09:17:31.645+0000 I NETWORK [conn4079748] end connection 10.0.179.209:39466 (1504 connections now open) 2016-03-13T09:17:31.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40456 #4080735 (1505 connections now open) 2016-03-13T09:17:31.646+0000 I NETWORK [conn4069600] end connection 10.0.216.90:45608 (1504 connections now open) 2016-03-13T09:17:31.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57402 #4080736 (1505 connections now open) 2016-03-13T09:17:31.647+0000 I NETWORK [conn4075223] end connection 10.0.179.209:34902 (1504 connections now open) 2016-03-13T09:17:31.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40457 #4080737 (1505 connections now open) 2016-03-13T09:17:31.648+0000 I NETWORK [conn4069930] end connection 10.0.216.90:45970 (1504 connections now open) 2016-03-13T09:17:31.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57403 #4080738 (1505 connections now open) 2016-03-13T09:17:31.648+0000 I NETWORK [conn4072344] end connection 10.0.7.113:50568 (1503 connections now open) 2016-03-13T09:17:31.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59386 #4080739 (1505 connections now open) 2016-03-13T09:17:31.649+0000 I NETWORK [conn4072928] end connection 10.0.216.90:49190 (1504 connections now open) 2016-03-13T09:17:31.650+0000 I NETWORK [conn4073014] end connection 10.0.7.113:51279 (1503 connections now open) 2016-03-13T09:17:31.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57404 #4080740 (1504 connections now open) 2016-03-13T09:17:31.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59387 #4080741 (1505 connections now open) 2016-03-13T09:17:31.652+0000 I NETWORK [conn4070585] end connection 10.0.179.209:58434 (1504 connections now open) 2016-03-13T09:17:31.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40460 #4080742 (1505 connections now open) 2016-03-13T09:17:31.653+0000 I NETWORK [conn4072469] end connection 10.0.7.113:50703 (1504 connections now open) 2016-03-13T09:17:31.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59389 #4080743 (1505 connections now open) 2016-03-13T09:17:31.654+0000 I NETWORK [conn4080344] end connection 10.0.216.90:57026 (1504 connections now open) 2016-03-13T09:17:31.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57406 #4080744 (1505 connections now open) 2016-03-13T09:17:31.654+0000 I NETWORK [conn4068137] end connection 10.0.179.209:55978 (1503 connections now open) 2016-03-13T09:17:31.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40461 #4080745 (1505 connections now open) 2016-03-13T09:17:31.660+0000 I NETWORK [conn4080216] end connection 10.0.179.209:39944 (1504 connections now open) 2016-03-13T09:17:31.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40462 #4080746 (1505 connections now open) 2016-03-13T09:17:31.662+0000 I NETWORK [conn4072639] end connection 10.0.216.90:48874 (1504 connections now open) 2016-03-13T09:17:31.663+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57408 #4080747 (1505 connections now open) 2016-03-13T09:17:31.664+0000 I NETWORK [conn4078909] end connection 10.0.179.209:38615 (1504 connections now open) 2016-03-13T09:17:31.664+0000 I NETWORK [conn4072004] end connection 10.0.216.90:48202 (1503 connections now open) 2016-03-13T09:17:31.664+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40464 #4080748 (1504 connections now open) 2016-03-13T09:17:31.665+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57409 #4080749 (1505 connections now open) 2016-03-13T09:17:31.666+0000 I NETWORK [conn4068051] end connection 10.0.179.209:55892 (1504 connections now open) 2016-03-13T09:17:31.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40465 #4080750 (1505 connections now open) 2016-03-13T09:17:31.668+0000 I NETWORK [conn4078236] end connection 10.0.216.90:54887 (1504 connections now open) 2016-03-13T09:17:31.669+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57411 #4080751 (1505 connections now open) 2016-03-13T09:17:31.669+0000 I NETWORK [conn4071584] end connection 10.0.7.113:49757 (1504 connections now open) 2016-03-13T09:17:31.669+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59394 #4080752 (1505 connections now open) 2016-03-13T09:17:31.671+0000 I NETWORK [conn4071943] end connection 10.0.179.209:59790 (1504 connections now open) 2016-03-13T09:17:31.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40467 #4080753 (1505 connections now open) 2016-03-13T09:17:31.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42422 #4080754 (1506 connections now open) 2016-03-13T09:17:31.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42423 #4080755 (1507 connections now open) 2016-03-13T09:17:31.672+0000 I NETWORK [conn4073224] end connection 10.0.216.90:49499 (1506 connections now open) 2016-03-13T09:17:31.672+0000 I NETWORK [conn4076658] end connection 10.0.179.209:36342 (1505 connections now open) 2016-03-13T09:17:31.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57413 #4080756 (1506 connections now open) 2016-03-13T09:17:31.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40468 #4080757 (1507 connections now open) 2016-03-13T09:17:31.673+0000 I NETWORK [conn4071626] end connection 10.0.7.113:49797 (1506 connections now open) 2016-03-13T09:17:31.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59396 #4080758 (1507 connections now open) 2016-03-13T09:17:31.674+0000 I NETWORK [conn4070607] end connection 10.0.216.90:46695 (1506 connections now open) 2016-03-13T09:17:31.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57414 #4080759 (1507 connections now open) 2016-03-13T09:17:31.675+0000 I NETWORK [conn4075814] end connection 10.0.179.209:35496 (1506 connections now open) 2016-03-13T09:17:31.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40469 #4080760 (1507 connections now open) 2016-03-13T09:17:31.677+0000 I NETWORK [conn4071625] end connection 10.0.216.90:47789 (1506 connections now open) 2016-03-13T09:17:31.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57415 #4080761 (1507 connections now open) 2016-03-13T09:17:31.678+0000 I NETWORK [conn4074931] end connection 10.0.7.113:53327 (1506 connections now open) 2016-03-13T09:17:31.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59398 #4080762 (1507 connections now open) 2016-03-13T09:17:31.680+0000 I NETWORK [conn4071672] end connection 10.0.7.113:49846 (1506 connections now open) 2016-03-13T09:17:31.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59399 #4080763 (1507 connections now open) 2016-03-13T09:17:31.683+0000 I NETWORK [conn4071716] end connection 10.0.7.113:49889 (1506 connections now open) 2016-03-13T09:17:31.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59400 #4080764 (1507 connections now open) 2016-03-13T09:17:31.685+0000 I NETWORK [conn4069537] end connection 10.0.179.209:57388 (1506 connections now open) 2016-03-13T09:17:31.686+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40473 #4080765 (1507 connections now open) 2016-03-13T09:17:31.688+0000 I NETWORK [conn4070459] end connection 10.0.179.209:58308 (1506 connections now open) 2016-03-13T09:17:31.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40474 #4080766 (1507 connections now open) 2016-03-13T09:17:31.689+0000 I NETWORK [conn4074437] end connection 10.0.216.90:50801 (1506 connections now open) 2016-03-13T09:17:31.689+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57420 #4080767 (1507 connections now open) 2016-03-13T09:17:31.690+0000 I NETWORK [conn4077039] end connection 10.0.179.209:36722 (1506 connections now open) 2016-03-13T09:17:31.690+0000 I NETWORK [conn4072970] end connection 10.0.7.113:51235 (1505 connections now open) 2016-03-13T09:17:31.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40475 #4080768 (1506 connections now open) 2016-03-13T09:17:31.691+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59403 #4080769 (1507 connections now open) 2016-03-13T09:17:31.691+0000 I NETWORK [conn4070690] end connection 10.0.216.90:46788 (1506 connections now open) 2016-03-13T09:17:31.691+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57421 #4080770 (1507 connections now open) 2016-03-13T09:17:31.692+0000 I NETWORK [conn4076019] end connection 10.0.179.209:35703 (1506 connections now open) 2016-03-13T09:17:31.692+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40476 #4080771 (1507 connections now open) 2016-03-13T09:17:31.693+0000 I NETWORK [conn4074775] end connection 10.0.216.90:51168 (1506 connections now open) 2016-03-13T09:17:31.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57422 #4080772 (1507 connections now open) 2016-03-13T09:17:31.695+0000 I NETWORK [conn4075558] end connection 10.0.7.113:54011 (1506 connections now open) 2016-03-13T09:17:31.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59405 #4080773 (1507 connections now open) 2016-03-13T09:17:31.696+0000 I NETWORK [conn4068180] end connection 10.0.179.209:56019 (1506 connections now open) 2016-03-13T09:17:31.697+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40478 #4080774 (1507 connections now open) 2016-03-13T09:17:31.699+0000 I NETWORK [conn4072493] end connection 10.0.179.209:60326 (1506 connections now open) 2016-03-13T09:17:31.699+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40479 #4080775 (1507 connections now open) 2016-03-13T09:17:31.700+0000 I NETWORK [conn4074928] end connection 10.0.216.90:51342 (1506 connections now open) 2016-03-13T09:17:31.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57426 #4080776 (1507 connections now open) 2016-03-13T09:17:31.701+0000 I NETWORK [conn4072241] end connection 10.0.179.209:60078 (1506 connections now open) 2016-03-13T09:17:31.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40480 #4080777 (1507 connections now open) 2016-03-13T09:17:31.702+0000 I NETWORK [conn4073350] end connection 10.0.216.90:49630 (1506 connections now open) 2016-03-13T09:17:31.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57427 #4080778 (1507 connections now open) 2016-03-13T09:17:31.703+0000 I NETWORK [conn4072134] end connection 10.0.7.113:50339 (1506 connections now open) 2016-03-13T09:17:31.703+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59408 #4080779 (1507 connections now open) 2016-03-13T09:17:31.704+0000 I NETWORK [conn4075014] end connection 10.0.216.90:51432 (1506 connections now open) 2016-03-13T09:17:31.704+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57428 #4080780 (1507 connections now open) 2016-03-13T09:17:31.705+0000 I NETWORK [conn4078735] end connection 10.0.179.209:38441 (1506 connections now open) 2016-03-13T09:17:31.705+0000 I NETWORK [conn4073267] end connection 10.0.7.113:51536 (1505 connections now open) 2016-03-13T09:17:31.705+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40482 #4080781 (1506 connections now open) 2016-03-13T09:17:31.705+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59409 #4080782 (1507 connections now open) 2016-03-13T09:17:31.706+0000 I NETWORK [conn4073080] end connection 10.0.179.209:60925 (1506 connections now open) 2016-03-13T09:17:31.707+0000 I NETWORK [conn4072217] end connection 10.0.7.113:50429 (1505 connections now open) 2016-03-13T09:17:31.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40483 #4080783 (1507 connections now open) 2016-03-13T09:17:31.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59410 #4080784 (1507 connections now open) 2016-03-13T09:17:31.708+0000 I NETWORK [conn4070266] end connection 10.0.216.90:46327 (1506 connections now open) 2016-03-13T09:17:31.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57430 #4080785 (1507 connections now open) 2016-03-13T09:17:31.708+0000 I NETWORK [conn4077429] end connection 10.0.179.209:37106 (1506 connections now open) 2016-03-13T09:17:31.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40484 #4080786 (1507 connections now open) 2016-03-13T09:17:31.709+0000 I NETWORK [conn4075477] end connection 10.0.7.113:53922 (1505 connections now open) 2016-03-13T09:17:31.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59411 #4080787 (1507 connections now open) 2016-03-13T09:17:31.710+0000 I NETWORK [conn4074601] end connection 10.0.216.90:50984 (1506 connections now open) 2016-03-13T09:17:31.710+0000 I NETWORK [conn4077556] end connection 10.0.179.209:37235 (1505 connections now open) 2016-03-13T09:17:31.710+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57431 #4080788 (1506 connections now open) 2016-03-13T09:17:31.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40485 #4080789 (1507 connections now open) 2016-03-13T09:17:31.711+0000 I NETWORK [conn4073837] end connection 10.0.7.113:52158 (1505 connections now open) 2016-03-13T09:17:31.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59412 #4080790 (1507 connections now open) 2016-03-13T09:17:31.712+0000 I NETWORK [conn4075141] end connection 10.0.216.90:51568 (1506 connections now open) 2016-03-13T09:17:31.712+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57432 #4080791 (1507 connections now open) 2016-03-13T09:17:31.713+0000 I NETWORK [conn4077772] end connection 10.0.7.113:56347 (1506 connections now open) 2016-03-13T09:17:31.713+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59413 #4080792 (1507 connections now open) 2016-03-13T09:17:31.714+0000 I NETWORK [conn4074686] end connection 10.0.216.90:51078 (1506 connections now open) 2016-03-13T09:17:31.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57433 #4080793 (1507 connections now open) 2016-03-13T09:17:31.715+0000 I NETWORK [conn4079456] end connection 10.0.7.113:58113 (1506 connections now open) 2016-03-13T09:17:31.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59414 #4080794 (1507 connections now open) 2016-03-13T09:17:31.717+0000 I NETWORK [conn4074130] end connection 10.0.7.113:52469 (1506 connections now open) 2016-03-13T09:17:31.717+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59415 #4080795 (1507 connections now open) 2016-03-13T09:17:31.718+0000 I NETWORK [conn4073498] end connection 10.0.179.209:33115 (1506 connections now open) 2016-03-13T09:17:31.718+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40489 #4080796 (1507 connections now open) 2016-03-13T09:17:31.719+0000 I NETWORK [conn4073599] end connection 10.0.216.90:49900 (1506 connections now open) 2016-03-13T09:17:31.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57436 #4080797 (1507 connections now open) 2016-03-13T09:17:31.722+0000 I NETWORK [conn4072846] end connection 10.0.7.113:51101 (1506 connections now open) 2016-03-13T09:17:31.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59418 #4080798 (1507 connections now open) 2016-03-13T09:17:31.723+0000 I NETWORK [conn4080044] end connection 10.0.216.90:56731 (1506 connections now open) 2016-03-13T09:17:31.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57438 #4080799 (1507 connections now open) 2016-03-13T09:17:31.725+0000 I NETWORK [conn4077082] end connection 10.0.216.90:53668 (1506 connections now open) 2016-03-13T09:17:31.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57439 #4080800 (1507 connections now open) 2016-03-13T09:17:31.726+0000 I NETWORK [conn4074501] end connection 10.0.7.113:52878 (1506 connections now open) 2016-03-13T09:17:31.727+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59420 #4080801 (1507 connections now open) 2016-03-13T09:17:31.729+0000 I NETWORK [conn4073374] end connection 10.0.179.209:32986 (1506 connections now open) 2016-03-13T09:17:31.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40495 #4080802 (1507 connections now open) 2016-03-13T09:17:31.730+0000 I NETWORK [conn4072091] end connection 10.0.7.113:50295 (1506 connections now open) 2016-03-13T09:17:31.730+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59422 #4080803 (1507 connections now open) 2016-03-13T09:17:31.731+0000 I NETWORK [conn4074887] end connection 10.0.216.90:51300 (1506 connections now open) 2016-03-13T09:17:31.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57442 #4080804 (1507 connections now open) 2016-03-13T09:17:31.733+0000 I NETWORK [conn4078106] end connection 10.0.179.209:37796 (1506 connections now open) 2016-03-13T09:17:31.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40497 #4080805 (1507 connections now open) 2016-03-13T09:17:31.735+0000 I NETWORK [conn4073225] end connection 10.0.7.113:51490 (1506 connections now open) 2016-03-13T09:17:31.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59424 #4080806 (1507 connections now open) 2016-03-13T09:17:31.736+0000 I NETWORK [conn4073293] end connection 10.0.7.113:51562 (1506 connections now open) 2016-03-13T09:17:31.737+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59425 #4080807 (1507 connections now open) 2016-03-13T09:17:31.738+0000 I NETWORK [conn4070734] end connection 10.0.7.113:48852 (1506 connections now open) 2016-03-13T09:17:31.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59426 #4080808 (1507 connections now open) 2016-03-13T09:17:31.742+0000 I NETWORK [conn4075057] end connection 10.0.7.113:53459 (1506 connections now open) 2016-03-13T09:17:31.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59428 #4080809 (1507 connections now open) 2016-03-13T09:17:31.743+0000 I NETWORK [conn4073833] end connection 10.0.179.209:33452 (1506 connections now open) 2016-03-13T09:17:31.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40502 #4080810 (1507 connections now open) 2016-03-13T09:17:31.747+0000 I NETWORK [conn4075939] end connection 10.0.216.90:52434 (1506 connections now open) 2016-03-13T09:17:31.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57450 #4080811 (1507 connections now open) 2016-03-13T09:17:31.747+0000 I NETWORK [conn4079962] end connection 10.0.7.113:58624 (1506 connections now open) 2016-03-13T09:17:31.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59431 #4080812 (1507 connections now open) 2016-03-13T09:17:31.767+0000 I NETWORK [conn4076020] end connection 10.0.216.90:52524 (1506 connections now open) 2016-03-13T09:17:31.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57451 #4080813 (1507 connections now open) 2016-03-13T09:17:31.771+0000 I NETWORK [conn4079409] end connection 10.0.179.209:39119 (1506 connections now open) 2016-03-13T09:17:31.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40507 #4080814 (1507 connections now open) 2016-03-13T09:17:31.773+0000 I NETWORK [conn4069995] end connection 10.0.179.209:57854 (1506 connections now open) 2016-03-13T09:17:31.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40508 #4080815 (1507 connections now open) 2016-03-13T09:17:31.774+0000 I NETWORK [conn4077471] end connection 10.0.7.113:56032 (1506 connections now open) 2016-03-13T09:17:31.774+0000 I NETWORK [conn4078489] end connection 10.0.216.90:55162 (1505 connections now open) 2016-03-13T09:17:31.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59435 #4080816 (1506 connections now open) 2016-03-13T09:17:31.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57454 #4080817 (1507 connections now open) 2016-03-13T09:17:31.777+0000 I NETWORK [conn4075353] end connection 10.0.7.113:53783 (1506 connections now open) 2016-03-13T09:17:31.778+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59437 #4080818 (1507 connections now open) 2016-03-13T09:17:31.779+0000 I NETWORK [conn4070417] end connection 10.0.179.209:58264 (1506 connections now open) 2016-03-13T09:17:31.780+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40511 #4080819 (1507 connections now open) 2016-03-13T09:17:31.780+0000 I NETWORK [conn4076062] end connection 10.0.216.90:52570 (1506 connections now open) 2016-03-13T09:17:31.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57456 #4080820 (1507 connections now open) 2016-03-13T09:17:31.781+0000 I NETWORK [conn4074297] end connection 10.0.179.209:33940 (1506 connections now open) 2016-03-13T09:17:31.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40512 #4080821 (1507 connections now open) 2016-03-13T09:17:31.784+0000 I NETWORK [conn4080302] end connection 10.0.179.209:40028 (1506 connections now open) 2016-03-13T09:17:31.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40514 #4080822 (1507 connections now open) 2016-03-13T09:17:31.785+0000 I NETWORK [conn4076269] end connection 10.0.7.113:54769 (1506 connections now open) 2016-03-13T09:17:31.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59441 #4080823 (1507 connections now open) 2016-03-13T09:17:31.786+0000 I NETWORK [conn4073181] end connection 10.0.216.90:49457 (1506 connections now open) 2016-03-13T09:17:31.786+0000 I NETWORK [conn4071796] end connection 10.0.179.209:59640 (1505 connections now open) 2016-03-13T09:17:31.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57459 #4080824 (1506 connections now open) 2016-03-13T09:17:31.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40516 #4080825 (1507 connections now open) 2016-03-13T09:17:31.787+0000 I NETWORK [conn4073625] end connection 10.0.7.113:51929 (1506 connections now open) 2016-03-13T09:17:31.788+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59442 #4080826 (1507 connections now open) 2016-03-13T09:17:31.788+0000 I NETWORK [conn4071394] end connection 10.0.179.209:59241 (1506 connections now open) 2016-03-13T09:17:31.788+0000 I NETWORK [conn4077554] end connection 10.0.216.90:54166 (1505 connections now open) 2016-03-13T09:17:31.789+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40517 #4080827 (1506 connections now open) 2016-03-13T09:17:31.789+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57460 #4080828 (1507 connections now open) 2016-03-13T09:17:31.791+0000 I NETWORK [conn4078952] end connection 10.0.179.209:38659 (1506 connections now open) 2016-03-13T09:17:31.791+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40518 #4080829 (1507 connections now open) 2016-03-13T09:17:31.792+0000 I NETWORK [conn4068747] end connection 10.0.179.209:56590 (1506 connections now open) 2016-03-13T09:17:31.793+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40519 #4080830 (1507 connections now open) 2016-03-13T09:17:31.793+0000 I NETWORK [conn4075555] end connection 10.0.179.209:35239 (1506 connections now open) 2016-03-13T09:17:31.794+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40520 #4080831 (1507 connections now open) 2016-03-13T09:17:31.795+0000 I NETWORK [conn4073057] end connection 10.0.7.113:51317 (1506 connections now open) 2016-03-13T09:17:31.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59446 #4080832 (1507 connections now open) 2016-03-13T09:17:31.796+0000 I NETWORK [conn4071819] end connection 10.0.179.209:59662 (1506 connections now open) 2016-03-13T09:17:31.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40522 #4080833 (1507 connections now open) 2016-03-13T09:17:31.798+0000 I NETWORK [conn4072385] end connection 10.0.179.209:60216 (1506 connections now open) 2016-03-13T09:17:31.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40523 #4080834 (1507 connections now open) 2016-03-13T09:17:31.800+0000 I NETWORK [conn4075859] end connection 10.0.7.113:54328 (1506 connections now open) 2016-03-13T09:17:31.800+0000 I NETWORK [conn4075224] end connection 10.0.216.90:51659 (1505 connections now open) 2016-03-13T09:17:31.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59449 #4080835 (1506 connections now open) 2016-03-13T09:17:31.800+0000 I NETWORK [conn4071010] end connection 10.0.179.209:58856 (1505 connections now open) 2016-03-13T09:17:31.801+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57465 #4080836 (1506 connections now open) 2016-03-13T09:17:31.801+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40526 #4080837 (1507 connections now open) 2016-03-13T09:17:31.802+0000 I NETWORK [conn4073053] end connection 10.0.179.209:60895 (1506 connections now open) 2016-03-13T09:17:31.802+0000 I NETWORK [conn4076160] end connection 10.0.7.113:54650 (1505 connections now open) 2016-03-13T09:17:31.802+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40527 #4080838 (1506 connections now open) 2016-03-13T09:17:31.802+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59451 #4080839 (1507 connections now open) 2016-03-13T09:17:31.804+0000 I NETWORK [conn4069110] end connection 10.0.179.209:56962 (1506 connections now open) 2016-03-13T09:17:31.804+0000 I NETWORK [conn4076485] end connection 10.0.216.90:53027 (1505 connections now open) 2016-03-13T09:17:31.804+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40529 #4080840 (1506 connections now open) 2016-03-13T09:17:31.804+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57467 #4080841 (1507 connections now open) 2016-03-13T09:17:31.805+0000 I NETWORK [conn4078779] end connection 10.0.179.209:38485 (1506 connections now open) 2016-03-13T09:17:31.805+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40530 #4080842 (1507 connections now open) 2016-03-13T09:17:31.806+0000 I NETWORK [conn4074145] end connection 10.0.179.209:33772 (1506 connections now open) 2016-03-13T09:17:31.806+0000 I NETWORK [conn4076975] end connection 10.0.7.113:55512 (1505 connections now open) 2016-03-13T09:17:31.806+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40531 #4080843 (1506 connections now open) 2016-03-13T09:17:31.807+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59454 #4080844 (1507 connections now open) 2016-03-13T09:17:31.808+0000 I NETWORK [conn4077385] end connection 10.0.216.90:53982 (1506 connections now open) 2016-03-13T09:17:31.808+0000 I NETWORK [conn4073390] end connection 10.0.179.209:32999 (1506 connections now open) 2016-03-13T09:17:31.808+0000 I NETWORK [conn4069997] end connection 10.0.7.113:48057 (1504 connections now open) 2016-03-13T09:17:31.808+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57469 #4080845 (1505 connections now open) 2016-03-13T09:17:31.808+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40533 #4080846 (1506 connections now open) 2016-03-13T09:17:31.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59456 #4080847 (1507 connections now open) 2016-03-13T09:17:31.810+0000 I NETWORK [conn4079198] end connection 10.0.179.209:38912 (1506 connections now open) 2016-03-13T09:17:31.810+0000 I NETWORK [conn4076225] end connection 10.0.216.90:52757 (1505 connections now open) 2016-03-13T09:17:31.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40534 #4080848 (1506 connections now open) 2016-03-13T09:17:31.810+0000 I NETWORK [conn4071099] end connection 10.0.7.113:49232 (1505 connections now open) 2016-03-13T09:17:31.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57470 #4080849 (1506 connections now open) 2016-03-13T09:17:31.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59458 #4080850 (1507 connections now open) 2016-03-13T09:17:31.811+0000 I NETWORK [conn4077856] end connection 10.0.7.113:56436 (1506 connections now open) 2016-03-13T09:17:31.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59459 #4080851 (1507 connections now open) 2016-03-13T09:17:31.815+0000 I NETWORK [conn4076102] end connection 10.0.7.113:54584 (1506 connections now open) 2016-03-13T09:17:31.816+0000 I NETWORK [conn4075476] end connection 10.0.216.90:51936 (1505 connections now open) 2016-03-13T09:17:31.816+0000 I NETWORK [conn4072157] end connection 10.0.179.209:59997 (1504 connections now open) 2016-03-13T09:17:31.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59463 #4080852 (1505 connections now open) 2016-03-13T09:17:31.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57473 #4080853 (1506 connections now open) 2016-03-13T09:17:31.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40540 #4080854 (1507 connections now open) 2016-03-13T09:17:31.816+0000 I NETWORK [conn4071413] end connection 10.0.179.209:59267 (1506 connections now open) 2016-03-13T09:17:31.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40541 #4080855 (1507 connections now open) 2016-03-13T09:17:31.818+0000 I NETWORK [conn4071609] end connection 10.0.179.209:59451 (1506 connections now open) 2016-03-13T09:17:31.818+0000 I NETWORK [conn4073958] end connection 10.0.7.113:52290 (1505 connections now open) 2016-03-13T09:17:31.818+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40542 #4080856 (1506 connections now open) 2016-03-13T09:17:31.818+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59465 #4080857 (1507 connections now open) 2016-03-13T09:17:31.818+0000 I NETWORK [conn4071372] end connection 10.0.179.209:59221 (1505 connections now open) 2016-03-13T09:17:31.819+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40543 #4080858 (1507 connections now open) 2016-03-13T09:17:31.820+0000 I NETWORK [conn4072721] end connection 10.0.179.209:60551 (1506 connections now open) 2016-03-13T09:17:31.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40545 #4080859 (1507 connections now open) 2016-03-13T09:17:31.821+0000 I NETWORK [conn4076697] end connection 10.0.216.90:53252 (1506 connections now open) 2016-03-13T09:17:31.822+0000 I NETWORK [conn4075306] end connection 10.0.179.209:34989 (1505 connections now open) 2016-03-13T09:17:31.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57476 #4080860 (1506 connections now open) 2016-03-13T09:17:31.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40546 #4080861 (1507 connections now open) 2016-03-13T09:17:31.823+0000 I NETWORK [conn4075900] end connection 10.0.216.90:52391 (1506 connections now open) 2016-03-13T09:17:31.824+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57477 #4080862 (1507 connections now open) 2016-03-13T09:17:31.824+0000 I NETWORK [conn4068617] end connection 10.0.179.209:56463 (1506 connections now open) 2016-03-13T09:17:31.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40549 #4080863 (1507 connections now open) 2016-03-13T09:17:31.825+0000 I NETWORK [conn4078447] end connection 10.0.216.90:55118 (1506 connections now open) 2016-03-13T09:17:31.825+0000 I NETWORK [conn4071056] end connection 10.0.7.113:49192 (1505 connections now open) 2016-03-13T09:17:31.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57478 #4080864 (1506 connections now open) 2016-03-13T09:17:31.826+0000 I NETWORK [conn4079454] end connection 10.0.179.209:39162 (1505 connections now open) 2016-03-13T09:17:31.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59472 #4080865 (1506 connections now open) 2016-03-13T09:17:31.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40550 #4080866 (1507 connections now open) 2016-03-13T09:17:31.827+0000 I NETWORK [conn4078865] end connection 10.0.216.90:55557 (1506 connections now open) 2016-03-13T09:17:31.827+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57479 #4080867 (1507 connections now open) 2016-03-13T09:17:31.828+0000 I NETWORK [conn4075643] end connection 10.0.179.209:35326 (1506 connections now open) 2016-03-13T09:17:31.828+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40552 #4080868 (1507 connections now open) 2016-03-13T09:17:31.829+0000 I NETWORK [conn4075727] end connection 10.0.216.90:52207 (1506 connections now open) 2016-03-13T09:17:31.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57480 #4080869 (1507 connections now open) 2016-03-13T09:17:31.830+0000 I NETWORK [conn4070969] end connection 10.0.7.113:49098 (1506 connections now open) 2016-03-13T09:17:31.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59476 #4080870 (1507 connections now open) 2016-03-13T09:17:31.833+0000 I NETWORK [conn4079669] end connection 10.0.216.90:56349 (1506 connections now open) 2016-03-13T09:17:31.833+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57482 #4080871 (1507 connections now open) 2016-03-13T09:17:31.834+0000 I NETWORK [conn4076288] end connection 10.0.7.113:54791 (1506 connections now open) 2016-03-13T09:17:31.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59480 #4080872 (1507 connections now open) 2016-03-13T09:17:31.834+0000 I NETWORK [conn4074046] end connection 10.0.7.113:52379 (1506 connections now open) 2016-03-13T09:17:31.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59481 #4080873 (1507 connections now open) 2016-03-13T09:17:31.836+0000 I NETWORK [conn4074089] end connection 10.0.7.113:52427 (1506 connections now open) 2016-03-13T09:17:31.837+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59483 #4080874 (1507 connections now open) 2016-03-13T09:17:31.838+0000 I NETWORK [conn4075155] end connection 10.0.7.113:53564 (1506 connections now open) 2016-03-13T09:17:31.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59484 #4080875 (1507 connections now open) 2016-03-13T09:17:31.840+0000 I NETWORK [conn4074172] end connection 10.0.7.113:52516 (1506 connections now open) 2016-03-13T09:17:31.840+0000 I NETWORK [conn4074274] end connection 10.0.7.113:52621 (1505 connections now open) 2016-03-13T09:17:31.840+0000 I NETWORK [conn4074104] end connection 10.0.216.90:50444 (1504 connections now open) 2016-03-13T09:17:31.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59485 #4080876 (1505 connections now open) 2016-03-13T09:17:31.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57485 #4080877 (1506 connections now open) 2016-03-13T09:17:31.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59486 #4080878 (1507 connections now open) 2016-03-13T09:17:31.842+0000 I NETWORK [conn4077257] end connection 10.0.216.90:53850 (1506 connections now open) 2016-03-13T09:17:31.843+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57486 #4080879 (1507 connections now open) 2016-03-13T09:17:31.843+0000 I NETWORK [conn4074544] end connection 10.0.26.69:35939 (1506 connections now open) 2016-03-13T09:17:31.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42426 #4080880 (1507 connections now open) 2016-03-13T09:17:31.844+0000 I NETWORK [conn4076742] end connection 10.0.7.113:55264 (1506 connections now open) 2016-03-13T09:17:31.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59490 #4080881 (1507 connections now open) 2016-03-13T09:17:31.846+0000 I NETWORK [conn4072869] end connection 10.0.7.113:51129 (1506 connections now open) 2016-03-13T09:17:31.847+0000 I NETWORK [conn4071139] end connection 10.0.179.209:58985 (1505 connections now open) 2016-03-13T09:17:31.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59491 #4080882 (1507 connections now open) 2016-03-13T09:17:31.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40569 #4080883 (1507 connections now open) 2016-03-13T09:17:31.848+0000 I NETWORK [conn4076716] end connection 10.0.179.209:36410 (1506 connections now open) 2016-03-13T09:17:31.848+0000 I NETWORK [conn4073475] end connection 10.0.7.113:51761 (1505 connections now open) 2016-03-13T09:17:31.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40570 #4080884 (1506 connections now open) 2016-03-13T09:17:31.849+0000 I NETWORK [conn4079834] end connection 10.0.179.209:39549 (1505 connections now open) 2016-03-13T09:17:31.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59493 #4080885 (1507 connections now open) 2016-03-13T09:17:31.849+0000 I NETWORK [conn4078780] end connection 10.0.216.90:55469 (1505 connections now open) 2016-03-13T09:17:31.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40571 #4080886 (1506 connections now open) 2016-03-13T09:17:31.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57489 #4080887 (1507 connections now open) 2016-03-13T09:17:31.850+0000 I NETWORK [conn4073768] end connection 10.0.179.209:33383 (1506 connections now open) 2016-03-13T09:17:31.850+0000 I NETWORK [conn4072995] end connection 10.0.7.113:51265 (1505 connections now open) 2016-03-13T09:17:31.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40572 #4080888 (1506 connections now open) 2016-03-13T09:17:31.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59495 #4080889 (1507 connections now open) 2016-03-13T09:17:31.852+0000 I NETWORK [conn4073291] end connection 10.0.179.209:32900 (1506 connections now open) 2016-03-13T09:17:31.852+0000 I NETWORK [conn4080004] end connection 10.0.7.113:58668 (1505 connections now open) 2016-03-13T09:17:31.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40573 #4080890 (1506 connections now open) 2016-03-13T09:17:31.852+0000 I NETWORK [conn4072885] end connection 10.0.179.209:60722 (1505 connections now open) 2016-03-13T09:17:31.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59496 #4080891 (1507 connections now open) 2016-03-13T09:17:31.852+0000 I NETWORK [conn4072495] end connection 10.0.7.113:50725 (1505 connections now open) 2016-03-13T09:17:31.853+0000 I NETWORK [conn4075600] end connection 10.0.216.90:52065 (1504 connections now open) 2016-03-13T09:17:31.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40574 #4080892 (1506 connections now open) 2016-03-13T09:17:31.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59497 #4080893 (1506 connections now open) 2016-03-13T09:17:31.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57491 #4080894 (1507 connections now open) 2016-03-13T09:17:31.854+0000 I NETWORK [conn4077346] end connection 10.0.7.113:55889 (1506 connections now open) 2016-03-13T09:17:31.854+0000 I NETWORK [conn4078420] end connection 10.0.179.209:38111 (1505 connections now open) 2016-03-13T09:17:31.854+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59498 #4080895 (1506 connections now open) 2016-03-13T09:17:31.854+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40575 #4080896 (1507 connections now open) 2016-03-13T09:17:31.854+0000 I NETWORK [conn4076309] end connection 10.0.216.90:52847 (1506 connections now open) 2016-03-13T09:17:31.855+0000 I NETWORK [conn4076226] end connection 10.0.26.69:37768 (1505 connections now open) 2016-03-13T09:17:31.855+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57492 #4080897 (1506 connections now open) 2016-03-13T09:17:31.855+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42430 #4080898 (1507 connections now open) 2016-03-13T09:17:31.856+0000 I NETWORK [conn4077388] end connection 10.0.7.113:55939 (1506 connections now open) 2016-03-13T09:17:31.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59500 #4080899 (1507 connections now open) 2016-03-13T09:17:31.857+0000 I NETWORK [conn4078279] end connection 10.0.216.90:54934 (1506 connections now open) 2016-03-13T09:17:31.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57493 #4080900 (1507 connections now open) 2016-03-13T09:17:31.857+0000 I NETWORK [conn4074756] end connection 10.0.179.209:34429 (1506 connections now open) 2016-03-13T09:17:31.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40578 #4080901 (1507 connections now open) 2016-03-13T09:17:31.859+0000 I NETWORK [conn4071861] end connection 10.0.179.209:59705 (1506 connections now open) 2016-03-13T09:17:31.859+0000 I NETWORK [conn4077318] end connection 10.0.20.53:47148 (1505 connections now open) 2016-03-13T09:17:31.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40580 #4080902 (1507 connections now open) 2016-03-13T09:17:31.859+0000 I NETWORK [conn4076266] end connection 10.0.216.90:52801 (1505 connections now open) 2016-03-13T09:17:31.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57494 #4080903 (1506 connections now open) 2016-03-13T09:17:31.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50689 #4080904 (1507 connections now open) 2016-03-13T09:17:31.860+0000 I NETWORK [conn4072218] end connection 10.0.27.237:55326 (1506 connections now open) 2016-03-13T09:17:31.860+0000 I NETWORK [conn4074859] end connection 10.0.179.209:34528 (1505 connections now open) 2016-03-13T09:17:31.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35757 #4080905 (1506 connections now open) 2016-03-13T09:17:31.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40581 #4080906 (1507 connections now open) 2016-03-13T09:17:31.861+0000 I NETWORK [conn4073250] end connection 10.0.7.113:51522 (1505 connections now open) 2016-03-13T09:17:31.861+0000 I NETWORK [conn4077641] end connection 10.0.179.209:37321 (1505 connections now open) 2016-03-13T09:17:31.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59505 #4080907 (1506 connections now open) 2016-03-13T09:17:31.861+0000 I NETWORK [conn4077686] end connection 10.0.216.90:54301 (1505 connections now open) 2016-03-13T09:17:31.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40582 #4080908 (1506 connections now open) 2016-03-13T09:17:31.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57495 #4080909 (1507 connections now open) 2016-03-13T09:17:31.862+0000 I NETWORK [conn4072592] end connection 10.0.179.209:60427 (1506 connections now open) 2016-03-13T09:17:31.863+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40583 #4080910 (1507 connections now open) 2016-03-13T09:17:31.863+0000 I NETWORK [conn4073541] end connection 10.0.7.113:51838 (1506 connections now open) 2016-03-13T09:17:31.863+0000 I NETWORK [conn4078446] end connection 10.0.179.209:38138 (1505 connections now open) 2016-03-13T09:17:31.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59507 #4080911 (1506 connections now open) 2016-03-13T09:17:31.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40584 #4080912 (1507 connections now open) 2016-03-13T09:17:31.865+0000 I NETWORK [conn4070121] end connection 10.0.26.69:59439 (1506 connections now open) 2016-03-13T09:17:31.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42435 #4080913 (1507 connections now open) 2016-03-13T09:17:31.866+0000 I NETWORK [conn4075226] end connection 10.0.7.113:53643 (1506 connections now open) 2016-03-13T09:17:31.866+0000 I NETWORK [conn4073683] end connection 10.0.216.90:49993 (1505 connections now open) 2016-03-13T09:17:31.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59509 #4080914 (1506 connections now open) 2016-03-13T09:17:31.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57497 #4080915 (1507 connections now open) 2016-03-13T09:17:31.867+0000 I NETWORK [conn4072952] end connection 10.0.26.69:34232 (1506 connections now open) 2016-03-13T09:17:31.867+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42436 #4080916 (1507 connections now open) 2016-03-13T09:17:31.867+0000 I NETWORK [conn4073310] end connection 10.0.7.113:51576 (1505 connections now open) 2016-03-13T09:17:31.868+0000 I NETWORK [conn4076868] end connection 10.0.216.90:53439 (1505 connections now open) 2016-03-13T09:17:31.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59510 #4080917 (1506 connections now open) 2016-03-13T09:17:31.868+0000 I NETWORK [conn4077643] end connection 10.0.7.113:56221 (1505 connections now open) 2016-03-13T09:17:31.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57498 #4080918 (1506 connections now open) 2016-03-13T09:17:31.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59511 #4080919 (1507 connections now open) 2016-03-13T09:17:31.868+0000 I NETWORK [conn4073137] end connection 10.0.179.209:60978 (1506 connections now open) 2016-03-13T09:17:31.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40588 #4080920 (1507 connections now open) 2016-03-13T09:17:31.872+0000 I NETWORK [conn4070246] end connection 10.0.26.69:59576 (1506 connections now open) 2016-03-13T09:17:31.872+0000 I NETWORK [conn4072791] end connection 10.0.7.113:51039 (1506 connections now open) 2016-03-13T09:17:31.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42437 #4080921 (1506 connections now open) 2016-03-13T09:17:31.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59512 #4080922 (1507 connections now open) 2016-03-13T09:17:31.873+0000 I NETWORK [conn4073265] end connection 10.0.179.209:32873 (1506 connections now open) 2016-03-13T09:17:31.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40590 #4080923 (1507 connections now open) 2016-03-13T09:17:31.875+0000 I NETWORK [conn4077813] end connection 10.0.216.90:54435 (1506 connections now open) 2016-03-13T09:17:31.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57500 #4080924 (1507 connections now open) 2016-03-13T09:17:31.875+0000 I NETWORK [conn4077897] end connection 10.0.179.209:37580 (1505 connections now open) 2016-03-13T09:17:31.875+0000 I NETWORK [conn4073640] end connection 10.0.179.209:33253 (1505 connections now open) 2016-03-13T09:17:31.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40591 #4080925 (1506 connections now open) 2016-03-13T09:17:31.876+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40592 #4080926 (1507 connections now open) 2016-03-13T09:17:31.876+0000 I NETWORK [conn4072258] end connection 10.0.27.237:55372 (1506 connections now open) 2016-03-13T09:17:31.876+0000 I NETWORK [conn4074846] end connection 10.0.26.69:36257 (1505 connections now open) 2016-03-13T09:17:31.876+0000 I NETWORK [conn4076911] end connection 10.0.216.90:53484 (1504 connections now open) 2016-03-13T09:17:31.876+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35762 #4080927 (1506 connections now open) 2016-03-13T09:17:31.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42439 #4080928 (1506 connections now open) 2016-03-13T09:17:31.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57501 #4080929 (1507 connections now open) 2016-03-13T09:17:31.877+0000 I NETWORK [conn4073914] end connection 10.0.7.113:52247 (1506 connections now open) 2016-03-13T09:17:31.877+0000 I NETWORK [conn4072535] end connection 10.0.179.209:60368 (1505 connections now open) 2016-03-13T09:17:31.877+0000 I NETWORK [conn4073431] end connection 10.0.179.209:33046 (1504 connections now open) 2016-03-13T09:17:31.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59517 #4080930 (1505 connections now open) 2016-03-13T09:17:31.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40593 #4080931 (1506 connections now open) 2016-03-13T09:17:31.878+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40594 #4080932 (1507 connections now open) 2016-03-13T09:17:31.878+0000 I NETWORK [conn4071864] end connection 10.0.26.69:33076 (1506 connections now open) 2016-03-13T09:17:31.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42440 #4080933 (1507 connections now open) 2016-03-13T09:17:31.879+0000 I NETWORK [conn4072215] end connection 10.0.20.53:41789 (1506 connections now open) 2016-03-13T09:17:31.879+0000 I NETWORK [conn4078237] end connection 10.0.7.113:56851 (1505 connections now open) 2016-03-13T09:17:31.879+0000 I NETWORK [conn4073750] end connection 10.0.179.209:33366 (1504 connections now open) 2016-03-13T09:17:31.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50692 #4080934 (1506 connections now open) 2016-03-13T09:17:31.880+0000 I NETWORK [conn4077211] end connection 10.0.216.90:53804 (1504 connections now open) 2016-03-13T09:17:31.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59519 #4080935 (1505 connections now open) 2016-03-13T09:17:31.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40595 #4080936 (1506 connections now open) 2016-03-13T09:17:31.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57502 #4080937 (1507 connections now open) 2016-03-13T09:17:31.880+0000 I NETWORK [conn4073667] end connection 10.0.26.69:34993 (1506 connections now open) 2016-03-13T09:17:31.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42441 #4080938 (1507 connections now open) 2016-03-13T09:17:31.881+0000 I NETWORK [conn4073641] end connection 10.0.20.53:43296 (1506 connections now open) 2016-03-13T09:17:31.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50693 #4080939 (1507 connections now open) 2016-03-13T09:17:31.881+0000 I NETWORK [conn4072281] end connection 10.0.179.209:60119 (1506 connections now open) 2016-03-13T09:17:31.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40597 #4080940 (1507 connections now open) 2016-03-13T09:17:31.883+0000 I NETWORK [conn4076400] end connection 10.0.179.209:36091 (1506 connections now open) 2016-03-13T09:17:31.883+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40599 #4080941 (1507 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [conn4069703] end connection 10.0.7.113:47742 (1506 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [conn4074716] end connection 10.0.7.113:53099 (1505 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [conn4080150] end connection 10.0.179.209:39870 (1504 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [conn4075618] end connection 10.0.27.237:58815 (1504 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59522 #4080942 (1504 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59523 #4080943 (1505 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [conn4080262] end connection 10.0.216.90:56944 (1504 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40600 #4080944 (1505 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35766 #4080945 (1506 connections now open) 2016-03-13T09:17:31.884+0000 I NETWORK [conn4072257] end connection 10.0.20.53:41830 (1504 connections now open) 2016-03-13T09:17:31.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57504 #4080946 (1506 connections now open) 2016-03-13T09:17:31.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50695 #4080947 (1507 connections now open) 2016-03-13T09:17:31.886+0000 I NETWORK [conn4072216] end connection 10.0.179.209:60060 (1506 connections now open) 2016-03-13T09:17:31.886+0000 I NETWORK [conn4077512] end connection 10.0.216.90:54121 (1505 connections now open) 2016-03-13T09:17:31.886+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40602 #4080948 (1506 connections now open) 2016-03-13T09:17:31.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57505 #4080949 (1507 connections now open) 2016-03-13T09:17:31.888+0000 I NETWORK [conn4069282] end connection 10.0.7.113:47283 (1506 connections now open) 2016-03-13T09:17:31.888+0000 I NETWORK [conn4072299] end connection 10.0.27.237:55415 (1505 connections now open) 2016-03-13T09:17:31.888+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59526 #4080950 (1506 connections now open) 2016-03-13T09:17:31.888+0000 I NETWORK [conn4077898] end connection 10.0.216.90:54524 (1505 connections now open) 2016-03-13T09:17:31.888+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35768 #4080951 (1506 connections now open) 2016-03-13T09:17:31.889+0000 I NETWORK [conn4076397] end connection 10.0.7.113:54901 (1505 connections now open) 2016-03-13T09:17:31.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57506 #4080952 (1506 connections now open) 2016-03-13T09:17:31.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59527 #4080953 (1507 connections now open) 2016-03-13T09:17:31.889+0000 I NETWORK [conn4080447] end connection 10.0.20.53:50398 (1506 connections now open) 2016-03-13T09:17:31.890+0000 I NETWORK [conn4074019] end connection 10.0.27.237:57209 (1505 connections now open) 2016-03-13T09:17:31.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50697 #4080954 (1506 connections now open) 2016-03-13T09:17:31.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35769 #4080955 (1507 connections now open) 2016-03-13T09:17:31.890+0000 I NETWORK [conn4073556] end connection 10.0.179.209:33169 (1506 connections now open) 2016-03-13T09:17:31.891+0000 I NETWORK [conn4073459] end connection 10.0.26.69:34762 (1505 connections now open) 2016-03-13T09:17:31.891+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40606 #4080956 (1507 connections now open) 2016-03-13T09:17:31.891+0000 I NETWORK [conn4072386] end connection 10.0.20.53:41953 (1505 connections now open) 2016-03-13T09:17:31.891+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42446 #4080957 (1507 connections now open) 2016-03-13T09:17:31.891+0000 I NETWORK [conn4076527] end connection 10.0.179.209:36217 (1505 connections now open) 2016-03-13T09:17:31.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50698 #4080958 (1506 connections now open) 2016-03-13T09:17:31.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40607 #4080959 (1507 connections now open) 2016-03-13T09:17:31.894+0000 I NETWORK [conn4080386] end connection 10.0.179.209:40115 (1506 connections now open) 2016-03-13T09:17:31.895+0000 I NETWORK [conn4071755] end connection 10.0.179.209:59601 (1505 connections now open) 2016-03-13T09:17:31.895+0000 I NETWORK [conn4070164] end connection 10.0.26.69:59484 (1504 connections now open) 2016-03-13T09:17:31.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40609 #4080960 (1505 connections now open) 2016-03-13T09:17:31.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40610 #4080961 (1506 connections now open) 2016-03-13T09:17:31.895+0000 I NETWORK [conn4075518] end connection 10.0.7.113:53970 (1504 connections now open) 2016-03-13T09:17:31.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42448 #4080962 (1506 connections now open) 2016-03-13T09:17:31.896+0000 I NETWORK [conn4071947] end connection 10.0.7.113:50129 (1505 connections now open) 2016-03-13T09:17:31.896+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59531 #4080963 (1506 connections now open) 2016-03-13T09:17:31.896+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59532 #4080964 (1507 connections now open) 2016-03-13T09:17:31.896+0000 I NETWORK [conn4077169] end connection 10.0.179.209:36853 (1506 connections now open) 2016-03-13T09:17:31.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40611 #4080965 (1507 connections now open) 2016-03-13T09:17:31.898+0000 I NETWORK [conn4078192] end connection 10.0.216.90:54842 (1506 connections now open) 2016-03-13T09:17:31.899+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57510 #4080966 (1507 connections now open) 2016-03-13T09:17:31.899+0000 I NETWORK [conn4069455] end connection 10.0.26.69:58707 (1506 connections now open) 2016-03-13T09:17:31.899+0000 I NETWORK [conn4068985] end connection 10.0.20.53:38421 (1505 connections now open) 2016-03-13T09:17:31.899+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42450 #4080967 (1506 connections now open) 2016-03-13T09:17:31.899+0000 I NETWORK [conn4070772] end connection 10.0.179.209:58624 (1504 connections now open) 2016-03-13T09:17:31.899+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50702 #4080968 (1506 connections now open) 2016-03-13T09:17:31.900+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40614 #4080969 (1507 connections now open) 2016-03-13T09:17:31.900+0000 I NETWORK [conn4075646] end connection 10.0.7.113:54099 (1506 connections now open) 2016-03-13T09:17:31.900+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59535 #4080970 (1507 connections now open) 2016-03-13T09:17:31.901+0000 I NETWORK [conn4073751] end connection 10.0.26.69:35083 (1506 connections now open) 2016-03-13T09:17:31.901+0000 I NETWORK [conn4078022] end connection 10.0.216.90:54661 (1505 connections now open) 2016-03-13T09:17:31.901+0000 I NETWORK [conn4072595] end connection 10.0.20.53:42173 (1504 connections now open) 2016-03-13T09:17:31.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42451 #4080971 (1505 connections now open) 2016-03-13T09:17:31.901+0000 I NETWORK [conn4072427] end connection 10.0.27.237:55554 (1504 connections now open) 2016-03-13T09:17:31.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57511 #4080972 (1506 connections now open) 2016-03-13T09:17:31.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50703 #4080973 (1506 connections now open) 2016-03-13T09:17:31.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35775 #4080974 (1507 connections now open) 2016-03-13T09:17:31.902+0000 I NETWORK [conn4076781] end connection 10.0.179.209:36467 (1506 connections now open) 2016-03-13T09:17:31.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40617 #4080975 (1507 connections now open) 2016-03-13T09:17:31.903+0000 I NETWORK [conn4079773] end connection 10.0.27.237:34757 (1506 connections now open) 2016-03-13T09:17:31.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35776 #4080976 (1507 connections now open) 2016-03-13T09:17:31.904+0000 I NETWORK [conn4073810] end connection 10.0.7.113:52125 (1506 connections now open) 2016-03-13T09:17:31.904+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59538 #4080977 (1507 connections now open) 2016-03-13T09:17:31.904+0000 I NETWORK [conn4077386] end connection 10.0.179.209:37063 (1506 connections now open) 2016-03-13T09:17:31.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40619 #4080978 (1507 connections now open) 2016-03-13T09:17:31.905+0000 I NETWORK [conn4069297] end connection 10.0.20.53:38740 (1506 connections now open) 2016-03-13T09:17:31.905+0000 I NETWORK [conn4078361] end connection 10.0.7.113:56986 (1506 connections now open) 2016-03-13T09:17:31.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50705 #4080979 (1506 connections now open) 2016-03-13T09:17:31.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59539 #4080980 (1507 connections now open) 2016-03-13T09:17:31.906+0000 I NETWORK [conn4073351] end connection 10.0.7.113:51626 (1505 connections now open) 2016-03-13T09:17:31.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59540 #4080981 (1507 connections now open) 2016-03-13T09:17:31.907+0000 I NETWORK [conn4072553] end connection 10.0.27.237:55691 (1506 connections now open) 2016-03-13T09:17:31.907+0000 I NETWORK [conn4068475] end connection 10.0.20.53:37886 (1505 connections now open) 2016-03-13T09:17:31.907+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35778 #4080982 (1506 connections now open) 2016-03-13T09:17:31.907+0000 I NETWORK [conn4078507] end connection 10.0.7.113:57141 (1505 connections now open) 2016-03-13T09:17:31.907+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50706 #4080983 (1506 connections now open) 2016-03-13T09:17:31.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59541 #4080984 (1507 connections now open) 2016-03-13T09:17:31.908+0000 I NETWORK [conn4073161] end connection 10.0.179.209:32773 (1506 connections now open) 2016-03-13T09:17:31.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40623 #4080985 (1507 connections now open) 2016-03-13T09:17:31.909+0000 I NETWORK [conn4074021] end connection 10.0.7.113:52350 (1506 connections now open) 2016-03-13T09:17:31.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59542 #4080986 (1507 connections now open) 2016-03-13T09:17:31.910+0000 I NETWORK [conn4071143] end connection 10.0.27.237:54182 (1506 connections now open) 2016-03-13T09:17:31.910+0000 I NETWORK [conn4074500] end connection 10.0.179.209:34165 (1505 connections now open) 2016-03-13T09:17:31.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35780 #4080987 (1506 connections now open) 2016-03-13T09:17:31.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40625 #4080988 (1507 connections now open) 2016-03-13T09:17:31.912+0000 I NETWORK [conn4077080] end connection 10.0.179.209:36765 (1506 connections now open) 2016-03-13T09:17:31.912+0000 I NETWORK [conn4073682] end connection 10.0.179.209:33299 (1505 connections now open) 2016-03-13T09:17:31.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40626 #4080989 (1506 connections now open) 2016-03-13T09:17:31.913+0000 I NETWORK [conn4069384] end connection 10.0.20.53:38832 (1505 connections now open) 2016-03-13T09:17:31.913+0000 I NETWORK [conn4076656] end connection 10.0.26.69:38228 (1504 connections now open) 2016-03-13T09:17:31.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40627 #4080990 (1505 connections now open) 2016-03-13T09:17:31.913+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-2-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:31.913+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-0-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:31.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50709 #4080991 (1506 connections now open) 2016-03-13T09:17:31.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42457 #4080992 (1507 connections now open) 2016-03-13T09:17:31.914+0000 I NETWORK [conn4074418] end connection 10.0.179.209:34074 (1506 connections now open) 2016-03-13T09:17:31.914+0000 I NETWORK [conn4073975] end connection 10.0.179.209:33595 (1505 connections now open) 2016-03-13T09:17:31.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40628 #4080993 (1506 connections now open) 2016-03-13T09:17:31.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40629 #4080994 (1507 connections now open) 2016-03-13T09:17:31.916+0000 I NETWORK [conn4073957] end connection 10.0.26.69:35303 (1506 connections now open) 2016-03-13T09:17:31.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42458 #4080995 (1507 connections now open) 2016-03-13T09:17:31.916+0000 I NETWORK [conn4073808] end connection 10.0.179.209:33422 (1506 connections now open) 2016-03-13T09:17:31.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40631 #4080996 (1507 connections now open) 2016-03-13T09:17:31.917+0000 I NETWORK [conn4073770] end connection 10.0.7.113:52082 (1506 connections now open) 2016-03-13T09:17:31.917+0000 I NETWORK [conn4080174] end connection 10.0.27.237:35183 (1505 connections now open) 2016-03-13T09:17:31.917+0000 I NETWORK [conn4072845] end connection 10.0.27.237:56010 (1504 connections now open) 2016-03-13T09:17:31.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59546 #4080997 (1505 connections now open) 2016-03-13T09:17:31.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35785 #4080998 (1506 connections now open) 2016-03-13T09:17:31.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35786 #4080999 (1507 connections now open) 2016-03-13T09:17:31.918+0000 I NETWORK [conn4076399] end connection 10.0.216.90:52938 (1506 connections now open) 2016-03-13T09:17:31.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57512 #4081000 (1507 connections now open) 2016-03-13T09:17:31.919+0000 I NETWORK [conn4076184] end connection 10.0.7.113:54678 (1506 connections now open) 2016-03-13T09:17:31.919+0000 I NETWORK [conn4072886] end connection 10.0.20.53:42483 (1505 connections now open) 2016-03-13T09:17:31.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59547 #4081001 (1506 connections now open) 2016-03-13T09:17:31.920+0000 I NETWORK [conn4075858] end connection 10.0.27.237:59043 (1505 connections now open) 2016-03-13T09:17:31.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50712 #4081002 (1507 connections now open) 2016-03-13T09:17:31.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35787 #4081003 (1507 connections now open) 2016-03-13T09:17:31.921+0000 I NETWORK [conn4069430] end connection 10.0.20.53:38877 (1506 connections now open) 2016-03-13T09:17:31.921+0000 I NETWORK [conn4076699] end connection 10.0.7.113:55221 (1505 connections now open) 2016-03-13T09:17:31.921+0000 I NETWORK [conn4073930] end connection 10.0.179.209:33559 (1504 connections now open) 2016-03-13T09:17:31.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50713 #4081004 (1505 connections now open) 2016-03-13T09:17:31.922+0000 I NETWORK [conn4077598] end connection 10.0.179.209:37279 (1503 connections now open) 2016-03-13T09:17:31.922+0000 I NETWORK [conn4073979] end connection 10.0.7.113:52303 (1503 connections now open) 2016-03-13T09:17:31.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59549 #4081005 (1504 connections now open) 2016-03-13T09:17:31.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40635 #4081006 (1505 connections now open) 2016-03-13T09:17:31.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40634 #4081007 (1506 connections now open) 2016-03-13T09:17:31.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59550 #4081008 (1507 connections now open) 2016-03-13T09:17:31.924+0000 I NETWORK [conn4077853] end connection 10.0.179.209:37536 (1506 connections now open) 2016-03-13T09:17:31.924+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:31.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40637 #4081009 (1507 connections now open) 2016-03-13T09:17:31.924+0000 I NETWORK [conn4069044] end connection 10.0.27.237:51950 (1506 connections now open) 2016-03-13T09:17:31.925+0000 I NETWORK [conn4069956] end connection 10.0.26.69:59254 (1505 connections now open) 2016-03-13T09:17:31.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35792 #4081010 (1506 connections now open) 2016-03-13T09:17:31.925+0000 I NETWORK [conn4072927] end connection 10.0.20.53:42531 (1504 connections now open) 2016-03-13T09:17:31.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50715 #4081011 (1506 connections now open) 2016-03-13T09:17:31.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42462 #4081012 (1507 connections now open) 2016-03-13T09:17:31.926+0000 I NETWORK [conn4076613] end connection 10.0.179.209:36300 (1506 connections now open) 2016-03-13T09:17:31.926+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40639 #4081013 (1507 connections now open) 2016-03-13T09:17:31.926+0000 I NETWORK [conn4075686] end connection 10.0.27.237:58874 (1506 connections now open) 2016-03-13T09:17:31.927+0000 I NETWORK [conn4069383] end connection 10.0.27.237:52306 (1505 connections now open) 2016-03-13T09:17:31.927+0000 I NETWORK [conn4078294] end connection 10.0.20.53:48140 (1504 connections now open) 2016-03-13T09:17:31.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35793 #4081014 (1505 connections now open) 2016-03-13T09:17:31.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35794 #4081015 (1506 connections now open) 2016-03-13T09:17:31.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50716 #4081016 (1507 connections now open) 2016-03-13T09:17:31.927+0000 I NETWORK [conn4078864] end connection 10.0.179.209:38573 (1506 connections now open) 2016-03-13T09:17:31.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40640 #4081017 (1507 connections now open) 2016-03-13T09:17:31.931+0000 I NETWORK [conn4069955] end connection 10.0.27.237:52915 (1506 connections now open) 2016-03-13T09:17:31.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35797 #4081018 (1507 connections now open) 2016-03-13T09:17:31.931+0000 I NETWORK [conn4078738] end connection 10.0.26.69:40437 (1506 connections now open) 2016-03-13T09:17:31.932+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42465 #4081019 (1507 connections now open) 2016-03-13T09:17:31.932+0000 I NETWORK [conn4074317] end connection 10.0.27.237:57503 (1506 connections now open) 2016-03-13T09:17:31.932+0000 I NETWORK [conn4075071] end connection 10.0.179.209:34741 (1505 connections now open) 2016-03-13T09:17:31.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35798 #4081020 (1506 connections now open) 2016-03-13T09:17:31.933+0000 I NETWORK [conn4073037] end connection 10.0.27.237:56216 (1505 connections now open) 2016-03-13T09:17:31.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40644 #4081021 (1507 connections now open) 2016-03-13T09:17:31.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35799 #4081022 (1507 connections now open) 2016-03-13T09:17:31.933+0000 I NETWORK [conn4077994] end connection 10.0.20.53:47840 (1506 connections now open) 2016-03-13T09:17:31.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50721 #4081023 (1507 connections now open) 2016-03-13T09:17:31.934+0000 I NETWORK [conn4075325] end connection 10.0.27.237:58511 (1506 connections now open) 2016-03-13T09:17:31.934+0000 I NETWORK [conn4080361] end connection 10.0.179.209:40085 (1505 connections now open) 2016-03-13T09:17:31.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35800 #4081024 (1507 connections now open) 2016-03-13T09:17:31.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40646 #4081025 (1507 connections now open) 2016-03-13T09:17:31.935+0000 I NETWORK [conn4077214] end connection 10.0.7.113:55758 (1506 connections now open) 2016-03-13T09:17:31.935+0000 I NETWORK [conn4077703] end connection 10.0.20.53:47549 (1505 connections now open) 2016-03-13T09:17:31.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59558 #4081026 (1507 connections now open) 2016-03-13T09:17:31.936+0000 I NETWORK [conn4071184] end connection 10.0.216.90:47308 (1505 connections now open) 2016-03-13T09:17:31.936+0000 I NETWORK [conn4074929] end connection 10.0.20.53:44642 (1504 connections now open) 2016-03-13T09:17:31.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50722 #4081027 (1506 connections now open) 2016-03-13T09:17:31.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57514 #4081028 (1506 connections now open) 2016-03-13T09:17:31.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50723 #4081029 (1507 connections now open) 2016-03-13T09:17:31.937+0000 I NETWORK [conn4078592] end connection 10.0.7.113:57230 (1506 connections now open) 2016-03-13T09:17:31.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59559 #4081030 (1507 connections now open) 2016-03-13T09:17:31.937+0000 I NETWORK [conn4071820] end connection 10.0.26.69:33031 (1506 connections now open) 2016-03-13T09:17:31.938+0000 I NETWORK [conn4072636] end connection 10.0.20.53:42219 (1505 connections now open) 2016-03-13T09:17:31.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42469 #4081031 (1506 connections now open) 2016-03-13T09:17:31.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50724 #4081032 (1507 connections now open) 2016-03-13T09:17:31.938+0000 I NETWORK [conn4073182] end connection 10.0.27.237:56366 (1506 connections now open) 2016-03-13T09:17:31.938+0000 I NETWORK [conn4077789] end connection 10.0.179.209:37465 (1506 connections now open) 2016-03-13T09:17:31.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35804 #4081033 (1506 connections now open) 2016-03-13T09:17:31.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40648 #4081034 (1507 connections now open) 2016-03-13T09:17:31.939+0000 I NETWORK [conn4080640] end connection 10.0.26.69:42354 (1506 connections now open) 2016-03-13T09:17:31.939+0000 I NETWORK [conn4075813] end connection 10.0.27.237:58995 (1506 connections now open) 2016-03-13T09:17:31.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35805 #4081035 (1506 connections now open) 2016-03-13T09:17:31.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42470 #4081036 (1507 connections now open) 2016-03-13T09:17:31.940+0000 I NETWORK [conn4079431] end connection 10.0.179.209:39138 (1506 connections now open) 2016-03-13T09:17:31.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40649 #4081037 (1507 connections now open) 2016-03-13T09:17:31.941+0000 I NETWORK [conn4073749] end connection 10.0.216.90:50069 (1506 connections now open) 2016-03-13T09:17:31.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57516 #4081038 (1507 connections now open) 2016-03-13T09:17:31.942+0000 I NETWORK [conn4073501] end connection 10.0.27.237:56693 (1506 connections now open) 2016-03-13T09:17:31.942+0000 I NETWORK [conn4071903] end connection 10.0.26.69:33125 (1505 connections now open) 2016-03-13T09:17:31.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35807 #4081039 (1506 connections now open) 2016-03-13T09:17:31.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42472 #4081040 (1507 connections now open) 2016-03-13T09:17:31.943+0000 I NETWORK [conn4074626] end connection 10.0.179.209:34301 (1506 connections now open) 2016-03-13T09:17:31.943+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40651 #4081041 (1507 connections now open) 2016-03-13T09:17:31.944+0000 I NETWORK [conn4073539] end connection 10.0.27.237:56733 (1506 connections now open) 2016-03-13T09:17:31.944+0000 I NETWORK [conn4074171] end connection 10.0.26.69:35532 (1505 connections now open) 2016-03-13T09:17:31.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35809 #4081042 (1506 connections now open) 2016-03-13T09:17:31.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42474 #4081043 (1507 connections now open) 2016-03-13T09:17:31.944+0000 I NETWORK [conn4069888] end connection 10.0.179.209:57739 (1506 connections now open) 2016-03-13T09:17:31.945+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40652 #4081044 (1507 connections now open) 2016-03-13T09:17:31.945+0000 I NETWORK [conn4071565] end connection 10.0.216.90:47722 (1506 connections now open) 2016-03-13T09:17:31.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57519 #4081045 (1507 connections now open) 2016-03-13T09:17:31.946+0000 I NETWORK [conn4076528] end connection 10.0.26.69:38094 (1506 connections now open) 2016-03-13T09:17:31.946+0000 I NETWORK [conn4069171] end connection 10.0.20.53:38612 (1505 connections now open) 2016-03-13T09:17:31.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42476 #4081046 (1506 connections now open) 2016-03-13T09:17:31.947+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50730 #4081047 (1507 connections now open) 2016-03-13T09:17:31.947+0000 I NETWORK [conn4078402] end connection 10.0.216.90:55071 (1506 connections now open) 2016-03-13T09:17:31.947+0000 I NETWORK [conn4072326] end connection 10.0.7.113:50544 (1505 connections now open) 2016-03-13T09:17:31.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59565 #4081048 (1506 connections now open) 2016-03-13T09:17:31.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57520 #4081049 (1507 connections now open) 2016-03-13T09:17:31.948+0000 I NETWORK [conn4069113] end connection 10.0.20.53:38552 (1506 connections now open) 2016-03-13T09:17:31.948+0000 I NETWORK [conn4074129] end connection 10.0.26.69:35483 (1505 connections now open) 2016-03-13T09:17:31.948+0000 I NETWORK [conn4079666] end connection 10.0.179.209:39379 (1505 connections now open) 2016-03-13T09:17:31.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42478 #4081050 (1505 connections now open) 2016-03-13T09:17:31.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50733 #4081051 (1506 connections now open) 2016-03-13T09:17:31.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40656 #4081052 (1507 connections now open) 2016-03-13T09:17:31.949+0000 I NETWORK [conn4073643] end connection 10.0.7.113:51942 (1506 connections now open) 2016-03-13T09:17:31.949+0000 I NETWORK [conn4074436] end connection 10.0.179.209:34087 (1505 connections now open) 2016-03-13T09:17:31.949+0000 I NETWORK [conn4075241] end connection 10.0.27.237:58430 (1504 connections now open) 2016-03-13T09:17:31.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59566 #4081053 (1506 connections now open) 2016-03-13T09:17:31.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40657 #4081054 (1506 connections now open) 2016-03-13T09:17:31.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35814 #4081055 (1507 connections now open) 2016-03-13T09:17:31.950+0000 I NETWORK [conn4073433] end connection 10.0.26.69:34743 (1506 connections now open) 2016-03-13T09:17:31.950+0000 I NETWORK [conn4071098] end connection 10.0.216.90:47217 (1506 connections now open) 2016-03-13T09:17:31.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42479 #4081056 (1506 connections now open) 2016-03-13T09:17:31.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57522 #4081057 (1507 connections now open) 2016-03-13T09:17:31.951+0000 I NETWORK [conn4076355] end connection 10.0.27.237:59547 (1505 connections now open) 2016-03-13T09:17:31.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35815 #4081058 (1507 connections now open) 2016-03-13T09:17:31.952+0000 I NETWORK [conn4074231] end connection 10.0.27.237:57420 (1506 connections now open) 2016-03-13T09:17:31.952+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35816 #4081059 (1507 connections now open) 2016-03-13T09:17:31.952+0000 I NETWORK [conn4073083] end connection 10.0.216.90:49355 (1506 connections now open) 2016-03-13T09:17:31.953+0000 I NETWORK [conn4076398] end connection 10.0.27.237:59586 (1505 connections now open) 2016-03-13T09:17:31.953+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57524 #4081060 (1506 connections now open) 2016-03-13T09:17:31.953+0000 I NETWORK [conn4078781] end connection 10.0.26.69:40475 (1504 connections now open) 2016-03-13T09:17:31.953+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35817 #4081061 (1506 connections now open) 2016-03-13T09:17:31.953+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42482 #4081062 (1507 connections now open) 2016-03-13T09:17:31.954+0000 I NETWORK [conn4070351] end connection 10.0.179.209:58195 (1506 connections now open) 2016-03-13T09:17:31.954+0000 I NETWORK [conn4070185] end connection 10.0.26.69:59499 (1505 connections now open) 2016-03-13T09:17:31.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40660 #4081063 (1506 connections now open) 2016-03-13T09:17:31.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42483 #4081064 (1507 connections now open) 2016-03-13T09:17:31.955+0000 I NETWORK [conn4078064] end connection 10.0.179.209:37753 (1506 connections now open) 2016-03-13T09:17:31.956+0000 I NETWORK [conn4078147] end connection 10.0.179.209:37838 (1505 connections now open) 2016-03-13T09:17:31.956+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40661 #4081065 (1506 connections now open) 2016-03-13T09:17:31.956+0000 I NETWORK [conn4074559] end connection 10.0.27.237:57756 (1505 connections now open) 2016-03-13T09:17:31.956+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40662 #4081066 (1506 connections now open) 2016-03-13T09:17:31.956+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35820 #4081067 (1507 connections now open) 2016-03-13T09:17:31.956+0000 I NETWORK [conn4072343] end connection 10.0.26.69:33587 (1506 connections now open) 2016-03-13T09:17:31.957+0000 I NETWORK [conn4080066] end connection 10.0.216.90:56742 (1505 connections now open) 2016-03-13T09:17:31.957+0000 I NETWORK [conn4071480] end connection 10.0.27.237:54543 (1504 connections now open) 2016-03-13T09:17:31.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42485 #4081068 (1506 connections now open) 2016-03-13T09:17:31.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57526 #4081069 (1506 connections now open) 2016-03-13T09:17:31.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35821 #4081070 (1507 connections now open) 2016-03-13T09:17:31.958+0000 I NETWORK [conn4069806] end connection 10.0.27.237:52752 (1506 connections now open) 2016-03-13T09:17:31.958+0000 I NETWORK [conn4070715] end connection 10.0.20.53:40229 (1505 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35822 #4081071 (1506 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [conn4073334] end connection 10.0.216.90:49617 (1504 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50742 #4081072 (1506 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [conn4073180] end connection 10.0.26.69:34473 (1505 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57527 #4081073 (1507 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [conn4079643] end connection 10.0.179.209:39346 (1504 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [conn4073915] end connection 10.0.27.237:57102 (1504 connections now open) 2016-03-13T09:17:31.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42487 #4081074 (1505 connections now open) 2016-03-13T09:17:31.960+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40666 #4081075 (1506 connections now open) 2016-03-13T09:17:31.960+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35823 #4081076 (1507 connections now open) 2016-03-13T09:17:31.960+0000 I NETWORK [conn4069511] end connection 10.0.20.53:38965 (1506 connections now open) 2016-03-13T09:17:31.960+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50743 #4081077 (1507 connections now open) 2016-03-13T09:17:31.960+0000 I NETWORK [conn4074731] end connection 10.0.27.237:57927 (1506 connections now open) 2016-03-13T09:17:31.961+0000 I NETWORK [conn4074003] end connection 10.0.26.69:35350 (1505 connections now open) 2016-03-13T09:17:31.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35824 #4081078 (1506 connections now open) 2016-03-13T09:17:31.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42488 #4081079 (1507 connections now open) 2016-03-13T09:17:31.961+0000 I NETWORK [conn4078712] end connection 10.0.7.113:57353 (1506 connections now open) 2016-03-13T09:17:31.961+0000 I NETWORK [conn4080615] end connection 10.0.27.237:35657 (1505 connections now open) 2016-03-13T09:17:31.962+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59572 #4081080 (1506 connections now open) 2016-03-13T09:17:31.962+0000 I NETWORK [conn4069257] end connection 10.0.20.53:38695 (1504 connections now open) 2016-03-13T09:17:31.962+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35825 #4081081 (1506 connections now open) 2016-03-13T09:17:31.962+0000 I NETWORK [conn4077995] end connection 10.0.27.237:32935 (1505 connections now open) 2016-03-13T09:17:31.962+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50745 #4081082 (1506 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35826 #4081083 (1507 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [conn4075857] end connection 10.0.26.69:37359 (1506 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [conn4072788] end connection 10.0.20.53:42374 (1505 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [conn4070877] end connection 10.0.216.90:46993 (1504 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42490 #4081084 (1505 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50746 #4081085 (1506 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57530 #4081086 (1507 connections now open) 2016-03-13T09:17:31.963+0000 I NETWORK [conn4076529] end connection 10.0.27.237:59716 (1505 connections now open) 2016-03-13T09:17:31.964+0000 I NETWORK [conn4074800] end connection 10.0.179.209:34473 (1505 connections now open) 2016-03-13T09:17:31.964+0000 I NETWORK [conn4076631] end connection 10.0.179.209:36326 (1504 connections now open) 2016-03-13T09:17:31.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35827 #4081087 (1505 connections now open) 2016-03-13T09:17:31.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40670 #4081088 (1506 connections now open) 2016-03-13T09:17:31.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40671 #4081089 (1507 connections now open) 2016-03-13T09:17:31.964+0000 I NETWORK [conn4072664] end connection 10.0.7.113:50905 (1506 connections now open) 2016-03-13T09:17:31.965+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59573 #4081090 (1507 connections now open) 2016-03-13T09:17:31.966+0000 I NETWORK [conn4078401] end connection 10.0.179.209:38094 (1506 connections now open) 2016-03-13T09:17:31.966+0000 I NETWORK [conn4070688] end connection 10.0.179.209:58539 (1505 connections now open) 2016-03-13T09:17:31.966+0000 I NETWORK [conn4070689] end connection 10.0.20.53:40204 (1504 connections now open) 2016-03-13T09:17:31.966+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40672 #4081091 (1506 connections now open) 2016-03-13T09:17:31.966+0000 I NETWORK [conn4074774] end connection 10.0.7.113:53157 (1504 connections now open) 2016-03-13T09:17:31.966+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40673 #4081092 (1505 connections now open) 2016-03-13T09:17:31.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50749 #4081093 (1506 connections now open) 2016-03-13T09:17:31.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59574 #4081094 (1507 connections now open) 2016-03-13T09:17:31.967+0000 I NETWORK [conn4074945] end connection 10.0.27.237:58138 (1505 connections now open) 2016-03-13T09:17:31.967+0000 I NETWORK [conn4078530] end connection 10.0.216.90:55208 (1506 connections now open) 2016-03-13T09:17:31.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35830 #4081095 (1506 connections now open) 2016-03-13T09:17:31.967+0000 I NETWORK [conn4068872] end connection 10.0.20.53:38313 (1504 connections now open) 2016-03-13T09:17:31.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57533 #4081096 (1507 connections now open) 2016-03-13T09:17:31.968+0000 I NETWORK [conn4074004] end connection 10.0.27.237:57187 (1505 connections now open) 2016-03-13T09:17:31.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50750 #4081097 (1506 connections now open) 2016-03-13T09:17:31.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35831 #4081098 (1507 connections now open) 2016-03-13T09:17:31.968+0000 I NETWORK [conn4070564] end connection 10.0.26.69:59912 (1506 connections now open) 2016-03-13T09:17:31.969+0000 I NETWORK [conn4077104] end connection 10.0.7.113:55642 (1505 connections now open) 2016-03-13T09:17:31.969+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42495 #4081099 (1506 connections now open) 2016-03-13T09:17:31.969+0000 I NETWORK [conn4078079] end connection 10.0.27.237:33016 (1505 connections now open) 2016-03-13T09:17:31.969+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59575 #4081100 (1507 connections now open) 2016-03-13T09:17:31.969+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35832 #4081101 (1507 connections now open) 2016-03-13T09:17:31.970+0000 I NETWORK [conn4074044] end connection 10.0.27.237:57226 (1506 connections now open) 2016-03-13T09:17:31.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35833 #4081102 (1507 connections now open) 2016-03-13T09:17:31.971+0000 I NETWORK [conn4079011] end connection 10.0.7.113:57659 (1506 connections now open) 2016-03-13T09:17:31.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59576 #4081103 (1507 connections now open) 2016-03-13T09:17:31.971+0000 I NETWORK [conn4074255] end connection 10.0.179.209:33897 (1506 connections now open) 2016-03-13T09:17:31.972+0000 I NETWORK [conn4076740] end connection 10.0.27.237:59929 (1505 connections now open) 2016-03-13T09:17:31.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40676 #4081104 (1506 connections now open) 2016-03-13T09:17:31.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35835 #4081105 (1507 connections now open) 2016-03-13T09:17:31.974+0000 I NETWORK [conn4079243] end connection 10.0.26.69:40948 (1506 connections now open) 2016-03-13T09:17:31.974+0000 I NETWORK [conn4073081] end connection 10.0.20.53:42693 (1505 connections now open) 2016-03-13T09:17:31.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42498 #4081106 (1506 connections now open) 2016-03-13T09:17:31.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50754 #4081107 (1507 connections now open) 2016-03-13T09:17:31.975+0000 I NETWORK [conn4074985] end connection 10.0.179.209:34652 (1506 connections now open) 2016-03-13T09:17:31.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40678 #4081108 (1507 connections now open) 2016-03-13T09:17:31.976+0000 I NETWORK [conn4073726] end connection 10.0.26.69:35059 (1506 connections now open) 2016-03-13T09:17:31.976+0000 I NETWORK [conn4074105] end connection 10.0.27.237:57293 (1505 connections now open) 2016-03-13T09:17:31.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42499 #4081109 (1506 connections now open) 2016-03-13T09:17:31.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35836 #4081110 (1507 connections now open) 2016-03-13T09:17:31.977+0000 I NETWORK [conn4073162] end connection 10.0.20.53:42782 (1506 connections now open) 2016-03-13T09:17:31.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50756 #4081111 (1507 connections now open) 2016-03-13T09:17:31.979+0000 I NETWORK [conn4074946] end connection 10.0.7.113:53337 (1506 connections now open) 2016-03-13T09:17:31.979+0000 I NETWORK [conn4071522] end connection 10.0.26.69:60945 (1505 connections now open) 2016-03-13T09:17:31.979+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59578 #4081112 (1506 connections now open) 2016-03-13T09:17:31.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42502 #4081113 (1507 connections now open) 2016-03-13T09:17:31.980+0000 I NETWORK [conn4077684] end connection 10.0.179.209:37364 (1506 connections now open) 2016-03-13T09:17:31.981+0000 I NETWORK [conn4075619] end connection 10.0.7.113:54069 (1505 connections now open) 2016-03-13T09:17:31.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40683 #4081114 (1506 connections now open) 2016-03-13T09:17:31.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59579 #4081115 (1507 connections now open) 2016-03-13T09:17:31.981+0000 I NETWORK [conn4078866] end connection 10.0.26.69:40560 (1506 connections now open) 2016-03-13T09:17:31.982+0000 I NETWORK [conn4072576] end connection 10.0.20.53:42152 (1505 connections now open) 2016-03-13T09:17:31.982+0000 I NETWORK [conn4071944] end connection 10.0.216.90:48135 (1504 connections now open) 2016-03-13T09:17:31.982+0000 I NETWORK [conn4075196] end connection 10.0.179.209:34872 (1503 connections now open) 2016-03-13T09:17:31.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42504 #4081116 (1504 connections now open) 2016-03-13T09:17:31.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50760 #4081117 (1505 connections now open) 2016-03-13T09:17:31.982+0000 I NETWORK [conn4075099] end connection 10.0.216.90:51522 (1504 connections now open) 2016-03-13T09:17:31.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57542 #4081118 (1505 connections now open) 2016-03-13T09:17:31.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57543 #4081119 (1506 connections now open) 2016-03-13T09:17:31.983+0000 I NETWORK [conn4076463] end connection 10.0.7.113:54970 (1505 connections now open) 2016-03-13T09:17:31.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59581 #4081120 (1506 connections now open) 2016-03-13T09:17:31.983+0000 I NETWORK [conn4070502] end connection 10.0.27.237:53503 (1505 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [conn4073333] end connection 10.0.20.53:42962 (1504 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [conn4072830] end connection 10.0.26.69:34114 (1503 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35842 #4081121 (1504 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [conn4080612] end connection 10.0.216.90:57296 (1502 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50762 #4081122 (1505 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42506 #4081123 (1505 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57544 #4081124 (1506 connections now open) 2016-03-13T09:17:31.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40685 #4081125 (1507 connections now open) 2016-03-13T09:17:31.985+0000 I NETWORK [conn4075240] end connection 10.0.26.69:36686 (1506 connections now open) 2016-03-13T09:17:31.985+0000 I NETWORK [conn4070015] end connection 10.0.20.53:39501 (1505 connections now open) 2016-03-13T09:17:31.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42507 #4081126 (1506 connections now open) 2016-03-13T09:17:31.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50763 #4081127 (1507 connections now open) 2016-03-13T09:17:31.986+0000 I NETWORK [conn4075642] end connection 10.0.20.53:45403 (1506 connections now open) 2016-03-13T09:17:31.986+0000 I NETWORK [conn4077317] end connection 10.0.216.90:53918 (1505 connections now open) 2016-03-13T09:17:31.986+0000 I NETWORK [conn4073416] end connection 10.0.26.69:34721 (1505 connections now open) 2016-03-13T09:17:31.986+0000 I NETWORK [conn4076022] end connection 10.0.7.113:54498 (1503 connections now open) 2016-03-13T09:17:31.986+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50764 #4081128 (1505 connections now open) 2016-03-13T09:17:31.986+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57546 #4081129 (1505 connections now open) 2016-03-13T09:17:31.986+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42508 #4081130 (1506 connections now open) 2016-03-13T09:17:31.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59584 #4081131 (1507 connections now open) 2016-03-13T09:17:31.987+0000 I NETWORK [conn4073852] end connection 10.0.27.237:57041 (1506 connections now open) 2016-03-13T09:17:31.987+0000 I NETWORK [conn4073724] end connection 10.0.20.53:43386 (1505 connections now open) 2016-03-13T09:17:31.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35845 #4081132 (1507 connections now open) 2016-03-13T09:17:31.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50765 #4081133 (1507 connections now open) 2016-03-13T09:17:31.988+0000 I NETWORK [conn4072844] end connection 10.0.179.209:60682 (1506 connections now open) 2016-03-13T09:17:31.988+0000 I NETWORK [conn4073458] end connection 10.0.20.53:43097 (1505 connections now open) 2016-03-13T09:17:31.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40687 #4081134 (1506 connections now open) 2016-03-13T09:17:31.988+0000 I NETWORK [conn4078739] end connection 10.0.7.113:57385 (1505 connections now open) 2016-03-13T09:17:31.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50766 #4081135 (1507 connections now open) 2016-03-13T09:17:31.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59586 #4081136 (1507 connections now open) 2016-03-13T09:17:31.989+0000 I NETWORK [conn4074643] end connection 10.0.26.69:36052 (1506 connections now open) 2016-03-13T09:17:31.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42512 #4081137 (1507 connections now open) 2016-03-13T09:17:31.990+0000 I NETWORK [conn4079412] end connection 10.0.27.237:34375 (1506 connections now open) 2016-03-13T09:17:31.991+0000 I NETWORK [conn4073624] end connection 10.0.216.90:49932 (1505 connections now open) 2016-03-13T09:17:31.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35848 #4081138 (1506 connections now open) 2016-03-13T09:17:31.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57550 #4081139 (1507 connections now open) 2016-03-13T09:17:31.991+0000 I NETWORK [conn4070817] end connection 10.0.27.237:53833 (1505 connections now open) 2016-03-13T09:17:31.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35849 #4081140 (1507 connections now open) 2016-03-13T09:17:31.992+0000 I NETWORK [conn4077573] end connection 10.0.179.209:37266 (1506 connections now open) 2016-03-13T09:17:31.992+0000 I NETWORK [conn4075029] end connection 10.0.7.113:53428 (1505 connections now open) 2016-03-13T09:17:31.992+0000 I NETWORK [conn4073499] end connection 10.0.20.53:43142 (1505 connections now open) 2016-03-13T09:17:31.992+0000 I NETWORK [conn4074543] end connection 10.0.27.237:57736 (1503 connections now open) 2016-03-13T09:17:31.992+0000 I NETWORK [conn4080231] end connection 10.0.179.209:39956 (1502 connections now open) 2016-03-13T09:17:31.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50770 #4081141 (1503 connections now open) 2016-03-13T09:17:31.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40689 #4081142 (1504 connections now open) 2016-03-13T09:17:31.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35850 #4081143 (1505 connections now open) 2016-03-13T09:17:31.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59590 #4081144 (1506 connections now open) 2016-03-13T09:17:31.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40690 #4081145 (1507 connections now open) 2016-03-13T09:17:31.993+0000 I NETWORK [conn4077746] end connection 10.0.27.237:60919 (1506 connections now open) 2016-03-13T09:17:31.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35851 #4081146 (1507 connections now open) 2016-03-13T09:17:31.994+0000 I NETWORK [conn4072910] end connection 10.0.20.53:42509 (1506 connections now open) 2016-03-13T09:17:31.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50772 #4081147 (1507 connections now open) 2016-03-13T09:17:31.995+0000 I NETWORK [conn4075281] end connection 10.0.7.113:53704 (1506 connections now open) 2016-03-13T09:17:31.995+0000 I NETWORK [conn4072115] end connection 10.0.216.90:48311 (1506 connections now open) 2016-03-13T09:17:31.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57552 #4081148 (1506 connections now open) 2016-03-13T09:17:31.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59591 #4081149 (1507 connections now open) 2016-03-13T09:17:31.996+0000 I NETWORK [conn4080363] end connection 10.0.26.69:42078 (1506 connections now open) 2016-03-13T09:17:31.996+0000 I NETWORK [conn4076572] end connection 10.0.7.113:55081 (1505 connections now open) 2016-03-13T09:17:31.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42518 #4081150 (1506 connections now open) 2016-03-13T09:17:31.996+0000 I NETWORK [conn4071052] end connection 10.0.20.53:40584 (1505 connections now open) 2016-03-13T09:17:31.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59592 #4081151 (1507 connections now open) 2016-03-13T09:17:31.996+0000 I NETWORK [conn4074802] end connection 10.0.27.237:57989 (1505 connections now open) 2016-03-13T09:17:31.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50774 #4081152 (1506 connections now open) 2016-03-13T09:17:31.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35854 #4081153 (1507 connections now open) 2016-03-13T09:17:31.997+0000 I NETWORK [conn4075701] end connection 10.0.7.113:54160 (1506 connections now open) 2016-03-13T09:17:31.997+0000 I NETWORK [conn4077514] end connection 10.0.26.69:39142 (1505 connections now open) 2016-03-13T09:17:31.997+0000 I NETWORK [conn4080193] end connection 10.0.27.237:35204 (1504 connections now open) 2016-03-13T09:17:31.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59593 #4081154 (1505 connections now open) 2016-03-13T09:17:31.998+0000 I NETWORK [conn4077516] end connection 10.0.7.113:56076 (1504 connections now open) 2016-03-13T09:17:31.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35855 #4081155 (1505 connections now open) 2016-03-13T09:17:31.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42519 #4081156 (1506 connections now open) 2016-03-13T09:17:31.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59594 #4081157 (1507 connections now open) 2016-03-13T09:17:31.998+0000 I NETWORK [conn4074378] end connection 10.0.20.53:44068 (1506 connections now open) 2016-03-13T09:17:31.999+0000 I NETWORK [conn4072283] end connection 10.0.216.90:48495 (1505 connections now open) 2016-03-13T09:17:31.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50776 #4081158 (1506 connections now open) 2016-03-13T09:17:31.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57554 #4081159 (1507 connections now open) 2016-03-13T09:17:32.000+0000 I NETWORK [conn4075786] end connection 10.0.179.209:35476 (1506 connections now open) 2016-03-13T09:17:32.000+0000 I NETWORK [conn4072763] end connection 10.0.27.237:55919 (1505 connections now open) 2016-03-13T09:17:32.000+0000 I NETWORK [conn4079242] end connection 10.0.7.113:57901 (1504 connections now open) 2016-03-13T09:17:32.000+0000 I NETWORK [conn4078590] end connection 10.0.26.69:40281 (1503 connections now open) 2016-03-13T09:17:32.000+0000 I NETWORK [conn4079036] end connection 10.0.216.90:55725 (1503 connections now open) 2016-03-13T09:17:32.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40695 #4081160 (1503 connections now open) 2016-03-13T09:17:32.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35857 #4081161 (1504 connections now open) 2016-03-13T09:17:32.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59597 #4081162 (1505 connections now open) 2016-03-13T09:17:32.001+0000 I NETWORK [conn4073707] end connection 10.0.20.53:43361 (1503 connections now open) 2016-03-13T09:17:32.001+0000 I NETWORK [conn4075435] end connection 10.0.27.237:58623 (1504 connections now open) 2016-03-13T09:17:32.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57555 #4081163 (1506 connections now open) 2016-03-13T09:17:32.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42522 #4081164 (1505 connections now open) 2016-03-13T09:17:32.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35858 #4081165 (1506 connections now open) 2016-03-13T09:17:32.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50778 #4081166 (1507 connections now open) 2016-03-13T09:17:32.002+0000 I NETWORK [conn4070921] end connection 10.0.26.69:60307 (1506 connections now open) 2016-03-13T09:17:32.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42523 #4081167 (1507 connections now open) 2016-03-13T09:17:32.003+0000 I NETWORK [conn4072368] end connection 10.0.216.90:48588 (1506 connections now open) 2016-03-13T09:17:32.003+0000 I NETWORK [conn4077407] end connection 10.0.27.237:60584 (1505 connections now open) 2016-03-13T09:17:32.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57556 #4081168 (1506 connections now open) 2016-03-13T09:17:32.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35859 #4081169 (1507 connections now open) 2016-03-13T09:17:32.003+0000 I NETWORK [conn4076311] end connection 10.0.27.237:59504 (1506 connections now open) 2016-03-13T09:17:32.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35860 #4081170 (1507 connections now open) 2016-03-13T09:17:32.005+0000 I NETWORK [conn4075368] end connection 10.0.7.113:53793 (1506 connections now open) 2016-03-13T09:17:32.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59601 #4081171 (1507 connections now open) 2016-03-13T09:17:32.006+0000 I NETWORK [conn4069786] end connection 10.0.20.53:39263 (1506 connections now open) 2016-03-13T09:17:32.006+0000 I NETWORK [conn4079476] end connection 10.0.7.113:58130 (1505 connections now open) 2016-03-13T09:17:32.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50781 #4081172 (1507 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [conn4076530] end connection 10.0.7.113:55040 (1505 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [conn4071756] end connection 10.0.26.69:32950 (1504 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59602 #4081173 (1505 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [conn4069744] end connection 10.0.216.90:45777 (1504 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59603 #4081174 (1505 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42527 #4081175 (1506 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [conn4077083] end connection 10.0.27.237:60272 (1505 connections now open) 2016-03-13T09:17:32.007+0000 I NETWORK [conn4075957] end connection 10.0.179.209:35646 (1505 connections now open) 2016-03-13T09:17:32.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57558 #4081176 (1506 connections now open) 2016-03-13T09:17:32.008+0000 I NETWORK [conn4074901] end connection 10.0.20.53:44613 (1504 connections now open) 2016-03-13T09:17:32.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35864 #4081177 (1505 connections now open) 2016-03-13T09:17:32.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40698 #4081178 (1506 connections now open) 2016-03-13T09:17:32.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50782 #4081179 (1507 connections now open) 2016-03-13T09:17:32.009+0000 I NETWORK [conn4071699] end connection 10.0.7.113:49862 (1506 connections now open) 2016-03-13T09:17:32.009+0000 I NETWORK [conn4074687] end connection 10.0.26.69:36101 (1505 connections now open) 2016-03-13T09:17:32.009+0000 I NETWORK [conn4080320] end connection 10.0.216.90:56996 (1504 connections now open) 2016-03-13T09:17:32.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42529 #4081180 (1505 connections now open) 2016-03-13T09:17:32.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57559 #4081181 (1506 connections now open) 2016-03-13T09:17:32.010+0000 I NETWORK [conn4074128] end connection 10.0.27.237:57312 (1505 connections now open) 2016-03-13T09:17:32.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59605 #4081182 (1507 connections now open) 2016-03-13T09:17:32.010+0000 I NETWORK [conn4073853] end connection 10.0.20.53:43519 (1505 connections now open) 2016-03-13T09:17:32.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35866 #4081183 (1506 connections now open) 2016-03-13T09:17:32.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50784 #4081184 (1507 connections now open) 2016-03-13T09:17:32.011+0000 I NETWORK [conn4076657] end connection 10.0.7.113:55170 (1506 connections now open) 2016-03-13T09:17:32.012+0000 I NETWORK [conn4077487] end connection 10.0.216.90:54102 (1505 connections now open) 2016-03-13T09:17:32.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59607 #4081185 (1506 connections now open) 2016-03-13T09:17:32.012+0000 I NETWORK [conn4073013] end connection 10.0.27.237:56189 (1505 connections now open) 2016-03-13T09:17:32.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57560 #4081186 (1507 connections now open) 2016-03-13T09:17:32.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35867 #4081187 (1507 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [conn4075876] end connection 10.0.7.113:54339 (1506 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [conn4080259] end connection 10.0.27.237:35270 (1505 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [conn4071962] end connection 10.0.20.53:41528 (1504 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [conn4069998] end connection 10.0.20.53:39484 (1503 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [conn4075996] end connection 10.0.179.209:35684 (1503 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59609 #4081188 (1503 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [conn4071396] end connection 10.0.26.69:60809 (1502 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35868 #4081189 (1504 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50786 #4081190 (1504 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50787 #4081191 (1505 connections now open) 2016-03-13T09:17:32.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40700 #4081192 (1506 connections now open) 2016-03-13T09:17:32.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42532 #4081193 (1507 connections now open) 2016-03-13T09:17:32.015+0000 I NETWORK [conn4071539] end connection 10.0.26.69:60954 (1506 connections now open) 2016-03-13T09:17:32.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42533 #4081194 (1507 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [conn4074902] end connection 10.0.27.237:58093 (1506 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [conn4074315] end connection 10.0.179.209:33949 (1505 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [conn4073933] end connection 10.0.20.53:43608 (1504 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [conn4075788] end connection 10.0.7.113:54248 (1504 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35869 #4081195 (1504 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [conn4075352] end connection 10.0.27.237:58537 (1503 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [conn4076419] end connection 10.0.179.209:36114 (1503 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50788 #4081196 (1503 connections now open) 2016-03-13T09:17:32.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40701 #4081197 (1504 connections now open) 2016-03-13T09:17:32.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59611 #4081198 (1505 connections now open) 2016-03-13T09:17:32.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35870 #4081199 (1506 connections now open) 2016-03-13T09:17:32.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40702 #4081200 (1507 connections now open) 2016-03-13T09:17:32.019+0000 I NETWORK [conn4074477] end connection 10.0.27.237:57672 (1506 connections now open) 2016-03-13T09:17:32.019+0000 I NETWORK [conn4073055] end connection 10.0.20.53:42670 (1505 connections now open) 2016-03-13T09:17:32.019+0000 I NETWORK [conn4075662] end connection 10.0.7.113:54111 (1504 connections now open) 2016-03-13T09:17:32.019+0000 I NETWORK [conn4076373] end connection 10.0.179.209:36074 (1503 connections now open) 2016-03-13T09:17:32.019+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35871 #4081201 (1504 connections now open) 2016-03-13T09:17:32.019+0000 I NETWORK [conn4077014] end connection 10.0.179.209:36703 (1502 connections now open) 2016-03-13T09:17:32.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50790 #4081202 (1504 connections now open) 2016-03-13T09:17:32.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40703 #4081203 (1505 connections now open) 2016-03-13T09:17:32.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40704 #4081204 (1506 connections now open) 2016-03-13T09:17:32.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59613 #4081205 (1507 connections now open) 2016-03-13T09:17:32.020+0000 I NETWORK [conn4076183] end connection 10.0.20.53:45987 (1506 connections now open) 2016-03-13T09:17:32.020+0000 I NETWORK [conn4070733] end connection 10.0.26.69:60098 (1505 connections now open) 2016-03-13T09:17:32.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50791 #4081206 (1506 connections now open) 2016-03-13T09:17:32.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42535 #4081207 (1507 connections now open) 2016-03-13T09:17:32.021+0000 I NETWORK [conn4077144] end connection 10.0.20.53:46974 (1506 connections now open) 2016-03-13T09:17:32.022+0000 I NETWORK [conn4076548] end connection 10.0.179.209:36242 (1505 connections now open) 2016-03-13T09:17:32.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50792 #4081208 (1506 connections now open) 2016-03-13T09:17:32.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40705 #4081209 (1507 connections now open) 2016-03-13T09:17:32.022+0000 I NETWORK [conn4075115] end connection 10.0.7.113:53520 (1506 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [conn4073709] end connection 10.0.27.237:56890 (1505 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [conn4076843] end connection 10.0.179.209:36532 (1504 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [conn4076267] end connection 10.0.20.53:46080 (1503 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [conn4080365] end connection 10.0.7.113:59022 (1502 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59615 #4081210 (1504 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35874 #4081211 (1504 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [conn4071331] end connection 10.0.26.69:60734 (1503 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40706 #4081212 (1505 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50793 #4081213 (1506 connections now open) 2016-03-13T09:17:32.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59616 #4081214 (1506 connections now open) 2016-03-13T09:17:32.024+0000 I NETWORK [conn4070968] end connection 10.0.26.69:60355 (1505 connections now open) 2016-03-13T09:17:32.024+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42537 #4081215 (1506 connections now open) 2016-03-13T09:17:32.024+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42538 #4081216 (1507 connections now open) 2016-03-13T09:17:32.025+0000 I NETWORK [conn4074772] end connection 10.0.179.209:34443 (1506 connections now open) 2016-03-13T09:17:32.025+0000 I NETWORK [conn4077298] end connection 10.0.27.237:60489 (1505 connections now open) 2016-03-13T09:17:32.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40707 #4081217 (1506 connections now open) 2016-03-13T09:17:32.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35876 #4081218 (1507 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [conn4075998] end connection 10.0.7.113:54471 (1506 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [conn4073959] end connection 10.0.20.53:43629 (1506 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [conn4079304] end connection 10.0.179.209:39011 (1504 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [conn4079386] end connection 10.0.20.53:49282 (1503 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [conn4070589] end connection 10.0.216.90:46670 (1502 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50795 #4081219 (1503 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40708 #4081220 (1504 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59617 #4081221 (1505 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50796 #4081222 (1506 connections now open) 2016-03-13T09:17:32.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57564 #4081223 (1507 connections now open) 2016-03-13T09:17:32.037+0000 I NETWORK [conn4080215] end connection 10.0.26.69:41926 (1506 connections now open) 2016-03-13T09:17:32.037+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42541 #4081224 (1507 connections now open) 2016-03-13T09:17:32.049+0000 I NETWORK [conn4072929] end connection 10.0.27.237:56101 (1506 connections now open) 2016-03-13T09:17:32.050+0000 I NETWORK [conn4079474] end connection 10.0.216.90:56157 (1505 connections now open) 2016-03-13T09:17:32.050+0000 I NETWORK [conn4080261] end connection 10.0.7.113:58924 (1505 connections now open) 2016-03-13T09:17:32.050+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57567 #4081225 (1505 connections now open) 2016-03-13T09:17:32.050+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59619 #4081226 (1506 connections now open) 2016-03-13T09:17:32.050+0000 I NETWORK [conn4076204] end connection 10.0.7.113:54699 (1505 connections now open) 2016-03-13T09:17:32.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35879 #4081227 (1506 connections now open) 2016-03-13T09:17:32.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59620 #4081228 (1507 connections now open) 2016-03-13T09:17:32.052+0000 I NETWORK [conn4077387] end connection 10.0.27.237:60572 (1506 connections now open) 2016-03-13T09:17:32.052+0000 I NETWORK [conn4074045] end connection 10.0.20.53:43716 (1505 connections now open) 2016-03-13T09:17:32.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35881 #4081229 (1506 connections now open) 2016-03-13T09:17:32.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50797 #4081230 (1507 connections now open) 2016-03-13T09:17:32.052+0000 I NETWORK [conn4075434] end connection 10.0.26.69:36902 (1506 connections now open) 2016-03-13T09:17:32.053+0000 I NETWORK [conn4080303] end connection 10.0.26.69:42009 (1505 connections now open) 2016-03-13T09:17:32.053+0000 I NETWORK [conn4072911] end connection 10.0.216.90:49175 (1505 connections now open) 2016-03-13T09:17:32.053+0000 I NETWORK [conn4074730] end connection 10.0.26.69:36145 (1503 connections now open) 2016-03-13T09:17:32.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42543 #4081231 (1504 connections now open) 2016-03-13T09:17:32.053+0000 I NETWORK [conn4075744] end connection 10.0.179.209:35431 (1502 connections now open) 2016-03-13T09:17:32.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57569 #4081232 (1504 connections now open) 2016-03-13T09:17:32.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42544 #4081233 (1505 connections now open) 2016-03-13T09:17:32.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42545 #4081234 (1506 connections now open) 2016-03-13T09:17:32.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40710 #4081235 (1507 connections now open) 2016-03-13T09:17:32.054+0000 I NETWORK [conn4070458] end connection 10.0.20.53:39970 (1506 connections now open) 2016-03-13T09:17:32.055+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50800 #4081236 (1507 connections now open) 2016-03-13T09:17:32.055+0000 I NETWORK [conn4073054] end connection 10.0.27.237:56230 (1506 connections now open) 2016-03-13T09:17:32.055+0000 I NETWORK [conn4080614] end connection 10.0.7.113:59282 (1505 connections now open) 2016-03-13T09:17:32.055+0000 I NETWORK [conn4075599] end connection 10.0.26.69:37086 (1504 connections now open) 2016-03-13T09:17:32.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35884 #4081237 (1505 connections now open) 2016-03-13T09:17:32.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59624 #4081238 (1506 connections now open) 2016-03-13T09:17:32.056+0000 I NETWORK [conn4074541] end connection 10.0.216.90:50928 (1505 connections now open) 2016-03-13T09:17:32.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42547 #4081239 (1506 connections now open) 2016-03-13T09:17:32.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57571 #4081240 (1507 connections now open) 2016-03-13T09:17:32.056+0000 I NETWORK [conn4072114] end connection 10.0.20.53:41689 (1506 connections now open) 2016-03-13T09:17:32.057+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50802 #4081241 (1507 connections now open) 2016-03-13T09:17:32.058+0000 I NETWORK [conn4075645] end connection 10.0.26.69:37127 (1506 connections now open) 2016-03-13T09:17:32.058+0000 I NETWORK [conn4070967] end connection 10.0.216.90:47081 (1505 connections now open) 2016-03-13T09:17:32.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42549 #4081242 (1506 connections now open) 2016-03-13T09:17:32.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57572 #4081243 (1507 connections now open) 2016-03-13T09:17:32.058+0000 I NETWORK [conn4076824] end connection 10.0.26.69:38411 (1506 connections now open) 2016-03-13T09:17:32.059+0000 I NETWORK [conn4074298] end connection 10.0.20.53:43981 (1505 connections now open) 2016-03-13T09:17:32.059+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42550 #4081244 (1506 connections now open) 2016-03-13T09:17:32.059+0000 I NETWORK [conn4075116] end connection 10.0.216.90:51533 (1505 connections now open) 2016-03-13T09:17:32.059+0000 I NETWORK [conn4073178] end connection 10.0.179.209:32789 (1504 connections now open) 2016-03-13T09:17:32.059+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50804 #4081245 (1505 connections now open) 2016-03-13T09:17:32.059+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57573 #4081246 (1506 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [conn4071116] end connection 10.0.26.69:60503 (1505 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40715 #4081247 (1506 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [conn4077873] end connection 10.0.27.237:32808 (1504 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [conn4075557] end connection 10.0.27.237:58745 (1504 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [conn4077531] end connection 10.0.216.90:54148 (1504 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42551 #4081248 (1506 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35888 #4081249 (1505 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [conn4075450] end connection 10.0.7.113:53886 (1504 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35889 #4081250 (1505 connections now open) 2016-03-13T09:17:32.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57574 #4081251 (1506 connections now open) 2016-03-13T09:17:32.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59626 #4081252 (1507 connections now open) 2016-03-13T09:17:32.061+0000 I NETWORK [conn4070140] end connection 10.0.20.53:39633 (1506 connections now open) 2016-03-13T09:17:32.061+0000 I NETWORK [conn4077701] end connection 10.0.179.209:37388 (1505 connections now open) 2016-03-13T09:17:32.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50805 #4081253 (1506 connections now open) 2016-03-13T09:17:32.062+0000 I NETWORK [conn4079897] end connection 10.0.216.90:56571 (1505 connections now open) 2016-03-13T09:17:32.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40716 #4081254 (1506 connections now open) 2016-03-13T09:17:32.062+0000 I NETWORK [conn4074903] end connection 10.0.216.90:51311 (1505 connections now open) 2016-03-13T09:17:32.062+0000 I NETWORK [conn4076443] end connection 10.0.26.69:38001 (1504 connections now open) 2016-03-13T09:17:32.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57575 #4081255 (1506 connections now open) 2016-03-13T09:17:32.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57576 #4081256 (1506 connections now open) 2016-03-13T09:17:32.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42555 #4081257 (1507 connections now open) 2016-03-13T09:17:32.064+0000 I NETWORK [conn4074684] end connection 10.0.179.209:34356 (1506 connections now open) 2016-03-13T09:17:32.064+0000 I NETWORK [conn4075154] end connection 10.0.27.237:58344 (1505 connections now open) 2016-03-13T09:17:32.064+0000 I NETWORK [conn4077187] end connection 10.0.179.209:36875 (1504 connections now open) 2016-03-13T09:17:32.064+0000 I NETWORK [conn4070375] end connection 10.0.20.53:39881 (1503 connections now open) 2016-03-13T09:17:32.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35892 #4081258 (1504 connections now open) 2016-03-13T09:17:32.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40718 #4081259 (1505 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50808 #4081260 (1506 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [conn4074419] end connection 10.0.216.90:50790 (1505 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40719 #4081261 (1507 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [conn4076420] end connection 10.0.7.113:54921 (1505 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [conn4077515] end connection 10.0.27.237:60696 (1504 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57577 #4081262 (1505 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [conn4075815] end connection 10.0.26.69:37308 (1503 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59629 #4081263 (1505 connections now open) 2016-03-13T09:17:32.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35893 #4081264 (1506 connections now open) 2016-03-13T09:17:32.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42558 #4081265 (1507 connections now open) 2016-03-13T09:17:32.066+0000 I NETWORK [conn4075239] end connection 10.0.216.90:51670 (1506 connections now open) 2016-03-13T09:17:32.066+0000 I NETWORK [conn4075280] end connection 10.0.27.237:58474 (1506 connections now open) 2016-03-13T09:17:32.066+0000 I NETWORK [conn4074757] end connection 10.0.20.53:44466 (1504 connections now open) 2016-03-13T09:17:32.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35894 #4081266 (1505 connections now open) 2016-03-13T09:17:32.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57578 #4081267 (1506 connections now open) 2016-03-13T09:17:32.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50810 #4081268 (1507 connections now open) 2016-03-13T09:17:32.067+0000 I NETWORK [conn4074273] end connection 10.0.26.69:35646 (1506 connections now open) 2016-03-13T09:17:32.067+0000 I NETWORK [conn4075198] end connection 10.0.216.90:51624 (1505 connections now open) 2016-03-13T09:17:32.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42560 #4081269 (1506 connections now open) 2016-03-13T09:17:32.068+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57579 #4081270 (1507 connections now open) 2016-03-13T09:17:32.068+0000 I NETWORK [conn4079837] end connection 10.0.7.113:58498 (1505 connections now open) 2016-03-13T09:17:32.068+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59631 #4081271 (1507 connections now open) 2016-03-13T09:17:32.068+0000 I NETWORK [conn4080172] end connection 10.0.26.69:41885 (1506 connections now open) 2016-03-13T09:17:32.068+0000 I NETWORK [conn4075875] end connection 10.0.216.90:52367 (1505 connections now open) 2016-03-13T09:17:32.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57580 #4081272 (1506 connections now open) 2016-03-13T09:17:32.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42562 #4081273 (1507 connections now open) 2016-03-13T09:17:32.069+0000 I NETWORK [conn4076592] end connection 10.0.7.113:55103 (1506 connections now open) 2016-03-13T09:17:32.069+0000 I NETWORK [conn4074828] end connection 10.0.216.90:51228 (1505 connections now open) 2016-03-13T09:17:32.069+0000 I NETWORK [conn4075601] end connection 10.0.27.237:58790 (1504 connections now open) 2016-03-13T09:17:32.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59632 #4081274 (1505 connections now open) 2016-03-13T09:17:32.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57581 #4081275 (1506 connections now open) 2016-03-13T09:17:32.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35897 #4081276 (1507 connections now open) 2016-03-13T09:17:32.070+0000 I NETWORK [conn4078377] end connection 10.0.27.237:33311 (1506 connections now open) 2016-03-13T09:17:32.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35898 #4081277 (1507 connections now open) 2016-03-13T09:17:32.071+0000 I NETWORK [conn4076913] end connection 10.0.26.69:38501 (1506 connections now open) 2016-03-13T09:17:32.071+0000 I NETWORK [conn4075408] end connection 10.0.216.90:51851 (1505 connections now open) 2016-03-13T09:17:32.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42564 #4081278 (1506 connections now open) 2016-03-13T09:17:32.071+0000 I NETWORK [conn4075448] end connection 10.0.216.90:51897 (1505 connections now open) 2016-03-13T09:17:32.072+0000 I NETWORK [conn4076872] end connection 10.0.7.113:55400 (1504 connections now open) 2016-03-13T09:17:32.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57582 #4081279 (1505 connections now open) 2016-03-13T09:17:32.072+0000 I NETWORK [conn4074229] end connection 10.0.20.53:43906 (1503 connections now open) 2016-03-13T09:17:32.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57583 #4081280 (1505 connections now open) 2016-03-13T09:17:32.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50814 #4081281 (1506 connections now open) 2016-03-13T09:17:32.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59634 #4081282 (1507 connections now open) 2016-03-13T09:17:32.072+0000 I NETWORK [conn4071204] end connection 10.0.26.69:60594 (1506 connections now open) 2016-03-13T09:17:32.073+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-2-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:32.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42566 #4081283 (1507 connections now open) 2016-03-13T09:17:32.074+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:32.074+0000 I NETWORK [conn4075197] end connection 10.0.26.69:36643 (1506 connections now open) 2016-03-13T09:17:32.075+0000 I NETWORK [conn4077363] end connection 10.0.179.209:37048 (1505 connections now open) 2016-03-13T09:17:32.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42568 #4081284 (1506 connections now open) 2016-03-13T09:17:32.075+0000 I NETWORK [conn4076915] end connection 10.0.7.113:55441 (1505 connections now open) 2016-03-13T09:17:32.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40723 #4081285 (1507 connections now open) 2016-03-13T09:17:32.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59636 #4081286 (1507 connections now open) 2016-03-13T09:17:32.076+0000 I NETWORK [conn4076823] end connection 10.0.20.53:46660 (1506 connections now open) 2016-03-13T09:17:32.076+0000 I NETWORK [conn4072620] end connection 10.0.26.69:33890 (1505 connections now open) 2016-03-13T09:17:32.076+0000 I NETWORK [conn4075787] end connection 10.0.216.90:52271 (1504 connections now open) 2016-03-13T09:17:32.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50816 #4081287 (1505 connections now open) 2016-03-13T09:17:32.077+0000 I NETWORK [conn4074340] end connection 10.0.216.90:50698 (1504 connections now open) 2016-03-13T09:17:32.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42569 #4081288 (1505 connections now open) 2016-03-13T09:17:32.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57585 #4081289 (1506 connections now open) 2016-03-13T09:17:32.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57586 #4081290 (1507 connections now open) 2016-03-13T09:17:32.077+0000 I NETWORK [conn4077558] end connection 10.0.7.113:56127 (1506 connections now open) 2016-03-13T09:17:32.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59637 #4081291 (1507 connections now open) 2016-03-13T09:17:32.078+0000 I NETWORK [conn4078884] end connection 10.0.7.113:57525 (1506 connections now open) 2016-03-13T09:17:32.078+0000 I NETWORK [conn4076021] end connection 10.0.26.69:37541 (1505 connections now open) 2016-03-13T09:17:32.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59638 #4081292 (1506 connections now open) 2016-03-13T09:17:32.078+0000 I NETWORK [conn4079686] end connection 10.0.20.53:49597 (1505 connections now open) 2016-03-13T09:17:32.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42571 #4081293 (1506 connections now open) 2016-03-13T09:17:32.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50817 #4081294 (1507 connections now open) 2016-03-13T09:17:32.080+0000 I NETWORK [conn4079771] end connection 10.0.20.53:49684 (1506 connections now open) 2016-03-13T09:17:32.080+0000 I NETWORK [conn4076506] end connection 10.0.7.113:55014 (1505 connections now open) 2016-03-13T09:17:32.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50819 #4081295 (1506 connections now open) 2016-03-13T09:17:32.080+0000 I NETWORK [conn4075830] end connection 10.0.216.90:52319 (1505 connections now open) 2016-03-13T09:17:32.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59639 #4081296 (1507 connections now open) 2016-03-13T09:17:32.080+0000 I NETWORK [conn4076203] end connection 10.0.26.69:37741 (1505 connections now open) 2016-03-13T09:17:32.081+0000 I NETWORK [conn4074642] end connection 10.0.179.209:34311 (1505 connections now open) 2016-03-13T09:17:32.081+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40725 #4081297 (1505 connections now open) 2016-03-13T09:17:32.081+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42573 #4081298 (1506 connections now open) 2016-03-13T09:17:32.081+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57587 #4081299 (1507 connections now open) 2016-03-13T09:17:32.082+0000 I NETWORK [conn4076063] end connection 10.0.26.69:37583 (1506 connections now open) 2016-03-13T09:17:32.082+0000 I NETWORK [conn4076802] end connection 10.0.7.113:55332 (1505 connections now open) 2016-03-13T09:17:32.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42574 #4081300 (1506 connections now open) 2016-03-13T09:17:32.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59640 #4081301 (1507 connections now open) 2016-03-13T09:17:32.082+0000 I NETWORK [conn4074600] end connection 10.0.179.209:34269 (1506 connections now open) 2016-03-13T09:17:32.083+0000 I NETWORK [conn4074558] end connection 10.0.26.69:35960 (1505 connections now open) 2016-03-13T09:17:32.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40726 #4081302 (1506 connections now open) 2016-03-13T09:17:32.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42575 #4081303 (1507 connections now open) 2016-03-13T09:17:32.084+0000 I NETWORK [conn4078797] end connection 10.0.20.53:48665 (1506 connections now open) 2016-03-13T09:17:32.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50822 #4081304 (1507 connections now open) 2016-03-13T09:17:32.085+0000 I NETWORK [conn4074420] end connection 10.0.27.237:57611 (1506 connections now open) 2016-03-13T09:17:32.085+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35903 #4081305 (1507 connections now open) 2016-03-13T09:17:32.086+0000 I NETWORK [conn4074803] end connection 10.0.20.53:44511 (1506 connections now open) 2016-03-13T09:17:32.086+0000 I NETWORK [conn4070184] end connection 10.0.20.53:39680 (1505 connections now open) 2016-03-13T09:17:32.086+0000 I NETWORK [conn4076502] end connection 10.0.216.90:53048 (1504 connections now open) 2016-03-13T09:17:32.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50825 #4081306 (1505 connections now open) 2016-03-13T09:17:32.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50824 #4081307 (1506 connections now open) 2016-03-13T09:17:32.087+0000 I NETWORK [conn4077231] end connection 10.0.179.209:36920 (1505 connections now open) 2016-03-13T09:17:32.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57592 #4081308 (1506 connections now open) 2016-03-13T09:17:32.087+0000 I NETWORK [conn4075770] end connection 10.0.27.237:58957 (1505 connections now open) 2016-03-13T09:17:32.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40728 #4081309 (1506 connections now open) 2016-03-13T09:17:32.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35904 #4081310 (1507 connections now open) 2016-03-13T09:17:32.089+0000 I NETWORK [conn4079008] end connection 10.0.20.53:48884 (1506 connections now open) 2016-03-13T09:17:32.089+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50827 #4081311 (1507 connections now open) 2016-03-13T09:17:32.089+0000 I NETWORK [conn4076103] end connection 10.0.26.69:37632 (1506 connections now open) 2016-03-13T09:17:32.090+0000 I NETWORK [conn4077618] end connection 10.0.7.113:56193 (1505 connections now open) 2016-03-13T09:17:32.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42580 #4081312 (1507 connections now open) 2016-03-13T09:17:32.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59644 #4081313 (1507 connections now open) 2016-03-13T09:17:32.091+0000 I NETWORK [conn4075393] end connection 10.0.179.209:35073 (1506 connections now open) 2016-03-13T09:17:32.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40729 #4081314 (1507 connections now open) 2016-03-13T09:17:32.091+0000 I NETWORK [conn4077730] end connection 10.0.27.237:60910 (1506 connections now open) 2016-03-13T09:17:32.091+0000 I NETWORK [conn4075901] end connection 10.0.27.237:59081 (1505 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35906 #4081315 (1506 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35907 #4081316 (1507 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [conn4070308] end connection 10.0.20.53:39807 (1506 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [conn4077871] end connection 10.0.179.209:37553 (1506 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [conn4073932] end connection 10.0.7.113:52260 (1504 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50831 #4081317 (1505 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40730 #4081318 (1506 connections now open) 2016-03-13T09:17:32.092+0000 I NETWORK [conn4075745] end connection 10.0.216.90:52227 (1505 connections now open) 2016-03-13T09:17:32.093+0000 I NETWORK [conn4077170] end connection 10.0.7.113:55711 (1504 connections now open) 2016-03-13T09:17:32.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59645 #4081319 (1505 connections now open) 2016-03-13T09:17:32.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57597 #4081320 (1506 connections now open) 2016-03-13T09:17:32.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59646 #4081321 (1507 connections now open) 2016-03-13T09:17:32.093+0000 I NETWORK [conn4076842] end connection 10.0.216.90:53416 (1506 connections now open) 2016-03-13T09:17:32.093+0000 I NETWORK [conn4076140] end connection 10.0.26.69:37674 (1505 connections now open) 2016-03-13T09:17:32.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57598 #4081322 (1506 connections now open) 2016-03-13T09:17:32.094+0000 I NETWORK [conn4074845] end connection 10.0.20.53:44554 (1505 connections now open) 2016-03-13T09:17:32.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42583 #4081323 (1507 connections now open) 2016-03-13T09:17:32.094+0000 I NETWORK [conn4077771] end connection 10.0.27.237:60950 (1505 connections now open) 2016-03-13T09:17:32.094+0000 I NETWORK [conn4073559] end connection 10.0.7.113:51852 (1504 connections now open) 2016-03-13T09:17:32.094+0000 I NETWORK [conn4075573] end connection 10.0.179.209:35259 (1504 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [conn4074585] end connection 10.0.216.90:50975 (1502 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35909 #4081324 (1503 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59647 #4081325 (1504 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [conn4071160] end connection 10.0.26.69:60551 (1502 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50833 #4081326 (1505 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40732 #4081327 (1506 connections now open) 2016-03-13T09:17:32.095+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-0-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:32.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57599 #4081328 (1506 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42584 #4081329 (1507 connections now open) 2016-03-13T09:17:32.095+0000 I NETWORK [conn4072870] end connection 10.0.26.69:34153 (1506 connections now open) 2016-03-13T09:17:32.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42585 #4081330 (1507 connections now open) 2016-03-13T09:17:32.097+0000 I NETWORK [conn4075979] end connection 10.0.27.237:59165 (1506 connections now open) 2016-03-13T09:17:32.098+0000 I NETWORK [conn4076461] end connection 10.0.216.90:53003 (1505 connections now open) 2016-03-13T09:17:32.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35911 #4081331 (1506 connections now open) 2016-03-13T09:17:32.098+0000 I NETWORK [conn4076354] end connection 10.0.26.69:37909 (1504 connections now open) 2016-03-13T09:17:32.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57601 #4081332 (1506 connections now open) 2016-03-13T09:17:32.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42587 #4081333 (1507 connections now open) 2016-03-13T09:17:32.099+0000 I NETWORK [conn4073084] end connection 10.0.7.113:51348 (1506 connections now open) 2016-03-13T09:17:32.099+0000 I NETWORK [conn4080507] end connection 10.0.7.113:59181 (1505 connections now open) 2016-03-13T09:17:32.099+0000 I NETWORK [conn4077954] end connection 10.0.179.209:37637 (1504 connections now open) 2016-03-13T09:17:32.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59649 #4081334 (1505 connections now open) 2016-03-13T09:17:32.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59650 #4081335 (1506 connections now open) 2016-03-13T09:17:32.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40734 #4081336 (1507 connections now open) 2016-03-13T09:17:32.100+0000 I NETWORK [conn4080616] end connection 10.0.27.237:35659 (1506 connections now open) 2016-03-13T09:17:32.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35914 #4081337 (1507 connections now open) 2016-03-13T09:17:32.101+0000 I NETWORK [conn4076549] end connection 10.0.216.90:53092 (1506 connections now open) 2016-03-13T09:17:32.101+0000 I NETWORK [conn4072993] end connection 10.0.20.53:42601 (1505 connections now open) 2016-03-13T09:17:32.102+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57604 #4081338 (1506 connections now open) 2016-03-13T09:17:32.102+0000 I NETWORK [conn4074232] end connection 10.0.7.113:52572 (1505 connections now open) 2016-03-13T09:17:32.102+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50838 #4081339 (1506 connections now open) 2016-03-13T09:17:32.102+0000 I NETWORK [conn4076632] end connection 10.0.216.90:53183 (1505 connections now open) 2016-03-13T09:17:32.102+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59651 #4081340 (1507 connections now open) 2016-03-13T09:17:32.102+0000 I NETWORK [conn4078077] end connection 10.0.179.209:37769 (1505 connections now open) 2016-03-13T09:17:32.102+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40735 #4081341 (1506 connections now open) 2016-03-13T09:17:32.103+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57605 #4081342 (1507 connections now open) 2016-03-13T09:17:32.103+0000 I NETWORK [conn4075517] end connection 10.0.27.237:58707 (1506 connections now open) 2016-03-13T09:17:32.103+0000 I NETWORK [conn4075366] end connection 10.0.26.69:36822 (1505 connections now open) 2016-03-13T09:17:32.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42593 #4081343 (1506 connections now open) 2016-03-13T09:17:32.104+0000 I NETWORK [conn4074972] end connection 10.0.20.53:44688 (1505 connections now open) 2016-03-13T09:17:32.104+0000 I NETWORK [conn4075028] end connection 10.0.216.90:51443 (1504 connections now open) 2016-03-13T09:17:32.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50840 #4081344 (1505 connections now open) 2016-03-13T09:17:32.105+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35916 #4081345 (1506 connections now open) 2016-03-13T09:17:32.105+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57607 #4081346 (1507 connections now open) 2016-03-13T09:17:32.105+0000 I NETWORK [conn4072367] end connection 10.0.20.53:41934 (1506 connections now open) 2016-03-13T09:17:32.106+0000 I NETWORK [conn4077321] end connection 10.0.7.113:55869 (1505 connections now open) 2016-03-13T09:17:32.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50841 #4081347 (1506 connections now open) 2016-03-13T09:17:32.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59653 #4081348 (1507 connections now open) 2016-03-13T09:17:32.107+0000 I NETWORK [conn4071862] end connection 10.0.27.237:54951 (1506 connections now open) 2016-03-13T09:17:32.108+0000 I NETWORK [conn4076741] end connection 10.0.26.69:38322 (1505 connections now open) 2016-03-13T09:17:32.108+0000 I NETWORK [conn4077408] end connection 10.0.7.113:55961 (1504 connections now open) 2016-03-13T09:17:32.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42597 #4081349 (1505 connections now open) 2016-03-13T09:17:32.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35919 #4081350 (1506 connections now open) 2016-03-13T09:17:32.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59654 #4081351 (1507 connections now open) 2016-03-13T09:17:32.108+0000 I NETWORK [conn4079939] end connection 10.0.26.69:41655 (1506 connections now open) 2016-03-13T09:17:32.108+0000 I NETWORK [conn4076800] end connection 10.0.216.90:53369 (1506 connections now open) 2016-03-13T09:17:32.109+0000 I NETWORK [conn4080591] end connection 10.0.7.113:59270 (1505 connections now open) 2016-03-13T09:17:32.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42598 #4081352 (1505 connections now open) 2016-03-13T09:17:32.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59655 #4081353 (1506 connections now open) 2016-03-13T09:17:32.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57610 #4081354 (1507 connections now open) 2016-03-13T09:17:32.110+0000 I NETWORK [conn4076268] end connection 10.0.27.237:59462 (1506 connections now open) 2016-03-13T09:17:32.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35920 #4081355 (1507 connections now open) 2016-03-13T09:17:32.110+0000 I NETWORK [conn4071456] end connection 10.0.20.53:40996 (1506 connections now open) 2016-03-13T09:17:32.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50847 #4081356 (1507 connections now open) 2016-03-13T09:17:32.111+0000 I NETWORK [conn4075266] end connection 10.0.20.53:44996 (1506 connections now open) 2016-03-13T09:17:32.112+0000 I NETWORK [conn4071628] end connection 10.0.26.69:32811 (1505 connections now open) 2016-03-13T09:17:32.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50848 #4081357 (1506 connections now open) 2016-03-13T09:17:32.112+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42600 #4081358 (1507 connections now open) 2016-03-13T09:17:32.113+0000 I NETWORK [conn4076550] end connection 10.0.7.113:55061 (1506 connections now open) 2016-03-13T09:17:32.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59657 #4081359 (1507 connections now open) 2016-03-13T09:17:32.113+0000 I NETWORK [conn4072662] end connection 10.0.216.90:48905 (1505 connections now open) 2016-03-13T09:17:32.113+0000 I NETWORK [conn4072496] end connection 10.0.26.69:33760 (1505 connections now open) 2016-03-13T09:17:32.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57613 #4081360 (1506 connections now open) 2016-03-13T09:17:32.113+0000 I NETWORK [conn4078296] end connection 10.0.179.209:37979 (1504 connections now open) 2016-03-13T09:17:32.114+0000 I NETWORK [conn4074816] end connection 10.0.20.53:44525 (1504 connections now open) 2016-03-13T09:17:32.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42602 #4081361 (1505 connections now open) 2016-03-13T09:17:32.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40739 #4081362 (1506 connections now open) 2016-03-13T09:17:32.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50850 #4081363 (1507 connections now open) 2016-03-13T09:17:32.114+0000 I NETWORK [conn4077661] end connection 10.0.7.113:56241 (1506 connections now open) 2016-03-13T09:17:32.115+0000 I NETWORK [conn4077342] end connection 10.0.20.53:47170 (1505 connections now open) 2016-03-13T09:17:32.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59659 #4081364 (1506 connections now open) 2016-03-13T09:17:32.115+0000 I NETWORK [conn4072048] end connection 10.0.26.69:33269 (1505 connections now open) 2016-03-13T09:17:32.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50853 #4081365 (1506 connections now open) 2016-03-13T09:17:32.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42604 #4081366 (1507 connections now open) 2016-03-13T09:17:32.115+0000 I NETWORK [conn4078378] end connection 10.0.179.209:38064 (1505 connections now open) 2016-03-13T09:17:32.115+0000 I NETWORK [conn4077126] end connection 10.0.26.69:38733 (1505 connections now open) 2016-03-13T09:17:32.116+0000 I NETWORK [conn4072722] end connection 10.0.20.53:42306 (1504 connections now open) 2016-03-13T09:17:32.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40740 #4081367 (1505 connections now open) 2016-03-13T09:17:32.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42605 #4081368 (1506 connections now open) 2016-03-13T09:17:32.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50854 #4081369 (1507 connections now open) 2016-03-13T09:17:32.116+0000 I NETWORK [conn4079220] end connection 10.0.7.113:57871 (1506 connections now open) 2016-03-13T09:17:32.117+0000 I NETWORK [conn4076332] end connection 10.0.27.237:59518 (1505 connections now open) 2016-03-13T09:17:32.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59660 #4081370 (1506 connections now open) 2016-03-13T09:17:32.117+0000 I NETWORK [conn4074986] end connection 10.0.20.53:44703 (1505 connections now open) 2016-03-13T09:17:32.117+0000 I NETWORK [conn4072994] end connection 10.0.216.90:49266 (1504 connections now open) 2016-03-13T09:17:32.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50855 #4081371 (1505 connections now open) 2016-03-13T09:17:32.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57614 #4081372 (1506 connections now open) 2016-03-13T09:17:32.118+0000 I NETWORK [conn4080191] end connection 10.0.20.53:50131 (1505 connections now open) 2016-03-13T09:17:32.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50856 #4081373 (1506 connections now open) 2016-03-13T09:17:32.118+0000 I NETWORK [conn4074438] end connection 10.0.7.113:52803 (1504 connections now open) 2016-03-13T09:17:32.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59661 #4081374 (1506 connections now open) 2016-03-13T09:17:32.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35924 #4081375 (1507 connections now open) 2016-03-13T09:17:32.119+0000 I NETWORK [conn4076801] end connection 10.0.27.237:59987 (1505 connections now open) 2016-03-13T09:17:32.119+0000 I NETWORK [conn4073500] end connection 10.0.216.90:49790 (1505 connections now open) 2016-03-13T09:17:32.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35925 #4081376 (1506 connections now open) 2016-03-13T09:17:32.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57615 #4081377 (1507 connections now open) 2016-03-13T09:17:32.120+0000 I NETWORK [conn4077748] end connection 10.0.7.113:56330 (1506 connections now open) 2016-03-13T09:17:32.120+0000 I NETWORK [conn4075407] end connection 10.0.20.53:45146 (1506 connections now open) 2016-03-13T09:17:32.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59662 #4081378 (1506 connections now open) 2016-03-13T09:17:32.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50860 #4081379 (1507 connections now open) 2016-03-13T09:17:32.121+0000 I NETWORK [conn4076417] end connection 10.0.216.90:52959 (1506 connections now open) 2016-03-13T09:17:32.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57616 #4081380 (1507 connections now open) 2016-03-13T09:17:32.122+0000 I NETWORK [conn4073474] end connection 10.0.27.237:56665 (1506 connections now open) 2016-03-13T09:17:32.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35929 #4081381 (1507 connections now open) 2016-03-13T09:17:32.123+0000 I NETWORK [conn4076974] end connection 10.0.216.90:53553 (1506 connections now open) 2016-03-13T09:17:32.123+0000 I NETWORK [conn4075916] end connection 10.0.26.69:37422 (1505 connections now open) 2016-03-13T09:17:32.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57617 #4081382 (1506 connections now open) 2016-03-13T09:17:32.123+0000 I NETWORK [conn4076870] end connection 10.0.26.69:38460 (1505 connections now open) 2016-03-13T09:17:32.124+0000 I NETWORK [conn4076158] end connection 10.0.27.237:59348 (1504 connections now open) 2016-03-13T09:17:32.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42612 #4081383 (1506 connections now open) 2016-03-13T09:17:32.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42613 #4081384 (1506 connections now open) 2016-03-13T09:17:32.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35932 #4081385 (1507 connections now open) 2016-03-13T09:17:32.125+0000 I NETWORK [conn4077874] end connection 10.0.216.90:54498 (1506 connections now open) 2016-03-13T09:17:32.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57618 #4081386 (1507 connections now open) 2016-03-13T09:17:32.126+0000 I NETWORK [conn4079499] end connection 10.0.27.237:34467 (1506 connections now open) 2016-03-13T09:17:32.126+0000 I NETWORK [conn4071671] end connection 10.0.27.237:54738 (1505 connections now open) 2016-03-13T09:17:32.126+0000 I NETWORK [conn4078039] end connection 10.0.179.209:37724 (1505 connections now open) 2016-03-13T09:17:32.126+0000 I NETWORK [conn4073292] end connection 10.0.27.237:56481 (1503 connections now open) 2016-03-13T09:17:32.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35933 #4081387 (1504 connections now open) 2016-03-13T09:17:32.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40745 #4081388 (1505 connections now open) 2016-03-13T09:17:32.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35934 #4081389 (1506 connections now open) 2016-03-13T09:17:32.127+0000 I NETWORK [conn4073415] end connection 10.0.216.90:49701 (1505 connections now open) 2016-03-13T09:17:32.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57619 #4081390 (1506 connections now open) 2016-03-13T09:17:32.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35935 #4081391 (1507 connections now open) 2016-03-13T09:17:32.129+0000 I NETWORK [conn4074801] end connection 10.0.216.90:51203 (1506 connections now open) 2016-03-13T09:17:32.129+0000 I NETWORK [conn4076039] end connection 10.0.20.53:45822 (1506 connections now open) 2016-03-13T09:17:32.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57620 #4081392 (1506 connections now open) 2016-03-13T09:17:32.130+0000 I NETWORK [conn4076041] end connection 10.0.27.237:59226 (1505 connections now open) 2016-03-13T09:17:32.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50870 #4081393 (1506 connections now open) 2016-03-13T09:17:32.130+0000 I NETWORK [conn4074773] end connection 10.0.26.69:36193 (1505 connections now open) 2016-03-13T09:17:32.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42619 #4081394 (1506 connections now open) 2016-03-13T09:17:32.131+0000 I NETWORK [conn4074671] end connection 10.0.26.69:36079 (1505 connections now open) 2016-03-13T09:17:32.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42620 #4081395 (1506 connections now open) 2016-03-13T09:17:32.131+0000 I NETWORK [conn4077685] end connection 10.0.20.53:47528 (1504 connections now open) 2016-03-13T09:17:32.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50871 #4081396 (1506 connections now open) 2016-03-13T09:17:32.132+0000 I NETWORK [conn4071919] end connection 10.0.26.69:33136 (1505 connections now open) 2016-03-13T09:17:32.132+0000 I NETWORK [conn4076484] end connection 10.0.27.237:59670 (1504 connections now open) 2016-03-13T09:17:32.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42621 #4081397 (1505 connections now open) 2016-03-13T09:17:32.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35939 #4081398 (1506 connections now open) 2016-03-13T09:17:32.133+0000 I NETWORK [conn4079176] end connection 10.0.7.113:57832 (1505 connections now open) 2016-03-13T09:17:32.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59668 #4081399 (1506 connections now open) 2016-03-13T09:17:32.133+0000 I NETWORK [conn4072005] end connection 10.0.26.69:33229 (1505 connections now open) 2016-03-13T09:17:32.134+0000 I NETWORK [conn4077728] end connection 10.0.20.53:47569 (1504 connections now open) 2016-03-13T09:17:32.134+0000 I NETWORK [conn4074063] end connection 10.0.26.69:35418 (1503 connections now open) 2016-03-13T09:17:32.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50873 #4081400 (1504 connections now open) 2016-03-13T09:17:32.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42623 #4081401 (1505 connections now open) 2016-03-13T09:17:32.134+0000 I NETWORK [conn4078822] end connection 10.0.27.237:33764 (1504 connections now open) 2016-03-13T09:17:32.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42624 #4081402 (1506 connections now open) 2016-03-13T09:17:32.135+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35940 #4081403 (1506 connections now open) 2016-03-13T09:17:32.135+0000 I NETWORK [conn4078740] end connection 10.0.27.237:33688 (1505 connections now open) 2016-03-13T09:17:32.136+0000 I NETWORK [conn4071329] end connection 10.0.20.53:40862 (1504 connections now open) 2016-03-13T09:17:32.136+0000 I NETWORK [conn4078165] end connection 10.0.216.90:54812 (1503 connections now open) 2016-03-13T09:17:32.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50875 #4081404 (1504 connections now open) 2016-03-13T09:17:32.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35941 #4081405 (1505 connections now open) 2016-03-13T09:17:32.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57623 #4081406 (1506 connections now open) 2016-03-13T09:17:32.137+0000 I NETWORK [conn4079160] end connection 10.0.27.237:34108 (1505 connections now open) 2016-03-13T09:17:32.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35942 #4081407 (1506 connections now open) 2016-03-13T09:17:32.137+0000 I NETWORK [conn4078465] end connection 10.0.179.209:38151 (1504 connections now open) 2016-03-13T09:17:32.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35943 #4081408 (1506 connections now open) 2016-03-13T09:17:32.137+0000 I NETWORK [conn4072789] end connection 10.0.26.69:34068 (1505 connections now open) 2016-03-13T09:17:32.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40748 #4081409 (1507 connections now open) 2016-03-13T09:17:32.138+0000 I NETWORK [conn4074088] end connection 10.0.216.90:50430 (1505 connections now open) 2016-03-13T09:17:32.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42627 #4081410 (1506 connections now open) 2016-03-13T09:17:32.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57624 #4081411 (1507 connections now open) 2016-03-13T09:17:32.139+0000 I NETWORK [conn4077213] end connection 10.0.27.237:60406 (1506 connections now open) 2016-03-13T09:17:32.139+0000 I NETWORK [conn4077854] end connection 10.0.20.53:47694 (1505 connections now open) 2016-03-13T09:17:32.139+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35944 #4081412 (1506 connections now open) 2016-03-13T09:17:32.139+0000 I NETWORK [conn4077938] end connection 10.0.20.53:47779 (1505 connections now open) 2016-03-13T09:17:32.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50878 #4081413 (1506 connections now open) 2016-03-13T09:17:32.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50879 #4081414 (1507 connections now open) 2016-03-13T09:17:32.141+0000 I NETWORK [conn4077747] end connection 10.0.26.69:39382 (1506 connections now open) 2016-03-13T09:17:32.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42630 #4081415 (1507 connections now open) 2016-03-13T09:17:32.142+0000 I NETWORK [conn4071563] end connection 10.0.20.53:41119 (1506 connections now open) 2016-03-13T09:17:32.142+0000 I NETWORK [conn4071879] end connection 10.0.20.53:41442 (1505 connections now open) 2016-03-13T09:17:32.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50880 #4081416 (1506 connections now open) 2016-03-13T09:17:32.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50881 #4081417 (1507 connections now open) 2016-03-13T09:17:32.143+0000 I NETWORK [conn4077472] end connection 10.0.27.237:60658 (1506 connections now open) 2016-03-13T09:17:32.144+0000 I NETWORK [conn4078448] end connection 10.0.20.53:48293 (1505 connections now open) 2016-03-13T09:17:32.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35948 #4081418 (1506 connections now open) 2016-03-13T09:17:32.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50882 #4081419 (1507 connections now open) 2016-03-13T09:17:32.144+0000 I NETWORK [conn4077790] end connection 10.0.216.90:54408 (1506 connections now open) 2016-03-13T09:17:32.145+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57627 #4081420 (1507 connections now open) 2016-03-13T09:17:32.146+0000 I NETWORK [conn4075726] end connection 10.0.20.53:45493 (1506 connections now open) 2016-03-13T09:17:32.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50884 #4081421 (1507 connections now open) 2016-03-13T09:17:32.146+0000 I NETWORK [conn4074213] end connection 10.0.27.237:57398 (1506 connections now open) 2016-03-13T09:17:32.146+0000 I NETWORK [conn4077081] end connection 10.0.26.69:38683 (1505 connections now open) 2016-03-13T09:17:32.147+0000 I NETWORK [conn4080658] end connection 10.0.20.53:50637 (1504 connections now open) 2016-03-13T09:17:32.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35949 #4081422 (1506 connections now open) 2016-03-13T09:17:32.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42634 #4081423 (1506 connections now open) 2016-03-13T09:17:32.147+0000 I NETWORK [conn4072300] end connection 10.0.26.69:33539 (1505 connections now open) 2016-03-13T09:17:32.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50885 #4081424 (1506 connections now open) 2016-03-13T09:17:32.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42635 #4081425 (1507 connections now open) 2016-03-13T09:17:32.148+0000 I NETWORK [conn4077981] end connection 10.0.20.53:47819 (1506 connections now open) 2016-03-13T09:17:32.148+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50886 #4081426 (1507 connections now open) 2016-03-13T09:17:32.149+0000 I NETWORK [conn4073011] end connection 10.0.20.53:42623 (1506 connections now open) 2016-03-13T09:17:32.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50887 #4081427 (1507 connections now open) 2016-03-13T09:17:32.150+0000 I NETWORK [conn4075856] end connection 10.0.20.53:45631 (1506 connections now open) 2016-03-13T09:17:32.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50888 #4081428 (1507 connections now open) 2016-03-13T09:17:32.150+0000 I NETWORK [conn4073684] end connection 10.0.7.113:51990 (1506 connections now open) 2016-03-13T09:17:32.150+0000 I NETWORK [conn4077557] end connection 10.0.27.237:60742 (1505 connections now open) 2016-03-13T09:17:32.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59675 #4081429 (1506 connections now open) 2016-03-13T09:17:32.151+0000 I NETWORK [conn4072387] end connection 10.0.26.69:33630 (1504 connections now open) 2016-03-13T09:17:32.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42637 #4081430 (1506 connections now open) 2016-03-13T09:17:32.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35952 #4081431 (1507 connections now open) 2016-03-13T09:17:32.152+0000 I NETWORK [conn4073290] end connection 10.0.20.53:42917 (1506 connections now open) 2016-03-13T09:17:32.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50891 #4081432 (1507 connections now open) 2016-03-13T09:17:32.153+0000 I NETWORK [conn4072680] end connection 10.0.20.53:42261 (1506 connections now open) 2016-03-13T09:17:32.154+0000 I NETWORK [conn4076287] end connection 10.0.26.69:37835 (1505 connections now open) 2016-03-13T09:17:32.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50892 #4081433 (1506 connections now open) 2016-03-13T09:17:32.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42641 #4081434 (1507 connections now open) 2016-03-13T09:17:32.155+0000 I NETWORK [conn4080566] end connection 10.0.20.53:50542 (1506 connections now open) 2016-03-13T09:17:32.156+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50894 #4081435 (1507 connections now open) 2016-03-13T09:17:32.156+0000 I NETWORK [conn4072468] end connection 10.0.26.69:33720 (1506 connections now open) 2016-03-13T09:17:32.156+0000 I NETWORK [conn4074173] end connection 10.0.27.237:57358 (1505 connections now open) 2016-03-13T09:17:32.156+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42644 #4081436 (1506 connections now open) 2016-03-13T09:17:32.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35954 #4081437 (1507 connections now open) 2016-03-13T09:17:32.157+0000 I NETWORK [conn4075899] end connection 10.0.20.53:45675 (1506 connections now open) 2016-03-13T09:17:32.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50895 #4081438 (1507 connections now open) 2016-03-13T09:17:32.157+0000 I NETWORK [conn4072047] end connection 10.0.20.53:41619 (1505 connections now open) 2016-03-13T09:17:32.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50896 #4081439 (1507 connections now open) 2016-03-13T09:17:32.158+0000 I NETWORK [conn4075409] end connection 10.0.27.237:58597 (1506 connections now open) 2016-03-13T09:17:32.158+0000 I NETWORK [conn4073308] end connection 10.0.26.69:34607 (1505 connections now open) 2016-03-13T09:17:32.159+0000 I NETWORK [conn4076185] end connection 10.0.27.237:59377 (1504 connections now open) 2016-03-13T09:17:32.159+0000 I NETWORK [conn4077915] end connection 10.0.7.113:56511 (1503 connections now open) 2016-03-13T09:17:32.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35955 #4081440 (1505 connections now open) 2016-03-13T09:17:32.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35956 #4081441 (1505 connections now open) 2016-03-13T09:17:32.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42647 #4081442 (1506 connections now open) 2016-03-13T09:17:32.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59679 #4081443 (1507 connections now open) 2016-03-13T09:17:32.160+0000 I NETWORK [conn4078632] end connection 10.0.26.69:40323 (1506 connections now open) 2016-03-13T09:17:32.161+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42648 #4081444 (1507 connections now open) 2016-03-13T09:17:32.162+0000 I NETWORK [conn4076245] end connection 10.0.27.237:59431 (1506 connections now open) 2016-03-13T09:17:32.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35959 #4081445 (1507 connections now open) 2016-03-13T09:17:32.162+0000 I NETWORK [conn4076634] end connection 10.0.26.69:38202 (1506 connections now open) 2016-03-13T09:17:32.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42650 #4081446 (1507 connections now open) 2016-03-13T09:17:32.163+0000 I NETWORK [conn4077575] end connection 10.0.27.237:60752 (1506 connections now open) 2016-03-13T09:17:32.163+0000 I NETWORK [conn4072969] end connection 10.0.26.69:34254 (1506 connections now open) 2016-03-13T09:17:32.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35960 #4081447 (1506 connections now open) 2016-03-13T09:17:32.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42651 #4081448 (1507 connections now open) 2016-03-13T09:17:32.164+0000 I NETWORK [conn4078023] end connection 10.0.27.237:32964 (1506 connections now open) 2016-03-13T09:17:32.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35961 #4081449 (1507 connections now open) 2016-03-13T09:17:32.165+0000 I NETWORK [conn4078105] end connection 10.0.20.53:47945 (1506 connections now open) 2016-03-13T09:17:32.165+0000 I NETWORK [conn4071582] end connection 10.0.20.53:41129 (1505 connections now open) 2016-03-13T09:17:32.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50900 #4081450 (1506 connections now open) 2016-03-13T09:17:32.165+0000 I NETWORK [conn4077914] end connection 10.0.216.90:54539 (1505 connections now open) 2016-03-13T09:17:32.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50901 #4081451 (1506 connections now open) 2016-03-13T09:17:32.166+0000 I NETWORK [conn4075978] end connection 10.0.20.53:45764 (1505 connections now open) 2016-03-13T09:17:32.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57636 #4081452 (1506 connections now open) 2016-03-13T09:17:32.166+0000 I NETWORK [conn4078037] end connection 10.0.7.113:56647 (1505 connections now open) 2016-03-13T09:17:32.166+0000 I NETWORK [conn4075055] end connection 10.0.27.237:58240 (1504 connections now open) 2016-03-13T09:17:32.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50902 #4081453 (1505 connections now open) 2016-03-13T09:17:32.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59681 #4081454 (1506 connections now open) 2016-03-13T09:17:32.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35963 #4081455 (1507 connections now open) 2016-03-13T09:17:32.167+0000 I NETWORK [conn4075491] end connection 10.0.216.90:51948 (1506 connections now open) 2016-03-13T09:17:32.167+0000 I NETWORK [conn4078573] end connection 10.0.20.53:48421 (1505 connections now open) 2016-03-13T09:17:32.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57637 #4081456 (1506 connections now open) 2016-03-13T09:17:32.168+0000 I NETWORK [conn4075702] end connection 10.0.26.69:37191 (1505 connections now open) 2016-03-13T09:17:32.168+0000 I NETWORK [conn4077617] end connection 10.0.27.237:60791 (1504 connections now open) 2016-03-13T09:17:32.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50903 #4081457 (1506 connections now open) 2016-03-13T09:17:32.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35965 #4081458 (1506 connections now open) 2016-03-13T09:17:32.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42656 #4081459 (1507 connections now open) 2016-03-13T09:17:32.169+0000 I NETWORK [conn4073309] end connection 10.0.20.53:42938 (1506 connections now open) 2016-03-13T09:17:32.169+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50904 #4081460 (1507 connections now open) 2016-03-13T09:17:32.169+0000 I NETWORK [conn4073771] end connection 10.0.27.237:56959 (1506 connections now open) 2016-03-13T09:17:32.169+0000 I NETWORK [conn4073876] end connection 10.0.20.53:43540 (1505 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35967 #4081461 (1506 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [conn4076396] end connection 10.0.20.53:46214 (1505 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [conn4074689] end connection 10.0.7.113:53068 (1504 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [conn4073623] end connection 10.0.26.69:34946 (1503 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50905 #4081462 (1504 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [conn4073099] end connection 10.0.26.69:34384 (1502 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50906 #4081463 (1505 connections now open) 2016-03-13T09:17:32.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59682 #4081464 (1506 connections now open) 2016-03-13T09:17:32.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42658 #4081465 (1506 connections now open) 2016-03-13T09:17:32.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42659 #4081466 (1507 connections now open) 2016-03-13T09:17:32.171+0000 I NETWORK [conn4075114] end connection 10.0.20.53:44837 (1506 connections now open) 2016-03-13T09:17:32.171+0000 I NETWORK [conn4078295] end connection 10.0.216.90:54951 (1505 connections now open) 2016-03-13T09:17:32.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50907 #4081467 (1506 connections now open) 2016-03-13T09:17:32.171+0000 I NETWORK [conn4072511] end connection 10.0.27.237:55648 (1504 connections now open) 2016-03-13T09:17:32.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57639 #4081468 (1506 connections now open) 2016-03-13T09:17:32.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35970 #4081469 (1507 connections now open) 2016-03-13T09:17:32.173+0000 I NETWORK [conn4077125] end connection 10.0.20.53:46954 (1506 connections now open) 2016-03-13T09:17:32.173+0000 I NETWORK [conn4076635] end connection 10.0.27.237:59818 (1505 connections now open) 2016-03-13T09:17:32.173+0000 I NETWORK [conn4080234] end connection 10.0.26.69:41951 (1504 connections now open) 2016-03-13T09:17:32.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50909 #4081470 (1506 connections now open) 2016-03-13T09:17:32.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35971 #4081471 (1506 connections now open) 2016-03-13T09:17:32.174+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42662 #4081472 (1507 connections now open) 2016-03-13T09:17:32.174+0000 I NETWORK [conn4072594] end connection 10.0.27.237:55739 (1506 connections now open) 2016-03-13T09:17:32.174+0000 I NETWORK [conn4076867] end connection 10.0.20.53:46701 (1505 connections now open) 2016-03-13T09:17:32.174+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35972 #4081473 (1506 connections now open) 2016-03-13T09:17:32.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50911 #4081474 (1507 connections now open) 2016-03-13T09:17:32.175+0000 I NETWORK [conn4079327] end connection 10.0.20.53:49218 (1506 connections now open) 2016-03-13T09:17:32.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50912 #4081475 (1507 connections now open) 2016-03-13T09:17:32.176+0000 I NETWORK [conn4078334] end connection 10.0.216.90:54996 (1506 connections now open) 2016-03-13T09:17:32.176+0000 I NETWORK [conn4078253] end connection 10.0.27.237:33186 (1505 connections now open) 2016-03-13T09:17:32.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57641 #4081476 (1506 connections now open) 2016-03-13T09:17:32.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35975 #4081477 (1507 connections now open) 2016-03-13T09:17:32.177+0000 I NETWORK [conn4077788] end connection 10.0.26.69:39422 (1506 connections now open) 2016-03-13T09:17:32.178+0000 I NETWORK [conn4074147] end connection 10.0.7.113:52482 (1505 connections now open) 2016-03-13T09:17:32.178+0000 I NETWORK [conn4076912] end connection 10.0.20.53:46745 (1504 connections now open) 2016-03-13T09:17:32.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42666 #4081478 (1505 connections now open) 2016-03-13T09:17:32.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59684 #4081479 (1506 connections now open) 2016-03-13T09:17:32.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50914 #4081480 (1507 connections now open) 2016-03-13T09:17:32.179+0000 I NETWORK [conn4076845] end connection 10.0.27.237:60029 (1506 connections now open) 2016-03-13T09:17:32.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35977 #4081481 (1507 connections now open) 2016-03-13T09:17:32.179+0000 I NETWORK [conn4078591] end connection 10.0.27.237:33526 (1506 connections now open) 2016-03-13T09:17:32.179+0000 I NETWORK [conn4073082] end connection 10.0.26.69:34363 (1506 connections now open) 2016-03-13T09:17:32.180+0000 I NETWORK [conn4077319] end connection 10.0.26.69:38931 (1504 connections now open) 2016-03-13T09:17:32.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42667 #4081482 (1505 connections now open) 2016-03-13T09:17:32.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35978 #4081483 (1506 connections now open) 2016-03-13T09:17:32.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42668 #4081484 (1507 connections now open) 2016-03-13T09:17:32.183+0000 I NETWORK [conn4074316] end connection 10.0.20.53:43994 (1506 connections now open) 2016-03-13T09:17:32.184+0000 I NETWORK [conn4075367] end connection 10.0.27.237:58558 (1505 connections now open) 2016-03-13T09:17:32.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50918 #4081485 (1506 connections now open) 2016-03-13T09:17:32.184+0000 I NETWORK [conn4074886] end connection 10.0.26.69:36299 (1505 connections now open) 2016-03-13T09:17:32.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35980 #4081486 (1507 connections now open) 2016-03-13T09:17:32.185+0000 I NETWORK [conn4078968] end connection 10.0.27.237:33915 (1505 connections now open) 2016-03-13T09:17:32.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42670 #4081487 (1507 connections now open) 2016-03-13T09:17:32.185+0000 I NETWORK [conn4076101] end connection 10.0.20.53:45896 (1505 connections now open) 2016-03-13T09:17:32.185+0000 I NETWORK [conn4078336] end connection 10.0.7.113:56956 (1504 connections now open) 2016-03-13T09:17:32.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35982 #4081488 (1505 connections now open) 2016-03-13T09:17:32.186+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59686 #4081489 (1506 connections now open) 2016-03-13T09:17:32.186+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50920 #4081490 (1507 connections now open) 2016-03-13T09:17:32.187+0000 I NETWORK [conn4078254] end connection 10.0.7.113:56865 (1506 connections now open) 2016-03-13T09:17:32.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59687 #4081491 (1507 connections now open) 2016-03-13T09:17:32.189+0000 I NETWORK [conn4077405] end connection 10.0.26.69:39024 (1506 connections now open) 2016-03-13T09:17:32.189+0000 I NETWORK [conn4078464] end connection 10.0.7.113:57092 (1505 connections now open) 2016-03-13T09:17:32.189+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42675 #4081492 (1506 connections now open) 2016-03-13T09:17:32.189+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59688 #4081493 (1507 connections now open) 2016-03-13T09:17:32.190+0000 I NETWORK [conn4078360] end connection 10.0.27.237:33301 (1506 connections now open) 2016-03-13T09:17:32.190+0000 I NETWORK [conn4076460] end connection 10.0.20.53:46273 (1505 connections now open) 2016-03-13T09:17:32.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35984 #4081494 (1506 connections now open) 2016-03-13T09:17:32.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50924 #4081495 (1507 connections now open) 2016-03-13T09:17:32.191+0000 I NETWORK [conn4077018] end connection 10.0.27.237:60198 (1506 connections now open) 2016-03-13T09:17:32.191+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35985 #4081496 (1507 connections now open) 2016-03-13T09:17:32.191+0000 I NETWORK [conn4077489] end connection 10.0.26.69:39111 (1506 connections now open) 2016-03-13T09:17:32.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42676 #4081497 (1507 connections now open) 2016-03-13T09:17:32.192+0000 I NETWORK [conn4078449] end connection 10.0.27.237:33387 (1506 connections now open) 2016-03-13T09:17:32.192+0000 I NETWORK [conn4076547] end connection 10.0.20.53:46360 (1505 connections now open) 2016-03-13T09:17:32.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35986 #4081498 (1506 connections now open) 2016-03-13T09:17:32.192+0000 I NETWORK [conn4078281] end connection 10.0.26.69:39951 (1505 connections now open) 2016-03-13T09:17:32.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50926 #4081499 (1506 connections now open) 2016-03-13T09:17:32.193+0000 I NETWORK [conn4075725] end connection 10.0.179.209:35411 (1505 connections now open) 2016-03-13T09:17:32.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42677 #4081500 (1506 connections now open) 2016-03-13T09:17:32.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40749 #4081501 (1507 connections now open) 2016-03-13T09:17:32.194+0000 I NETWORK [conn4078550] end connection 10.0.7.113:57184 (1506 connections now open) 2016-03-13T09:17:32.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59690 #4081502 (1507 connections now open) 2016-03-13T09:17:32.194+0000 I NETWORK [conn4078150] end connection 10.0.26.69:39830 (1506 connections now open) 2016-03-13T09:17:32.195+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42679 #4081503 (1507 connections now open) 2016-03-13T09:17:32.195+0000 I NETWORK [conn4073248] end connection 10.0.20.53:42871 (1506 connections now open) 2016-03-13T09:17:32.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50929 #4081504 (1507 connections now open) 2016-03-13T09:17:32.198+0000 I NETWORK [conn4078633] end connection 10.0.27.237:33566 (1506 connections now open) 2016-03-13T09:17:32.198+0000 I NETWORK [conn4074230] end connection 10.0.26.69:35600 (1505 connections now open) 2016-03-13T09:17:32.198+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35991 #4081505 (1506 connections now open) 2016-03-13T09:17:32.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42682 #4081506 (1507 connections now open) 2016-03-13T09:17:32.199+0000 I NETWORK [conn4074062] end connection 10.0.20.53:43731 (1505 connections now open) 2016-03-13T09:17:32.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50932 #4081507 (1507 connections now open) 2016-03-13T09:17:32.200+0000 I NETWORK [conn4077343] end connection 10.0.179.209:37025 (1506 connections now open) 2016-03-13T09:17:32.201+0000 I NETWORK [conn4077513] end connection 10.0.20.53:47347 (1505 connections now open) 2016-03-13T09:17:32.201+0000 I NETWORK [conn4078490] end connection 10.0.27.237:33433 (1504 connections now open) 2016-03-13T09:17:32.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40751 #4081508 (1505 connections now open) 2016-03-13T09:17:32.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50933 #4081509 (1506 connections now open) 2016-03-13T09:17:32.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35993 #4081510 (1507 connections now open) 2016-03-13T09:17:32.201+0000 I NETWORK [conn4077147] end connection 10.0.27.237:60331 (1506 connections now open) 2016-03-13T09:17:32.201+0000 I NETWORK [conn4073809] end connection 10.0.26.69:35150 (1505 connections now open) 2016-03-13T09:17:32.202+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35994 #4081511 (1506 connections now open) 2016-03-13T09:17:32.202+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42685 #4081512 (1507 connections now open) 2016-03-13T09:17:32.203+0000 I NETWORK [conn4077431] end connection 10.0.27.237:60611 (1506 connections now open) 2016-03-13T09:17:32.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35995 #4081513 (1507 connections now open) 2016-03-13T09:17:32.203+0000 I NETWORK [conn4075323] end connection 10.0.20.53:45058 (1506 connections now open) 2016-03-13T09:17:32.203+0000 I NETWORK [conn4075180] end connection 10.0.179.209:34858 (1505 connections now open) 2016-03-13T09:17:32.204+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50936 #4081514 (1506 connections now open) 2016-03-13T09:17:32.204+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40752 #4081515 (1507 connections now open) 2016-03-13T09:17:32.204+0000 I NETWORK [conn4077979] end connection 10.0.26.69:39647 (1506 connections now open) 2016-03-13T09:17:32.204+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42687 #4081516 (1507 connections now open) 2016-03-13T09:17:32.206+0000 I NETWORK [conn4077015] end connection 10.0.20.53:46844 (1506 connections now open) 2016-03-13T09:17:32.206+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50938 #4081517 (1507 connections now open) 2016-03-13T09:17:32.206+0000 I NETWORK [conn4079329] end connection 10.0.26.69:41035 (1506 connections now open) 2016-03-13T09:17:32.207+0000 I NETWORK [conn4075532] end connection 10.0.27.237:58730 (1505 connections now open) 2016-03-13T09:17:32.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42689 #4081518 (1507 connections now open) 2016-03-13T09:17:32.207+0000 I NETWORK [conn4079749] end connection 10.0.20.53:49668 (1505 connections now open) 2016-03-13T09:17:32.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35997 #4081519 (1506 connections now open) 2016-03-13T09:17:32.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50939 #4081520 (1507 connections now open) 2016-03-13T09:17:32.209+0000 I NETWORK [conn4073558] end connection 10.0.26.69:34877 (1506 connections now open) 2016-03-13T09:17:32.210+0000 I NETWORK [conn4078531] end connection 10.0.27.237:33472 (1505 connections now open) 2016-03-13T09:17:32.210+0000 I NETWORK [conn4077277] end connection 10.0.7.113:55821 (1504 connections now open) 2016-03-13T09:17:32.210+0000 I NETWORK [conn4078657] end connection 10.0.179.209:38354 (1503 connections now open) 2016-03-13T09:17:32.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42691 #4081521 (1504 connections now open) 2016-03-13T09:17:32.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:35999 #4081522 (1505 connections now open) 2016-03-13T09:17:32.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59693 #4081523 (1506 connections now open) 2016-03-13T09:17:32.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40754 #4081524 (1507 connections now open) 2016-03-13T09:17:32.211+0000 I NETWORK [conn4072968] end connection 10.0.27.237:56143 (1506 connections now open) 2016-03-13T09:17:32.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36000 #4081525 (1507 connections now open) 2016-03-13T09:17:32.211+0000 I NETWORK [conn4077599] end connection 10.0.20.53:47440 (1506 connections now open) 2016-03-13T09:17:32.212+0000 I NETWORK [conn4075279] end connection 10.0.20.53:45013 (1505 connections now open) 2016-03-13T09:17:32.212+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50941 #4081526 (1506 connections now open) 2016-03-13T09:17:32.212+0000 I NETWORK [conn4074587] end connection 10.0.27.237:57782 (1505 connections now open) 2016-03-13T09:17:32.212+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50942 #4081527 (1506 connections now open) 2016-03-13T09:17:32.212+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36001 #4081528 (1507 connections now open) 2016-03-13T09:17:32.213+0000 I NETWORK [conn4075575] end connection 10.0.27.237:58769 (1506 connections now open) 2016-03-13T09:17:32.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36002 #4081529 (1507 connections now open) 2016-03-13T09:17:32.213+0000 I NETWORK [conn4079879] end connection 10.0.20.53:49800 (1506 connections now open) 2016-03-13T09:17:32.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50943 #4081530 (1507 connections now open) 2016-03-13T09:17:32.215+0000 I NETWORK [conn4078506] end connection 10.0.27.237:33443 (1506 connections now open) 2016-03-13T09:17:32.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36003 #4081531 (1507 connections now open) 2016-03-13T09:17:32.216+0000 I NETWORK [conn4079921] end connection 10.0.20.53:49842 (1506 connections now open) 2016-03-13T09:17:32.216+0000 I NETWORK [conn4077939] end connection 10.0.27.237:32880 (1505 connections now open) 2016-03-13T09:17:32.216+0000 I NETWORK [conn4077168] end connection 10.0.26.69:38777 (1504 connections now open) 2016-03-13T09:17:32.216+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50945 #4081532 (1505 connections now open) 2016-03-13T09:17:32.216+0000 I NETWORK [conn4076331] end connection 10.0.26.69:37879 (1504 connections now open) 2016-03-13T09:17:32.216+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36004 #4081533 (1505 connections now open) 2016-03-13T09:17:32.216+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42695 #4081534 (1506 connections now open) 2016-03-13T09:17:32.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42696 #4081535 (1507 connections now open) 2016-03-13T09:17:32.217+0000 I NETWORK [conn4074688] end connection 10.0.27.237:57889 (1505 connections now open) 2016-03-13T09:17:32.217+0000 I NETWORK [conn4075685] end connection 10.0.179.209:35369 (1505 connections now open) 2016-03-13T09:17:32.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36005 #4081536 (1506 connections now open) 2016-03-13T09:17:32.217+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40755 #4081537 (1507 connections now open) 2016-03-13T09:17:32.218+0000 I NETWORK [conn4074087] end connection 10.0.20.53:43760 (1506 connections now open) 2016-03-13T09:17:32.219+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50947 #4081538 (1507 connections now open) 2016-03-13T09:17:32.219+0000 I NETWORK [conn4078038] end connection 10.0.26.69:39701 (1506 connections now open) 2016-03-13T09:17:32.219+0000 I NETWORK [conn4077952] end connection 10.0.27.237:32889 (1505 connections now open) 2016-03-13T09:17:32.219+0000 I NETWORK [conn4078841] end connection 10.0.7.113:57483 (1504 connections now open) 2016-03-13T09:17:32.219+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42698 #4081539 (1505 connections now open) 2016-03-13T09:17:32.219+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36007 #4081540 (1506 connections now open) 2016-03-13T09:17:32.219+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59695 #4081541 (1507 connections now open) 2016-03-13T09:17:32.220+0000 I NETWORK [conn4074715] end connection 10.0.27.237:57907 (1506 connections now open) 2016-03-13T09:17:32.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36008 #4081542 (1507 connections now open) 2016-03-13T09:17:32.221+0000 I NETWORK [conn4073931] end connection 10.0.26.69:35284 (1506 connections now open) 2016-03-13T09:17:32.221+0000 I NETWORK [conn4080527] end connection 10.0.7.113:59194 (1505 connections now open) 2016-03-13T09:17:32.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42700 #4081543 (1506 connections now open) 2016-03-13T09:17:32.221+0000 I NETWORK [conn4079175] end connection 10.0.27.237:34129 (1504 connections now open) 2016-03-13T09:17:32.221+0000 I NETWORK [conn4074557] end connection 10.0.20.53:44262 (1504 connections now open) 2016-03-13T09:17:32.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59696 #4081544 (1505 connections now open) 2016-03-13T09:17:32.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36009 #4081545 (1506 connections now open) 2016-03-13T09:17:32.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50950 #4081546 (1507 connections now open) 2016-03-13T09:17:32.222+0000 I NETWORK [conn4079285] end connection 10.0.26.69:40989 (1506 connections now open) 2016-03-13T09:17:32.223+0000 I NETWORK [conn4073977] end connection 10.0.26.69:35327 (1505 connections now open) 2016-03-13T09:17:32.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42701 #4081547 (1506 connections now open) 2016-03-13T09:17:32.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42702 #4081548 (1507 connections now open) 2016-03-13T09:17:32.225+0000 I NETWORK [conn4078824] end connection 10.0.216.90:55514 (1506 connections now open) 2016-03-13T09:17:32.225+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57657 #4081549 (1507 connections now open) 2016-03-13T09:17:32.225+0000 I NETWORK [conn4076719] end connection 10.0.27.237:59905 (1506 connections now open) 2016-03-13T09:17:32.226+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36012 #4081550 (1507 connections now open) 2016-03-13T09:17:32.226+0000 I NETWORK [conn4075771] end connection 10.0.26.69:37265 (1506 connections now open) 2016-03-13T09:17:32.227+0000 I NETWORK [conn4076783] end connection 10.0.27.237:59973 (1505 connections now open) 2016-03-13T09:17:32.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42705 #4081551 (1506 connections now open) 2016-03-13T09:17:32.227+0000 I NETWORK [conn4078548] end connection 10.0.26.69:40235 (1505 connections now open) 2016-03-13T09:17:32.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36013 #4081552 (1507 connections now open) 2016-03-13T09:17:32.227+0000 I NETWORK [conn4080405] end connection 10.0.27.237:35424 (1505 connections now open) 2016-03-13T09:17:32.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42706 #4081553 (1506 connections now open) 2016-03-13T09:17:32.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36014 #4081554 (1507 connections now open) 2016-03-13T09:17:32.228+0000 I NETWORK [conn4077600] end connection 10.0.26.69:39235 (1506 connections now open) 2016-03-13T09:17:32.229+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42707 #4081555 (1507 connections now open) 2016-03-13T09:17:32.229+0000 I NETWORK [conn4075812] end connection 10.0.216.90:52297 (1506 connections now open) 2016-03-13T09:17:32.229+0000 I NETWORK [conn4078335] end connection 10.0.26.69:40020 (1505 connections now open) 2016-03-13T09:17:32.229+0000 I NETWORK [conn4074394] end connection 10.0.27.237:57588 (1504 connections now open) 2016-03-13T09:17:32.229+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57659 #4081556 (1505 connections now open) 2016-03-13T09:17:32.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42708 #4081557 (1506 connections now open) 2016-03-13T09:17:32.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36015 #4081558 (1507 connections now open) 2016-03-13T09:17:32.230+0000 I NETWORK [conn4074759] end connection 10.0.27.237:57951 (1506 connections now open) 2016-03-13T09:17:32.230+0000 I NETWORK [conn4074461] end connection 10.0.20.53:44156 (1505 connections now open) 2016-03-13T09:17:32.230+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36016 #4081559 (1506 connections now open) 2016-03-13T09:17:32.231+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50958 #4081560 (1507 connections now open) 2016-03-13T09:17:32.231+0000 I NETWORK [conn4074020] end connection 10.0.20.53:43686 (1506 connections now open) 2016-03-13T09:17:32.231+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50959 #4081561 (1507 connections now open) 2016-03-13T09:17:32.231+0000 I NETWORK [conn4078421] end connection 10.0.27.237:33356 (1506 connections now open) 2016-03-13T09:17:32.232+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36017 #4081562 (1507 connections now open) 2016-03-13T09:17:32.233+0000 I NETWORK [conn4075183] end connection 10.0.216.90:51614 (1506 connections now open) 2016-03-13T09:17:32.233+0000 I NETWORK [conn4078617] end connection 10.0.26.69:40303 (1505 connections now open) 2016-03-13T09:17:32.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57661 #4081563 (1506 connections now open) 2016-03-13T09:17:32.233+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42711 #4081564 (1507 connections now open) 2016-03-13T09:17:32.234+0000 I NETWORK [conn4078634] end connection 10.0.179.209:38325 (1506 connections now open) 2016-03-13T09:17:32.234+0000 I NETWORK [conn4075833] end connection 10.0.7.113:54290 (1505 connections now open) 2016-03-13T09:17:32.234+0000 I NETWORK [conn4074435] end connection 10.0.20.53:44126 (1504 connections now open) 2016-03-13T09:17:32.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40756 #4081565 (1505 connections now open) 2016-03-13T09:17:32.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59697 #4081566 (1506 connections now open) 2016-03-13T09:17:32.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50962 #4081567 (1507 connections now open) 2016-03-13T09:17:32.236+0000 I NETWORK [conn4080448] end connection 10.0.26.69:42165 (1506 connections now open) 2016-03-13T09:17:32.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42714 #4081568 (1507 connections now open) 2016-03-13T09:17:32.237+0000 I NETWORK [conn4080660] end connection 10.0.7.113:59322 (1506 connections now open) 2016-03-13T09:17:32.237+0000 I NETWORK [conn4076590] end connection 10.0.179.209:36282 (1505 connections now open) 2016-03-13T09:17:32.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59698 #4081569 (1506 connections now open) 2016-03-13T09:17:32.237+0000 I NETWORK [conn4079077] end connection 10.0.27.237:34022 (1504 connections now open) 2016-03-13T09:17:32.237+0000 I NETWORK [conn4074586] end connection 10.0.26.69:35987 (1504 connections now open) 2016-03-13T09:17:32.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40757 #4081570 (1505 connections now open) 2016-03-13T09:17:32.237+0000 I NETWORK [conn4076914] end connection 10.0.179.209:36595 (1504 connections now open) 2016-03-13T09:17:32.238+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42715 #4081571 (1505 connections now open) 2016-03-13T09:17:32.238+0000 I NETWORK [conn4074188] end connection 10.0.27.237:57380 (1503 connections now open) 2016-03-13T09:17:32.238+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36022 #4081572 (1506 connections now open) 2016-03-13T09:17:32.238+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40758 #4081573 (1506 connections now open) 2016-03-13T09:17:32.238+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36023 #4081574 (1507 connections now open) 2016-03-13T09:17:32.239+0000 I NETWORK [conn4073557] end connection 10.0.20.53:43207 (1506 connections now open) 2016-03-13T09:17:32.239+0000 I NETWORK [conn4078166] end connection 10.0.7.113:56776 (1506 connections now open) 2016-03-13T09:17:32.239+0000 I NETWORK [conn4080258] end connection 10.0.20.53:50201 (1504 connections now open) 2016-03-13T09:17:32.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50965 #4081575 (1505 connections now open) 2016-03-13T09:17:32.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59699 #4081576 (1506 connections now open) 2016-03-13T09:17:32.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50966 #4081577 (1507 connections now open) 2016-03-13T09:17:32.239+0000 I NETWORK [conn4078910] end connection 10.0.26.69:40604 (1506 connections now open) 2016-03-13T09:17:32.240+0000 I NETWORK [conn4075013] end connection 10.0.27.237:58203 (1505 connections now open) 2016-03-13T09:17:32.240+0000 I NETWORK [conn4075072] end connection 10.0.27.237:58259 (1504 connections now open) 2016-03-13T09:17:32.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42717 #4081578 (1506 connections now open) 2016-03-13T09:17:32.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36024 #4081579 (1506 connections now open) 2016-03-13T09:17:32.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36025 #4081580 (1507 connections now open) 2016-03-13T09:17:32.241+0000 I NETWORK [conn4080387] end connection 10.0.20.53:50337 (1506 connections now open) 2016-03-13T09:17:32.241+0000 I NETWORK [conn4078164] end connection 10.0.20.53:48005 (1505 connections now open) 2016-03-13T09:17:32.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50967 #4081581 (1506 connections now open) 2016-03-13T09:17:32.242+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50968 #4081582 (1507 connections now open) 2016-03-13T09:17:32.242+0000 I NETWORK [conn4079388] end connection 10.0.7.113:58042 (1506 connections now open) 2016-03-13T09:17:32.242+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59700 #4081583 (1507 connections now open) 2016-03-13T09:17:32.243+0000 I NETWORK [conn4078969] end connection 10.0.20.53:48840 (1506 connections now open) 2016-03-13T09:17:32.243+0000 I NETWORK [conn4078549] end connection 10.0.27.237:33481 (1506 connections now open) 2016-03-13T09:17:32.243+0000 I NETWORK [conn4078883] end connection 10.0.26.69:40579 (1504 connections now open) 2016-03-13T09:17:32.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50970 #4081584 (1505 connections now open) 2016-03-13T09:17:32.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36027 #4081585 (1506 connections now open) 2016-03-13T09:17:32.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42718 #4081586 (1507 connections now open) 2016-03-13T09:17:32.244+0000 I NETWORK [conn4076932] end connection 10.0.179.209:36618 (1506 connections now open) 2016-03-13T09:17:32.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40761 #4081587 (1507 connections now open) 2016-03-13T09:17:32.245+0000 I NETWORK [conn4079602] end connection 10.0.27.237:34576 (1506 connections now open) 2016-03-13T09:17:32.246+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36029 #4081588 (1507 connections now open) 2016-03-13T09:17:32.246+0000 I NETWORK [conn4079306] end connection 10.0.7.113:57957 (1506 connections now open) 2016-03-13T09:17:32.246+0000 I NETWORK [conn4079200] end connection 10.0.27.237:34150 (1505 connections now open) 2016-03-13T09:17:32.246+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59701 #4081589 (1506 connections now open) 2016-03-13T09:17:32.247+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36030 #4081590 (1507 connections now open) 2016-03-13T09:17:32.249+0000 I NETWORK [conn4079900] end connection 10.0.7.113:58550 (1506 connections now open) 2016-03-13T09:17:32.249+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59702 #4081591 (1507 connections now open) 2016-03-13T09:17:32.250+0000 I NETWORK [conn4076157] end connection 10.0.179.209:35853 (1506 connections now open) 2016-03-13T09:17:32.250+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40764 #4081592 (1507 connections now open) 2016-03-13T09:17:32.250+0000 I NETWORK [conn4075225] end connection 10.0.26.69:36670 (1506 connections now open) 2016-03-13T09:17:32.250+0000 I NETWORK [conn4080706] end connection 10.0.27.237:35753 (1505 connections now open) 2016-03-13T09:17:32.251+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42722 #4081593 (1506 connections now open) 2016-03-13T09:17:32.251+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36034 #4081594 (1507 connections now open) 2016-03-13T09:17:32.251+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-1-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:32.252+0000 I NETWORK [conn4079010] end connection 10.0.26.69:40710 (1506 connections now open) 2016-03-13T09:17:32.252+0000 I NETWORK [conn4079009] end connection 10.0.179.209:38721 (1505 connections now open) 2016-03-13T09:17:32.252+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42723 #4081595 (1506 connections now open) 2016-03-13T09:17:32.252+0000 I NETWORK [conn4075100] end connection 10.0.27.237:58286 (1505 connections now open) 2016-03-13T09:17:32.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40765 #4081596 (1506 connections now open) 2016-03-13T09:17:32.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36036 #4081597 (1507 connections now open) 2016-03-13T09:17:32.253+0000 I NETWORK [conn4078321] end connection 10.0.20.53:48162 (1505 connections now open) 2016-03-13T09:17:32.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50976 #4081598 (1507 connections now open) 2016-03-13T09:17:32.254+0000 I NETWORK [conn4080661] end connection 10.0.26.69:42368 (1506 connections now open) 2016-03-13T09:17:32.254+0000 I NETWORK [conn4079052] end connection 10.0.27.237:33997 (1506 connections now open) 2016-03-13T09:17:32.254+0000 I NETWORK [conn4075516] end connection 10.0.26.69:36996 (1504 connections now open) 2016-03-13T09:17:32.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36037 #4081599 (1506 connections now open) 2016-03-13T09:17:32.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42725 #4081600 (1506 connections now open) 2016-03-13T09:17:32.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42726 #4081601 (1507 connections now open) 2016-03-13T09:17:32.255+0000 I NETWORK [conn4079561] end connection 10.0.7.113:58214 (1506 connections now open) 2016-03-13T09:17:32.255+0000 I NETWORK [conn4079812] end connection 10.0.20.53:49728 (1505 connections now open) 2016-03-13T09:17:32.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59703 #4081602 (1506 connections now open) 2016-03-13T09:17:32.255+0000 I NETWORK [conn4080089] end connection 10.0.179.209:39813 (1505 connections now open) 2016-03-13T09:17:32.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50979 #4081603 (1506 connections now open) 2016-03-13T09:17:32.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40768 #4081604 (1507 connections now open) 2016-03-13T09:17:32.256+0000 I NETWORK [conn4075743] end connection 10.0.27.237:58936 (1506 connections now open) 2016-03-13T09:17:32.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36039 #4081605 (1507 connections now open) 2016-03-13T09:17:32.257+0000 I NETWORK [conn4079118] end connection 10.0.27.237:34062 (1506 connections now open) 2016-03-13T09:17:32.257+0000 I NETWORK [conn4078404] end connection 10.0.20.53:48248 (1505 connections now open) 2016-03-13T09:17:32.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36040 #4081606 (1506 connections now open) 2016-03-13T09:17:32.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50980 #4081607 (1507 connections now open) 2016-03-13T09:17:32.258+0000 I NETWORK [conn4077102] end connection 10.0.26.69:38697 (1506 connections now open) 2016-03-13T09:17:32.259+0000 I NETWORK [conn4078993] end connection 10.0.27.237:33938 (1505 connections now open) 2016-03-13T09:17:32.259+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42729 #4081608 (1507 connections now open) 2016-03-13T09:17:32.259+0000 I NETWORK [conn4075351] end connection 10.0.20.53:45084 (1505 connections now open) 2016-03-13T09:17:32.259+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36042 #4081609 (1506 connections now open) 2016-03-13T09:17:32.259+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50981 #4081610 (1507 connections now open) 2016-03-13T09:17:32.259+0000 I NETWORK [conn4077659] end connection 10.0.20.53:47506 (1506 connections now open) 2016-03-13T09:17:32.260+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50982 #4081611 (1507 connections now open) 2016-03-13T09:17:32.261+0000 I NETWORK [conn4079035] end connection 10.0.20.53:48902 (1506 connections now open) 2016-03-13T09:17:32.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50983 #4081612 (1507 connections now open) 2016-03-13T09:17:32.262+0000 I NETWORK [conn4080109] end connection 10.0.179.209:39829 (1506 connections now open) 2016-03-13T09:17:32.262+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40770 #4081613 (1507 connections now open) 2016-03-13T09:17:32.262+0000 I NETWORK [conn4075728] end connection 10.0.26.69:37216 (1506 connections now open) 2016-03-13T09:17:32.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42731 #4081614 (1507 connections now open) 2016-03-13T09:17:32.266+0000 I NETWORK [conn4078992] end connection 10.0.20.53:48857 (1506 connections now open) 2016-03-13T09:17:32.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50987 #4081615 (1507 connections now open) 2016-03-13T09:17:32.267+0000 I NETWORK [conn4076503] end connection 10.0.20.53:46318 (1506 connections now open) 2016-03-13T09:17:32.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50988 #4081616 (1507 connections now open) 2016-03-13T09:17:32.268+0000 I NETWORK [conn4079727] end connection 10.0.27.237:34709 (1506 connections now open) 2016-03-13T09:17:32.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36050 #4081617 (1507 connections now open) 2016-03-13T09:17:32.268+0000 I NETWORK [conn4079411] end connection 10.0.26.69:41119 (1506 connections now open) 2016-03-13T09:17:32.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42736 #4081618 (1507 connections now open) 2016-03-13T09:17:32.270+0000 I NETWORK [conn4076698] end connection 10.0.26.69:38278 (1506 connections now open) 2016-03-13T09:17:32.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42738 #4081619 (1507 connections now open) 2016-03-13T09:17:32.271+0000 I NETWORK [conn4078251] end connection 10.0.20.53:48094 (1506 connections now open) 2016-03-13T09:17:32.271+0000 I NETWORK [conn4079940] end connection 10.0.27.237:34935 (1505 connections now open) 2016-03-13T09:17:32.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50991 #4081620 (1506 connections now open) 2016-03-13T09:17:32.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36052 #4081621 (1507 connections now open) 2016-03-13T09:17:32.272+0000 I NETWORK [conn4077210] end connection 10.0.20.53:47039 (1505 connections now open) 2016-03-13T09:17:32.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50992 #4081622 (1507 connections now open) 2016-03-13T09:17:32.274+0000 I NETWORK [conn4079076] end connection 10.0.20.53:48943 (1506 connections now open) 2016-03-13T09:17:32.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50994 #4081623 (1507 connections now open) 2016-03-13T09:17:32.275+0000 I NETWORK [conn4080279] end connection 10.0.27.237:35290 (1506 connections now open) 2016-03-13T09:17:32.275+0000 I NETWORK [conn4080045] end connection 10.0.26.69:41760 (1505 connections now open) 2016-03-13T09:17:32.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36055 #4081624 (1507 connections now open) 2016-03-13T09:17:32.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42740 #4081625 (1507 connections now open) 2016-03-13T09:17:32.276+0000 I NETWORK [conn4079707] end connection 10.0.27.237:34687 (1506 connections now open) 2016-03-13T09:17:32.276+0000 I NETWORK [conn4074900] end connection 10.0.26.69:36325 (1505 connections now open) 2016-03-13T09:17:32.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36056 #4081626 (1507 connections now open) 2016-03-13T09:17:32.277+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42741 #4081627 (1507 connections now open) 2016-03-13T09:17:32.278+0000 I NETWORK [conn4078148] end connection 10.0.20.53:47986 (1506 connections now open) 2016-03-13T09:17:32.278+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50997 #4081628 (1507 connections now open) 2016-03-13T09:17:32.278+0000 I NETWORK [conn4078078] end connection 10.0.20.53:47922 (1506 connections now open) 2016-03-13T09:17:32.279+0000 I NETWORK [conn4079432] end connection 10.0.26.69:41137 (1505 connections now open) 2016-03-13T09:17:32.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:50998 #4081629 (1506 connections now open) 2016-03-13T09:17:32.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42743 #4081630 (1507 connections now open) 2016-03-13T09:17:32.279+0000 I NETWORK [conn4076825] end connection 10.0.27.237:60015 (1506 connections now open) 2016-03-13T09:17:32.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36057 #4081631 (1507 connections now open) 2016-03-13T09:17:32.281+0000 I NETWORK [conn4076871] end connection 10.0.27.237:60059 (1506 connections now open) 2016-03-13T09:17:32.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36059 #4081632 (1507 connections now open) 2016-03-13T09:17:32.282+0000 I NETWORK [conn4075324] end connection 10.0.26.69:36774 (1506 connections now open) 2016-03-13T09:17:32.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42745 #4081633 (1507 connections now open) 2016-03-13T09:17:32.283+0000 I NETWORK [conn4079159] end connection 10.0.20.53:49033 (1506 connections now open) 2016-03-13T09:17:32.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51002 #4081634 (1507 connections now open) 2016-03-13T09:17:32.284+0000 I NETWORK [conn4079772] end connection 10.0.26.69:41486 (1506 connections now open) 2016-03-13T09:17:32.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42746 #4081635 (1507 connections now open) 2016-03-13T09:17:32.285+0000 I NETWORK [conn4078376] end connection 10.0.20.53:48226 (1506 connections now open) 2016-03-13T09:17:32.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51003 #4081636 (1507 connections now open) 2016-03-13T09:17:32.285+0000 I NETWORK [conn4077127] end connection 10.0.27.237:60317 (1506 connections now open) 2016-03-13T09:17:32.286+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36061 #4081637 (1507 connections now open) 2016-03-13T09:17:32.287+0000 I NETWORK [conn4080091] end connection 10.0.26.69:41805 (1506 connections now open) 2016-03-13T09:17:32.287+0000 I NETWORK [conn4079241] end connection 10.0.20.53:49127 (1505 connections now open) 2016-03-13T09:17:32.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42747 #4081638 (1506 connections now open) 2016-03-13T09:17:32.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51004 #4081639 (1507 connections now open) 2016-03-13T09:17:32.288+0000 I NETWORK [conn4078737] end connection 10.0.216.90:55428 (1506 connections now open) 2016-03-13T09:17:32.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57673 #4081640 (1507 connections now open) 2016-03-13T09:17:32.291+0000 I NETWORK [conn4075574] end connection 10.0.26.69:37054 (1506 connections now open) 2016-03-13T09:17:32.291+0000 I NETWORK [conn4078658] end connection 10.0.27.237:33602 (1505 connections now open) 2016-03-13T09:17:32.291+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42749 #4081641 (1506 connections now open) 2016-03-13T09:17:32.292+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36064 #4081642 (1507 connections now open) 2016-03-13T09:17:32.292+0000 I NETWORK [conn4078546] end connection 10.0.20.53:48402 (1506 connections now open) 2016-03-13T09:17:32.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51007 #4081643 (1507 connections now open) 2016-03-13T09:17:32.293+0000 I NETWORK [conn4080002] end connection 10.0.27.237:35005 (1506 connections now open) 2016-03-13T09:17:32.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36065 #4081644 (1507 connections now open) 2016-03-13T09:17:32.294+0000 I NETWORK [conn4077555] end connection 10.0.26.69:39192 (1506 connections now open) 2016-03-13T09:17:32.294+0000 I NETWORK [conn4074355] end connection 10.0.20.53:44036 (1505 connections now open) 2016-03-13T09:17:32.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42751 #4081645 (1506 connections now open) 2016-03-13T09:17:32.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51009 #4081646 (1507 connections now open) 2016-03-13T09:17:32.295+0000 I NETWORK [conn4079413] end connection 10.0.20.53:49309 (1506 connections now open) 2016-03-13T09:17:32.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51010 #4081647 (1507 connections now open) 2016-03-13T09:17:32.297+0000 I NETWORK [conn4079501] end connection 10.0.20.53:49399 (1506 connections now open) 2016-03-13T09:17:32.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51011 #4081648 (1507 connections now open) 2016-03-13T09:17:32.298+0000 I NETWORK [conn4079078] end connection 10.0.26.69:40775 (1506 connections now open) 2016-03-13T09:17:32.299+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42754 #4081649 (1507 connections now open) 2016-03-13T09:17:32.300+0000 I NETWORK [conn4075661] end connection 10.0.26.69:37145 (1506 connections now open) 2016-03-13T09:17:32.300+0000 I NETWORK [conn4076182] end connection 10.0.26.69:37723 (1505 connections now open) 2016-03-13T09:17:32.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42755 #4081650 (1506 connections now open) 2016-03-13T09:17:32.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42756 #4081651 (1507 connections now open) 2016-03-13T09:17:32.301+0000 I NETWORK [conn4077855] end connection 10.0.27.237:32800 (1506 connections now open) 2016-03-13T09:17:32.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36069 #4081652 (1507 connections now open) 2016-03-13T09:17:32.302+0000 I NETWORK [conn4075789] end connection 10.0.26.69:37283 (1506 connections now open) 2016-03-13T09:17:32.302+0000 I NETWORK [conn4080685] end connection 10.0.26.69:42397 (1505 connections now open) 2016-03-13T09:17:32.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42757 #4081653 (1506 connections now open) 2016-03-13T09:17:32.303+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42758 #4081654 (1507 connections now open) 2016-03-13T09:17:32.306+0000 I NETWORK [conn4078122] end connection 10.0.26.69:39794 (1506 connections now open) 2016-03-13T09:17:32.306+0000 I NETWORK [conn4079667] end connection 10.0.20.53:49579 (1505 connections now open) 2016-03-13T09:17:32.306+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42760 #4081655 (1506 connections now open) 2016-03-13T09:17:32.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51019 #4081656 (1507 connections now open) 2016-03-13T09:17:32.307+0000 I NETWORK [conn4080090] end connection 10.0.27.237:35096 (1506 connections now open) 2016-03-13T09:17:32.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36071 #4081657 (1507 connections now open) 2016-03-13T09:17:32.308+0000 I NETWORK [conn4076614] end connection 10.0.26.69:38185 (1506 connections now open) 2016-03-13T09:17:32.309+0000 I NETWORK [conn4079794] end connection 10.0.20.53:49712 (1505 connections now open) 2016-03-13T09:17:32.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42761 #4081658 (1506 connections now open) 2016-03-13T09:17:32.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51021 #4081659 (1507 connections now open) 2016-03-13T09:17:32.309+0000 I NETWORK [conn4077146] end connection 10.0.26.69:38746 (1506 connections now open) 2016-03-13T09:17:32.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42762 #4081660 (1507 connections now open) 2016-03-13T09:17:32.310+0000 I NETWORK [conn4080428] end connection 10.0.27.237:35449 (1506 connections now open) 2016-03-13T09:17:32.310+0000 I NETWORK [conn4078238] end connection 10.0.26.69:39920 (1505 connections now open) 2016-03-13T09:17:32.311+0000 I NETWORK [conn4079961] end connection 10.0.20.53:49890 (1505 connections now open) 2016-03-13T09:17:32.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36072 #4081661 (1507 connections now open) 2016-03-13T09:17:32.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51022 #4081662 (1506 connections now open) 2016-03-13T09:17:32.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42763 #4081663 (1507 connections now open) 2016-03-13T09:17:32.312+0000 I NETWORK [conn4076957] end connection 10.0.27.237:60144 (1506 connections now open) 2016-03-13T09:17:32.312+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36073 #4081664 (1507 connections now open) 2016-03-13T09:17:32.312+0000 I NETWORK [conn4080171] end connection 10.0.20.53:50115 (1506 connections now open) 2016-03-13T09:17:32.313+0000 I NETWORK [conn4077345] end connection 10.0.26.69:38962 (1505 connections now open) 2016-03-13T09:17:32.313+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51023 #4081665 (1506 connections now open) 2016-03-13T09:17:32.313+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42765 #4081666 (1507 connections now open) 2016-03-13T09:17:32.315+0000 I NETWORK [conn4079835] end connection 10.0.26.69:41551 (1506 connections now open) 2016-03-13T09:17:32.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42767 #4081667 (1507 connections now open) 2016-03-13T09:17:32.316+0000 I NETWORK [conn4079750] end connection 10.0.27.237:34736 (1506 connections now open) 2016-03-13T09:17:32.316+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36075 #4081668 (1507 connections now open) 2016-03-13T09:17:32.317+0000 I NETWORK [conn4080590] end connection 10.0.26.69:42313 (1506 connections now open) 2016-03-13T09:17:32.317+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42768 #4081669 (1507 connections now open) 2016-03-13T09:17:32.317+0000 I NETWORK [conn4076418] end connection 10.0.26.69:37970 (1506 connections now open) 2016-03-13T09:17:32.318+0000 I NETWORK [conn4079922] end connection 10.0.27.237:34914 (1505 connections now open) 2016-03-13T09:17:32.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42769 #4081670 (1506 connections now open) 2016-03-13T09:17:32.318+0000 I NETWORK [conn4080232] end connection 10.0.20.53:50174 (1505 connections now open) 2016-03-13T09:17:32.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36076 #4081671 (1507 connections now open) 2016-03-13T09:17:32.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51026 #4081672 (1507 connections now open) 2016-03-13T09:17:32.320+0000 I NETWORK [conn4077233] end connection 10.0.20.53:47060 (1506 connections now open) 2016-03-13T09:17:32.320+0000 I NETWORK [conn4076244] end connection 10.0.26.69:37786 (1505 connections now open) 2016-03-13T09:17:32.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51028 #4081673 (1506 connections now open) 2016-03-13T09:17:32.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42770 #4081674 (1507 connections now open) 2016-03-13T09:17:32.322+0000 I NETWORK [conn3015122] end connection 10.0.217.145:55688 (1506 connections now open) 2016-03-13T09:17:32.323+0000 I NETWORK [conn4076375] end connection 10.0.26.69:37928 (1505 connections now open) 2016-03-13T09:17:32.323+0000 I NETWORK [conn3687674] end connection 10.0.217.145:58514 (1504 connections now open) 2016-03-13T09:17:32.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42772 #4081675 (1505 connections now open) 2016-03-13T09:17:32.324+0000 I NETWORK [conn4076955] end connection 10.0.20.53:46787 (1504 connections now open) 2016-03-13T09:17:32.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51030 #4081676 (1505 connections now open) 2016-03-13T09:17:32.325+0000 I NETWORK [conn4077212] end connection 10.0.26.69:38826 (1504 connections now open) 2016-03-13T09:17:32.325+0000 I NETWORK [conn4076504] end connection 10.0.26.69:38063 (1503 connections now open) 2016-03-13T09:17:32.326+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42773 #4081677 (1504 connections now open) 2016-03-13T09:17:32.326+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42774 #4081678 (1505 connections now open) 2016-03-13T09:17:32.327+0000 I NETWORK [conn4076676] end connection 10.0.20.53:46495 (1504 connections now open) 2016-03-13T09:17:32.327+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51031 #4081679 (1505 connections now open) 2016-03-13T09:17:32.327+0000 I NETWORK [conn4079582] end connection 10.0.26.69:41291 (1504 connections now open) 2016-03-13T09:17:32.327+0000 I NETWORK [conn4079095] end connection 10.0.26.69:40794 (1503 connections now open) 2016-03-13T09:17:32.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42775 #4081680 (1504 connections now open) 2016-03-13T09:17:32.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42776 #4081681 (1505 connections now open) 2016-03-13T09:17:32.329+0000 I NETWORK [conn4079519] end connection 10.0.20.53:49415 (1504 connections now open) 2016-03-13T09:17:32.329+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51033 #4081682 (1505 connections now open) 2016-03-13T09:17:32.329+0000 I NETWORK [conn4080607] end connection 10.0.20.53:50586 (1504 connections now open) 2016-03-13T09:17:32.330+0000 I NETWORK [conn4079368] end connection 10.0.27.237:34327 (1503 connections now open) 2016-03-13T09:17:32.330+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51034 #4081683 (1505 connections now open) 2016-03-13T09:17:32.330+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36080 #4081684 (1505 connections now open) 2016-03-13T09:17:32.332+0000 I NETWORK [mongosMain] connection accepted from 10.0.217.145:55627 #4081685 (1506 connections now open) 2016-03-13T09:17:32.332+0000 I NETWORK [mongosMain] connection accepted from 10.0.217.145:55628 #4081686 (1507 connections now open) 2016-03-13T09:17:32.333+0000 I NETWORK [conn4077300] end connection 10.0.26.69:38917 (1506 connections now open) 2016-03-13T09:17:32.333+0000 I NETWORK [conn4079855] end connection 10.0.26.69:41572 (1505 connections now open) 2016-03-13T09:17:32.333+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42778 #4081687 (1506 connections now open) 2016-03-13T09:17:32.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42779 #4081688 (1507 connections now open) 2016-03-13T09:17:32.334+0000 I NETWORK [conn4080608] end connection 10.0.20.53:50587 (1506 connections now open) 2016-03-13T09:17:32.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51036 #4081689 (1507 connections now open) 2016-03-13T09:17:32.335+0000 I NETWORK [conn4079880] end connection 10.0.26.69:41589 (1506 connections now open) 2016-03-13T09:17:32.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42780 #4081690 (1507 connections now open) 2016-03-13T09:17:32.336+0000 I NETWORK [conn4079898] end connection 10.0.20.53:49816 (1506 connections now open) 2016-03-13T09:17:32.337+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51037 #4081691 (1507 connections now open) 2016-03-13T09:17:32.337+0000 I NETWORK [conn4077299] end connection 10.0.20.53:47128 (1506 connections now open) 2016-03-13T09:17:32.338+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51038 #4081692 (1507 connections now open) 2016-03-13T09:17:32.338+0000 I NETWORK [conn4075832] end connection 10.0.20.53:45598 (1506 connections now open) 2016-03-13T09:17:32.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51039 #4081693 (1507 connections now open) 2016-03-13T09:17:32.339+0000 I NETWORK [conn4078107] end connection 10.0.27.237:33046 (1506 connections now open) 2016-03-13T09:17:32.339+0000 I NETWORK [conn4077256] end connection 10.0.20.53:47084 (1505 connections now open) 2016-03-13T09:17:32.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36084 #4081694 (1507 connections now open) 2016-03-13T09:17:32.340+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51040 #4081695 (1507 connections now open) 2016-03-13T09:17:32.342+0000 I NETWORK [conn4076890] end connection 10.0.26.69:38473 (1506 connections now open) 2016-03-13T09:17:32.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42784 #4081696 (1507 connections now open) 2016-03-13T09:17:32.343+0000 I NETWORK [conn4075954] end connection 10.0.20.53:45732 (1506 connections now open) 2016-03-13T09:17:32.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51042 #4081697 (1507 connections now open) 2016-03-13T09:17:32.344+0000 I NETWORK [conn4078280] end connection 10.0.27.237:33217 (1506 connections now open) 2016-03-13T09:17:32.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36086 #4081698 (1507 connections now open) 2016-03-13T09:17:32.345+0000 I NETWORK [conn4079430] end connection 10.0.20.53:49326 (1506 connections now open) 2016-03-13T09:17:32.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51044 #4081699 (1507 connections now open) 2016-03-13T09:17:32.346+0000 I NETWORK [conn4080659] end connection 10.0.27.237:35706 (1506 connections now open) 2016-03-13T09:17:32.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36087 #4081700 (1507 connections now open) 2016-03-13T09:17:32.349+0000 I NETWORK [conn4080388] end connection 10.0.27.237:35404 (1506 connections now open) 2016-03-13T09:17:32.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36088 #4081701 (1507 connections now open) 2016-03-13T09:17:32.351+0000 I NETWORK [conn4079645] end connection 10.0.20.53:49551 (1506 connections now open) 2016-03-13T09:17:32.351+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51047 #4081702 (1507 connections now open) 2016-03-13T09:17:32.353+0000 I NETWORK [conn4075874] end connection 10.0.20.53:45643 (1506 connections now open) 2016-03-13T09:17:32.353+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51048 #4081703 (1507 connections now open) 2016-03-13T09:17:32.355+0000 I NETWORK [conn4077017] end connection 10.0.26.69:38604 (1506 connections now open) 2016-03-13T09:17:32.355+0000 I NETWORK [conn4077100] end connection 10.0.20.53:46931 (1505 connections now open) 2016-03-13T09:17:32.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42789 #4081704 (1506 connections now open) 2016-03-13T09:17:32.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51049 #4081705 (1507 connections now open) 2016-03-13T09:17:32.357+0000 I NETWORK [conn4078615] end connection 10.0.27.237:33557 (1506 connections now open) 2016-03-13T09:17:32.357+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 16825 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:17:32.357+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36091 #4081706 (1507 connections now open) 2016-03-13T09:17:32.358+0000 I NETWORK [conn4078839] end connection 10.0.20.53:48710 (1506 connections now open) 2016-03-13T09:17:32.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51050 #4081707 (1507 connections now open) 2016-03-13T09:17:32.360+0000 I NETWORK [conn4078575] end connection 10.0.27.237:33516 (1506 connections now open) 2016-03-13T09:17:32.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36092 #4081708 (1507 connections now open) 2016-03-13T09:17:32.360+0000 I NETWORK [conn4076675] end connection 10.0.26.69:38245 (1505 connections now open) 2016-03-13T09:17:32.360+0000 I NETWORK [conn4080321] end connection 10.0.20.53:50262 (1505 connections now open) 2016-03-13T09:17:32.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42791 #4081709 (1506 connections now open) 2016-03-13T09:17:32.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51051 #4081710 (1507 connections now open) 2016-03-13T09:17:32.362+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.ActionReports: 5ms sequenceNumber: 16826 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:17:32.364+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 16827 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:17:32.367+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 16828 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:17:32.367+0000 I NETWORK [conn4077189] end connection 10.0.26.69:38790 (1506 connections now open) 2016-03-13T09:17:32.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42793 #4081711 (1507 connections now open) 2016-03-13T09:17:32.367+0000 I NETWORK [conn4077787] end connection 10.0.20.53:47630 (1506 connections now open) 2016-03-13T09:17:32.367+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 16829 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:17:32.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51052 #4081712 (1507 connections now open) 2016-03-13T09:17:32.368+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 16830 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:17:32.369+0000 I NETWORK [conn4077232] end connection 10.0.26.69:38839 (1506 connections now open) 2016-03-13T09:17:32.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42794 #4081713 (1507 connections now open) 2016-03-13T09:17:32.373+0000 I NETWORK [conn4079625] end connection 10.0.27.237:34596 (1506 connections now open) 2016-03-13T09:17:32.373+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36094 #4081714 (1507 connections now open) 2016-03-13T09:17:32.375+0000 I NETWORK [conn4077660] end connection 10.0.26.69:39293 (1506 connections now open) 2016-03-13T09:17:32.375+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 6ms sequenceNumber: 16831 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:17:32.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42795 #4081715 (1507 connections now open) 2016-03-13T09:17:32.376+0000 I NETWORK [conn4080064] end connection 10.0.20.53:49997 (1506 connections now open) 2016-03-13T09:17:32.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51053 #4081716 (1507 connections now open) 2016-03-13T09:17:32.377+0000 I NETWORK [conn4077704] end connection 10.0.26.69:39336 (1506 connections now open) 2016-03-13T09:17:32.377+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42796 #4081717 (1507 connections now open) 2016-03-13T09:17:32.379+0000 I NETWORK [conn4077872] end connection 10.0.26.69:39513 (1506 connections now open) 2016-03-13T09:17:32.379+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42797 #4081718 (1507 connections now open) 2016-03-13T09:17:32.379+0000 I NETWORK [conn4079037] end connection 10.0.26.69:40730 (1505 connections now open) 2016-03-13T09:17:32.379+0000 I NETWORK [conn4077913] end connection 10.0.20.53:47756 (1505 connections now open) 2016-03-13T09:17:32.380+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42798 #4081719 (1506 connections now open) 2016-03-13T09:17:32.380+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51054 #4081720 (1507 connections now open) 2016-03-13T09:17:32.381+0000 I NETWORK [conn4079521] end connection 10.0.26.69:41225 (1506 connections now open) 2016-03-13T09:17:32.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42799 #4081721 (1507 connections now open) 2016-03-13T09:17:32.386+0000 I NETWORK [conn4078674] end connection 10.0.26.69:40368 (1506 connections now open) 2016-03-13T09:17:32.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42801 #4081722 (1507 connections now open) 2016-03-13T09:17:32.387+0000 I NETWORK [conn4077729] end connection 10.0.7.113:56309 (1506 connections now open) 2016-03-13T09:17:32.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59706 #4081723 (1507 connections now open) 2016-03-13T09:17:32.388+0000 I NETWORK [conn4077996] end connection 10.0.26.69:39659 (1506 connections now open) 2016-03-13T09:17:32.388+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42802 #4081724 (1507 connections now open) 2016-03-13T09:17:32.389+0000 I NETWORK [conn4079857] end connection 10.0.7.113:58511 (1506 connections now open) 2016-03-13T09:17:32.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59707 #4081725 (1507 connections now open) 2016-03-13T09:17:32.392+0000 I NETWORK [conn4078868] end connection 10.0.27.237:33807 (1506 connections now open) 2016-03-13T09:17:32.393+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36099 #4081726 (1507 connections now open) 2016-03-13T09:17:32.394+0000 I NETWORK [conn4078207] end connection 10.0.26.69:39885 (1506 connections now open) 2016-03-13T09:17:32.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42804 #4081727 (1507 connections now open) 2016-03-13T09:17:32.396+0000 I NETWORK [conn4079134] end connection 10.0.26.69:40834 (1506 connections now open) 2016-03-13T09:17:32.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42805 #4081728 (1507 connections now open) 2016-03-13T09:17:32.401+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.Clients: 25ms sequenceNumber: 16832 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:17:32.404+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 16833 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:17:32.405+0000 I NETWORK [conn4078711] end connection 10.0.26.69:40409 (1506 connections now open) 2016-03-13T09:17:32.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42807 #4081729 (1507 connections now open) 2016-03-13T09:17:32.406+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 16834 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:17:32.407+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 16835 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:17:32.408+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 16836 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:17:32.421+0000 I NETWORK [conn4080133] end connection 10.0.27.237:35136 (1506 connections now open) 2016-03-13T09:17:32.421+0000 I NETWORK [conn4078928] end connection 10.0.26.69:40625 (1505 connections now open) 2016-03-13T09:17:32.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36105 #4081730 (1506 connections now open) 2016-03-13T09:17:32.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42811 #4081731 (1507 connections now open) 2016-03-13T09:17:32.424+0000 I NETWORK [conn4079177] end connection 10.0.26.69:40879 (1506 connections now open) 2016-03-13T09:17:32.425+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42812 #4081732 (1507 connections now open) 2016-03-13T09:17:32.428+0000 I NETWORK [conn4079262] end connection 10.0.26.69:40968 (1506 connections now open) 2016-03-13T09:17:32.428+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42813 #4081733 (1507 connections now open) 2016-03-13T09:17:32.431+0000 I NETWORK [conn4080619] end connection 10.0.27.237:35663 (1506 connections now open) 2016-03-13T09:17:32.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36107 #4081734 (1507 connections now open) 2016-03-13T09:17:32.437+0000 I NETWORK [conn4080617] end connection 10.0.27.237:35660 (1506 connections now open) 2016-03-13T09:17:32.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36108 #4081735 (1507 connections now open) 2016-03-13T09:17:32.442+0000 I NETWORK [conn4080618] end connection 10.0.27.237:35661 (1506 connections now open) 2016-03-13T09:17:32.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36109 #4081736 (1507 connections now open) 2016-03-13T09:17:32.458+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 49ms sequenceNumber: 16837 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:17:32.463+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.CustomEventReports: 1ms sequenceNumber: 16838 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:17:32.464+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 16839 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:17:32.470+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 16840 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:17:32.474+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 16841 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:17:32.475+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 16842 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:17:32.476+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 16843 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:17:32.480+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 16844 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:17:32.517+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 36ms sequenceNumber: 16845 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:17:32.605+0000 I NETWORK [conn4076181] end connection 10.0.179.209:35873 (1506 connections now open) 2016-03-13T09:17:32.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40774 #4081737 (1507 connections now open) 2016-03-13T09:17:32.802+0000 I NETWORK [conn4077829] end connection 10.0.179.209:37512 (1506 connections now open) 2016-03-13T09:17:32.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40775 #4081738 (1507 connections now open) 2016-03-13T09:17:32.960+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.HeatmapReports: 439ms sequenceNumber: 16846 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:17:33.014+0000 I NETWORK [conn4076633] end connection 10.0.20.53:46449 (1506 connections now open) 2016-03-13T09:17:33.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51057 #4081739 (1507 connections now open) 2016-03-13T09:17:33.033+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 59ms sequenceNumber: 16847 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:17:33.036+0000 I NETWORK [conn4077059] end connection 10.0.7.113:55600 (1506 connections now open) 2016-03-13T09:17:33.037+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59709 #4081740 (1507 connections now open) 2016-03-13T09:17:33.058+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.NavigationReports: 20ms sequenceNumber: 16848 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:17:33.060+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 16849 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:17:33.064+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 16850 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:17:33.065+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 16851 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:17:33.066+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 1ms sequenceNumber: 16852 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:17:33.069+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 16853 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:17:33.070+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 16854 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:17:33.071+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 16855 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:17:33.475+0000 I NETWORK [conn4077470] end connection 10.0.26.69:39099 (1506 connections now open) 2016-03-13T09:17:33.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42817 #4081741 (1507 connections now open) 2016-03-13T09:17:33.484+0000 I NETWORK [conn4081390] end connection 10.0.216.90:57619 (1506 connections now open) 2016-03-13T09:17:33.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57693 #4081742 (1507 connections now open) 2016-03-13T09:17:33.698+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 627ms sequenceNumber: 16856 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:17:33.948+0000 I NETWORK [conn4075769] end connection 10.0.216.90:52250 (1506 connections now open) 2016-03-13T09:17:33.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57695 #4081743 (1507 connections now open) 2016-03-13T09:17:34.103+0000 I NETWORK [conn4081570] end connection 10.0.179.209:40757 (1506 connections now open) 2016-03-13T09:17:34.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40779 #4081744 (1507 connections now open) 2016-03-13T09:17:34.336+0000 I NETWORK [conn4080787] end connection 10.0.7.113:59411 (1506 connections now open) 2016-03-13T09:17:34.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59715 #4081745 (1507 connections now open) 2016-03-13T09:17:34.442+0000 I NETWORK [conn4080697] end connection 10.0.27.237:35737 (1506 connections now open) 2016-03-13T09:17:34.442+0000 I NETWORK [conn4080698] end connection 10.0.27.237:35738 (1506 connections now open) 2016-03-13T09:17:34.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36114 #4081746 (1506 connections now open) 2016-03-13T09:17:34.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36115 #4081747 (1507 connections now open) 2016-03-13T09:17:34.604+0000 I NETWORK [conn4077642] end connection 10.0.26.69:39282 (1506 connections now open) 2016-03-13T09:17:34.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42821 #4081748 (1507 connections now open) 2016-03-13T09:17:35.049+0000 I NETWORK [conn4081316] end connection 10.0.27.237:35907 (1506 connections now open) 2016-03-13T09:17:35.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36118 #4081749 (1507 connections now open) 2016-03-13T09:17:35.343+0000 I NETWORK [conn4080406] end connection 10.0.7.113:59060 (1506 connections now open) 2016-03-13T09:17:35.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59717 #4081750 (1507 connections now open) 2016-03-13T09:17:35.456+0000 I NETWORK [conn4079981] end connection 10.0.27.237:34978 (1506 connections now open) 2016-03-13T09:17:35.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36120 #4081751 (1507 connections now open) 2016-03-13T09:17:36.256+0000 I NETWORK [conn4081629] end connection 10.0.20.53:50998 (1506 connections now open) 2016-03-13T09:17:36.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51065 #4081752 (1507 connections now open) 2016-03-13T09:17:36.419+0000 I NETWORK [conn4079157] end connection 10.0.216.90:55848 (1506 connections now open) 2016-03-13T09:17:36.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57703 #4081753 (1507 connections now open) 2016-03-13T09:17:36.570+0000 I NETWORK [conn4080699] end connection 10.0.7.113:59360 (1506 connections now open) 2016-03-13T09:17:36.570+0000 I NETWORK [conn4080700] end connection 10.0.7.113:59361 (1506 connections now open) 2016-03-13T09:17:36.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59721 #4081754 (1506 connections now open) 2016-03-13T09:17:36.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59722 #4081755 (1507 connections now open) 2016-03-13T09:17:36.648+0000 I NETWORK [conn4078235] end connection 10.0.20.53:48071 (1506 connections now open) 2016-03-13T09:17:36.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51068 #4081756 (1507 connections now open) 2016-03-13T09:17:36.932+0000 I NETWORK [conn4079623] end connection 10.0.179.209:39334 (1506 connections now open) 2016-03-13T09:17:36.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40790 #4081757 (1507 connections now open) 2016-03-13T09:17:37.346+0000 I NETWORK [conn4079218] end connection 10.0.26.69:40921 (1506 connections now open) 2016-03-13T09:17:37.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42827 #4081758 (1507 connections now open) 2016-03-13T09:17:37.613+0000 I NETWORK [conn4080701] end connection 10.0.216.90:57376 (1506 connections now open) 2016-03-13T09:17:37.613+0000 I NETWORK [conn4080702] end connection 10.0.216.90:57377 (1506 connections now open) 2016-03-13T09:17:37.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57705 #4081759 (1506 connections now open) 2016-03-13T09:17:37.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57704 #4081760 (1507 connections now open) 2016-03-13T09:17:37.999+0000 I NETWORK [conn4080194] end connection 10.0.7.113:58851 (1506 connections now open) 2016-03-13T09:17:38.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59728 #4081761 (1507 connections now open) 2016-03-13T09:17:38.145+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.Sessions: 4357ms sequenceNumber: 16857 version: 30649|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:17:38.364+0000 I NETWORK [conn4078616] end connection 10.0.20.53:48466 (1506 connections now open) 2016-03-13T09:17:38.364+0000 I NETWORK [conn4080834] end connection 10.0.179.209:40523 (1505 connections now open) 2016-03-13T09:17:38.365+0000 I NETWORK [conn4079284] end connection 10.0.216.90:55975 (1504 connections now open) 2016-03-13T09:17:38.365+0000 I NETWORK [conn4081060] end connection 10.0.216.90:57524 (1504 connections now open) 2016-03-13T09:17:38.365+0000 I NETWORK [conn4080943] end connection 10.0.7.113:59523 (1504 connections now open) 2016-03-13T09:17:38.365+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40794 #4081762 (1505 connections now open) 2016-03-13T09:17:38.365+0000 I NETWORK [conn4078736] end connection 10.0.20.53:48596 (1502 connections now open) 2016-03-13T09:17:38.365+0000 I NETWORK [conn4079709] end connection 10.0.26.69:41420 (1501 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [conn4076717] end connection 10.0.20.53:46540 (1500 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [conn4076286] end connection 10.0.179.209:35987 (1500 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42830 #4081763 (1500 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [conn4081262] end connection 10.0.216.90:57577 (1499 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [conn4079283] end connection 10.0.20.53:49174 (1498 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [conn4081549] end connection 10.0.216.90:57657 (1497 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [conn4079475] end connection 10.0.26.69:41180 (1497 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51079 #4081764 (1497 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40795 #4081765 (1498 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51080 #4081766 (1499 connections now open) 2016-03-13T09:17:38.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51081 #4081767 (1500 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59729 #4081768 (1501 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57708 #4081769 (1502 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [conn4078488] end connection 10.0.179.209:38181 (1501 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57710 #4081770 (1503 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57709 #4081771 (1503 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57711 #4081772 (1504 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42831 #4081773 (1505 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51082 #4081774 (1506 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40796 #4081775 (1507 connections now open) 2016-03-13T09:17:38.367+0000 I NETWORK [conn4081003] end connection 10.0.27.237:35787 (1506 connections now open) 2016-03-13T09:17:38.368+0000 I NETWORK [conn4081257] end connection 10.0.26.69:42555 (1505 connections now open) 2016-03-13T09:17:38.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36129 #4081776 (1506 connections now open) 2016-03-13T09:17:38.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42833 #4081777 (1507 connections now open) 2016-03-13T09:17:38.382+0000 I NETWORK [conn4078927] end connection 10.0.216.90:55612 (1506 connections now open) 2016-03-13T09:17:38.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57714 #4081778 (1507 connections now open) 2016-03-13T09:17:38.384+0000 I NETWORK [conn4081636] end connection 10.0.20.53:51003 (1506 connections now open) 2016-03-13T09:17:38.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51085 #4081779 (1507 connections now open) 2016-03-13T09:17:38.456+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.ShardTest: 0ms sequenceNumber: 16858 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:17:38.457+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 16859 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:17:38.458+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 16860 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:17:38.474+0000 I SHARDING [conn3180465] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 15ms sequenceNumber: 16861 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:17:38.478+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-2-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:38.478+0000 I NETWORK [conn4080926] end connection 10.0.179.209:40592 (1506 connections now open) 2016-03-13T09:17:38.478+0000 I NETWORK [conn4078951] end connection 10.0.20.53:48813 (1505 connections now open) 2016-03-13T09:17:38.478+0000 I NETWORK [conn4081556] end connection 10.0.216.90:57659 (1505 connections now open) 2016-03-13T09:17:38.479+0000 I NETWORK [conn4081696] end connection 10.0.26.69:42784 (1503 connections now open) 2016-03-13T09:17:38.479+0000 I NETWORK [conn4081229] end connection 10.0.27.237:35881 (1502 connections now open) 2016-03-13T09:17:38.479+0000 I NETWORK [conn4079410] end connection 10.0.216.90:56098 (1501 connections now open) 2016-03-13T09:17:38.479+0000 I NETWORK [conn4081756] end connection 10.0.20.53:51068 (1501 connections now open) 2016-03-13T09:17:38.479+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51087 #4081780 (1502 connections now open) 2016-03-13T09:17:38.479+0000 I NETWORK [conn4081477] end connection 10.0.27.237:35975 (1499 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36131 #4081781 (1501 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42835 #4081782 (1502 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [conn4078403] end connection 10.0.26.69:40087 (1501 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [conn4080111] end connection 10.0.7.113:58767 (1500 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [conn4077320] end connection 10.0.27.237:60502 (1499 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [conn4080922] end connection 10.0.7.113:59512 (1499 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42836 #4081783 (1499 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [conn4077705] end connection 10.0.7.113:56283 (1498 connections now open) 2016-03-13T09:17:38.480+0000 I NETWORK [conn4079286] end connection 10.0.179.209:38996 (1498 connections now open) 2016-03-13T09:17:38.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36132 #4081784 (1498 connections now open) 2016-03-13T09:17:38.481+0000 I NETWORK [conn4081354] end connection 10.0.216.90:57610 (1496 connections now open) 2016-03-13T09:17:38.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59731 #4081785 (1498 connections now open) 2016-03-13T09:17:38.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59732 #4081786 (1499 connections now open) 2016-03-13T09:17:38.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59733 #4081787 (1500 connections now open) 2016-03-13T09:17:38.481+0000 I NETWORK [conn4081191] end connection 10.0.20.53:50787 (1499 connections now open) 2016-03-13T09:17:38.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36133 #4081788 (1500 connections now open) 2016-03-13T09:17:38.482+0000 I NETWORK [conn4077432] end connection 10.0.7.113:55983 (1499 connections now open) 2016-03-13T09:17:38.482+0000 I NETWORK [conn4081476] end connection 10.0.216.90:57641 (1499 connections now open) 2016-03-13T09:17:38.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57715 #4081789 (1499 connections now open) 2016-03-13T09:17:38.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57716 #4081790 (1500 connections now open) 2016-03-13T09:17:38.482+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:38.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40801 #4081791 (1501 connections now open) 2016-03-13T09:17:38.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40802 #4081792 (1502 connections now open) 2016-03-13T09:17:38.483+0000 I NETWORK [conn4078821] end connection 10.0.20.53:48681 (1501 connections now open) 2016-03-13T09:17:38.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51090 #4081793 (1502 connections now open) 2016-03-13T09:17:38.483+0000 I NETWORK [conn4081185] end connection 10.0.7.113:59607 (1501 connections now open) 2016-03-13T09:17:38.483+0000 I NETWORK [conn4081022] end connection 10.0.27.237:35799 (1501 connections now open) 2016-03-13T09:17:38.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51091 #4081794 (1503 connections now open) 2016-03-13T09:17:38.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57717 #4081795 (1502 connections now open) 2016-03-13T09:17:38.484+0000 I NETWORK [conn4081471] end connection 10.0.27.237:35971 (1501 connections now open) 2016-03-13T09:17:38.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59735 #4081796 (1502 connections now open) 2016-03-13T09:17:38.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51092 #4081797 (1503 connections now open) 2016-03-13T09:17:38.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36135 #4081798 (1504 connections now open) 2016-03-13T09:17:38.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59736 #4081799 (1505 connections now open) 2016-03-13T09:17:38.485+0000 I NETWORK [conn4079133] end connection 10.0.216.90:55820 (1504 connections now open) 2016-03-13T09:17:38.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57719 #4081800 (1505 connections now open) 2016-03-13T09:17:38.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57721 #4081801 (1506 connections now open) 2016-03-13T09:17:38.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36136 #4081802 (1507 connections now open) 2016-03-13T09:17:38.489+0000 I NETWORK [conn4078108] end connection 10.0.7.113:56713 (1506 connections now open) 2016-03-13T09:17:38.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59738 #4081803 (1507 connections now open) 2016-03-13T09:17:38.490+0000 I NETWORK [conn4081346] end connection 10.0.216.90:57607 (1506 connections now open) 2016-03-13T09:17:38.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57726 #4081804 (1507 connections now open) 2016-03-13T09:17:38.491+0000 I NETWORK [conn4081580] end connection 10.0.27.237:36025 (1506 connections now open) 2016-03-13T09:17:38.491+0000 I NETWORK [conn4078193] end connection 10.0.7.113:56803 (1505 connections now open) 2016-03-13T09:17:38.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36138 #4081805 (1506 connections now open) 2016-03-13T09:17:38.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59739 #4081806 (1507 connections now open) 2016-03-13T09:17:38.492+0000 I NETWORK [conn4080606] end connection 10.0.20.53:50585 (1506 connections now open) 2016-03-13T09:17:38.493+0000 I NETWORK [conn4081193] end connection 10.0.26.69:42532 (1505 connections now open) 2016-03-13T09:17:38.493+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42841 #4081807 (1506 connections now open) 2016-03-13T09:17:38.493+0000 I NETWORK [conn4079584] end connection 10.0.7.113:58243 (1504 connections now open) 2016-03-13T09:17:38.493+0000 I NETWORK [conn4079199] end connection 10.0.20.53:49080 (1504 connections now open) 2016-03-13T09:17:38.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51098 #4081808 (1505 connections now open) 2016-03-13T09:17:38.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59741 #4081809 (1506 connections now open) 2016-03-13T09:17:38.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51099 #4081810 (1507 connections now open) 2016-03-13T09:17:38.495+0000 I NETWORK [conn4081085] end connection 10.0.20.53:50746 (1506 connections now open) 2016-03-13T09:17:38.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51100 #4081811 (1507 connections now open) 2016-03-13T09:17:38.496+0000 I NETWORK [conn4080898] end connection 10.0.26.69:42430 (1506 connections now open) 2016-03-13T09:17:38.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42842 #4081812 (1507 connections now open) 2016-03-13T09:17:38.498+0000 I NETWORK [conn4081231] end connection 10.0.26.69:42543 (1506 connections now open) 2016-03-13T09:17:38.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42843 #4081813 (1507 connections now open) 2016-03-13T09:17:38.501+0000 I NETWORK [conn4081288] end connection 10.0.26.69:42569 (1506 connections now open) 2016-03-13T09:17:38.501+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42844 #4081814 (1507 connections now open) 2016-03-13T09:17:38.539+0000 I NETWORK [conn4079813] end connection 10.0.26.69:41526 (1506 connections now open) 2016-03-13T09:17:38.540+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42846 #4081815 (1507 connections now open) 2016-03-13T09:17:38.542+0000 I NETWORK [conn4080065] end connection 10.0.26.69:41780 (1506 connections now open) 2016-03-13T09:17:38.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42847 #4081816 (1507 connections now open) 2016-03-13T09:17:38.558+0000 I NETWORK [conn4076996] end connection 10.0.179.209:36680 (1506 connections now open) 2016-03-13T09:17:38.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40807 #4081817 (1507 connections now open) 2016-03-13T09:17:38.560+0000 I NETWORK [conn4076954] end connection 10.0.179.209:36637 (1506 connections now open) 2016-03-13T09:17:38.561+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40808 #4081818 (1507 connections now open) 2016-03-13T09:17:38.645+0000 I NETWORK [conn4080708] end connection 10.0.20.53:50681 (1506 connections now open) 2016-03-13T09:17:38.645+0000 I NETWORK [conn4080709] end connection 10.0.20.53:50682 (1506 connections now open) 2016-03-13T09:17:38.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51103 #4081819 (1506 connections now open) 2016-03-13T09:17:38.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51104 #4081820 (1507 connections now open) 2016-03-13T09:17:39.192+0000 I SHARDING [conn3466873] ChunkManager: time to load chunks for mydomain.Sessions: 714ms sequenceNumber: 16862 version: 30650|1||563ba74d869758a4542b9075 based on: 30649|1||563ba74d869758a4542b9075 2016-03-13T09:17:39.375+0000 I NETWORK [conn4079367] end connection 10.0.20.53:49265 (1506 connections now open) 2016-03-13T09:17:39.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51107 #4081821 (1507 connections now open) 2016-03-13T09:17:39.825+0000 I NETWORK [conn4080063] end connection 10.0.179.209:39779 (1506 connections now open) 2016-03-13T09:17:39.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40810 #4081822 (1507 connections now open) 2016-03-13T09:17:39.828+0000 I NETWORK [conn4080277] end connection 10.0.179.209:40001 (1506 connections now open) 2016-03-13T09:17:39.828+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40811 #4081823 (1507 connections now open) 2016-03-13T09:17:40.288+0000 I NETWORK [conn4081398] end connection 10.0.27.237:35939 (1506 connections now open) 2016-03-13T09:17:40.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36142 #4081824 (1507 connections now open) 2016-03-13T09:17:40.290+0000 I NETWORK [conn4081042] end connection 10.0.27.237:35809 (1506 connections now open) 2016-03-13T09:17:40.291+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36143 #4081825 (1507 connections now open) 2016-03-13T09:17:40.415+0000 I NETWORK [conn4081805] end connection 10.0.27.237:36138 (1506 connections now open) 2016-03-13T09:17:40.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36144 #4081826 (1507 connections now open) 2016-03-13T09:17:40.553+0000 I NETWORK [conn4081643] end connection 10.0.20.53:51007 (1506 connections now open) 2016-03-13T09:17:40.553+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51110 #4081827 (1507 connections now open) 2016-03-13T09:17:40.731+0000 I NETWORK [conn4080092] end connection 10.0.216.90:56775 (1506 connections now open) 2016-03-13T09:17:40.732+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57733 #4081828 (1507 connections now open) 2016-03-13T09:17:40.734+0000 I NETWORK [conn4078912] end connection 10.0.216.90:55598 (1506 connections now open) 2016-03-13T09:17:40.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57734 #4081829 (1507 connections now open) 2016-03-13T09:17:40.990+0000 I NETWORK [conn4080711] end connection 10.0.179.209:40440 (1506 connections now open) 2016-03-13T09:17:40.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40814 #4081830 (1507 connections now open) 2016-03-13T09:17:40.991+0000 I NETWORK [conn4080710] end connection 10.0.179.209:40439 (1506 connections now open) 2016-03-13T09:17:40.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40815 #4081831 (1507 connections now open) 2016-03-13T09:17:41.684+0000 I NETWORK [conn4080754] end connection 10.0.26.69:42422 (1506 connections now open) 2016-03-13T09:17:41.684+0000 I NETWORK [conn4080755] end connection 10.0.26.69:42423 (1506 connections now open) 2016-03-13T09:17:41.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42854 #4081832 (1507 connections now open) 2016-03-13T09:17:41.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42855 #4081833 (1507 connections now open) 2016-03-13T09:17:42.212+0000 I NETWORK [conn4076040] end connection 10.0.179.209:35729 (1506 connections now open) 2016-03-13T09:17:42.212+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40818 #4081834 (1507 connections now open) 2016-03-13T09:17:42.214+0000 I NETWORK [conn4080990] end connection 10.0.179.209:40627 (1506 connections now open) 2016-03-13T09:17:42.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40819 #4081835 (1507 connections now open) 2016-03-13T09:17:42.415+0000 I NETWORK [conn4079542] end connection 10.0.7.113:58200 (1506 connections now open) 2016-03-13T09:17:42.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59751 #4081836 (1507 connections now open) 2016-03-13T09:17:42.417+0000 I NETWORK [conn4078659] end connection 10.0.7.113:57299 (1506 connections now open) 2016-03-13T09:17:42.418+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59752 #4081837 (1507 connections now open) 2016-03-13T09:17:42.850+0000 I NETWORK [conn4081542] end connection 10.0.27.237:36008 (1506 connections now open) 2016-03-13T09:17:42.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36147 #4081838 (1507 connections now open) 2016-03-13T09:17:43.553+0000 I NETWORK [conn4081394] end connection 10.0.26.69:42619 (1506 connections now open) 2016-03-13T09:17:43.553+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42858 #4081839 (1507 connections now open) 2016-03-13T09:17:43.556+0000 I NETWORK [conn4081711] end connection 10.0.26.69:42793 (1506 connections now open) 2016-03-13T09:17:43.556+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42859 #4081840 (1507 connections now open) 2016-03-13T09:17:43.558+0000 I NETWORK [conn4079601] end connection 10.0.26.69:41310 (1506 connections now open) 2016-03-13T09:17:43.559+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42860 #4081841 (1507 connections now open) 2016-03-13T09:17:43.579+0000 I NETWORK [conn4081519] end connection 10.0.27.237:35997 (1506 connections now open) 2016-03-13T09:17:43.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36149 #4081842 (1507 connections now open) 2016-03-13T09:17:44.038+0000 I NETWORK [conn4078208] end connection 10.0.27.237:33142 (1506 connections now open) 2016-03-13T09:17:44.038+0000 I NETWORK [conn4080936] end connection 10.0.179.209:40595 (1505 connections now open) 2016-03-13T09:17:44.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36151 #4081843 (1507 connections now open) 2016-03-13T09:17:44.039+0000 I NETWORK [conn4081076] end connection 10.0.27.237:35823 (1505 connections now open) 2016-03-13T09:17:44.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40821 #4081844 (1506 connections now open) 2016-03-13T09:17:44.039+0000 I NETWORK [conn4078885] end connection 10.0.20.53:48754 (1505 connections now open) 2016-03-13T09:17:44.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36152 #4081845 (1506 connections now open) 2016-03-13T09:17:44.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51117 #4081846 (1507 connections now open) 2016-03-13T09:17:44.041+0000 I NETWORK [conn4080568] end connection 10.0.27.237:35606 (1506 connections now open) 2016-03-13T09:17:44.041+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 16863 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:17:44.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36154 #4081847 (1507 connections now open) 2016-03-13T09:17:44.046+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.ActionReports: 5ms sequenceNumber: 16864 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:17:44.048+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 16865 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:17:44.050+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 16866 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:17:44.051+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 16867 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:17:44.052+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 16868 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:17:44.055+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 2ms sequenceNumber: 16869 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:17:44.087+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.Clients: 32ms sequenceNumber: 16870 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:17:44.090+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 16871 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:17:44.092+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 16872 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:17:44.093+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 16873 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:17:44.094+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 16874 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:17:44.135+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 41ms sequenceNumber: 16875 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:17:44.141+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.CustomEventReports: 2ms sequenceNumber: 16876 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:17:44.142+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 16877 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:17:44.148+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 16878 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:17:44.152+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 16879 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:17:44.153+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 16880 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:17:44.154+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 16881 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:17:44.158+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 16882 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:17:44.194+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 35ms sequenceNumber: 16883 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:17:44.391+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.HeatmapReports: 194ms sequenceNumber: 16884 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:17:44.446+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 42ms sequenceNumber: 16885 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:17:44.455+0000 I NETWORK [conn4081746] end connection 10.0.27.237:36114 (1506 connections now open) 2016-03-13T09:17:44.455+0000 I NETWORK [conn4081747] end connection 10.0.27.237:36115 (1505 connections now open) 2016-03-13T09:17:44.455+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36156 #4081848 (1506 connections now open) 2016-03-13T09:17:44.455+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36155 #4081849 (1507 connections now open) 2016-03-13T09:17:44.458+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.NavigationReports: 7ms sequenceNumber: 16886 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:17:44.461+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 16887 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:17:44.464+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 16888 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:17:44.465+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 16889 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:17:44.467+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 1ms sequenceNumber: 16890 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:17:44.469+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 16891 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:17:44.471+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 16892 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:17:44.471+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 16893 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:17:44.542+0000 I NETWORK [conn4079502] end connection 10.0.26.69:41205 (1506 connections now open) 2016-03-13T09:17:44.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42864 #4081850 (1507 connections now open) 2016-03-13T09:17:44.992+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5303818fa188537623d68 2016-03-13T09:17:45.159+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 687ms sequenceNumber: 16894 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:17:46.117+0000 I NETWORK [conn4081241] end connection 10.0.20.53:50802 (1506 connections now open) 2016-03-13T09:17:46.117+0000 I NETWORK [conn4081282] end connection 10.0.7.113:59634 (1505 connections now open) 2016-03-13T09:17:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51120 #4081851 (1506 connections now open) 2016-03-13T09:17:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59765 #4081852 (1507 connections now open) 2016-03-13T09:17:46.583+0000 I NETWORK [conn4081754] end connection 10.0.7.113:59721 (1506 connections now open) 2016-03-13T09:17:46.584+0000 I NETWORK [conn4081755] end connection 10.0.7.113:59722 (1506 connections now open) 2016-03-13T09:17:46.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59767 #4081853 (1506 connections now open) 2016-03-13T09:17:46.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59766 #4081854 (1507 connections now open) 2016-03-13T09:17:46.716+0000 I NETWORK [conn4079626] end connection 10.0.26.69:41331 (1506 connections now open) 2016-03-13T09:17:46.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42871 #4081855 (1507 connections now open) 2016-03-13T09:17:47.627+0000 I NETWORK [conn4081759] end connection 10.0.216.90:57705 (1506 connections now open) 2016-03-13T09:17:47.627+0000 I NETWORK [conn4081760] end connection 10.0.216.90:57704 (1505 connections now open) 2016-03-13T09:17:47.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57756 #4081856 (1506 connections now open) 2016-03-13T09:17:47.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57755 #4081857 (1507 connections now open) 2016-03-13T09:17:48.607+0000 I NETWORK [conn4080022] end connection 10.0.179.209:39738 (1506 connections now open) 2016-03-13T09:17:48.608+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40842 #4081858 (1507 connections now open) 2016-03-13T09:17:48.630+0000 I NETWORK [conn4081081] end connection 10.0.27.237:35825 (1506 connections now open) 2016-03-13T09:17:48.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36168 #4081859 (1507 connections now open) 2016-03-13T09:17:48.660+0000 I NETWORK [conn4081820] end connection 10.0.20.53:51104 (1506 connections now open) 2016-03-13T09:17:48.660+0000 I NETWORK [conn4081819] end connection 10.0.20.53:51103 (1506 connections now open) 2016-03-13T09:17:48.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51121 #4081860 (1506 connections now open) 2016-03-13T09:17:48.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51122 #4081861 (1507 connections now open) 2016-03-13T09:17:49.315+0000 I NETWORK [conn4080469] end connection 10.0.20.53:50430 (1506 connections now open) 2016-03-13T09:17:49.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51126 #4081862 (1507 connections now open) 2016-03-13T09:17:49.341+0000 I NETWORK [conn4081537] end connection 10.0.179.209:40755 (1506 connections now open) 2016-03-13T09:17:49.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40844 #4081863 (1507 connections now open) 2016-03-13T09:17:50.044+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:17:49.774+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:17:50.044+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:17:50.180+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5302418fa188537623d66 2016-03-13T09:17:50.317+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5302e18fa188537623d67 2016-03-13T09:17:50.487+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.Sessions: 5234ms sequenceNumber: 16895 version: 30650|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:17:50.792+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 16896 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:17:50.793+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 16897 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:17:50.795+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 16898 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:17:50.825+0000 I SHARDING [conn3361460] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 30ms sequenceNumber: 16899 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:17:50.828+0000 I NETWORK [conn4076202] end connection 10.0.27.237:59392 (1506 connections now open) 2016-03-13T09:17:50.829+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-1-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:50.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36170 #4081864 (1507 connections now open) 2016-03-13T09:17:50.829+0000 I NETWORK [conn4081025] end connection 10.0.179.209:40646 (1506 connections now open) 2016-03-13T09:17:50.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40846 #4081865 (1507 connections now open) 2016-03-13T09:17:50.830+0000 I NETWORK [conn4080720] end connection 10.0.179.209:40448 (1506 connections now open) 2016-03-13T09:17:50.831+0000 I NETWORK [conn4081105] end connection 10.0.27.237:35835 (1505 connections now open) 2016-03-13T09:17:50.831+0000 I NETWORK [conn4080589] end connection 10.0.179.209:40335 (1504 connections now open) 2016-03-13T09:17:50.831+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40847 #4081866 (1505 connections now open) 2016-03-13T09:17:50.831+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36171 #4081867 (1506 connections now open) 2016-03-13T09:17:50.831+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40848 #4081868 (1507 connections now open) 2016-03-13T09:17:50.831+0000 I NETWORK [conn4081102] end connection 10.0.27.237:35833 (1506 connections now open) 2016-03-13T09:17:50.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36173 #4081869 (1507 connections now open) 2016-03-13T09:17:50.837+0000 I NETWORK [conn4081437] end connection 10.0.27.237:35954 (1506 connections now open) 2016-03-13T09:17:50.837+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36175 #4081870 (1507 connections now open) 2016-03-13T09:17:50.947+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('56346357584ad10f1f9aaa8c')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56346357584ad10f1f9aaa8c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634669d55418c0ec3f9ca7d') }, shard: "shmydomain1", lastmod: Timestamp 30650000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:17:50.970+0000 I NETWORK [conn4079158] end connection 10.0.179.209:38871 (1506 connections now open) 2016-03-13T09:17:50.971+0000 I NETWORK [conn4078163] end connection 10.0.179.209:37853 (1505 connections now open) 2016-03-13T09:17:50.971+0000 I NETWORK [conn4080966] end connection 10.0.216.90:57510 (1504 connections now open) 2016-03-13T09:17:50.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40852 #4081871 (1506 connections now open) 2016-03-13T09:17:50.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57761 #4081872 (1506 connections now open) 2016-03-13T09:17:50.972+0000 I NETWORK [conn4080844] end connection 10.0.7.113:59454 (1505 connections now open) 2016-03-13T09:17:50.972+0000 I NETWORK [conn4081425] end connection 10.0.26.69:42635 (1504 connections now open) 2016-03-13T09:17:50.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42880 #4081873 (1505 connections now open) 2016-03-13T09:17:50.972+0000 I NETWORK [conn4081634] end connection 10.0.20.53:51002 (1504 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [conn4081349] end connection 10.0.26.69:42597 (1503 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [conn4080548] end connection 10.0.179.209:40293 (1502 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [conn4079708] end connection 10.0.216.90:56390 (1501 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59776 #4081874 (1502 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [conn4078867] end connection 10.0.7.113:57510 (1500 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [conn4081411] end connection 10.0.216.90:57624 (1500 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40854 #4081875 (1501 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [conn4081473] end connection 10.0.27.237:35972 (1500 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57763 #4081876 (1501 connections now open) 2016-03-13T09:17:50.973+0000 I NETWORK [conn4078886] end connection 10.0.179.209:38588 (1499 connections now open) 2016-03-13T09:17:50.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51129 #4081877 (1501 connections now open) 2016-03-13T09:17:50.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40855 #4081878 (1502 connections now open) 2016-03-13T09:17:50.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36178 #4081879 (1503 connections now open) 2016-03-13T09:17:50.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40856 #4081880 (1504 connections now open) 2016-03-13T09:17:50.974+0000 I NETWORK [conn4079752] end connection 10.0.216.90:56433 (1503 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57764 #4081881 (1504 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [conn4080839] end connection 10.0.7.113:59451 (1503 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [conn4081441] end connection 10.0.27.237:35956 (1502 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59778 #4081882 (1503 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57765 #4081883 (1504 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36179 #4081884 (1505 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [conn4079473] end connection 10.0.179.209:39173 (1503 connections now open) 2016-03-13T09:17:50.975+0000 I NETWORK [conn4081604] end connection 10.0.179.209:40768 (1504 connections now open) 2016-03-13T09:17:50.976+0000 I NETWORK [conn4081520] end connection 10.0.20.53:50939 (1502 connections now open) 2016-03-13T09:17:50.976+0000 I NETWORK [conn4079079] end connection 10.0.7.113:57732 (1501 connections now open) 2016-03-13T09:17:50.976+0000 I NETWORK [conn4081786] end connection 10.0.7.113:59732 (1500 connections now open) 2016-03-13T09:17:50.976+0000 I NETWORK [conn4081617] end connection 10.0.27.237:36050 (1499 connections now open) 2016-03-13T09:17:50.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59779 #4081885 (1500 connections now open) 2016-03-13T09:17:50.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59780 #4081886 (1501 connections now open) 2016-03-13T09:17:50.977+0000 I NETWORK [conn4078911] end connection 10.0.7.113:57557 (1499 connections now open) 2016-03-13T09:17:50.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42881 #4081887 (1502 connections now open) 2016-03-13T09:17:50.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59781 #4081888 (1502 connections now open) 2016-03-13T09:17:50.977+0000 I NETWORK [conn4079201] end connection 10.0.7.113:57855 (1500 connections now open) 2016-03-13T09:17:50.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59782 #4081889 (1502 connections now open) 2016-03-13T09:17:50.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59783 #4081890 (1503 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [conn4077344] end connection 10.0.216.90:53937 (1502 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [conn4081038] end connection 10.0.216.90:57516 (1501 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [conn4080742] end connection 10.0.179.209:40460 (1501 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [conn4081045] end connection 10.0.216.90:57519 (1499 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [conn4080281] end connection 10.0.7.113:58937 (1498 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40858 #4081891 (1499 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57768 #4081892 (1500 connections now open) 2016-03-13T09:17:50.978+0000 I NETWORK [conn4081782] end connection 10.0.26.69:42835 (1499 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [conn4081300] end connection 10.0.26.69:42574 (1498 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57769 #4081893 (1499 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [conn4081698] end connection 10.0.27.237:36086 (1498 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [conn4080192] end connection 10.0.26.69:41910 (1498 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [conn4081057] end connection 10.0.216.90:57522 (1496 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [conn4077997] end connection 10.0.7.113:56602 (1497 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [conn4080748] end connection 10.0.179.209:40464 (1496 connections now open) 2016-03-13T09:17:50.979+0000 I NETWORK [conn4077188] end connection 10.0.20.53:47018 (1493 connections now open) 2016-03-13T09:17:50.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57770 #4081894 (1494 connections now open) 2016-03-13T09:17:50.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36181 #4081895 (1495 connections now open) 2016-03-13T09:17:50.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40860 #4081896 (1496 connections now open) 2016-03-13T09:17:50.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36182 #4081897 (1497 connections now open) 2016-03-13T09:17:50.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51134 #4081898 (1498 connections now open) 2016-03-13T09:17:50.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57771 #4081899 (1499 connections now open) 2016-03-13T09:17:50.980+0000 I NETWORK [conn4081099] end connection 10.0.26.69:42495 (1497 connections now open) 2016-03-13T09:17:50.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42882 #4081900 (1499 connections now open) 2016-03-13T09:17:50.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42883 #4081901 (1500 connections now open) 2016-03-13T09:17:50.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42885 #4081902 (1501 connections now open) 2016-03-13T09:17:50.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59786 #4081903 (1502 connections now open) 2016-03-13T09:17:50.982+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59787 #4081904 (1503 connections now open) 2016-03-13T09:17:50.983+0000 I NETWORK [conn4081494] end connection 10.0.27.237:35984 (1502 connections now open) 2016-03-13T09:17:50.983+0000 I NETWORK [conn4081510] end connection 10.0.27.237:35993 (1501 connections now open) 2016-03-13T09:17:50.984+0000 I NETWORK [conn4081356] end connection 10.0.20.53:50847 (1500 connections now open) 2016-03-13T09:17:50.984+0000 I NETWORK [conn4081565] end connection 10.0.179.209:40756 (1499 connections now open) 2016-03-13T09:17:50.984+0000 I NETWORK [conn4081062] end connection 10.0.26.69:42482 (1498 connections now open) 2016-03-13T09:17:50.984+0000 I NETWORK [conn4076462] end connection 10.0.27.237:59644 (1497 connections now open) 2016-03-13T09:17:50.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51136 #4081905 (1498 connections now open) 2016-03-13T09:17:50.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36184 #4081906 (1499 connections now open) 2016-03-13T09:17:50.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36185 #4081907 (1500 connections now open) 2016-03-13T09:17:50.985+0000 I NETWORK [conn4080927] end connection 10.0.27.237:35762 (1499 connections now open) 2016-03-13T09:17:50.985+0000 I NETWORK [conn4080925] end connection 10.0.179.209:40591 (1498 connections now open) 2016-03-13T09:17:50.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51138 #4081908 (1499 connections now open) 2016-03-13T09:17:50.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40863 #4081909 (1500 connections now open) 2016-03-13T09:17:50.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42887 #4081910 (1501 connections now open) 2016-03-13T09:17:50.986+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36186 #4081911 (1502 connections now open) 2016-03-13T09:17:50.986+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36187 #4081912 (1503 connections now open) 2016-03-13T09:17:50.987+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:50.987+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-2-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:50.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40864 #4081913 (1504 connections now open) 2016-03-13T09:17:50.988+0000 I NETWORK [conn4081004] end connection 10.0.20.53:50713 (1503 connections now open) 2016-03-13T09:17:50.988+0000 I NETWORK [conn4081146] end connection 10.0.27.237:35851 (1503 connections now open) 2016-03-13T09:17:50.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42892 #4081914 (1504 connections now open) 2016-03-13T09:17:50.988+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51140 #4081915 (1504 connections now open) 2016-03-13T09:17:50.988+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:50.989+0000 I NETWORK [conn4081496] end connection 10.0.27.237:35985 (1503 connections now open) 2016-03-13T09:17:50.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36190 #4081916 (1504 connections now open) 2016-03-13T09:17:50.990+0000 I NETWORK [conn4078121] end connection 10.0.20.53:47965 (1503 connections now open) 2016-03-13T09:17:50.990+0000 I NETWORK [conn4081072] end connection 10.0.20.53:50742 (1502 connections now open) 2016-03-13T09:17:50.990+0000 I NETWORK [conn4081056] end connection 10.0.26.69:42479 (1501 connections now open) 2016-03-13T09:17:50.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51142 #4081917 (1503 connections now open) 2016-03-13T09:17:50.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36192 #4081918 (1503 connections now open) 2016-03-13T09:17:50.990+0000 I NETWORK [conn4080905] end connection 10.0.27.237:35757 (1502 connections now open) 2016-03-13T09:17:50.991+0000 I NETWORK [conn4081674] end connection 10.0.26.69:42770 (1501 connections now open) 2016-03-13T09:17:50.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36193 #4081919 (1503 connections now open) 2016-03-13T09:17:50.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42893 #4081920 (1503 connections now open) 2016-03-13T09:17:50.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40867 #4081921 (1504 connections now open) 2016-03-13T09:17:50.991+0000 I NETWORK [conn4081183] end connection 10.0.27.237:35866 (1503 connections now open) 2016-03-13T09:17:50.992+0000 I NETWORK [conn4081527] end connection 10.0.20.53:50942 (1502 connections now open) 2016-03-13T09:17:50.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42895 #4081922 (1503 connections now open) 2016-03-13T09:17:50.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51144 #4081923 (1504 connections now open) 2016-03-13T09:17:50.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51146 #4081924 (1505 connections now open) 2016-03-13T09:17:50.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40869 #4081925 (1506 connections now open) 2016-03-13T09:17:50.993+0000 I NETWORK [conn4081041] end connection 10.0.179.209:40651 (1505 connections now open) 2016-03-13T09:17:50.994+0000 I NETWORK [conn4081463] end connection 10.0.20.53:50906 (1504 connections now open) 2016-03-13T09:17:50.994+0000 I NETWORK [conn4081178] end connection 10.0.179.209:40698 (1503 connections now open) 2016-03-13T09:17:50.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36195 #4081926 (1504 connections now open) 2016-03-13T09:17:50.994+0000 I NETWORK [conn4079878] end connection 10.0.216.90:56560 (1502 connections now open) 2016-03-13T09:17:50.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40870 #4081927 (1505 connections now open) 2016-03-13T09:17:50.994+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 16900 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:17:50.995+0000 I NETWORK [conn4077980] end connection 10.0.216.90:54616 (1503 connections now open) 2016-03-13T09:17:50.995+0000 I NETWORK [conn4081808] end connection 10.0.20.53:51098 (1502 connections now open) 2016-03-13T09:17:50.995+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40873 #4081928 (1503 connections now open) 2016-03-13T09:17:50.995+0000 I NETWORK [conn4079287] end connection 10.0.7.113:57941 (1502 connections now open) 2016-03-13T09:17:50.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57782 #4081929 (1503 connections now open) 2016-03-13T09:17:50.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57783 #4081930 (1504 connections now open) 2016-03-13T09:17:50.996+0000 I NETWORK [conn4081375] end connection 10.0.27.237:35924 (1503 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36197 #4081931 (1504 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [conn4081315] end connection 10.0.27.237:35906 (1503 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51149 #4081932 (1505 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51150 #4081933 (1505 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [conn4081744] end connection 10.0.179.209:40779 (1504 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [conn4080803] end connection 10.0.7.113:59422 (1503 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [conn4081097] end connection 10.0.20.53:50750 (1502 connections now open) 2016-03-13T09:17:50.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59790 #4081934 (1504 connections now open) 2016-03-13T09:17:50.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40876 #4081935 (1504 connections now open) 2016-03-13T09:17:50.998+0000 I NETWORK [conn4077488] end connection 10.0.27.237:60671 (1502 connections now open) 2016-03-13T09:17:50.998+0000 I NETWORK [conn4081323] end connection 10.0.26.69:42583 (1502 connections now open) 2016-03-13T09:17:50.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59792 #4081936 (1503 connections now open) 2016-03-13T09:17:50.998+0000 I NETWORK [conn4081023] end connection 10.0.20.53:50721 (1501 connections now open) 2016-03-13T09:17:50.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36198 #4081937 (1503 connections now open) 2016-03-13T09:17:50.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42900 #4081938 (1504 connections now open) 2016-03-13T09:17:50.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51152 #4081939 (1505 connections now open) 2016-03-13T09:17:50.999+0000 I NETWORK [conn4081764] end connection 10.0.20.53:51079 (1504 connections now open) 2016-03-13T09:17:50.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57789 #4081940 (1505 connections now open) 2016-03-13T09:17:50.999+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.ActionReports: 5ms sequenceNumber: 16901 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:17:51.000+0000 I NETWORK [conn4081804] end connection 10.0.216.90:57726 (1504 connections now open) 2016-03-13T09:17:51.001+0000 I NETWORK [conn4080427] end connection 10.0.20.53:50383 (1503 connections now open) 2016-03-13T09:17:51.001+0000 I NETWORK [conn4080067] end connection 10.0.7.113:58719 (1502 connections now open) 2016-03-13T09:17:51.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51153 #4081941 (1503 connections now open) 2016-03-13T09:17:51.001+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51154 #4081942 (1504 connections now open) 2016-03-13T09:17:51.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59795 #4081943 (1505 connections now open) 2016-03-13T09:17:51.002+0000 I NETWORK [conn4080005] end connection 10.0.216.90:56689 (1504 connections now open) 2016-03-13T09:17:51.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57793 #4081944 (1505 connections now open) 2016-03-13T09:17:51.003+0000 I NETWORK [conn4081831] end connection 10.0.179.209:40815 (1504 connections now open) 2016-03-13T09:17:51.003+0000 I NETWORK [conn4081830] end connection 10.0.179.209:40814 (1503 connections now open) 2016-03-13T09:17:51.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40880 #4081945 (1504 connections now open) 2016-03-13T09:17:51.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40881 #4081946 (1505 connections now open) 2016-03-13T09:17:51.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51156 #4081947 (1506 connections now open) 2016-03-13T09:17:51.007+0000 I NETWORK [conn4081199] end connection 10.0.27.237:35870 (1505 connections now open) 2016-03-13T09:17:51.007+0000 I NETWORK [conn4081218] end connection 10.0.27.237:35876 (1504 connections now open) 2016-03-13T09:17:51.007+0000 I NETWORK [conn4081758] end connection 10.0.26.69:42827 (1503 connections now open) 2016-03-13T09:17:51.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42904 #4081948 (1504 connections now open) 2016-03-13T09:17:51.007+0000 I NETWORK [conn4081553] end connection 10.0.26.69:42706 (1503 connections now open) 2016-03-13T09:17:51.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42905 #4081949 (1504 connections now open) 2016-03-13T09:17:51.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36199 #4081950 (1505 connections now open) 2016-03-13T09:17:51.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36200 #4081951 (1506 connections now open) 2016-03-13T09:17:51.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36201 #4081952 (1507 connections now open) 2016-03-13T09:17:51.009+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.AppReports: 9ms sequenceNumber: 16902 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:17:51.012+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 16903 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:17:51.013+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 16904 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:17:51.013+0000 I NETWORK [conn4080974] end connection 10.0.27.237:35775 (1506 connections now open) 2016-03-13T09:17:51.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36202 #4081953 (1507 connections now open) 2016-03-13T09:17:51.056+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.AppVersionReports: 43ms sequenceNumber: 16905 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:17:51.070+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 13ms sequenceNumber: 16906 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:17:51.076+0000 I NETWORK [conn4080980] end connection 10.0.7.113:59539 (1506 connections now open) 2016-03-13T09:17:51.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59797 #4081954 (1507 connections now open) 2016-03-13T09:17:51.086+0000 I NETWORK [conn4081438] end connection 10.0.20.53:50895 (1506 connections now open) 2016-03-13T09:17:51.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51159 #4081955 (1507 connections now open) 2016-03-13T09:17:51.097+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.Clients: 27ms sequenceNumber: 16907 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:17:51.100+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 16908 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:17:51.137+0000 I NETWORK [conn4080781] end connection 10.0.179.209:40482 (1506 connections now open) 2016-03-13T09:17:51.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40883 #4081956 (1507 connections now open) 2016-03-13T09:17:51.138+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.CountryReports: 37ms sequenceNumber: 16909 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:17:51.141+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 3ms sequenceNumber: 16910 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:17:51.150+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.CrashedSessions: 8ms sequenceNumber: 16911 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:17:51.197+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 47ms sequenceNumber: 16912 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:17:51.207+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.CustomEventReports: 6ms sequenceNumber: 16913 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:17:51.208+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 16914 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:17:51.214+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 16915 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:17:51.228+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.DeviceReports: 13ms sequenceNumber: 16916 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:17:51.229+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 16917 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:17:51.230+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 16918 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:17:51.243+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.GestureReports: 13ms sequenceNumber: 16919 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:17:51.313+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 69ms sequenceNumber: 16920 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:17:51.595+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.HeatmapReports: 279ms sequenceNumber: 16921 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:17:51.665+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 57ms sequenceNumber: 16922 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:17:51.677+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.NavigationReports: 7ms sequenceNumber: 16923 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:17:51.679+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 16924 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:17:51.683+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 16925 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:17:51.684+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.ResolutionReports: 1ms sequenceNumber: 16926 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:17:51.685+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 1ms sequenceNumber: 16927 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:17:51.688+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 16928 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:17:51.689+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 16929 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:17:51.690+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 16930 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:17:51.697+0000 I NETWORK [conn4081832] end connection 10.0.26.69:42854 (1506 connections now open) 2016-03-13T09:17:51.698+0000 I NETWORK [conn4081833] end connection 10.0.26.69:42855 (1505 connections now open) 2016-03-13T09:17:51.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42909 #4081957 (1506 connections now open) 2016-03-13T09:17:51.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42910 #4081958 (1507 connections now open) 2016-03-13T09:17:52.354+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 663ms sequenceNumber: 16931 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:17:52.714+0000 I NETWORK [conn4081478] end connection 10.0.26.69:42666 (1506 connections now open) 2016-03-13T09:17:52.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42911 #4081959 (1507 connections now open) 2016-03-13T09:17:52.999+0000 I NETWORK [conn4081324] end connection 10.0.27.237:35909 (1506 connections now open) 2016-03-13T09:17:52.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36207 #4081960 (1507 connections now open) 2016-03-13T09:17:53.319+0000 I NETWORK [conn4080716] end connection 10.0.7.113:59373 (1506 connections now open) 2016-03-13T09:17:53.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59800 #4081961 (1507 connections now open) 2016-03-13T09:17:54.468+0000 I NETWORK [conn4081849] end connection 10.0.27.237:36155 (1506 connections now open) 2016-03-13T09:17:54.468+0000 I NETWORK [conn4081848] end connection 10.0.27.237:36156 (1505 connections now open) 2016-03-13T09:17:54.468+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36210 #4081962 (1506 connections now open) 2016-03-13T09:17:54.468+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36211 #4081963 (1507 connections now open) 2016-03-13T09:17:56.054+0000 I NETWORK [conn4080278] end connection 10.0.20.53:50216 (1506 connections now open) 2016-03-13T09:17:56.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51163 #4081964 (1507 connections now open) 2016-03-13T09:17:56.129+0000 I NETWORK [conn4080770] end connection 10.0.216.90:57421 (1506 connections now open) 2016-03-13T09:17:56.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57798 #4081965 (1507 connections now open) 2016-03-13T09:17:56.597+0000 I NETWORK [conn4081854] end connection 10.0.7.113:59766 (1506 connections now open) 2016-03-13T09:17:56.597+0000 I NETWORK [conn4081853] end connection 10.0.7.113:59767 (1506 connections now open) 2016-03-13T09:17:56.597+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59804 #4081966 (1506 connections now open) 2016-03-13T09:17:56.597+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59805 #4081967 (1507 connections now open) 2016-03-13T09:17:56.763+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.Sessions: 4320ms sequenceNumber: 16932 version: 30650|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:17:57.015+0000 I NETWORK [conn4080728] end connection 10.0.179.209:40453 (1506 connections now open) 2016-03-13T09:17:57.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40887 #4081968 (1507 connections now open) 2016-03-13T09:17:57.080+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.ShardTest: 0ms sequenceNumber: 16933 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:17:57.081+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 16934 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:17:57.082+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 16935 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:17:57.098+0000 I SHARDING [conn3454527] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 16ms sequenceNumber: 16936 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:17:57.102+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-1-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:57.102+0000 I NETWORK [conn4081169] end connection 10.0.27.237:35859 (1506 connections now open) 2016-03-13T09:17:57.103+0000 I NETWORK [conn4081263] end connection 10.0.7.113:59629 (1505 connections now open) 2016-03-13T09:17:57.103+0000 I NETWORK [conn4081883] end connection 10.0.216.90:57765 (1504 connections now open) 2016-03-13T09:17:57.103+0000 I NETWORK [conn4076782] end connection 10.0.216.90:53345 (1503 connections now open) 2016-03-13T09:17:57.103+0000 I NETWORK [conn4081184] end connection 10.0.20.53:50784 (1502 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [conn4079455] end connection 10.0.216.90:56141 (1501 connections now open) 2016-03-13T09:17:57.104+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-1-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:57.104+0000 I NETWORK [conn4080796] end connection 10.0.179.209:40489 (1500 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36229 #4081969 (1501 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [conn4080967] end connection 10.0.26.69:42450 (1500 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59806 #4081970 (1502 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57801 #4081971 (1502 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57802 #4081972 (1503 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57803 #4081973 (1504 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51166 #4081974 (1505 connections now open) 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40891 #4081975 (1506 connections now open) 2016-03-13T09:17:57.104+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-0-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:57.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42918 #4081976 (1507 connections now open) 2016-03-13T09:17:57.105+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-1-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:17:57.105+0000 I NETWORK [conn4081172] end connection 10.0.20.53:50781 (1506 connections now open) 2016-03-13T09:17:57.105+0000 I NETWORK [conn4080774] end connection 10.0.179.209:40478 (1506 connections now open) 2016-03-13T09:17:57.105+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 16937 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:17:57.105+0000 I NETWORK [conn4081892] end connection 10.0.216.90:57768 (1504 connections now open) 2016-03-13T09:17:57.105+0000 I NETWORK [conn4079751] end connection 10.0.7.113:58412 (1503 connections now open) 2016-03-13T09:17:57.106+0000 I NETWORK [conn4079500] end connection 10.0.7.113:58159 (1502 connections now open) 2016-03-13T09:17:57.106+0000 I NETWORK [conn4079094] end connection 10.0.179.209:38806 (1501 connections now open) 2016-03-13T09:17:57.106+0000 I NETWORK [conn4081310] end connection 10.0.27.237:35904 (1500 connections now open) 2016-03-13T09:17:57.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57805 #4081977 (1501 connections now open) 2016-03-13T09:17:57.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59809 #4081978 (1502 connections now open) 2016-03-13T09:17:57.106+0000 I NETWORK [conn4081899] end connection 10.0.216.90:57771 (1501 connections now open) 2016-03-13T09:17:57.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40893 #4081979 (1502 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [conn4081948] end connection 10.0.26.69:42904 (1501 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51169 #4081980 (1502 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36231 #4081981 (1503 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59810 #4081982 (1504 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42920 #4081983 (1505 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [conn4080982] end connection 10.0.27.237:35778 (1503 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [conn4079710] end connection 10.0.7.113:58368 (1503 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57806 #4081984 (1504 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [conn4081397] end connection 10.0.26.69:42621 (1503 connections now open) 2016-03-13T09:17:57.107+0000 I NETWORK [conn4081550] end connection 10.0.27.237:36012 (1503 connections now open) 2016-03-13T09:17:57.108+0000 I NETWORK [conn4081877] end connection 10.0.20.53:51129 (1501 connections now open) 2016-03-13T09:17:57.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36233 #4081985 (1503 connections now open) 2016-03-13T09:17:57.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59811 #4081986 (1504 connections now open) 2016-03-13T09:17:57.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36234 #4081987 (1504 connections now open) 2016-03-13T09:17:57.109+0000 I NETWORK [conn4080684] end connection 10.0.20.53:50667 (1503 connections now open) 2016-03-13T09:17:57.109+0000 I NETWORK [conn4080802] end connection 10.0.179.209:40495 (1502 connections now open) 2016-03-13T09:17:57.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42922 #4081988 (1503 connections now open) 2016-03-13T09:17:57.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40894 #4081989 (1504 connections now open) 2016-03-13T09:17:57.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51172 #4081990 (1505 connections now open) 2016-03-13T09:17:57.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40895 #4081991 (1506 connections now open) 2016-03-13T09:17:57.110+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.ActionReports: 4ms sequenceNumber: 16938 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:17:57.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51173 #4081992 (1507 connections now open) 2016-03-13T09:17:57.112+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 16939 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:17:57.112+0000 I NETWORK [conn4081206] end connection 10.0.20.53:50791 (1506 connections now open) 2016-03-13T09:17:57.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51175 #4081993 (1507 connections now open) 2016-03-13T09:17:57.114+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 16940 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:17:57.115+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 16941 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:17:57.115+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 16942 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:17:57.118+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 2ms sequenceNumber: 16943 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:17:57.120+0000 I NETWORK [conn4079856] end connection 10.0.27.237:34844 (1506 connections now open) 2016-03-13T09:17:57.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36236 #4081994 (1507 connections now open) 2016-03-13T09:17:57.142+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.Clients: 23ms sequenceNumber: 16944 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:17:57.145+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 16945 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:17:57.147+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 16946 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:17:57.148+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 16947 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:17:57.149+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 16948 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:17:57.163+0000 I NETWORK [conn4080567] end connection 10.0.216.90:57255 (1506 connections now open) 2016-03-13T09:17:57.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57807 #4081995 (1507 connections now open) 2016-03-13T09:17:57.190+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 40ms sequenceNumber: 16949 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:17:57.195+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.CustomEventReports: 1ms sequenceNumber: 16950 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:17:57.196+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 16951 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:17:57.201+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 16952 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:17:57.206+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 16953 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:17:57.207+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 16954 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:17:57.207+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 16955 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:17:57.212+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 16956 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:17:57.246+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 34ms sequenceNumber: 16957 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:17:57.415+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.HeatmapReports: 165ms sequenceNumber: 16958 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:17:57.468+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 41ms sequenceNumber: 16959 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:17:57.479+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.NavigationReports: 7ms sequenceNumber: 16960 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:17:57.481+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 16961 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:17:57.485+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 16962 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:17:57.486+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 16963 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:17:57.487+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 0ms sequenceNumber: 16964 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:17:57.490+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 16965 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:17:57.491+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 16966 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:17:57.492+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 16967 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:17:57.639+0000 I NETWORK [conn4081856] end connection 10.0.216.90:57756 (1506 connections now open) 2016-03-13T09:17:57.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57808 #4081996 (1507 connections now open) 2016-03-13T09:17:57.640+0000 I NETWORK [conn4081857] end connection 10.0.216.90:57755 (1506 connections now open) 2016-03-13T09:17:57.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57809 #4081997 (1507 connections now open) 2016-03-13T09:17:57.672+0000 I NETWORK [conn4080997] end connection 10.0.7.113:59546 (1506 connections now open) 2016-03-13T09:17:57.672+0000 I NETWORK [conn4077056] end connection 10.0.179.209:36748 (1505 connections now open) 2016-03-13T09:17:57.673+0000 I NETWORK [conn4081126] end connection 10.0.26.69:42507 (1504 connections now open) 2016-03-13T09:17:57.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40897 #4081998 (1505 connections now open) 2016-03-13T09:17:57.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42926 #4081999 (1506 connections now open) 2016-03-13T09:17:57.674+0000 I NETWORK [conn4081952] end connection 10.0.27.237:36201 (1505 connections now open) 2016-03-13T09:17:57.674+0000 I NETWORK [conn4081069] end connection 10.0.216.90:57526 (1504 connections now open) 2016-03-13T09:17:57.674+0000 I NETWORK [conn4078359] end connection 10.0.216.90:55024 (1504 connections now open) 2016-03-13T09:17:57.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36237 #4082000 (1504 connections now open) 2016-03-13T09:17:57.675+0000 I NETWORK [conn4080906] end connection 10.0.179.209:40581 (1503 connections now open) 2016-03-13T09:17:57.675+0000 I NETWORK [conn4080235] end connection 10.0.7.113:58891 (1502 connections now open) 2016-03-13T09:17:57.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57812 #4082001 (1503 connections now open) 2016-03-13T09:17:57.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40899 #4082002 (1504 connections now open) 2016-03-13T09:17:57.676+0000 I NETWORK [conn4080946] end connection 10.0.216.90:57504 (1503 connections now open) 2016-03-13T09:17:57.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59812 #4082003 (1504 connections now open) 2016-03-13T09:17:57.676+0000 I NETWORK [conn4080280] end connection 10.0.216.90:56954 (1503 connections now open) 2016-03-13T09:17:57.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57814 #4082004 (1504 connections now open) 2016-03-13T09:17:57.676+0000 I NETWORK [conn4080863] end connection 10.0.179.209:40549 (1503 connections now open) 2016-03-13T09:17:57.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57815 #4082005 (1504 connections now open) 2016-03-13T09:17:57.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57816 #4082006 (1505 connections now open) 2016-03-13T09:17:57.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40900 #4082007 (1506 connections now open) 2016-03-13T09:17:57.677+0000 I NETWORK [conn4081155] end connection 10.0.27.237:35855 (1505 connections now open) 2016-03-13T09:17:57.677+0000 I NETWORK [conn4081572] end connection 10.0.27.237:36022 (1505 connections now open) 2016-03-13T09:17:57.678+0000 I NETWORK [conn4080805] end connection 10.0.179.209:40497 (1503 connections now open) 2016-03-13T09:17:57.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36239 #4082008 (1504 connections now open) 2016-03-13T09:17:57.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59813 #4082009 (1505 connections now open) 2016-03-13T09:17:57.678+0000 I NETWORK [conn4081358] end connection 10.0.26.69:42600 (1504 connections now open) 2016-03-13T09:17:57.678+0000 I NETWORK [conn4080715] end connection 10.0.179.209:40445 (1503 connections now open) 2016-03-13T09:17:57.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40901 #4082010 (1504 connections now open) 2016-03-13T09:17:57.678+0000 I NETWORK [conn4081376] end connection 10.0.27.237:35925 (1503 connections now open) 2016-03-13T09:17:57.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36240 #4082011 (1504 connections now open) 2016-03-13T09:17:57.679+0000 I NETWORK [conn4079498] end connection 10.0.216.90:56183 (1503 connections now open) 2016-03-13T09:17:57.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42929 #4082012 (1504 connections now open) 2016-03-13T09:17:57.679+0000 I NETWORK [conn4081260] end connection 10.0.20.53:50808 (1502 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [conn4081066] end connection 10.0.179.209:40662 (1502 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40902 #4082013 (1504 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57818 #4082014 (1504 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [conn4080260] end connection 10.0.26.69:41971 (1502 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51184 #4082015 (1504 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40903 #4082016 (1505 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36241 #4082017 (1506 connections now open) 2016-03-13T09:17:57.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42931 #4082018 (1507 connections now open) 2016-03-13T09:17:57.681+0000 I NETWORK [conn4080944] end connection 10.0.179.209:40600 (1506 connections now open) 2016-03-13T09:17:57.681+0000 I NETWORK [conn4080772] end connection 10.0.216.90:57422 (1505 connections now open) 2016-03-13T09:17:57.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57819 #4082019 (1506 connections now open) 2016-03-13T09:17:57.682+0000 I NETWORK [conn4081361] end connection 10.0.26.69:42602 (1505 connections now open) 2016-03-13T09:17:57.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40904 #4082020 (1506 connections now open) 2016-03-13T09:17:57.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42932 #4082021 (1507 connections now open) 2016-03-13T09:17:57.683+0000 I NETWORK [conn4081402] end connection 10.0.26.69:42624 (1506 connections now open) 2016-03-13T09:17:57.684+0000 I NETWORK [conn4081118] end connection 10.0.216.90:57542 (1505 connections now open) 2016-03-13T09:17:57.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57821 #4082022 (1506 connections now open) 2016-03-13T09:17:57.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42933 #4082023 (1507 connections now open) 2016-03-13T09:17:57.685+0000 I NETWORK [conn4080827] end connection 10.0.179.209:40517 (1506 connections now open) 2016-03-13T09:17:57.686+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40906 #4082024 (1507 connections now open) 2016-03-13T09:17:57.690+0000 I NETWORK [conn4081166] end connection 10.0.20.53:50778 (1506 connections now open) 2016-03-13T09:17:57.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51189 #4082025 (1507 connections now open) 2016-03-13T09:17:57.694+0000 I NETWORK [conn4081248] end connection 10.0.26.69:42551 (1506 connections now open) 2016-03-13T09:17:57.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42936 #4082026 (1507 connections now open) 2016-03-13T09:17:58.089+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 597ms sequenceNumber: 16968 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:17:58.113+0000 I NETWORK [conn4081089] end connection 10.0.179.209:40671 (1506 connections now open) 2016-03-13T09:17:58.113+0000 I NETWORK [conn4081139] end connection 10.0.216.90:57550 (1506 connections now open) 2016-03-13T09:17:58.113+0000 I NETWORK [conn4080916] end connection 10.0.26.69:42436 (1504 connections now open) 2016-03-13T09:17:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40910 #4082027 (1506 connections now open) 2016-03-13T09:17:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57825 #4082028 (1506 connections now open) 2016-03-13T09:17:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42938 #4082029 (1507 connections now open) 2016-03-13T09:17:58.674+0000 I NETWORK [conn4081860] end connection 10.0.20.53:51121 (1506 connections now open) 2016-03-13T09:17:58.674+0000 I NETWORK [conn4081861] end connection 10.0.20.53:51122 (1506 connections now open) 2016-03-13T09:17:58.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51193 #4082030 (1506 connections now open) 2016-03-13T09:17:58.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51192 #4082031 (1507 connections now open) 2016-03-13T09:18:00.485+0000 I NETWORK [conn4081068] end connection 10.0.26.69:42485 (1506 connections now open) 2016-03-13T09:18:00.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42948 #4082032 (1507 connections now open) 2016-03-13T09:18:01.015+0000 I NETWORK [conn4081945] end connection 10.0.179.209:40880 (1506 connections now open) 2016-03-13T09:18:01.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40913 #4082033 (1507 connections now open) 2016-03-13T09:18:01.015+0000 I NETWORK [conn4081946] end connection 10.0.179.209:40881 (1505 connections now open) 2016-03-13T09:18:01.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40914 #4082034 (1507 connections now open) 2016-03-13T09:18:01.711+0000 I NETWORK [conn4081957] end connection 10.0.26.69:42909 (1506 connections now open) 2016-03-13T09:18:01.711+0000 I NETWORK [conn4081958] end connection 10.0.26.69:42910 (1506 connections now open) 2016-03-13T09:18:01.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42950 #4082035 (1506 connections now open) 2016-03-13T09:18:01.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42951 #4082036 (1507 connections now open) 2016-03-13T09:18:02.172+0000 I NETWORK [conn4081384] end connection 10.0.26.69:42613 (1506 connections now open) 2016-03-13T09:18:02.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42955 #4082037 (1507 connections now open) 2016-03-13T09:18:02.204+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.Sessions: 4025ms sequenceNumber: 16969 version: 30650|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:18:02.540+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 16970 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:18:02.540+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 16971 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:18:02.550+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 9ms sequenceNumber: 16972 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:18:02.566+0000 I SHARDING [conn3687672] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 16ms sequenceNumber: 16973 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:18:02.568+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30650|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56346357584ad10f1f9aaa8c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634669d55418c0ec3f9ca7d') }) shmydomain1 -> shmydomain2 2016-03-13T09:18:02.571+0000 I NETWORK [conn4080907] end connection 10.0.7.113:59505 (1506 connections now open) 2016-03-13T09:18:02.571+0000 I NETWORK [conn4080752] end connection 10.0.7.113:59394 (1505 connections now open) 2016-03-13T09:18:02.571+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-0-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:02.571+0000 I NETWORK [conn4080955] end connection 10.0.27.237:35769 (1504 connections now open) 2016-03-13T09:18:02.571+0000 I NETWORK [conn4078574] end connection 10.0.216.90:55252 (1503 connections now open) 2016-03-13T09:18:02.571+0000 I NETWORK [conn4081065] end connection 10.0.179.209:40661 (1502 connections now open) 2016-03-13T09:18:02.572+0000 I NETWORK [conn4080767] end connection 10.0.216.90:57420 (1501 connections now open) 2016-03-13T09:18:02.572+0000 I NETWORK [conn4081841] end connection 10.0.26.69:42860 (1500 connections now open) 2016-03-13T09:18:02.572+0000 I NETWORK [conn4081624] end connection 10.0.27.237:36055 (1499 connections now open) 2016-03-13T09:18:02.572+0000 I NETWORK [conn4078782] end connection 10.0.7.113:57425 (1498 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59831 #4082038 (1499 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57831 #4082039 (1500 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40919 #4082040 (1501 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36251 #4082041 (1502 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59832 #4082042 (1503 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [conn4080878] end connection 10.0.7.113:59486 (1502 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36252 #4082043 (1504 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59834 #4082044 (1504 connections now open) 2016-03-13T09:18:02.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59835 #4082045 (1505 connections now open) 2016-03-13T09:18:02.574+0000 I NETWORK [conn4081791] end connection 10.0.179.209:40801 (1504 connections now open) 2016-03-13T09:18:02.574+0000 I NETWORK [conn4081106] end connection 10.0.26.69:42498 (1503 connections now open) 2016-03-13T09:18:02.574+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-0-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:02.575+0000 I NETWORK [conn4079795] end connection 10.0.7.113:58451 (1502 connections now open) 2016-03-13T09:18:02.575+0000 I NETWORK [conn4081995] end connection 10.0.216.90:57807 (1501 connections now open) 2016-03-13T09:18:02.575+0000 I NETWORK [conn4079261] end connection 10.0.7.113:57917 (1501 connections now open) 2016-03-13T09:18:02.575+0000 I NETWORK [conn4081051] end connection 10.0.20.53:50733 (1501 connections now open) 2016-03-13T09:18:02.576+0000 I NETWORK [conn4080707] end connection 10.0.7.113:59367 (1498 connections now open) 2016-03-13T09:18:02.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57832 #4082046 (1499 connections now open) 2016-03-13T09:18:02.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40921 #4082047 (1500 connections now open) 2016-03-13T09:18:02.576+0000 I NETWORK [conn4081020] end connection 10.0.27.237:35798 (1498 connections now open) 2016-03-13T09:18:02.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59836 #4082048 (1500 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59837 #4082049 (1501 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [conn4081977] end connection 10.0.216.90:57805 (1500 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42958 #4082050 (1501 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [conn4081431] end connection 10.0.27.237:35952 (1499 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59838 #4082051 (1501 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57836 #4082052 (1502 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [conn4078209] end connection 10.0.179.209:37894 (1500 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [conn4081195] end connection 10.0.27.237:35869 (1500 connections now open) 2016-03-13T09:18:02.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36254 #4082053 (1502 connections now open) 2016-03-13T09:18:02.578+0000 I NETWORK [conn4080609] end connection 10.0.20.53:50588 (1500 connections now open) 2016-03-13T09:18:02.578+0000 I NETWORK [conn4080429] end connection 10.0.216.90:57110 (1499 connections now open) 2016-03-13T09:18:02.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40925 #4082054 (1501 connections now open) 2016-03-13T09:18:02.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51202 #4082055 (1501 connections now open) 2016-03-13T09:18:02.578+0000 I NETWORK [conn4081074] end connection 10.0.26.69:42487 (1500 connections now open) 2016-03-13T09:18:02.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57837 #4082056 (1501 connections now open) 2016-03-13T09:18:02.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51203 #4082057 (1502 connections now open) 2016-03-13T09:18:02.579+0000 I NETWORK [conn4081265] end connection 10.0.26.69:42558 (1501 connections now open) 2016-03-13T09:18:02.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42959 #4082058 (1502 connections now open) 2016-03-13T09:18:02.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42960 #4082059 (1503 connections now open) 2016-03-13T09:18:02.580+0000 I NETWORK [conn4081307] end connection 10.0.20.53:50824 (1502 connections now open) 2016-03-13T09:18:02.580+0000 I NETWORK [conn4081797] end connection 10.0.20.53:51092 (1501 connections now open) 2016-03-13T09:18:02.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51205 #4082060 (1502 connections now open) 2016-03-13T09:18:02.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51206 #4082061 (1503 connections now open) 2016-03-13T09:18:02.581+0000 I NETWORK [conn4080969] end connection 10.0.179.209:40614 (1502 connections now open) 2016-03-13T09:18:02.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42961 #4082062 (1503 connections now open) 2016-03-13T09:18:02.581+0000 I NETWORK [conn4080799] end connection 10.0.216.90:57438 (1502 connections now open) 2016-03-13T09:18:02.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40926 #4082063 (1503 connections now open) 2016-03-13T09:18:02.582+0000 I NETWORK [conn4081303] end connection 10.0.26.69:42575 (1502 connections now open) 2016-03-13T09:18:02.582+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:02.582+0000 I NETWORK [conn4081984] end connection 10.0.216.90:57806 (1501 connections now open) 2016-03-13T09:18:02.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57840 #4082064 (1502 connections now open) 2016-03-13T09:18:02.582+0000 I NETWORK [conn4081357] end connection 10.0.20.53:50848 (1500 connections now open) 2016-03-13T09:18:02.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57841 #4082065 (1503 connections now open) 2016-03-13T09:18:02.583+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42965 #4082066 (1503 connections now open) 2016-03-13T09:18:02.584+0000 I NETWORK [conn4081719] end connection 10.0.26.69:42798 (1502 connections now open) 2016-03-13T09:18:02.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42966 #4082067 (1503 connections now open) 2016-03-13T09:18:02.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51207 #4082068 (1504 connections now open) 2016-03-13T09:18:02.585+0000 I NETWORK [conn4080747] end connection 10.0.216.90:57408 (1503 connections now open) 2016-03-13T09:18:02.585+0000 I NETWORK [conn4081092] end connection 10.0.179.209:40673 (1502 connections now open) 2016-03-13T09:18:02.585+0000 I NETWORK [conn4081326] end connection 10.0.20.53:50833 (1501 connections now open) 2016-03-13T09:18:02.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57842 #4082069 (1502 connections now open) 2016-03-13T09:18:02.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51208 #4082070 (1503 connections now open) 2016-03-13T09:18:02.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40927 #4082071 (1504 connections now open) 2016-03-13T09:18:02.586+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:02.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57843 #4082072 (1505 connections now open) 2016-03-13T09:18:02.587+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:02.588+0000 I NETWORK [conn4080828] end connection 10.0.216.90:57460 (1504 connections now open) 2016-03-13T09:18:02.589+0000 I NETWORK [conn4081213] end connection 10.0.20.53:50793 (1503 connections now open) 2016-03-13T09:18:02.589+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51209 #4082073 (1504 connections now open) 2016-03-13T09:18:02.590+0000 I NETWORK [conn4081563] end connection 10.0.216.90:57661 (1503 connections now open) 2016-03-13T09:18:02.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57846 #4082074 (1504 connections now open) 2016-03-13T09:18:02.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57847 #4082075 (1505 connections now open) 2016-03-13T09:18:02.594+0000 I NETWORK [conn4081111] end connection 10.0.20.53:50756 (1504 connections now open) 2016-03-13T09:18:02.594+0000 I NETWORK [conn4081971] end connection 10.0.216.90:57801 (1503 connections now open) 2016-03-13T09:18:02.594+0000 I NETWORK [conn4078823] end connection 10.0.7.113:57469 (1502 connections now open) 2016-03-13T09:18:02.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51211 #4082076 (1503 connections now open) 2016-03-13T09:18:02.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57848 #4082077 (1504 connections now open) 2016-03-13T09:18:02.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59843 #4082078 (1505 connections now open) 2016-03-13T09:18:02.597+0000 I NETWORK [conn4081122] end connection 10.0.20.53:50762 (1504 connections now open) 2016-03-13T09:18:02.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51213 #4082079 (1505 connections now open) 2016-03-13T09:18:02.599+0000 I NETWORK [conn4080508] end connection 10.0.216.90:57196 (1504 connections now open) 2016-03-13T09:18:02.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57849 #4082080 (1505 connections now open) 2016-03-13T09:18:02.602+0000 I NETWORK [conn4081135] end connection 10.0.20.53:50766 (1504 connections now open) 2016-03-13T09:18:02.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51214 #4082081 (1505 connections now open) 2016-03-13T09:18:02.603+0000 I NETWORK [conn4080788] end connection 10.0.216.90:57431 (1504 connections now open) 2016-03-13T09:18:02.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57853 #4082082 (1505 connections now open) 2016-03-13T09:18:02.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36257 #4082083 (1506 connections now open) 2016-03-13T09:18:02.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36259 #4082084 (1507 connections now open) 2016-03-13T09:18:02.615+0000 I NETWORK [conn4080999] end connection 10.0.27.237:35786 (1506 connections now open) 2016-03-13T09:18:02.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36260 #4082085 (1507 connections now open) 2016-03-13T09:18:02.617+0000 I NETWORK [conn4081095] end connection 10.0.27.237:35830 (1506 connections now open) 2016-03-13T09:18:02.618+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36262 #4082086 (1507 connections now open) 2016-03-13T09:18:02.620+0000 I NETWORK [conn4081266] end connection 10.0.27.237:35894 (1506 connections now open) 2016-03-13T09:18:02.623+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36264 #4082087 (1507 connections now open) 2016-03-13T09:18:02.629+0000 I NETWORK [conn4080977] end connection 10.0.7.113:59538 (1506 connections now open) 2016-03-13T09:18:02.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59846 #4082088 (1507 connections now open) 2016-03-13T09:18:02.646+0000 I NETWORK [conn4080810] end connection 10.0.179.209:40502 (1506 connections now open) 2016-03-13T09:18:02.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40934 #4082089 (1507 connections now open) 2016-03-13T09:18:02.685+0000 I NETWORK [conn4081091] end connection 10.0.179.209:40672 (1506 connections now open) 2016-03-13T09:18:02.686+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40935 #4082090 (1507 connections now open) 2016-03-13T09:18:02.688+0000 I NETWORK [conn4080901] end connection 10.0.179.209:40578 (1506 connections now open) 2016-03-13T09:18:02.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40937 #4082091 (1507 connections now open) 2016-03-13T09:18:02.690+0000 I NETWORK [conn4080771] end connection 10.0.179.209:40476 (1506 connections now open) 2016-03-13T09:18:02.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40938 #4082092 (1507 connections now open) 2016-03-13T09:18:02.692+0000 I NETWORK [conn4080814] end connection 10.0.179.209:40507 (1506 connections now open) 2016-03-13T09:18:02.692+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40939 #4082093 (1507 connections now open) 2016-03-13T09:18:03.647+0000 I NETWORK [conn4081198] end connection 10.0.7.113:59611 (1506 connections now open) 2016-03-13T09:18:03.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59851 #4082094 (1507 connections now open) 2016-03-13T09:18:03.648+0000 I NETWORK [conn4082006] end connection 10.0.216.90:57816 (1506 connections now open) 2016-03-13T09:18:03.649+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57857 #4082095 (1507 connections now open) 2016-03-13T09:18:03.649+0000 I NETWORK [conn4081590] end connection 10.0.27.237:36030 (1506 connections now open) 2016-03-13T09:18:03.649+0000 I NETWORK [conn4082022] end connection 10.0.216.90:57821 (1505 connections now open) 2016-03-13T09:18:03.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36271 #4082096 (1506 connections now open) 2016-03-13T09:18:03.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57858 #4082097 (1507 connections now open) 2016-03-13T09:18:03.652+0000 I NETWORK [conn4079980] end connection 10.0.179.209:39693 (1506 connections now open) 2016-03-13T09:18:03.652+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40941 #4082098 (1507 connections now open) 2016-03-13T09:18:03.653+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 16974 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:18:03.655+0000 I NETWORK [conn4081594] end connection 10.0.27.237:36034 (1506 connections now open) 2016-03-13T09:18:03.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36272 #4082099 (1507 connections now open) 2016-03-13T09:18:03.658+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.ActionReports: 5ms sequenceNumber: 16975 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:18:03.659+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 16976 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:18:03.662+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 16977 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:18:03.663+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 16978 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:18:03.664+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 16979 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:18:03.667+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 3ms sequenceNumber: 16980 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:18:03.672+0000 I NETWORK [conn4081391] end connection 10.0.27.237:35935 (1506 connections now open) 2016-03-13T09:18:03.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36273 #4082100 (1507 connections now open) 2016-03-13T09:18:03.696+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.Clients: 28ms sequenceNumber: 16981 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:18:03.699+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 1ms sequenceNumber: 16982 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:18:03.700+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 16983 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:18:03.702+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 1ms sequenceNumber: 16984 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:18:03.703+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.CrashedSessions: 1ms sequenceNumber: 16985 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:18:03.742+0000 I NETWORK [conn4081410] end connection 10.0.26.69:42627 (1506 connections now open) 2016-03-13T09:18:03.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42973 #4082101 (1507 connections now open) 2016-03-13T09:18:03.743+0000 I NETWORK [conn4081816] end connection 10.0.26.69:42847 (1506 connections now open) 2016-03-13T09:18:03.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42974 #4082102 (1507 connections now open) 2016-03-13T09:18:03.744+0000 I NETWORK [conn4081809] end connection 10.0.7.113:59741 (1506 connections now open) 2016-03-13T09:18:03.745+0000 I NETWORK [conn4081942] end connection 10.0.20.53:51154 (1505 connections now open) 2016-03-13T09:18:03.745+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51225 #4082103 (1506 connections now open) 2016-03-13T09:18:03.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59855 #4082104 (1507 connections now open) 2016-03-13T09:18:03.748+0000 I NETWORK [conn4081403] end connection 10.0.27.237:35940 (1506 connections now open) 2016-03-13T09:18:03.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36276 #4082105 (1507 connections now open) 2016-03-13T09:18:03.749+0000 I NETWORK [conn4079075] end connection 10.0.216.90:55766 (1506 connections now open) 2016-03-13T09:18:03.749+0000 I NETWORK [conn4081015] end connection 10.0.27.237:35794 (1505 connections now open) 2016-03-13T09:18:03.749+0000 I NETWORK [conn4081405] end connection 10.0.27.237:35941 (1504 connections now open) 2016-03-13T09:18:03.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36277 #4082106 (1505 connections now open) 2016-03-13T09:18:03.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57861 #4082107 (1506 connections now open) 2016-03-13T09:18:03.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36278 #4082108 (1507 connections now open) 2016-03-13T09:18:03.750+0000 I NETWORK [conn4082028] end connection 10.0.216.90:57825 (1506 connections now open) 2016-03-13T09:18:03.750+0000 I NETWORK [conn4081771] end connection 10.0.216.90:57709 (1505 connections now open) 2016-03-13T09:18:03.750+0000 I NETWORK [conn4081152] end connection 10.0.20.53:50774 (1504 connections now open) 2016-03-13T09:18:03.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51227 #4082109 (1505 connections now open) 2016-03-13T09:18:03.751+0000 I NETWORK [conn4081412] end connection 10.0.27.237:35944 (1504 connections now open) 2016-03-13T09:18:03.752+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36280 #4082110 (1505 connections now open) 2016-03-13T09:18:03.752+0000 I NETWORK [conn4078149] end connection 10.0.216.90:54796 (1504 connections now open) 2016-03-13T09:18:03.752+0000 I NETWORK [conn4081298] end connection 10.0.26.69:42573 (1503 connections now open) 2016-03-13T09:18:03.752+0000 I NETWORK [conn4080765] end connection 10.0.179.209:40473 (1503 connections now open) 2016-03-13T09:18:03.752+0000 I NETWORK [conn4081417] end connection 10.0.20.53:50881 (1501 connections now open) 2016-03-13T09:18:03.752+0000 I NETWORK [conn4081158] end connection 10.0.20.53:50776 (1501 connections now open) 2016-03-13T09:18:03.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57865 #4082111 (1501 connections now open) 2016-03-13T09:18:03.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51229 #4082112 (1502 connections now open) 2016-03-13T09:18:03.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51230 #4082113 (1503 connections now open) 2016-03-13T09:18:03.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57866 #4082114 (1504 connections now open) 2016-03-13T09:18:03.753+0000 I NETWORK [conn4080343] end connection 10.0.179.209:40071 (1503 connections now open) 2016-03-13T09:18:03.753+0000 I NETWORK [conn4081641] end connection 10.0.26.69:42749 (1502 connections now open) 2016-03-13T09:18:03.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57868 #4082115 (1503 connections now open) 2016-03-13T09:18:03.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40950 #4082116 (1504 connections now open) 2016-03-13T09:18:03.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42977 #4082117 (1505 connections now open) 2016-03-13T09:18:03.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40951 #4082118 (1506 connections now open) 2016-03-13T09:18:03.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42978 #4082119 (1507 connections now open) 2016-03-13T09:18:03.754+0000 I NETWORK [conn4081531] end connection 10.0.27.237:36003 (1506 connections now open) 2016-03-13T09:18:03.755+0000 I NETWORK [conn4081269] end connection 10.0.26.69:42560 (1505 connections now open) 2016-03-13T09:18:03.755+0000 I NETWORK [conn4081626] end connection 10.0.27.237:36056 (1504 connections now open) 2016-03-13T09:18:03.755+0000 I NETWORK [conn4080470] end connection 10.0.7.113:59132 (1503 connections now open) 2016-03-13T09:18:03.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42979 #4082120 (1504 connections now open) 2016-03-13T09:18:03.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36282 #4082121 (1505 connections now open) 2016-03-13T09:18:03.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59861 #4082122 (1506 connections now open) 2016-03-13T09:18:03.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36283 #4082123 (1507 connections now open) 2016-03-13T09:18:03.757+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 54ms sequenceNumber: 16986 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:18:03.763+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.CustomEventReports: 1ms sequenceNumber: 16987 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:18:03.763+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 16988 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:18:03.769+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 6ms sequenceNumber: 16989 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:18:03.773+0000 I NETWORK [conn4081396] end connection 10.0.20.53:50871 (1506 connections now open) 2016-03-13T09:18:03.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51232 #4082124 (1507 connections now open) 2016-03-13T09:18:03.774+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 16990 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:18:03.775+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.DurationReports: 1ms sequenceNumber: 16991 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:18:03.777+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.FunnelReports: 1ms sequenceNumber: 16992 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:18:03.778+0000 I NETWORK [conn4081889] end connection 10.0.7.113:59782 (1506 connections now open) 2016-03-13T09:18:03.778+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59862 #4082125 (1507 connections now open) 2016-03-13T09:18:03.781+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 16993 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:18:03.784+0000 I NETWORK [conn4081140] end connection 10.0.27.237:35849 (1506 connections now open) 2016-03-13T09:18:03.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36285 #4082126 (1507 connections now open) 2016-03-13T09:18:03.793+0000 I NETWORK [conn4081383] end connection 10.0.26.69:42612 (1506 connections now open) 2016-03-13T09:18:03.793+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42980 #4082127 (1507 connections now open) 2016-03-13T09:18:03.801+0000 I NETWORK [conn4081418] end connection 10.0.27.237:35948 (1506 connections now open) 2016-03-13T09:18:03.801+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36287 #4082128 (1507 connections now open) 2016-03-13T09:18:03.838+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 56ms sequenceNumber: 16994 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:18:03.858+0000 I NETWORK [conn4080875] end connection 10.0.7.113:59484 (1506 connections now open) 2016-03-13T09:18:03.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59863 #4082129 (1507 connections now open) 2016-03-13T09:18:04.044+0000 I NETWORK [conn4080738] end connection 10.0.216.90:57403 (1506 connections now open) 2016-03-13T09:18:04.044+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57872 #4082130 (1507 connections now open) 2016-03-13T09:18:04.075+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.HeatmapReports: 234ms sequenceNumber: 16995 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:18:04.142+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 55ms sequenceNumber: 16996 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:18:04.155+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.NavigationReports: 7ms sequenceNumber: 16997 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:18:04.157+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 16998 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:18:04.162+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.PendingSessions: 5ms sequenceNumber: 16999 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:18:04.163+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 17000 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:18:04.165+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 0ms sequenceNumber: 17001 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:18:04.168+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.RetentionReports: 3ms sequenceNumber: 17002 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:18:04.169+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 1ms sequenceNumber: 17003 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:18:04.170+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17004 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:18:04.480+0000 I NETWORK [conn4081962] end connection 10.0.27.237:36210 (1506 connections now open) 2016-03-13T09:18:04.480+0000 I NETWORK [conn4081963] end connection 10.0.27.237:36211 (1505 connections now open) 2016-03-13T09:18:04.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36289 #4082131 (1506 connections now open) 2016-03-13T09:18:04.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36290 #4082132 (1507 connections now open) 2016-03-13T09:18:04.947+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 777ms sequenceNumber: 17005 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:18:05.574+0000 I NETWORK [conn4080704] end connection 10.0.20.53:50680 (1506 connections now open) 2016-03-13T09:18:05.574+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51233 #4082133 (1507 connections now open) 2016-03-13T09:18:06.610+0000 I NETWORK [conn4081967] end connection 10.0.7.113:59805 (1506 connections now open) 2016-03-13T09:18:06.610+0000 I NETWORK [conn4081966] end connection 10.0.7.113:59804 (1506 connections now open) 2016-03-13T09:18:06.610+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59866 #4082134 (1506 connections now open) 2016-03-13T09:18:06.610+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59865 #4082135 (1507 connections now open) 2016-03-13T09:18:07.397+0000 I NETWORK [conn4080921] end connection 10.0.26.69:42437 (1506 connections now open) 2016-03-13T09:18:07.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42985 #4082136 (1507 connections now open) 2016-03-13T09:18:07.652+0000 I NETWORK [conn4081996] end connection 10.0.216.90:57808 (1506 connections now open) 2016-03-13T09:18:07.652+0000 I NETWORK [conn4081997] end connection 10.0.216.90:57809 (1506 connections now open) 2016-03-13T09:18:07.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57880 #4082137 (1507 connections now open) 2016-03-13T09:18:07.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57879 #4082138 (1507 connections now open) 2016-03-13T09:18:08.367+0000 I NETWORK [conn4080819] end connection 10.0.179.209:40511 (1506 connections now open) 2016-03-13T09:18:08.368+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40962 #4082139 (1507 connections now open) 2016-03-13T09:18:08.687+0000 I NETWORK [conn4082031] end connection 10.0.20.53:51192 (1506 connections now open) 2016-03-13T09:18:08.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51246 #4082140 (1507 connections now open) 2016-03-13T09:18:08.689+0000 I NETWORK [conn4082030] end connection 10.0.20.53:51193 (1506 connections now open) 2016-03-13T09:18:08.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51247 #4082141 (1507 connections now open) 2016-03-13T09:18:09.406+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.Sessions: 4369ms sequenceNumber: 17006 version: 30650|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:18:09.618+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.ShardTest: 0ms sequenceNumber: 17007 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:18:09.619+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 17008 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:18:09.620+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17009 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:18:09.637+0000 I SHARDING [conn2882870] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 16ms sequenceNumber: 17010 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:18:09.640+0000 I NETWORK [conn4081220] end connection 10.0.179.209:40708 (1506 connections now open) 2016-03-13T09:18:09.640+0000 I NETWORK [conn4081637] end connection 10.0.27.237:36061 (1505 connections now open) 2016-03-13T09:18:09.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40964 #4082142 (1506 connections now open) 2016-03-13T09:18:09.642+0000 I NETWORK [conn4081450] end connection 10.0.20.53:50900 (1505 connections now open) 2016-03-13T09:18:09.642+0000 I NETWORK [conn4082077] end connection 10.0.216.90:57848 (1504 connections now open) 2016-03-13T09:18:09.643+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36296 #4082143 (1505 connections now open) 2016-03-13T09:18:09.643+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57882 #4082144 (1506 connections now open) 2016-03-13T09:18:09.643+0000 I NETWORK [conn4081642] end connection 10.0.27.237:36064 (1505 connections now open) 2016-03-13T09:18:09.644+0000 I NETWORK [conn4081180] end connection 10.0.26.69:42529 (1504 connections now open) 2016-03-13T09:18:09.644+0000 I NETWORK [conn4081413] end connection 10.0.20.53:50878 (1503 connections now open) 2016-03-13T09:18:09.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51250 #4082145 (1504 connections now open) 2016-03-13T09:18:09.644+0000 I NETWORK [conn4080833] end connection 10.0.179.209:40522 (1502 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36297 #4082146 (1504 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [conn4082080] end connection 10.0.216.90:57849 (1502 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [conn4080703] end connection 10.0.179.209:40437 (1502 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51251 #4082147 (1503 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40966 #4082148 (1504 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40967 #4082149 (1505 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57884 #4082150 (1506 connections now open) 2016-03-13T09:18:09.645+0000 I NETWORK [conn4081522] end connection 10.0.27.237:35999 (1505 connections now open) 2016-03-13T09:18:09.646+0000 I NETWORK [conn4080641] end connection 10.0.7.113:59308 (1504 connections now open) 2016-03-13T09:18:09.646+0000 I NETWORK [conn4081422] end connection 10.0.27.237:35949 (1503 connections now open) 2016-03-13T09:18:09.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42991 #4082151 (1504 connections now open) 2016-03-13T09:18:09.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59878 #4082152 (1505 connections now open) 2016-03-13T09:18:09.647+0000 I NETWORK [conn4081219] end connection 10.0.20.53:50795 (1504 connections now open) 2016-03-13T09:18:09.647+0000 I NETWORK [conn4080824] end connection 10.0.216.90:57459 (1503 connections now open) 2016-03-13T09:18:09.647+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51254 #4082153 (1504 connections now open) 2016-03-13T09:18:09.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57887 #4082154 (1505 connections now open) 2016-03-13T09:18:09.648+0000 I NETWORK [conn4081230] end connection 10.0.20.53:50797 (1504 connections now open) 2016-03-13T09:18:09.648+0000 I NETWORK [conn4081455] end connection 10.0.27.237:35963 (1503 connections now open) 2016-03-13T09:18:09.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36300 #4082155 (1504 connections now open) 2016-03-13T09:18:09.649+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51256 #4082156 (1505 connections now open) 2016-03-13T09:18:09.649+0000 I NETWORK [conn4081935] end connection 10.0.179.209:40876 (1504 connections now open) 2016-03-13T09:18:09.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40968 #4082157 (1505 connections now open) 2016-03-13T09:18:09.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36301 #4082158 (1506 connections now open) 2016-03-13T09:18:09.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36302 #4082159 (1507 connections now open) 2016-03-13T09:18:09.654+0000 I NETWORK [conn4081236] end connection 10.0.20.53:50800 (1506 connections now open) 2016-03-13T09:18:09.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51257 #4082160 (1507 connections now open) 2016-03-13T09:18:09.654+0000 I NETWORK [conn4080740] end connection 10.0.216.90:57404 (1506 connections now open) 2016-03-13T09:18:09.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57888 #4082161 (1507 connections now open) 2016-03-13T09:18:09.657+0000 I NETWORK [conn4081652] end connection 10.0.27.237:36069 (1506 connections now open) 2016-03-13T09:18:09.659+0000 I NETWORK [conn4081644] end connection 10.0.27.237:36065 (1505 connections now open) 2016-03-13T09:18:09.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36305 #4082162 (1506 connections now open) 2016-03-13T09:18:09.660+0000 I NETWORK [conn4078798] end connection 10.0.179.209:38499 (1505 connections now open) 2016-03-13T09:18:09.660+0000 I NETWORK [conn4081668] end connection 10.0.27.237:36075 (1504 connections now open) 2016-03-13T09:18:09.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40970 #4082163 (1505 connections now open) 2016-03-13T09:18:09.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36306 #4082164 (1506 connections now open) 2016-03-13T09:18:09.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36307 #4082165 (1507 connections now open) 2016-03-13T09:18:09.663+0000 I NETWORK [conn4081597] end connection 10.0.27.237:36036 (1506 connections now open) 2016-03-13T09:18:09.665+0000 I NETWORK [conn4081014] end connection 10.0.27.237:35793 (1505 connections now open) 2016-03-13T09:18:09.665+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36308 #4082166 (1506 connections now open) 2016-03-13T09:18:09.665+0000 I NETWORK [conn4081657] end connection 10.0.27.237:36071 (1505 connections now open) 2016-03-13T09:18:09.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36309 #4082167 (1506 connections now open) 2016-03-13T09:18:09.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36310 #4082168 (1507 connections now open) 2016-03-13T09:18:09.670+0000 I NETWORK [conn4081498] end connection 10.0.27.237:35986 (1506 connections now open) 2016-03-13T09:18:09.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36312 #4082169 (1507 connections now open) 2016-03-13T09:18:09.677+0000 I NETWORK [conn4081789] end connection 10.0.216.90:57715 (1506 connections now open) 2016-03-13T09:18:09.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57889 #4082170 (1507 connections now open) 2016-03-13T09:18:09.681+0000 I NETWORK [conn4080811] end connection 10.0.216.90:57450 (1506 connections now open) 2016-03-13T09:18:09.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57890 #4082171 (1507 connections now open) 2016-03-13T09:18:09.736+0000 I NETWORK [conn4081340] end connection 10.0.7.113:59651 (1506 connections now open) 2016-03-13T09:18:09.737+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59880 #4082172 (1507 connections now open) 2016-03-13T09:18:09.739+0000 I NETWORK [conn4081943] end connection 10.0.7.113:59795 (1506 connections now open) 2016-03-13T09:18:09.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59881 #4082173 (1507 connections now open) 2016-03-13T09:18:09.741+0000 I NETWORK [conn4081443] end connection 10.0.7.113:59679 (1506 connections now open) 2016-03-13T09:18:09.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59882 #4082174 (1507 connections now open) 2016-03-13T09:18:09.762+0000 I NETWORK [conn4080903] end connection 10.0.216.90:57494 (1506 connections now open) 2016-03-13T09:18:09.762+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57892 #4082175 (1507 connections now open) 2016-03-13T09:18:10.066+0000 I NETWORK [conn4080717] end connection 10.0.179.209:40446 (1506 connections now open) 2016-03-13T09:18:10.067+0000 I NETWORK [conn4081359] end connection 10.0.7.113:59657 (1505 connections now open) 2016-03-13T09:18:10.067+0000 I NETWORK [conn4081207] end connection 10.0.26.69:42535 (1504 connections now open) 2016-03-13T09:18:10.067+0000 I NETWORK [conn4081215] end connection 10.0.26.69:42537 (1503 connections now open) 2016-03-13T09:18:10.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59885 #4082176 (1505 connections now open) 2016-03-13T09:18:10.069+0000 I NETWORK [conn4081423] end connection 10.0.26.69:42634 (1503 connections now open) 2016-03-13T09:18:10.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42995 #4082177 (1505 connections now open) 2016-03-13T09:18:10.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42996 #4082178 (1505 connections now open) 2016-03-13T09:18:10.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42997 #4082179 (1506 connections now open) 2016-03-13T09:18:10.072+0000 I NETWORK [conn4080957] end connection 10.0.26.69:42446 (1505 connections now open) 2016-03-13T09:18:10.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:42998 #4082180 (1506 connections now open) 2016-03-13T09:18:10.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40972 #4082181 (1507 connections now open) 2016-03-13T09:18:10.081+0000 I NETWORK [conn4081277] end connection 10.0.27.237:35898 (1506 connections now open) 2016-03-13T09:18:10.081+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36316 #4082182 (1507 connections now open) 2016-03-13T09:18:10.082+0000 I NETWORK [conn4080842] end connection 10.0.179.209:40530 (1506 connections now open) 2016-03-13T09:18:10.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40973 #4082183 (1507 connections now open) 2016-03-13T09:18:10.083+0000 I NETWORK [conn4080933] end connection 10.0.26.69:42440 (1506 connections now open) 2016-03-13T09:18:10.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43001 #4082184 (1507 connections now open) 2016-03-13T09:18:10.084+0000 I NETWORK [conn4081179] end connection 10.0.20.53:50782 (1506 connections now open) 2016-03-13T09:18:10.085+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51260 #4082185 (1507 connections now open) 2016-03-13T09:18:10.143+0000 I NETWORK [conn4081459] end connection 10.0.26.69:42656 (1506 connections now open) 2016-03-13T09:18:10.143+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43002 #4082186 (1507 connections now open) 2016-03-13T09:18:10.146+0000 I NETWORK [conn4081150] end connection 10.0.26.69:42518 (1506 connections now open) 2016-03-13T09:18:10.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43003 #4082187 (1507 connections now open) 2016-03-13T09:18:10.149+0000 I NETWORK [conn4081513] end connection 10.0.27.237:35995 (1506 connections now open) 2016-03-13T09:18:10.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36317 #4082188 (1507 connections now open) 2016-03-13T09:18:10.151+0000 I NETWORK [conn4081528] end connection 10.0.27.237:36001 (1506 connections now open) 2016-03-13T09:18:10.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36318 #4082189 (1507 connections now open) 2016-03-13T09:18:10.160+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 17011 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:18:10.166+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.ActionReports: 5ms sequenceNumber: 17012 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:18:10.167+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17013 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:18:10.170+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.AppScreenReports: 3ms sequenceNumber: 17014 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:18:10.171+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 17015 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:18:10.172+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 17016 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:18:10.176+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 3ms sequenceNumber: 17017 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:18:10.213+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.Clients: 36ms sequenceNumber: 17018 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:18:10.217+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 1ms sequenceNumber: 17019 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:18:10.224+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.CountryReports: 6ms sequenceNumber: 17020 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:18:10.225+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 1ms sequenceNumber: 17021 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:18:10.226+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 17022 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:18:10.267+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 40ms sequenceNumber: 17023 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:18:10.276+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.CustomEventReports: 5ms sequenceNumber: 17024 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:18:10.277+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 17025 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:18:10.282+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 17026 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:18:10.294+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.DeviceReports: 11ms sequenceNumber: 17027 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:18:10.295+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 17028 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:18:10.296+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17029 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:18:10.307+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.GestureReports: 10ms sequenceNumber: 17030 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:18:10.357+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 49ms sequenceNumber: 17031 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:18:10.532+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.HeatmapReports: 172ms sequenceNumber: 17032 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:18:10.598+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 53ms sequenceNumber: 17033 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:18:10.620+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.NavigationReports: 17ms sequenceNumber: 17034 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:18:10.627+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.OSVersionReports: 6ms sequenceNumber: 17035 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:18:10.644+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.PendingSessions: 16ms sequenceNumber: 17036 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:18:10.645+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 17037 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:18:10.652+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 7ms sequenceNumber: 17038 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:18:10.655+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 17039 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:18:10.656+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 17040 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:18:10.657+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17041 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:18:11.027+0000 I NETWORK [conn4082034] end connection 10.0.179.209:40914 (1506 connections now open) 2016-03-13T09:18:11.028+0000 I NETWORK [conn4082033] end connection 10.0.179.209:40913 (1505 connections now open) 2016-03-13T09:18:11.028+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40976 #4082190 (1506 connections now open) 2016-03-13T09:18:11.028+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40977 #4082191 (1507 connections now open) 2016-03-13T09:18:11.329+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 671ms sequenceNumber: 17042 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:18:11.724+0000 I NETWORK [conn4082035] end connection 10.0.26.69:42950 (1506 connections now open) 2016-03-13T09:18:11.724+0000 I NETWORK [conn4082036] end connection 10.0.26.69:42951 (1506 connections now open) 2016-03-13T09:18:11.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43009 #4082192 (1506 connections now open) 2016-03-13T09:18:11.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43010 #4082193 (1507 connections now open) 2016-03-13T09:18:11.999+0000 I NETWORK [conn4082023] end connection 10.0.26.69:42933 (1506 connections now open) 2016-03-13T09:18:11.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43012 #4082194 (1507 connections now open) 2016-03-13T09:18:13.825+0000 I NETWORK [conn4081055] end connection 10.0.27.237:35814 (1506 connections now open) 2016-03-13T09:18:13.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36324 #4082195 (1507 connections now open) 2016-03-13T09:18:14.492+0000 I NETWORK [conn4082131] end connection 10.0.27.237:36289 (1506 connections now open) 2016-03-13T09:18:14.493+0000 I NETWORK [conn4082132] end connection 10.0.27.237:36290 (1505 connections now open) 2016-03-13T09:18:14.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36326 #4082196 (1506 connections now open) 2016-03-13T09:18:14.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36327 #4082197 (1507 connections now open) 2016-03-13T09:18:15.698+0000 I NETWORK [conn4079559] end connection 10.0.179.209:39260 (1506 connections now open) 2016-03-13T09:18:15.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40982 #4082198 (1507 connections now open) 2016-03-13T09:18:15.701+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.Sessions: 4287ms sequenceNumber: 17043 version: 30651|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:18:16.031+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 17044 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:18:16.032+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 17045 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:18:16.034+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17046 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:18:16.052+0000 I SHARDING [conn3432256] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 18ms sequenceNumber: 17047 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:18:16.055+0000 I NETWORK [conn4081201] end connection 10.0.27.237:35871 (1506 connections now open) 2016-03-13T09:18:16.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36330 #4082199 (1507 connections now open) 2016-03-13T09:18:16.058+0000 I NETWORK [conn4081083] end connection 10.0.27.237:35826 (1506 connections now open) 2016-03-13T09:18:16.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36331 #4082200 (1507 connections now open) 2016-03-13T09:18:16.078+0000 I NETWORK [conn4079583] end connection 10.0.216.90:56266 (1506 connections now open) 2016-03-13T09:18:16.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57909 #4082201 (1507 connections now open) 2016-03-13T09:18:16.117+0000 I NETWORK [conn4082095] end connection 10.0.216.90:57857 (1506 connections now open) 2016-03-13T09:18:16.117+0000 I NETWORK [conn4081432] end connection 10.0.20.53:50891 (1506 connections now open) 2016-03-13T09:18:16.117+0000 I NETWORK [conn4081321] end connection 10.0.7.113:59646 (1504 connections now open) 2016-03-13T09:18:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51268 #4082202 (1505 connections now open) 2016-03-13T09:18:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57912 #4082203 (1506 connections now open) 2016-03-13T09:18:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59900 #4082204 (1507 connections now open) 2016-03-13T09:18:16.182+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:18:16.182+0000-56e5305818fa188537623d69", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860696182), what: "balancer.round", ns: "", details: { executionTimeMillis: 31583, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:18:16.302+0000 I NETWORK [conn4082153] end connection 10.0.20.53:51254 (1506 connections now open) 2016-03-13T09:18:16.303+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51269 #4082205 (1507 connections now open) 2016-03-13T09:18:16.305+0000 I NETWORK [conn4081245] end connection 10.0.20.53:50804 (1506 connections now open) 2016-03-13T09:18:16.305+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51270 #4082206 (1507 connections now open) 2016-03-13T09:18:16.307+0000 I NETWORK [conn4081268] end connection 10.0.20.53:50810 (1506 connections now open) 2016-03-13T09:18:16.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51271 #4082207 (1507 connections now open) 2016-03-13T09:18:16.415+0000 I NETWORK [conn4081497] end connection 10.0.26.69:42676 (1506 connections now open) 2016-03-13T09:18:16.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43018 #4082208 (1507 connections now open) 2016-03-13T09:18:16.431+0000 I NETWORK [conn4081064] end connection 10.0.26.69:42483 (1506 connections now open) 2016-03-13T09:18:16.432+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43019 #4082209 (1507 connections now open) 2016-03-13T09:18:16.575+0000 I NETWORK [conn4080938] end connection 10.0.26.69:42441 (1506 connections now open) 2016-03-13T09:18:16.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43021 #4082210 (1507 connections now open) 2016-03-13T09:18:16.576+0000 I NETWORK [conn4081292] end connection 10.0.7.113:59638 (1506 connections now open) 2016-03-13T09:18:16.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59902 #4082211 (1507 connections now open) 2016-03-13T09:18:16.577+0000 I NETWORK [conn4080732] end connection 10.0.216.90:57400 (1506 connections now open) 2016-03-13T09:18:16.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57915 #4082212 (1507 connections now open) 2016-03-13T09:18:16.578+0000 I NETWORK [conn4080837] end connection 10.0.179.209:40526 (1506 connections now open) 2016-03-13T09:18:16.579+0000 I NETWORK [conn4081669] end connection 10.0.26.69:42768 (1505 connections now open) 2016-03-13T09:18:16.579+0000 I NETWORK [conn4082111] end connection 10.0.216.90:57865 (1505 connections now open) 2016-03-13T09:18:16.579+0000 I NETWORK [conn4081148] end connection 10.0.216.90:57552 (1503 connections now open) 2016-03-13T09:18:16.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43024 #4082213 (1504 connections now open) 2016-03-13T09:18:16.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57916 #4082214 (1505 connections now open) 2016-03-13T09:18:16.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:40996 #4082215 (1506 connections now open) 2016-03-13T09:18:16.580+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57917 #4082216 (1507 connections now open) 2016-03-13T09:18:16.580+0000 I NETWORK [conn4081745] end connection 10.0.7.113:59715 (1505 connections now open) 2016-03-13T09:18:16.580+0000 I NETWORK [conn4081825] end connection 10.0.27.237:36143 (1505 connections now open) 2016-03-13T09:18:16.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59904 #4082217 (1506 connections now open) 2016-03-13T09:18:16.582+0000 I NETWORK [conn4081869] end connection 10.0.27.237:36173 (1505 connections now open) 2016-03-13T09:18:16.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36337 #4082218 (1506 connections now open) 2016-03-13T09:18:16.582+0000 I NETWORK [conn4082048] end connection 10.0.7.113:59836 (1505 connections now open) 2016-03-13T09:18:16.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36338 #4082219 (1506 connections now open) 2016-03-13T09:18:16.583+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59905 #4082220 (1507 connections now open) 2016-03-13T09:18:16.583+0000 I NETWORK [conn4081159] end connection 10.0.216.90:57554 (1506 connections now open) 2016-03-13T09:18:16.583+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57921 #4082221 (1507 connections now open) 2016-03-13T09:18:16.583+0000 I NETWORK [conn4081284] end connection 10.0.26.69:42568 (1506 connections now open) 2016-03-13T09:18:16.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43027 #4082222 (1507 connections now open) 2016-03-13T09:18:16.584+0000 I NETWORK [conn4080714] end connection 10.0.179.209:40444 (1506 connections now open) 2016-03-13T09:18:16.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41000 #4082223 (1507 connections now open) 2016-03-13T09:18:16.585+0000 I NETWORK [conn4081290] end connection 10.0.216.90:57586 (1506 connections now open) 2016-03-13T09:18:16.585+0000 I NETWORK [conn4081882] end connection 10.0.7.113:59778 (1505 connections now open) 2016-03-13T09:18:16.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57922 #4082224 (1506 connections now open) 2016-03-13T09:18:16.586+0000 I NETWORK [conn4080132] end connection 10.0.216.90:56817 (1505 connections now open) 2016-03-13T09:18:16.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57923 #4082225 (1506 connections now open) 2016-03-13T09:18:16.586+0000 I NETWORK [conn4081793] end connection 10.0.20.53:51090 (1505 connections now open) 2016-03-13T09:18:16.587+0000 I NETWORK [conn4080730] end connection 10.0.179.209:40454 (1504 connections now open) 2016-03-13T09:18:16.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59906 #4082226 (1506 connections now open) 2016-03-13T09:18:16.587+0000 I NETWORK [conn4081924] end connection 10.0.20.53:51146 (1504 connections now open) 2016-03-13T09:18:16.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41002 #4082227 (1505 connections now open) 2016-03-13T09:18:16.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51276 #4082228 (1506 connections now open) 2016-03-13T09:18:16.588+0000 I NETWORK [conn4080733] end connection 10.0.179.209:40455 (1505 connections now open) 2016-03-13T09:18:16.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51277 #4082229 (1506 connections now open) 2016-03-13T09:18:16.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41005 #4082230 (1507 connections now open) 2016-03-13T09:18:16.590+0000 I NETWORK [conn4081253] end connection 10.0.20.53:50805 (1506 connections now open) 2016-03-13T09:18:16.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51279 #4082231 (1507 connections now open) 2016-03-13T09:18:16.591+0000 I NETWORK [conn4082042] end connection 10.0.7.113:59832 (1506 connections now open) 2016-03-13T09:18:16.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59908 #4082232 (1507 connections now open) 2016-03-13T09:18:16.591+0000 I NETWORK [conn4081176] end connection 10.0.216.90:57558 (1505 connections now open) 2016-03-13T09:18:16.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57926 #4082233 (1507 connections now open) 2016-03-13T09:18:16.592+0000 I NETWORK [conn4081612] end connection 10.0.20.53:50983 (1506 connections now open) 2016-03-13T09:18:16.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51280 #4082234 (1507 connections now open) 2016-03-13T09:18:16.594+0000 I NETWORK [conn4081028] end connection 10.0.216.90:57514 (1506 connections now open) 2016-03-13T09:18:16.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57927 #4082235 (1507 connections now open) 2016-03-13T09:18:16.595+0000 I NETWORK [conn4081451] end connection 10.0.20.53:50901 (1506 connections now open) 2016-03-13T09:18:16.595+0000 I NETWORK [conn4080959] end connection 10.0.179.209:40607 (1505 connections now open) 2016-03-13T09:18:16.595+0000 I NETWORK [conn4080046] end connection 10.0.7.113:58707 (1504 connections now open) 2016-03-13T09:18:16.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51283 #4082236 (1505 connections now open) 2016-03-13T09:18:16.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41007 #4082237 (1506 connections now open) 2016-03-13T09:18:16.596+0000 I NETWORK [conn4081289] end connection 10.0.216.90:57585 (1505 connections now open) 2016-03-13T09:18:16.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59909 #4082238 (1506 connections now open) 2016-03-13T09:18:16.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57928 #4082239 (1507 connections now open) 2016-03-13T09:18:16.596+0000 I NETWORK [conn4081312] end connection 10.0.26.69:42580 (1506 connections now open) 2016-03-13T09:18:16.597+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43032 #4082240 (1507 connections now open) 2016-03-13T09:18:16.598+0000 I NETWORK [conn4081109] end connection 10.0.26.69:42499 (1506 connections now open) 2016-03-13T09:18:16.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43033 #4082241 (1507 connections now open) 2016-03-13T09:18:16.600+0000 I NETWORK [conn4081843] end connection 10.0.27.237:36151 (1506 connections now open) 2016-03-13T09:18:16.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36339 #4082242 (1507 connections now open) 2016-03-13T09:18:16.601+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:16.602+0000 I NETWORK [conn4081258] end connection 10.0.27.237:35892 (1506 connections now open) 2016-03-13T09:18:16.603+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:16.603+0000 I NETWORK [conn4081838] end connection 10.0.27.237:36147 (1505 connections now open) 2016-03-13T09:18:16.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36341 #4082243 (1506 connections now open) 2016-03-13T09:18:16.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36342 #4082244 (1507 connections now open) 2016-03-13T09:18:16.616+0000 I NETWORK [conn4080861] end connection 10.0.179.209:40546 (1506 connections now open) 2016-03-13T09:18:16.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41009 #4082245 (1507 connections now open) 2016-03-13T09:18:16.618+0000 I NETWORK [conn4081119] end connection 10.0.216.90:57543 (1506 connections now open) 2016-03-13T09:18:16.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57930 #4082246 (1507 connections now open) 2016-03-13T09:18:16.622+0000 I NETWORK [conn4080737] end connection 10.0.179.209:40457 (1506 connections now open) 2016-03-13T09:18:16.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41010 #4082247 (1507 connections now open) 2016-03-13T09:18:16.623+0000 I NETWORK [conn4082134] end connection 10.0.7.113:59866 (1506 connections now open) 2016-03-13T09:18:16.623+0000 I NETWORK [conn4082135] end connection 10.0.7.113:59865 (1506 connections now open) 2016-03-13T09:18:16.623+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59912 #4082248 (1506 connections now open) 2016-03-13T09:18:16.623+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59913 #4082249 (1507 connections now open) 2016-03-13T09:18:16.688+0000 I NETWORK [conn4081223] end connection 10.0.216.90:57564 (1506 connections now open) 2016-03-13T09:18:16.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57931 #4082250 (1507 connections now open) 2016-03-13T09:18:16.714+0000 I NETWORK [conn4081751] end connection 10.0.27.237:36120 (1506 connections now open) 2016-03-13T09:18:16.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36348 #4082251 (1507 connections now open) 2016-03-13T09:18:16.715+0000 I NETWORK [conn4080784] end connection 10.0.7.113:59410 (1506 connections now open) 2016-03-13T09:18:16.715+0000 I NETWORK [conn4081706] end connection 10.0.27.237:36091 (1505 connections now open) 2016-03-13T09:18:16.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36349 #4082252 (1506 connections now open) 2016-03-13T09:18:16.717+0000 I NETWORK [conn4079385] end connection 10.0.179.209:39093 (1505 connections now open) 2016-03-13T09:18:16.718+0000 I NETWORK [conn4080745] end connection 10.0.179.209:40461 (1504 connections now open) 2016-03-13T09:18:16.718+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41011 #4082253 (1505 connections now open) 2016-03-13T09:18:16.718+0000 I NETWORK [conn4080971] end connection 10.0.26.69:42451 (1504 connections now open) 2016-03-13T09:18:16.718+0000 I NETWORK [conn4081602] end connection 10.0.7.113:59703 (1503 connections now open) 2016-03-13T09:18:16.718+0000 I NETWORK [conn4080746] end connection 10.0.179.209:40462 (1502 connections now open) 2016-03-13T09:18:16.718+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59914 #4082254 (1503 connections now open) 2016-03-13T09:18:16.719+0000 I NETWORK [conn4080852] end connection 10.0.7.113:59463 (1502 connections now open) 2016-03-13T09:18:16.719+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59915 #4082255 (1503 connections now open) 2016-03-13T09:18:16.720+0000 I NETWORK [conn4081424] end connection 10.0.20.53:50885 (1502 connections now open) 2016-03-13T09:18:16.720+0000 I NETWORK [conn4081655] end connection 10.0.26.69:42760 (1501 connections now open) 2016-03-13T09:18:16.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51285 #4082256 (1502 connections now open) 2016-03-13T09:18:16.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59917 #4082257 (1503 connections now open) 2016-03-13T09:18:16.721+0000 I NETWORK [conn4081113] end connection 10.0.26.69:42502 (1502 connections now open) 2016-03-13T09:18:16.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43035 #4082258 (1504 connections now open) 2016-03-13T09:18:16.721+0000 I NETWORK [conn4081115] end connection 10.0.7.113:59579 (1502 connections now open) 2016-03-13T09:18:16.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43037 #4082259 (1503 connections now open) 2016-03-13T09:18:16.722+0000 I NETWORK [conn4081885] end connection 10.0.7.113:59779 (1502 connections now open) 2016-03-13T09:18:16.722+0000 I NETWORK [conn4080750] end connection 10.0.179.209:40465 (1501 connections now open) 2016-03-13T09:18:16.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43038 #4082260 (1502 connections now open) 2016-03-13T09:18:16.722+0000 I NETWORK [conn4080995] end connection 10.0.26.69:42458 (1501 connections now open) 2016-03-13T09:18:16.723+0000 I NETWORK [conn4081093] end connection 10.0.20.53:50749 (1500 connections now open) 2016-03-13T09:18:16.723+0000 I NETWORK [conn4080981] end connection 10.0.7.113:59540 (1499 connections now open) 2016-03-13T09:18:16.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51289 #4082261 (1500 connections now open) 2016-03-13T09:18:16.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41013 #4082262 (1501 connections now open) 2016-03-13T09:18:16.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41015 #4082263 (1502 connections now open) 2016-03-13T09:18:16.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41016 #4082264 (1503 connections now open) 2016-03-13T09:18:16.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43040 #4082265 (1504 connections now open) 2016-03-13T09:18:16.725+0000 I NETWORK [conn4081098] end connection 10.0.27.237:35831 (1503 connections now open) 2016-03-13T09:18:16.726+0000 I NETWORK [conn4081047] end connection 10.0.20.53:50730 (1502 connections now open) 2016-03-13T09:18:16.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36354 #4082266 (1503 connections now open) 2016-03-13T09:18:16.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59919 #4082267 (1504 connections now open) 2016-03-13T09:18:16.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51290 #4082268 (1505 connections now open) 2016-03-13T09:18:16.727+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59923 #4082269 (1506 connections now open) 2016-03-13T09:18:16.727+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59924 #4082270 (1507 connections now open) 2016-03-13T09:18:16.727+0000 I NETWORK [conn4081829] end connection 10.0.216.90:57734 (1506 connections now open) 2016-03-13T09:18:16.727+0000 I NETWORK [conn4082114] end connection 10.0.216.90:57866 (1505 connections now open) 2016-03-13T09:18:16.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57938 #4082271 (1506 connections now open) 2016-03-13T09:18:16.729+0000 I NETWORK [conn4082162] end connection 10.0.27.237:36305 (1505 connections now open) 2016-03-13T09:18:16.730+0000 I NETWORK [conn4081225] end connection 10.0.216.90:57567 (1504 connections now open) 2016-03-13T09:18:16.730+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36355 #4082272 (1505 connections now open) 2016-03-13T09:18:16.730+0000 I NETWORK [conn4080865] end connection 10.0.7.113:59472 (1504 connections now open) 2016-03-13T09:18:16.730+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57940 #4082273 (1505 connections now open) 2016-03-13T09:18:16.731+0000 I NETWORK [conn4081559] end connection 10.0.27.237:36016 (1504 connections now open) 2016-03-13T09:18:16.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59925 #4082274 (1505 connections now open) 2016-03-13T09:18:16.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57941 #4082275 (1506 connections now open) 2016-03-13T09:18:16.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36357 #4082276 (1507 connections now open) 2016-03-13T09:18:16.732+0000 I NETWORK [conn4081059] end connection 10.0.27.237:35816 (1506 connections now open) 2016-03-13T09:18:16.732+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36358 #4082277 (1507 connections now open) 2016-03-13T09:18:16.733+0000 I NETWORK [conn4081544] end connection 10.0.7.113:59696 (1506 connections now open) 2016-03-13T09:18:16.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59928 #4082278 (1507 connections now open) 2016-03-13T09:18:16.734+0000 I NETWORK [conn4081749] end connection 10.0.27.237:36118 (1506 connections now open) 2016-03-13T09:18:16.734+0000 I NETWORK [conn4080849] end connection 10.0.216.90:57470 (1505 connections now open) 2016-03-13T09:18:16.734+0000 I NETWORK [conn4082088] end connection 10.0.7.113:59846 (1504 connections now open) 2016-03-13T09:18:16.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36360 #4082279 (1505 connections now open) 2016-03-13T09:18:16.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59929 #4082280 (1506 connections now open) 2016-03-13T09:18:16.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57943 #4082281 (1507 connections now open) 2016-03-13T09:18:16.736+0000 I NETWORK [conn4081579] end connection 10.0.27.237:36024 (1506 connections now open) 2016-03-13T09:18:16.736+0000 I NETWORK [conn4080778] end connection 10.0.216.90:57427 (1505 connections now open) 2016-03-13T09:18:16.737+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36361 #4082282 (1506 connections now open) 2016-03-13T09:18:16.737+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:16.737+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:16.737+0000 I NETWORK [conn4080962] end connection 10.0.26.69:42448 (1505 connections now open) 2016-03-13T09:18:16.738+0000 I NETWORK [conn4081714] end connection 10.0.27.237:36094 (1504 connections now open) 2016-03-13T09:18:16.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43046 #4082283 (1505 connections now open) 2016-03-13T09:18:16.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57944 #4082284 (1506 connections now open) 2016-03-13T09:18:16.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36363 #4082285 (1507 connections now open) 2016-03-13T09:18:16.739+0000 I NETWORK [conn4081609] end connection 10.0.27.237:36042 (1506 connections now open) 2016-03-13T09:18:16.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36364 #4082286 (1507 connections now open) 2016-03-13T09:18:16.741+0000 I NETWORK [conn4081781] end connection 10.0.27.237:36131 (1506 connections now open) 2016-03-13T09:18:16.741+0000 I NETWORK [conn4081734] end connection 10.0.27.237:36107 (1505 connections now open) 2016-03-13T09:18:16.742+0000 I NETWORK [conn4081017] end connection 10.0.179.209:40640 (1504 connections now open) 2016-03-13T09:18:16.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36365 #4082287 (1506 connections now open) 2016-03-13T09:18:16.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36366 #4082288 (1506 connections now open) 2016-03-13T09:18:16.756+0000 I NETWORK [conn4081196] end connection 10.0.20.53:50788 (1505 connections now open) 2016-03-13T09:18:16.759+0000 I NETWORK [conn4081404] end connection 10.0.20.53:50875 (1504 connections now open) 2016-03-13T09:18:16.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51295 #4082289 (1505 connections now open) 2016-03-13T09:18:16.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51296 #4082290 (1506 connections now open) 2016-03-13T09:18:16.760+0000 I NETWORK [conn4081521] end connection 10.0.26.69:42691 (1505 connections now open) 2016-03-13T09:18:16.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43048 #4082291 (1506 connections now open) 2016-03-13T09:18:16.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41019 #4082292 (1507 connections now open) 2016-03-13T09:18:16.767+0000 I NETWORK [conn4081344] end connection 10.0.20.53:50840 (1506 connections now open) 2016-03-13T09:18:16.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51297 #4082293 (1507 connections now open) 2016-03-13T09:18:16.769+0000 I NETWORK [conn4080757] end connection 10.0.179.209:40468 (1506 connections now open) 2016-03-13T09:18:16.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41020 #4082294 (1507 connections now open) 2016-03-13T09:18:16.772+0000 I NETWORK [conn4082105] end connection 10.0.27.237:36276 (1506 connections now open) 2016-03-13T09:18:16.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36368 #4082295 (1507 connections now open) 2016-03-13T09:18:16.774+0000 I NETWORK [conn4080760] end connection 10.0.179.209:40469 (1506 connections now open) 2016-03-13T09:18:16.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41024 #4082296 (1507 connections now open) 2016-03-13T09:18:16.775+0000 I NETWORK [conn4081631] end connection 10.0.27.237:36057 (1506 connections now open) 2016-03-13T09:18:16.775+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36369 #4082297 (1507 connections now open) 2016-03-13T09:18:16.776+0000 I NETWORK [conn4080883] end connection 10.0.179.209:40569 (1506 connections now open) 2016-03-13T09:18:16.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41026 #4082298 (1507 connections now open) 2016-03-13T09:18:17.013+0000 I NETWORK [conn4080942] end connection 10.0.7.113:59522 (1506 connections now open) 2016-03-13T09:18:17.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59931 #4082299 (1507 connections now open) 2016-03-13T09:18:17.016+0000 I NETWORK [conn4080847] end connection 10.0.7.113:59456 (1506 connections now open) 2016-03-13T09:18:17.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59932 #4082300 (1507 connections now open) 2016-03-13T09:18:17.018+0000 I NETWORK [conn4080850] end connection 10.0.7.113:59458 (1506 connections now open) 2016-03-13T09:18:17.019+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59933 #4082301 (1507 connections now open) 2016-03-13T09:18:17.025+0000 I NETWORK [conn4080766] end connection 10.0.179.209:40474 (1506 connections now open) 2016-03-13T09:18:17.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41028 #4082302 (1507 connections now open) 2016-03-13T09:18:17.031+0000 I NETWORK [conn4081562] end connection 10.0.27.237:36017 (1506 connections now open) 2016-03-13T09:18:17.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36370 #4082303 (1507 connections now open) 2016-03-13T09:18:17.034+0000 I NETWORK [conn4081987] end connection 10.0.27.237:36234 (1506 connections now open) 2016-03-13T09:18:17.034+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36371 #4082304 (1507 connections now open) 2016-03-13T09:18:17.038+0000 I NETWORK [conn4081545] end connection 10.0.27.237:36009 (1506 connections now open) 2016-03-13T09:18:17.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36372 #4082305 (1507 connections now open) 2016-03-13T09:18:17.040+0000 I NETWORK [conn4081540] end connection 10.0.27.237:36007 (1506 connections now open) 2016-03-13T09:18:17.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36373 #4082306 (1507 connections now open) 2016-03-13T09:18:17.041+0000 I NETWORK [conn4081222] end connection 10.0.20.53:50796 (1506 connections now open) 2016-03-13T09:18:17.042+0000 I NETWORK [conn4081071] end connection 10.0.27.237:35822 (1505 connections now open) 2016-03-13T09:18:17.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51299 #4082307 (1506 connections now open) 2016-03-13T09:18:17.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36374 #4082308 (1507 connections now open) 2016-03-13T09:18:17.042+0000 I NETWORK [conn4081482] end connection 10.0.26.69:42667 (1506 connections now open) 2016-03-13T09:18:17.043+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43050 #4082309 (1507 connections now open) 2016-03-13T09:18:17.045+0000 I NETWORK [conn4081036] end connection 10.0.26.69:42470 (1506 connections now open) 2016-03-13T09:18:17.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43051 #4082310 (1507 connections now open) 2016-03-13T09:18:17.053+0000 I NETWORK [conn4081164] end connection 10.0.26.69:42522 (1506 connections now open) 2016-03-13T09:18:17.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43053 #4082311 (1507 connections now open) 2016-03-13T09:18:17.061+0000 I NETWORK [conn4081153] end connection 10.0.27.237:35854 (1506 connections now open) 2016-03-13T09:18:17.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36376 #4082312 (1507 connections now open) 2016-03-13T09:18:17.062+0000 I NETWORK [conn4081408] end connection 10.0.27.237:35943 (1506 connections now open) 2016-03-13T09:18:17.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36377 #4082313 (1507 connections now open) 2016-03-13T09:18:17.416+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:18:17.429+0000 I NETWORK [conn4080173] end connection 10.0.7.113:58840 (1506 connections now open) 2016-03-13T09:18:17.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59935 #4082314 (1507 connections now open) 2016-03-13T09:18:17.665+0000 I NETWORK [conn4082138] end connection 10.0.216.90:57879 (1506 connections now open) 2016-03-13T09:18:17.665+0000 I NETWORK [conn4082137] end connection 10.0.216.90:57880 (1506 connections now open) 2016-03-13T09:18:17.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57952 #4082315 (1506 connections now open) 2016-03-13T09:18:17.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57951 #4082316 (1507 connections now open) 2016-03-13T09:18:18.260+0000 I NETWORK [conn4081232] end connection 10.0.216.90:57569 (1506 connections now open) 2016-03-13T09:18:18.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57954 #4082317 (1507 connections now open) 2016-03-13T09:18:18.614+0000 I NETWORK [conn4081040] end connection 10.0.26.69:42472 (1506 connections now open) 2016-03-13T09:18:18.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43056 #4082318 (1507 connections now open) 2016-03-13T09:18:18.617+0000 I NETWORK [conn4081043] end connection 10.0.26.69:42474 (1506 connections now open) 2016-03-13T09:18:18.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43057 #4082319 (1507 connections now open) 2016-03-13T09:18:18.701+0000 I NETWORK [conn4082140] end connection 10.0.20.53:51246 (1506 connections now open) 2016-03-13T09:18:18.702+0000 I NETWORK [conn4082141] end connection 10.0.20.53:51247 (1505 connections now open) 2016-03-13T09:18:18.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51303 #4082320 (1506 connections now open) 2016-03-13T09:18:18.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51302 #4082321 (1507 connections now open) 2016-03-13T09:18:18.749+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5305a18fa188537623d6a 2016-03-13T09:18:18.980+0000 I NETWORK [conn4081485] end connection 10.0.20.53:50918 (1506 connections now open) 2016-03-13T09:18:18.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51305 #4082322 (1507 connections now open) 2016-03-13T09:18:19.048+0000 I NETWORK [conn4080889] end connection 10.0.7.113:59495 (1506 connections now open) 2016-03-13T09:18:19.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59942 #4082323 (1507 connections now open) 2016-03-13T09:18:19.050+0000 I NETWORK [conn4080950] end connection 10.0.7.113:59526 (1506 connections now open) 2016-03-13T09:18:19.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59943 #4082324 (1507 connections now open) 2016-03-13T09:18:19.053+0000 I NETWORK [conn4080885] end connection 10.0.7.113:59493 (1506 connections now open) 2016-03-13T09:18:19.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59944 #4082325 (1507 connections now open) 2016-03-13T09:18:19.751+0000 I NETWORK [conn4080727] end connection 10.0.216.90:57398 (1506 connections now open) 2016-03-13T09:18:19.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57956 #4082326 (1507 connections now open) 2016-03-13T09:18:19.753+0000 I NETWORK [conn4080718] end connection 10.0.216.90:57391 (1506 connections now open) 2016-03-13T09:18:19.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57957 #4082327 (1507 connections now open) 2016-03-13T09:18:19.872+0000 I NETWORK [conn4080768] end connection 10.0.179.209:40475 (1506 connections now open) 2016-03-13T09:18:19.872+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41037 #4082328 (1507 connections now open) 2016-03-13T09:18:20.656+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:18:20.317+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:18:21.040+0000 I NETWORK [conn4082190] end connection 10.0.179.209:40976 (1506 connections now open) 2016-03-13T09:18:21.040+0000 I NETWORK [conn4082191] end connection 10.0.179.209:40977 (1506 connections now open) 2016-03-13T09:18:21.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41042 #4082329 (1506 connections now open) 2016-03-13T09:18:21.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41041 #4082330 (1507 connections now open) 2016-03-13T09:18:21.329+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634669d55418c0ec3f9ca7d')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634669d55418c0ec3f9ca7d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56346a2a55418c0ec59aacce') }, shard: "shmydomain1", lastmod: Timestamp 30651000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:18:21.467+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30651|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634669d55418c0ec3f9ca7d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56346a2a55418c0ec59aacce') }) shmydomain1 -> shmydomain3 2016-03-13T09:18:21.737+0000 I NETWORK [conn4082192] end connection 10.0.26.69:43009 (1506 connections now open) 2016-03-13T09:18:21.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43064 #4082331 (1507 connections now open) 2016-03-13T09:18:21.741+0000 I NETWORK [conn4082193] end connection 10.0.26.69:43010 (1506 connections now open) 2016-03-13T09:18:21.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43065 #4082332 (1507 connections now open) 2016-03-13T09:18:22.572+0000 I NETWORK [conn4081447] end connection 10.0.27.237:35960 (1506 connections now open) 2016-03-13T09:18:22.574+0000 I NETWORK [conn4081255] end connection 10.0.216.90:57575 (1505 connections now open) 2016-03-13T09:18:22.574+0000 I NETWORK [conn4081475] end connection 10.0.20.53:50912 (1504 connections now open) 2016-03-13T09:18:22.575+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57962 #4082333 (1505 connections now open) 2016-03-13T09:18:22.575+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51313 #4082334 (1506 connections now open) 2016-03-13T09:18:22.576+0000 I NETWORK [conn4081731] end connection 10.0.26.69:42811 (1505 connections now open) 2016-03-13T09:18:22.576+0000 I NETWORK [conn4081046] end connection 10.0.26.69:42476 (1504 connections now open) 2016-03-13T09:18:22.577+0000 I NETWORK [conn4081156] end connection 10.0.26.69:42519 (1503 connections now open) 2016-03-13T09:18:22.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43068 #4082335 (1504 connections now open) 2016-03-13T09:18:22.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43069 #4082336 (1505 connections now open) 2016-03-13T09:18:22.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43070 #4082337 (1506 connections now open) 2016-03-13T09:18:22.577+0000 I NETWORK [conn4080719] end connection 10.0.7.113:59375 (1505 connections now open) 2016-03-13T09:18:22.578+0000 I NETWORK [conn4081632] end connection 10.0.27.237:36059 (1504 connections now open) 2016-03-13T09:18:22.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59948 #4082338 (1505 connections now open) 2016-03-13T09:18:22.580+0000 I NETWORK [conn4082226] end connection 10.0.7.113:59906 (1504 connections now open) 2016-03-13T09:18:22.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59949 #4082339 (1505 connections now open) 2016-03-13T09:18:22.581+0000 I NETWORK [conn4080722] end connection 10.0.216.90:57393 (1504 connections now open) 2016-03-13T09:18:22.582+0000 I NETWORK [conn4081730] end connection 10.0.27.237:36105 (1503 connections now open) 2016-03-13T09:18:22.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57963 #4082340 (1504 connections now open) 2016-03-13T09:18:22.582+0000 I NETWORK [conn4081192] end connection 10.0.179.209:40700 (1503 connections now open) 2016-03-13T09:18:22.582+0000 I NETWORK [conn4082325] end connection 10.0.7.113:59944 (1502 connections now open) 2016-03-13T09:18:22.583+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41045 #4082341 (1503 connections now open) 2016-03-13T09:18:22.583+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59950 #4082342 (1504 connections now open) 2016-03-13T09:18:22.583+0000 I NETWORK [conn4081801] end connection 10.0.216.90:57721 (1503 connections now open) 2016-03-13T09:18:22.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57964 #4082343 (1504 connections now open) 2016-03-13T09:18:22.585+0000 I NETWORK [conn4081401] end connection 10.0.26.69:42623 (1503 connections now open) 2016-03-13T09:18:22.586+0000 I NETWORK [conn4081050] end connection 10.0.26.69:42478 (1502 connections now open) 2016-03-13T09:18:22.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43072 #4082344 (1504 connections now open) 2016-03-13T09:18:22.586+0000 I NETWORK [conn4081011] end connection 10.0.20.53:50715 (1502 connections now open) 2016-03-13T09:18:22.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43073 #4082345 (1503 connections now open) 2016-03-13T09:18:22.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51315 #4082346 (1504 connections now open) 2016-03-13T09:18:22.586+0000 I NETWORK [conn4080798] end connection 10.0.7.113:59418 (1503 connections now open) 2016-03-13T09:18:22.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59951 #4082347 (1504 connections now open) 2016-03-13T09:18:22.587+0000 I NETWORK [conn4080932] end connection 10.0.179.209:40594 (1503 connections now open) 2016-03-13T09:18:22.587+0000 I NETWORK [conn4080775] end connection 10.0.179.209:40479 (1502 connections now open) 2016-03-13T09:18:22.588+0000 I NETWORK [conn4081688] end connection 10.0.26.69:42779 (1501 connections now open) 2016-03-13T09:18:22.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41049 #4082348 (1502 connections now open) 2016-03-13T09:18:22.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41050 #4082349 (1503 connections now open) 2016-03-13T09:18:22.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43074 #4082350 (1504 connections now open) 2016-03-13T09:18:22.588+0000 I NETWORK [conn4082216] end connection 10.0.216.90:57917 (1503 connections now open) 2016-03-13T09:18:22.589+0000 I NETWORK [conn4082009] end connection 10.0.7.113:59813 (1502 connections now open) 2016-03-13T09:18:22.589+0000 I NETWORK [conn4082051] end connection 10.0.7.113:59838 (1501 connections now open) 2016-03-13T09:18:22.589+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59952 #4082351 (1502 connections now open) 2016-03-13T09:18:22.589+0000 I NETWORK [conn4080830] end connection 10.0.179.209:40519 (1501 connections now open) 2016-03-13T09:18:22.589+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59953 #4082352 (1502 connections now open) 2016-03-13T09:18:22.590+0000 I NETWORK [conn4081922] end connection 10.0.26.69:42895 (1501 connections now open) 2016-03-13T09:18:22.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57967 #4082353 (1502 connections now open) 2016-03-13T09:18:22.590+0000 I NETWORK [conn4081347] end connection 10.0.20.53:50841 (1500 connections now open) 2016-03-13T09:18:22.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41052 #4082354 (1502 connections now open) 2016-03-13T09:18:22.590+0000 I NETWORK [conn4082049] end connection 10.0.7.113:59837 (1501 connections now open) 2016-03-13T09:18:22.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43075 #4082355 (1502 connections now open) 2016-03-13T09:18:22.591+0000 I NETWORK [conn4082094] end connection 10.0.7.113:59851 (1501 connections now open) 2016-03-13T09:18:22.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51316 #4082356 (1502 connections now open) 2016-03-13T09:18:22.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59955 #4082357 (1503 connections now open) 2016-03-13T09:18:22.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59956 #4082358 (1504 connections now open) 2016-03-13T09:18:22.591+0000 I NETWORK [conn4081073] end connection 10.0.216.90:57527 (1502 connections now open) 2016-03-13T09:18:22.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57968 #4082359 (1504 connections now open) 2016-03-13T09:18:22.592+0000 I NETWORK [conn4082117] end connection 10.0.26.69:42977 (1503 connections now open) 2016-03-13T09:18:22.592+0000 I NETWORK [conn4080910] end connection 10.0.179.209:40583 (1502 connections now open) 2016-03-13T09:18:22.592+0000 I NETWORK [conn4081005] end connection 10.0.7.113:59549 (1501 connections now open) 2016-03-13T09:18:22.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43077 #4082360 (1502 connections now open) 2016-03-13T09:18:22.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41053 #4082361 (1503 connections now open) 2016-03-13T09:18:22.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59957 #4082362 (1504 connections now open) 2016-03-13T09:18:22.594+0000 I NETWORK [conn4081774] end connection 10.0.20.53:51082 (1503 connections now open) 2016-03-13T09:18:22.595+0000 I NETWORK [conn4082125] end connection 10.0.7.113:59862 (1502 connections now open) 2016-03-13T09:18:22.595+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51318 #4082363 (1503 connections now open) 2016-03-13T09:18:22.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59959 #4082364 (1504 connections now open) 2016-03-13T09:18:22.596+0000 I NETWORK [conn4082202] end connection 10.0.20.53:51268 (1502 connections now open) 2016-03-13T09:18:22.596+0000 I NETWORK [conn4081726] end connection 10.0.27.237:36099 (1502 connections now open) 2016-03-13T09:18:22.596+0000 I NETWORK [conn4080724] end connection 10.0.216.90:57394 (1501 connections now open) 2016-03-13T09:18:22.597+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51320 #4082365 (1502 connections now open) 2016-03-13T09:18:22.597+0000 I NETWORK [conn4081226] end connection 10.0.7.113:59619 (1501 connections now open) 2016-03-13T09:18:22.597+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57970 #4082366 (1503 connections now open) 2016-03-13T09:18:22.597+0000 I NETWORK [conn4081175] end connection 10.0.26.69:42527 (1501 connections now open) 2016-03-13T09:18:22.597+0000 I NETWORK [conn4081516] end connection 10.0.26.69:42687 (1500 connections now open) 2016-03-13T09:18:22.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43080 #4082367 (1501 connections now open) 2016-03-13T09:18:22.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43081 #4082368 (1502 connections now open) 2016-03-13T09:18:22.598+0000 I NETWORK [conn4080726] end connection 10.0.216.90:57395 (1501 connections now open) 2016-03-13T09:18:22.598+0000 I NETWORK [conn4081237] end connection 10.0.27.237:35884 (1500 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [conn4080954] end connection 10.0.20.53:50697 (1499 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59963 #4082369 (1500 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [conn4082261] end connection 10.0.20.53:51289 (1498 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [conn4082000] end connection 10.0.27.237:36237 (1498 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57972 #4082370 (1500 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [conn4081736] end connection 10.0.27.237:36109 (1498 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51322 #4082371 (1499 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51323 #4082372 (1500 connections now open) 2016-03-13T09:18:22.599+0000 I NETWORK [conn4081414] end connection 10.0.20.53:50879 (1499 connections now open) 2016-03-13T09:18:22.600+0000 I NETWORK [conn4081075] end connection 10.0.179.209:40666 (1498 connections now open) 2016-03-13T09:18:22.600+0000 I NETWORK [conn4081601] end connection 10.0.26.69:42726 (1498 connections now open) 2016-03-13T09:18:22.600+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51324 #4082373 (1498 connections now open) 2016-03-13T09:18:22.600+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41057 #4082374 (1499 connections now open) 2016-03-13T09:18:22.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43084 #4082375 (1500 connections now open) 2016-03-13T09:18:22.602+0000 I NETWORK [conn4081596] end connection 10.0.179.209:40765 (1499 connections now open) 2016-03-13T09:18:22.602+0000 I NETWORK [conn4081445] end connection 10.0.27.237:35959 (1498 connections now open) 2016-03-13T09:18:22.602+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41058 #4082376 (1499 connections now open) 2016-03-13T09:18:22.602+0000 I NETWORK [conn4082201] end connection 10.0.216.90:57909 (1497 connections now open) 2016-03-13T09:18:22.604+0000 I NETWORK [conn4082233] end connection 10.0.216.90:57926 (1497 connections now open) 2016-03-13T09:18:22.604+0000 I NETWORK [conn4081194] end connection 10.0.26.69:42533 (1496 connections now open) 2016-03-13T09:18:22.604+0000 I NETWORK [conn4081930] end connection 10.0.216.90:57783 (1496 connections now open) 2016-03-13T09:18:22.604+0000 I NETWORK [conn4079260] end connection 10.0.179.209:38972 (1495 connections now open) 2016-03-13T09:18:22.604+0000 I NETWORK [conn4081606] end connection 10.0.27.237:36040 (1493 connections now open) 2016-03-13T09:18:22.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41060 #4082377 (1494 connections now open) 2016-03-13T09:18:22.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43085 #4082378 (1495 connections now open) 2016-03-13T09:18:22.605+0000 I NETWORK [conn4081187] end connection 10.0.27.237:35867 (1494 connections now open) 2016-03-13T09:18:22.605+0000 I NETWORK [conn4080744] end connection 10.0.216.90:57406 (1493 connections now open) 2016-03-13T09:18:22.605+0000 I NETWORK [conn4081079] end connection 10.0.26.69:42488 (1492 connections now open) 2016-03-13T09:18:22.605+0000 I NETWORK [conn4081124] end connection 10.0.216.90:57544 (1492 connections now open) 2016-03-13T09:18:22.605+0000 I NETWORK [conn4081842] end connection 10.0.27.237:36149 (1492 connections now open) 2016-03-13T09:18:22.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57975 #4082379 (1491 connections now open) 2016-03-13T09:18:22.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43087 #4082380 (1492 connections now open) 2016-03-13T09:18:22.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57976 #4082381 (1493 connections now open) 2016-03-13T09:18:22.606+0000 I NETWORK [conn4081430] end connection 10.0.26.69:42637 (1492 connections now open) 2016-03-13T09:18:22.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57977 #4082382 (1493 connections now open) 2016-03-13T09:18:22.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57979 #4082383 (1494 connections now open) 2016-03-13T09:18:22.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57981 #4082384 (1495 connections now open) 2016-03-13T09:18:22.608+0000 I NETWORK [conn4081490] end connection 10.0.20.53:50920 (1494 connections now open) 2016-03-13T09:18:22.608+0000 I NETWORK [conn4081251] end connection 10.0.216.90:57574 (1493 connections now open) 2016-03-13T09:18:22.608+0000 I NETWORK [conn4081707] end connection 10.0.20.53:51050 (1492 connections now open) 2016-03-13T09:18:22.608+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57982 #4082385 (1493 connections now open) 2016-03-13T09:18:22.609+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43089 #4082386 (1494 connections now open) 2016-03-13T09:18:22.609+0000 I NETWORK [conn4080734] end connection 10.0.216.90:57401 (1493 connections now open) 2016-03-13T09:18:22.610+0000 I NETWORK [conn4081458] end connection 10.0.27.237:35965 (1492 connections now open) 2016-03-13T09:18:22.610+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57985 #4082387 (1493 connections now open) 2016-03-13T09:18:22.611+0000 I NETWORK [conn4081506] end connection 10.0.26.69:42682 (1492 connections now open) 2016-03-13T09:18:22.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43093 #4082388 (1493 connections now open) 2016-03-13T09:18:22.612+0000 I NETWORK [conn4081141] end connection 10.0.20.53:50770 (1492 connections now open) 2016-03-13T09:18:22.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51332 #4082389 (1493 connections now open) 2016-03-13T09:18:22.613+0000 I NETWORK [conn4080930] end connection 10.0.7.113:59517 (1492 connections now open) 2016-03-13T09:18:22.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51333 #4082390 (1493 connections now open) 2016-03-13T09:18:22.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59968 #4082391 (1494 connections now open) 2016-03-13T09:18:22.614+0000 I NETWORK [conn4081129] end connection 10.0.216.90:57546 (1493 connections now open) 2016-03-13T09:18:22.614+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57987 #4082392 (1494 connections now open) 2016-03-13T09:18:22.615+0000 I NETWORK [conn4080934] end connection 10.0.20.53:50692 (1493 connections now open) 2016-03-13T09:18:22.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51334 #4082393 (1494 connections now open) 2016-03-13T09:18:22.615+0000 I NETWORK [conn4081539] end connection 10.0.26.69:42698 (1493 connections now open) 2016-03-13T09:18:22.615+0000 I NETWORK [conn4081338] end connection 10.0.216.90:57604 (1492 connections now open) 2016-03-13T09:18:22.616+0000 I NETWORK [conn4081444] end connection 10.0.26.69:42648 (1491 connections now open) 2016-03-13T09:18:22.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43096 #4082394 (1492 connections now open) 2016-03-13T09:18:22.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43097 #4082395 (1493 connections now open) 2016-03-13T09:18:22.617+0000 I NETWORK [conn4081400] end connection 10.0.20.53:50873 (1492 connections now open) 2016-03-13T09:18:22.617+0000 I NETWORK [conn4080777] end connection 10.0.179.209:40480 (1492 connections now open) 2016-03-13T09:18:22.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51336 #4082396 (1492 connections now open) 2016-03-13T09:18:22.618+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57991 #4082397 (1493 connections now open) 2016-03-13T09:18:22.618+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51337 #4082398 (1494 connections now open) 2016-03-13T09:18:22.619+0000 I NETWORK [conn4080525] end connection 10.0.179.209:40262 (1493 connections now open) 2016-03-13T09:18:22.620+0000 I NETWORK [conn4081168] end connection 10.0.216.90:57556 (1492 connections now open) 2016-03-13T09:18:22.620+0000 I NETWORK [conn4081181] end connection 10.0.216.90:57559 (1491 connections now open) 2016-03-13T09:18:22.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57992 #4082399 (1492 connections now open) 2016-03-13T09:18:22.620+0000 I NETWORK [conn4080736] end connection 10.0.216.90:57402 (1490 connections now open) 2016-03-13T09:18:22.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41067 #4082400 (1493 connections now open) 2016-03-13T09:18:22.622+0000 I NETWORK [conn4080939] end connection 10.0.20.53:50693 (1491 connections now open) 2016-03-13T09:18:22.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51341 #4082401 (1492 connections now open) 2016-03-13T09:18:22.622+0000 I NETWORK [conn4081084] end connection 10.0.26.69:42490 (1491 connections now open) 2016-03-13T09:18:22.623+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57993 #4082402 (1492 connections now open) 2016-03-13T09:18:22.623+0000 I NETWORK [conn4080958] end connection 10.0.20.53:50698 (1491 connections now open) 2016-03-13T09:18:22.623+0000 I NETWORK [conn4082170] end connection 10.0.216.90:57889 (1491 connections now open) 2016-03-13T09:18:22.624+0000 I NETWORK [conn4080756] end connection 10.0.216.90:57413 (1489 connections now open) 2016-03-13T09:18:22.624+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51343 #4082403 (1490 connections now open) 2016-03-13T09:18:22.624+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41068 #4082404 (1491 connections now open) 2016-03-13T09:18:22.624+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43099 #4082405 (1492 connections now open) 2016-03-13T09:18:22.625+0000 I NETWORK [conn4081029] end connection 10.0.20.53:50723 (1491 connections now open) 2016-03-13T09:18:22.626+0000 I NETWORK [conn4081440] end connection 10.0.27.237:35955 (1490 connections now open) 2016-03-13T09:18:22.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51345 #4082406 (1491 connections now open) 2016-03-13T09:18:22.626+0000 I NETWORK [conn4081283] end connection 10.0.26.69:42566 (1490 connections now open) 2016-03-13T09:18:22.626+0000 I NETWORK [conn4082139] end connection 10.0.179.209:40962 (1489 connections now open) 2016-03-13T09:18:22.627+0000 I NETWORK [conn4081167] end connection 10.0.26.69:42523 (1488 connections now open) 2016-03-13T09:18:22.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41069 #4082407 (1489 connections now open) 2016-03-13T09:18:22.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57996 #4082408 (1490 connections now open) 2016-03-13T09:18:22.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43100 #4082409 (1491 connections now open) 2016-03-13T09:18:22.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57997 #4082410 (1492 connections now open) 2016-03-13T09:18:22.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:57998 #4082411 (1493 connections now open) 2016-03-13T09:18:22.629+0000 I NETWORK [conn4081677] end connection 10.0.26.69:42773 (1492 connections now open) 2016-03-13T09:18:22.629+0000 I NETWORK [conn4081627] end connection 10.0.26.69:42741 (1491 connections now open) 2016-03-13T09:18:22.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43102 #4082412 (1492 connections now open) 2016-03-13T09:18:22.631+0000 I NETWORK [conn4082039] end connection 10.0.216.90:57831 (1491 connections now open) 2016-03-13T09:18:22.632+0000 I NETWORK [conn4082301] end connection 10.0.7.113:59933 (1490 connections now open) 2016-03-13T09:18:22.632+0000 I NETWORK [conn4080686] end connection 10.0.7.113:59353 (1489 connections now open) 2016-03-13T09:18:22.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59975 #4082413 (1490 connections now open) 2016-03-13T09:18:22.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43104 #4082414 (1491 connections now open) 2016-03-13T09:18:22.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58001 #4082415 (1492 connections now open) 2016-03-13T09:18:22.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43106 #4082416 (1493 connections now open) 2016-03-13T09:18:22.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59976 #4082417 (1494 connections now open) 2016-03-13T09:18:22.634+0000 I NETWORK [conn4082119] end connection 10.0.26.69:42978 (1493 connections now open) 2016-03-13T09:18:22.634+0000 I NETWORK [conn4081202] end connection 10.0.20.53:50790 (1492 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51349 #4082418 (1493 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36395 #4082419 (1494 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36397 #4082420 (1495 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36396 #4082421 (1496 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36398 #4082422 (1497 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43108 #4082423 (1498 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36399 #4082424 (1499 connections now open) 2016-03-13T09:18:22.635+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36401 #4082425 (1500 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36400 #4082426 (1501 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36402 #4082427 (1502 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [conn4081116] end connection 10.0.26.69:42504 (1501 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36403 #4082428 (1503 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36404 #4082429 (1503 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36405 #4082430 (1504 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36406 #4082431 (1505 connections now open) 2016-03-13T09:18:22.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43109 #4082432 (1506 connections now open) 2016-03-13T09:18:22.637+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36407 #4082433 (1507 connections now open) 2016-03-13T09:18:22.639+0000 I NETWORK [conn4080783] end connection 10.0.179.209:40483 (1506 connections now open) 2016-03-13T09:18:22.640+0000 I NETWORK [conn4080789] end connection 10.0.179.209:40485 (1505 connections now open) 2016-03-13T09:18:22.640+0000 I NETWORK [conn4080786] end connection 10.0.179.209:40484 (1504 connections now open) 2016-03-13T09:18:22.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41072 #4082434 (1505 connections now open) 2016-03-13T09:18:22.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41073 #4082435 (1506 connections now open) 2016-03-13T09:18:22.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41074 #4082436 (1507 connections now open) 2016-03-13T09:18:22.641+0000 I NETWORK [conn4081123] end connection 10.0.26.69:42506 (1505 connections now open) 2016-03-13T09:18:22.641+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 17048 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:18:22.641+0000 I NETWORK [conn4080854] end connection 10.0.179.209:40540 (1505 connections now open) 2016-03-13T09:18:22.642+0000 I NETWORK [conn4082253] end connection 10.0.179.209:41011 (1504 connections now open) 2016-03-13T09:18:22.642+0000 I NETWORK [conn4081661] end connection 10.0.27.237:36072 (1503 connections now open) 2016-03-13T09:18:22.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41075 #4082437 (1504 connections now open) 2016-03-13T09:18:22.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36408 #4082438 (1505 connections now open) 2016-03-13T09:18:22.643+0000 I NETWORK [conn4082264] end connection 10.0.179.209:41016 (1504 connections now open) 2016-03-13T09:18:22.643+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41076 #4082439 (1505 connections now open) 2016-03-13T09:18:22.643+0000 I NETWORK [conn4081416] end connection 10.0.20.53:50880 (1504 connections now open) 2016-03-13T09:18:22.644+0000 I NETWORK [conn4080965] end connection 10.0.179.209:40611 (1503 connections now open) 2016-03-13T09:18:22.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41077 #4082440 (1504 connections now open) 2016-03-13T09:18:22.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43112 #4082441 (1505 connections now open) 2016-03-13T09:18:22.644+0000 I NETWORK [conn4081419] end connection 10.0.20.53:50882 (1504 connections now open) 2016-03-13T09:18:22.645+0000 I NETWORK [conn4081534] end connection 10.0.26.69:42695 (1503 connections now open) 2016-03-13T09:18:22.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41078 #4082442 (1504 connections now open) 2016-03-13T09:18:22.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43113 #4082443 (1505 connections now open) 2016-03-13T09:18:22.645+0000 I NETWORK [conn4080565] end connection 10.0.179.209:40305 (1504 connections now open) 2016-03-13T09:18:22.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41079 #4082444 (1505 connections now open) 2016-03-13T09:18:22.646+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.ActionReports: 5ms sequenceNumber: 17049 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:18:22.648+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17050 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:18:22.648+0000 I NETWORK [conn4080973] end connection 10.0.20.53:50703 (1504 connections now open) 2016-03-13T09:18:22.648+0000 I NETWORK [conn4081837] end connection 10.0.7.113:59752 (1504 connections now open) 2016-03-13T09:18:22.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59980 #4082445 (1504 connections now open) 2016-03-13T09:18:22.648+0000 I NETWORK [conn4081329] end connection 10.0.26.69:42584 (1502 connections now open) 2016-03-13T09:18:22.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51352 #4082446 (1505 connections now open) 2016-03-13T09:18:22.649+0000 I NETWORK [conn4081839] end connection 10.0.26.69:42858 (1503 connections now open) 2016-03-13T09:18:22.649+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43115 #4082447 (1504 connections now open) 2016-03-13T09:18:22.650+0000 I NETWORK [conn4080446] end connection 10.0.179.209:40172 (1503 connections now open) 2016-03-13T09:18:22.650+0000 I NETWORK [conn4080749] end connection 10.0.216.90:57409 (1502 connections now open) 2016-03-13T09:18:22.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43116 #4082448 (1503 connections now open) 2016-03-13T09:18:22.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41080 #4082449 (1504 connections now open) 2016-03-13T09:18:22.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58005 #4082450 (1505 connections now open) 2016-03-13T09:18:22.650+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 17051 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:18:22.652+0000 I NETWORK [conn4081186] end connection 10.0.216.90:57560 (1504 connections now open) 2016-03-13T09:18:22.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58006 #4082451 (1505 connections now open) 2016-03-13T09:18:22.656+0000 I NETWORK [conn4082311] end connection 10.0.26.69:43053 (1504 connections now open) 2016-03-13T09:18:22.657+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43121 #4082452 (1505 connections now open) 2016-03-13T09:18:22.657+0000 I NETWORK [conn4081130] end connection 10.0.26.69:42508 (1504 connections now open) 2016-03-13T09:18:22.658+0000 I NETWORK [conn4081352] end connection 10.0.26.69:42598 (1503 connections now open) 2016-03-13T09:18:22.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43122 #4082453 (1504 connections now open) 2016-03-13T09:18:22.659+0000 I NETWORK [conn4080776] end connection 10.0.216.90:57426 (1503 connections now open) 2016-03-13T09:18:22.659+0000 I NETWORK [conn4081484] end connection 10.0.26.69:42668 (1502 connections now open) 2016-03-13T09:18:22.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58008 #4082454 (1503 connections now open) 2016-03-13T09:18:22.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43123 #4082455 (1504 connections now open) 2016-03-13T09:18:22.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43124 #4082456 (1505 connections now open) 2016-03-13T09:18:22.663+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.AppUsage: 12ms sequenceNumber: 17052 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:18:22.664+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 17053 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:18:22.667+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 2ms sequenceNumber: 17054 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:18:22.669+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51354 #4082457 (1506 connections now open) 2016-03-13T09:18:22.669+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51355 #4082458 (1507 connections now open) 2016-03-13T09:18:22.670+0000 I NETWORK [conn4081372] end connection 10.0.216.90:57614 (1506 connections now open) 2016-03-13T09:18:22.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58009 #4082459 (1507 connections now open) 2016-03-13T09:18:22.672+0000 I NETWORK [conn4081360] end connection 10.0.216.90:57613 (1506 connections now open) 2016-03-13T09:18:22.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58010 #4082460 (1507 connections now open) 2016-03-13T09:18:22.682+0000 I NETWORK [conn4082174] end connection 10.0.7.113:59882 (1506 connections now open) 2016-03-13T09:18:22.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59981 #4082461 (1507 connections now open) 2016-03-13T09:18:22.682+0000 I NETWORK [conn4080785] end connection 10.0.216.90:57430 (1506 connections now open) 2016-03-13T09:18:22.683+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58011 #4082462 (1507 connections now open) 2016-03-13T09:18:22.701+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.Clients: 33ms sequenceNumber: 17055 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:18:22.703+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 17056 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:18:22.705+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 17057 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:18:22.706+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 17058 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:18:22.707+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 17059 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:18:22.710+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db1m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:22.735+0000 I NETWORK [conn4081174] end connection 10.0.7.113:59603 (1506 connections now open) 2016-03-13T09:18:22.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59982 #4082463 (1507 connections now open) 2016-03-13T09:18:22.737+0000 I NETWORK [conn4081246] end connection 10.0.216.90:57573 (1506 connections now open) 2016-03-13T09:18:22.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58012 #4082464 (1507 connections now open) 2016-03-13T09:18:22.740+0000 I NETWORK [conn4081125] end connection 10.0.179.209:40685 (1506 connections now open) 2016-03-13T09:18:22.740+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41083 #4082465 (1507 connections now open) 2016-03-13T09:18:22.759+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 51ms sequenceNumber: 17060 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:18:22.762+0000 I NETWORK [conn4081421] end connection 10.0.20.53:50884 (1506 connections now open) 2016-03-13T09:18:22.762+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51359 #4082466 (1507 connections now open) 2016-03-13T09:18:22.770+0000 I NETWORK [conn4082151] end connection 10.0.26.69:42991 (1506 connections now open) 2016-03-13T09:18:22.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43127 #4082467 (1507 connections now open) 2016-03-13T09:18:22.774+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.CustomEventReports: 10ms sequenceNumber: 17061 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:18:22.774+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 17062 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:18:22.775+0000 I NETWORK [conn4080797] end connection 10.0.216.90:57436 (1506 connections now open) 2016-03-13T09:18:22.775+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58013 #4082468 (1507 connections now open) 2016-03-13T09:18:22.780+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 17063 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:18:22.784+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 17064 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:18:22.793+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.DurationReports: 8ms sequenceNumber: 17065 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:18:22.794+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17066 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:18:22.801+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.GestureReports: 7ms sequenceNumber: 17067 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:18:22.840+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 37ms sequenceNumber: 17068 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:18:22.861+0000 I NETWORK [conn4081435] end connection 10.0.20.53:50894 (1506 connections now open) 2016-03-13T09:18:22.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51360 #4082469 (1507 connections now open) 2016-03-13T09:18:23.038+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.HeatmapReports: 195ms sequenceNumber: 17069 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:18:23.091+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 41ms sequenceNumber: 17070 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:18:23.104+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.NavigationReports: 7ms sequenceNumber: 17071 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:18:23.106+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 17072 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:18:23.109+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 17073 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:18:23.111+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 17074 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:18:23.112+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 0ms sequenceNumber: 17075 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:18:23.114+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 17076 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:18:23.115+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 17077 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:18:23.116+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17078 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:18:23.476+0000 I NETWORK [conn4081406] end connection 10.0.216.90:57623 (1506 connections now open) 2016-03-13T09:18:23.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58014 #4082470 (1507 connections now open) 2016-03-13T09:18:23.898+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 782ms sequenceNumber: 17079 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:18:24.412+0000 I NETWORK [conn4082323] end connection 10.0.7.113:59942 (1506 connections now open) 2016-03-13T09:18:24.413+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59986 #4082471 (1507 connections now open) 2016-03-13T09:18:24.505+0000 I NETWORK [conn4082197] end connection 10.0.27.237:36327 (1506 connections now open) 2016-03-13T09:18:24.505+0000 I NETWORK [conn4082196] end connection 10.0.27.237:36326 (1506 connections now open) 2016-03-13T09:18:24.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36412 #4082472 (1506 connections now open) 2016-03-13T09:18:24.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36411 #4082473 (1507 connections now open) 2016-03-13T09:18:26.636+0000 I NETWORK [conn4082249] end connection 10.0.7.113:59913 (1506 connections now open) 2016-03-13T09:18:26.636+0000 I NETWORK [conn4082248] end connection 10.0.7.113:59912 (1506 connections now open) 2016-03-13T09:18:26.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59987 #4082474 (1506 connections now open) 2016-03-13T09:18:26.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59988 #4082475 (1507 connections now open) 2016-03-13T09:18:27.687+0000 I NETWORK [conn4082316] end connection 10.0.216.90:57951 (1506 connections now open) 2016-03-13T09:18:27.687+0000 I NETWORK [conn4082315] end connection 10.0.216.90:57952 (1506 connections now open) 2016-03-13T09:18:27.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58017 #4082476 (1506 connections now open) 2016-03-13T09:18:27.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58018 #4082477 (1507 connections now open) 2016-03-13T09:18:28.067+0000 I NETWORK [conn4081426] end connection 10.0.20.53:50886 (1506 connections now open) 2016-03-13T09:18:28.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51366 #4082478 (1507 connections now open) 2016-03-13T09:18:28.113+0000 I NETWORK [conn4081200] end connection 10.0.179.209:40702 (1506 connections now open) 2016-03-13T09:18:28.113+0000 I NETWORK [conn4081512] end connection 10.0.26.69:42685 (1505 connections now open) 2016-03-13T09:18:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41090 #4082479 (1506 connections now open) 2016-03-13T09:18:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43134 #4082480 (1507 connections now open) 2016-03-13T09:18:28.146+0000 I NETWORK [conn4080983] end connection 10.0.20.53:50706 (1506 connections now open) 2016-03-13T09:18:28.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51370 #4082481 (1507 connections now open) 2016-03-13T09:18:28.269+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.Sessions: 4291ms sequenceNumber: 17080 version: 30651|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:18:28.287+0000 I NETWORK [conn4082255] end connection 10.0.7.113:59915 (1506 connections now open) 2016-03-13T09:18:28.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59994 #4082482 (1507 connections now open) 2016-03-13T09:18:28.484+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 17081 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:18:28.492+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 8ms sequenceNumber: 17082 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:18:28.493+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17083 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:18:28.511+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 17ms sequenceNumber: 17084 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:18:28.514+0000 I NETWORK [conn4082280] end connection 10.0.7.113:59929 (1506 connections now open) 2016-03-13T09:18:28.514+0000 I NETWORK [conn4080759] end connection 10.0.216.90:57414 (1505 connections now open) 2016-03-13T09:18:28.514+0000 I NETWORK [conn4082266] end connection 10.0.27.237:36354 (1504 connections now open) 2016-03-13T09:18:28.515+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58021 #4082483 (1505 connections now open) 2016-03-13T09:18:28.515+0000 I NETWORK [conn4081716] end connection 10.0.20.53:51053 (1504 connections now open) 2016-03-13T09:18:28.515+0000 I NETWORK [conn4081006] end connection 10.0.179.209:40635 (1503 connections now open) 2016-03-13T09:18:28.516+0000 I NETWORK [conn4082090] end connection 10.0.179.209:40935 (1502 connections now open) 2016-03-13T09:18:28.517+0000 I NETWORK [conn4081664] end connection 10.0.27.237:36073 (1501 connections now open) 2016-03-13T09:18:28.518+0000 I NETWORK [conn4081709] end connection 10.0.26.69:42791 (1500 connections now open) 2016-03-13T09:18:28.518+0000 I NETWORK [conn4080815] end connection 10.0.179.209:40508 (1499 connections now open) 2016-03-13T09:18:28.519+0000 I NETWORK [conn4082167] end connection 10.0.27.237:36309 (1498 connections now open) 2016-03-13T09:18:28.519+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:28.519+0000 I NETWORK [conn4081845] end connection 10.0.27.237:36152 (1497 connections now open) 2016-03-13T09:18:28.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51371 #4082484 (1498 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41091 #4082485 (1499 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:59996 #4082486 (1500 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41093 #4082487 (1501 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43137 #4082488 (1502 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36429 #4082489 (1503 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36430 #4082490 (1504 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58022 #4082491 (1505 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36434 #4082492 (1506 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58023 #4082493 (1507 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [conn4081387] end connection 10.0.27.237:35933 (1506 connections now open) 2016-03-13T09:18:28.520+0000 I NETWORK [conn4080804] end connection 10.0.216.90:57442 (1499 connections now open) 2016-03-13T09:18:28.519+0000 I NETWORK [conn4081256] end connection 10.0.216.90:57576 (1496 connections now open) 2016-03-13T09:18:28.521+0000 I NETWORK [conn4080807] end connection 10.0.7.113:59425 (1503 connections now open) 2016-03-13T09:18:28.521+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:28.521+0000 I NETWORK [conn4082299] end connection 10.0.7.113:59931 (1502 connections now open) 2016-03-13T09:18:28.521+0000 I NETWORK [conn4080761] end connection 10.0.216.90:57415 (1501 connections now open) 2016-03-13T09:18:28.521+0000 I NETWORK [conn4081428] end connection 10.0.20.53:50888 (1500 connections now open) 2016-03-13T09:18:28.522+0000 I NETWORK [conn4081671] end connection 10.0.27.237:36076 (1499 connections now open) 2016-03-13T09:18:28.522+0000 I NETWORK [conn4081267] end connection 10.0.216.90:57578 (1499 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [conn4082370] end connection 10.0.216.90:57972 (1497 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36438 #4082494 (1498 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60000 #4082495 (1499 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60001 #4082496 (1500 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36439 #4082497 (1501 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58024 #4082498 (1502 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36440 #4082499 (1503 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58025 #4082500 (1504 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58026 #4082501 (1505 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [conn4080947] end connection 10.0.20.53:50695 (1504 connections now open) 2016-03-13T09:18:28.523+0000 I NETWORK [conn4082300] end connection 10.0.7.113:59932 (1503 connections now open) 2016-03-13T09:18:28.524+0000 I NETWORK [conn4080991] end connection 10.0.20.53:50709 (1502 connections now open) 2016-03-13T09:18:28.524+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:28.524+0000 I NETWORK [conn4081270] end connection 10.0.216.90:57579 (1501 connections now open) 2016-03-13T09:18:28.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60002 #4082502 (1502 connections now open) 2016-03-13T09:18:28.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51373 #4082503 (1503 connections now open) 2016-03-13T09:18:28.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51374 #4082504 (1504 connections now open) 2016-03-13T09:18:28.525+0000 I NETWORK [conn4082165] end connection 10.0.27.237:36307 (1503 connections now open) 2016-03-13T09:18:28.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36441 #4082505 (1504 connections now open) 2016-03-13T09:18:28.528+0000 I NETWORK [conn4081776] end connection 10.0.27.237:36129 (1503 connections now open) 2016-03-13T09:18:28.528+0000 I NETWORK [conn4080868] end connection 10.0.179.209:40552 (1502 connections now open) 2016-03-13T09:18:28.529+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36442 #4082506 (1503 connections now open) 2016-03-13T09:18:28.529+0000 I NETWORK [conn4081134] end connection 10.0.179.209:40687 (1502 connections now open) 2016-03-13T09:18:28.531+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58030 #4082507 (1503 connections now open) 2016-03-13T09:18:28.532+0000 I NETWORK [conn4081453] end connection 10.0.20.53:50902 (1502 connections now open) 2016-03-13T09:18:28.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41095 #4082508 (1503 connections now open) 2016-03-13T09:18:28.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51376 #4082509 (1504 connections now open) 2016-03-13T09:18:28.533+0000 I NETWORK [conn4080893] end connection 10.0.7.113:59497 (1503 connections now open) 2016-03-13T09:18:28.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41096 #4082510 (1504 connections now open) 2016-03-13T09:18:28.533+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60005 #4082511 (1505 connections now open) 2016-03-13T09:18:28.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41097 #4082512 (1506 connections now open) 2016-03-13T09:18:28.534+0000 I NETWORK [conn4081750] end connection 10.0.7.113:59717 (1505 connections now open) 2016-03-13T09:18:28.535+0000 I NETWORK [conn4081272] end connection 10.0.216.90:57580 (1504 connections now open) 2016-03-13T09:18:28.535+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60006 #4082513 (1505 connections now open) 2016-03-13T09:18:28.535+0000 I NETWORK [conn4080877] end connection 10.0.216.90:57485 (1504 connections now open) 2016-03-13T09:18:28.535+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58033 #4082514 (1505 connections now open) 2016-03-13T09:18:28.535+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58035 #4082515 (1506 connections now open) 2016-03-13T09:18:28.536+0000 I NETWORK [conn4081457] end connection 10.0.20.53:50903 (1505 connections now open) 2016-03-13T09:18:28.536+0000 I NETWORK [conn4081462] end connection 10.0.20.53:50905 (1504 connections now open) 2016-03-13T09:18:28.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51377 #4082516 (1505 connections now open) 2016-03-13T09:18:28.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51378 #4082517 (1506 connections now open) 2016-03-13T09:18:28.537+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51379 #4082518 (1507 connections now open) 2016-03-13T09:18:28.537+0000 I NETWORK [conn4081275] end connection 10.0.216.90:57581 (1506 connections now open) 2016-03-13T09:18:28.537+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58036 #4082519 (1507 connections now open) 2016-03-13T09:18:28.539+0000 I NETWORK [conn4081452] end connection 10.0.216.90:57636 (1506 connections now open) 2016-03-13T09:18:28.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58037 #4082520 (1507 connections now open) 2016-03-13T09:18:28.544+0000 I NETWORK [conn4080821] end connection 10.0.179.209:40512 (1506 connections now open) 2016-03-13T09:18:28.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41100 #4082521 (1507 connections now open) 2016-03-13T09:18:28.545+0000 I NETWORK [conn4081470] end connection 10.0.20.53:50909 (1506 connections now open) 2016-03-13T09:18:28.546+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51381 #4082522 (1507 connections now open) 2016-03-13T09:18:28.546+0000 I NETWORK [conn4081027] end connection 10.0.20.53:50722 (1506 connections now open) 2016-03-13T09:18:28.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51382 #4082523 (1507 connections now open) 2016-03-13T09:18:28.551+0000 I NETWORK [conn4080822] end connection 10.0.179.209:40514 (1506 connections now open) 2016-03-13T09:18:28.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41101 #4082524 (1507 connections now open) 2016-03-13T09:18:28.572+0000 I NETWORK [conn4080813] end connection 10.0.216.90:57451 (1506 connections now open) 2016-03-13T09:18:28.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58039 #4082525 (1507 connections now open) 2016-03-13T09:18:28.572+0000 I NETWORK [conn4082352] end connection 10.0.7.113:59953 (1506 connections now open) 2016-03-13T09:18:28.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60008 #4082526 (1507 connections now open) 2016-03-13T09:18:28.574+0000 I NETWORK [conn4082084] end connection 10.0.27.237:36259 (1506 connections now open) 2016-03-13T09:18:28.574+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36446 #4082527 (1507 connections now open) 2016-03-13T09:18:28.575+0000 I NETWORK [conn4082358] end connection 10.0.7.113:59956 (1506 connections now open) 2016-03-13T09:18:28.575+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60009 #4082528 (1507 connections now open) 2016-03-13T09:18:28.576+0000 I NETWORK [conn4081694] end connection 10.0.27.237:36084 (1506 connections now open) 2016-03-13T09:18:28.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36447 #4082529 (1507 connections now open) 2016-03-13T09:18:28.577+0000 I NETWORK [conn4082357] end connection 10.0.7.113:59955 (1506 connections now open) 2016-03-13T09:18:28.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60010 #4082530 (1507 connections now open) 2016-03-13T09:18:28.633+0000 I NETWORK [conn4081032] end connection 10.0.20.53:50724 (1506 connections now open) 2016-03-13T09:18:28.634+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51385 #4082531 (1507 connections now open) 2016-03-13T09:18:28.636+0000 I NETWORK [conn4081495] end connection 10.0.20.53:50924 (1506 connections now open) 2016-03-13T09:18:28.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51386 #4082532 (1507 connections now open) 2016-03-13T09:18:28.639+0000 I NETWORK [conn4082340] end connection 10.0.216.90:57963 (1506 connections now open) 2016-03-13T09:18:28.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58041 #4082533 (1507 connections now open) 2016-03-13T09:18:28.641+0000 I NETWORK [conn4081000] end connection 10.0.216.90:57512 (1506 connections now open) 2016-03-13T09:18:28.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58042 #4082534 (1507 connections now open) 2016-03-13T09:18:28.643+0000 I NETWORK [conn4080829] end connection 10.0.179.209:40518 (1506 connections now open) 2016-03-13T09:18:28.644+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41105 #4082535 (1507 connections now open) 2016-03-13T09:18:28.678+0000 I NETWORK [conn4081299] end connection 10.0.216.90:57587 (1506 connections now open) 2016-03-13T09:18:28.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58043 #4082536 (1507 connections now open) 2016-03-13T09:18:28.680+0000 I NETWORK [conn4081986] end connection 10.0.7.113:59811 (1506 connections now open) 2016-03-13T09:18:28.680+0000 I NETWORK [conn4081320] end connection 10.0.216.90:57597 (1505 connections now open) 2016-03-13T09:18:28.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60012 #4082537 (1506 connections now open) 2016-03-13T09:18:28.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58044 #4082538 (1507 connections now open) 2016-03-13T09:18:28.683+0000 I NETWORK [conn4081322] end connection 10.0.216.90:57598 (1506 connections now open) 2016-03-13T09:18:28.683+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58045 #4082539 (1507 connections now open) 2016-03-13T09:18:28.685+0000 I NETWORK [conn4082421] end connection 10.0.27.237:36396 (1506 connections now open) 2016-03-13T09:18:28.686+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36448 #4082540 (1507 connections now open) 2016-03-13T09:18:28.722+0000 I NETWORK [conn4082321] end connection 10.0.20.53:51302 (1506 connections now open) 2016-03-13T09:18:28.722+0000 I NETWORK [conn4082320] end connection 10.0.20.53:51303 (1506 connections now open) 2016-03-13T09:18:28.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51389 #4082541 (1506 connections now open) 2016-03-13T09:18:28.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51390 #4082542 (1507 connections now open) 2016-03-13T09:18:29.026+0000 I NETWORK [conn4080838] end connection 10.0.179.209:40527 (1506 connections now open) 2016-03-13T09:18:29.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41107 #4082543 (1507 connections now open) 2016-03-13T09:18:29.286+0000 I NETWORK [conn4081203] end connection 10.0.179.209:40703 (1506 connections now open) 2016-03-13T09:18:29.286+0000 I NETWORK [conn4082057] end connection 10.0.20.53:51203 (1505 connections now open) 2016-03-13T09:18:29.286+0000 I NETWORK [conn4080831] end connection 10.0.179.209:40520 (1505 connections now open) 2016-03-13T09:18:29.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41108 #4082544 (1505 connections now open) 2016-03-13T09:18:29.287+0000 I NETWORK [conn4080985] end connection 10.0.179.209:40623 (1504 connections now open) 2016-03-13T09:18:29.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41109 #4082545 (1505 connections now open) 2016-03-13T09:18:29.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41110 #4082546 (1506 connections now open) 2016-03-13T09:18:29.288+0000 I NETWORK [conn4081687] end connection 10.0.26.69:42778 (1505 connections now open) 2016-03-13T09:18:29.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43146 #4082547 (1506 connections now open) 2016-03-13T09:18:29.289+0000 I NETWORK [conn4081613] end connection 10.0.179.209:40770 (1505 connections now open) 2016-03-13T09:18:29.290+0000 I NETWORK [conn4080780] end connection 10.0.216.90:57428 (1504 connections now open) 2016-03-13T09:18:29.290+0000 I NETWORK [conn4081560] end connection 10.0.20.53:50958 (1503 connections now open) 2016-03-13T09:18:29.290+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41111 #4082548 (1504 connections now open) 2016-03-13T09:18:29.290+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51392 #4082549 (1505 connections now open) 2016-03-13T09:18:29.290+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58050 #4082550 (1506 connections now open) 2016-03-13T09:18:29.291+0000 I NETWORK [conn4080951] end connection 10.0.27.237:35768 (1505 connections now open) 2016-03-13T09:18:29.291+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36453 #4082551 (1506 connections now open) 2016-03-13T09:18:29.292+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51394 #4082552 (1507 connections now open) 2016-03-13T09:18:29.292+0000 I NETWORK [conn4081160] end connection 10.0.179.209:40695 (1506 connections now open) 2016-03-13T09:18:29.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41113 #4082553 (1507 connections now open) 2016-03-13T09:18:29.293+0000 I NETWORK [conn4082406] end connection 10.0.20.53:51345 (1506 connections now open) 2016-03-13T09:18:29.293+0000 I NETWORK [conn4081721] end connection 10.0.26.69:42799 (1505 connections now open) 2016-03-13T09:18:29.294+0000 I NETWORK [conn4081474] end connection 10.0.20.53:50911 (1504 connections now open) 2016-03-13T09:18:29.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43148 #4082554 (1505 connections now open) 2016-03-13T09:18:29.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51395 #4082555 (1506 connections now open) 2016-03-13T09:18:29.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51396 #4082556 (1507 connections now open) 2016-03-13T09:18:29.295+0000 I NETWORK [conn4082461] end connection 10.0.7.113:59981 (1506 connections now open) 2016-03-13T09:18:29.295+0000 I NETWORK [conn4080948] end connection 10.0.179.209:40602 (1505 connections now open) 2016-03-13T09:18:29.296+0000 I NETWORK [conn4081177] end connection 10.0.27.237:35864 (1504 connections now open) 2016-03-13T09:18:29.296+0000 I NETWORK [conn4081007] end connection 10.0.179.209:40634 (1503 connections now open) 2016-03-13T09:18:29.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41114 #4082557 (1504 connections now open) 2016-03-13T09:18:29.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36456 #4082558 (1505 connections now open) 2016-03-13T09:18:29.297+0000 I NETWORK [conn4080887] end connection 10.0.216.90:57489 (1504 connections now open) 2016-03-13T09:18:29.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60018 #4082559 (1505 connections now open) 2016-03-13T09:18:29.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41116 #4082560 (1506 connections now open) 2016-03-13T09:18:29.298+0000 I NETWORK [conn4082107] end connection 10.0.216.90:57861 (1505 connections now open) 2016-03-13T09:18:29.298+0000 I NETWORK [conn4081547] end connection 10.0.26.69:42701 (1504 connections now open) 2016-03-13T09:18:29.299+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43152 #4082561 (1505 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [conn4081058] end connection 10.0.27.237:35815 (1504 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [conn4080855] end connection 10.0.179.209:40541 (1503 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58052 #4082562 (1504 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58053 #4082563 (1505 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [conn4081543] end connection 10.0.26.69:42700 (1503 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36458 #4082564 (1505 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41117 #4082565 (1506 connections now open) 2016-03-13T09:18:29.300+0000 I NETWORK [conn4081465] end connection 10.0.26.69:42658 (1505 connections now open) 2016-03-13T09:18:29.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43153 #4082566 (1506 connections now open) 2016-03-13T09:18:29.302+0000 I NETWORK [conn4081052] end connection 10.0.179.209:40656 (1504 connections now open) 2016-03-13T09:18:29.302+0000 I NETWORK [conn4081142] end connection 10.0.179.209:40689 (1504 connections now open) 2016-03-13T09:18:29.302+0000 I NETWORK [conn4081548] end connection 10.0.26.69:42702 (1503 connections now open) 2016-03-13T09:18:29.303+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43154 #4082567 (1504 connections now open) 2016-03-13T09:18:29.303+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41119 #4082568 (1505 connections now open) 2016-03-13T09:18:29.303+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43155 #4082569 (1506 connections now open) 2016-03-13T09:18:29.303+0000 I NETWORK [conn4081002] end connection 10.0.20.53:50712 (1505 connections now open) 2016-03-13T09:18:29.303+0000 I NETWORK [conn4082001] end connection 10.0.216.90:57812 (1504 connections now open) 2016-03-13T09:18:29.304+0000 I NETWORK [conn4080743] end connection 10.0.7.113:59389 (1503 connections now open) 2016-03-13T09:18:29.304+0000 I NETWORK [conn4082471] end connection 10.0.7.113:59986 (1503 connections now open) 2016-03-13T09:18:29.304+0000 I NETWORK [conn4080791] end connection 10.0.216.90:57432 (1503 connections now open) 2016-03-13T09:18:29.304+0000 I NETWORK [conn4080961] end connection 10.0.179.209:40610 (1502 connections now open) 2016-03-13T09:18:29.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51397 #4082570 (1501 connections now open) 2016-03-13T09:18:29.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60021 #4082571 (1502 connections now open) 2016-03-13T09:18:29.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58055 #4082572 (1503 connections now open) 2016-03-13T09:18:29.305+0000 I NETWORK [conn4081718] end connection 10.0.26.69:42797 (1502 connections now open) 2016-03-13T09:18:29.306+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41121 #4082573 (1503 connections now open) 2016-03-13T09:18:29.306+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60023 #4082574 (1504 connections now open) 2016-03-13T09:18:29.306+0000 I NETWORK [conn4080793] end connection 10.0.216.90:57433 (1503 connections now open) 2016-03-13T09:18:29.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58056 #4082575 (1504 connections now open) 2016-03-13T09:18:29.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43157 #4082576 (1505 connections now open) 2016-03-13T09:18:29.307+0000 I NETWORK [conn4081505] end connection 10.0.27.237:35991 (1504 connections now open) 2016-03-13T09:18:29.307+0000 I NETWORK [conn4081328] end connection 10.0.216.90:57599 (1503 connections now open) 2016-03-13T09:18:29.307+0000 I NETWORK [conn4080871] end connection 10.0.216.90:57482 (1502 connections now open) 2016-03-13T09:18:29.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58057 #4082577 (1503 connections now open) 2016-03-13T09:18:29.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36459 #4082578 (1504 connections now open) 2016-03-13T09:18:29.308+0000 I NETWORK [conn4081377] end connection 10.0.216.90:57615 (1503 connections now open) 2016-03-13T09:18:29.308+0000 I NETWORK [conn4081483] end connection 10.0.27.237:35978 (1502 connections now open) 2016-03-13T09:18:29.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58058 #4082579 (1503 connections now open) 2016-03-13T09:18:29.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41122 #4082580 (1504 connections now open) 2016-03-13T09:18:29.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36461 #4082581 (1505 connections now open) 2016-03-13T09:18:29.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58060 #4082582 (1506 connections now open) 2016-03-13T09:18:29.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58061 #4082583 (1507 connections now open) 2016-03-13T09:18:29.309+0000 I NETWORK [conn4082413] end connection 10.0.7.113:59975 (1506 connections now open) 2016-03-13T09:18:29.310+0000 I NETWORK [conn4080891] end connection 10.0.7.113:59496 (1505 connections now open) 2016-03-13T09:18:29.310+0000 I NETWORK [conn4081082] end connection 10.0.20.53:50745 (1504 connections now open) 2016-03-13T09:18:29.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60025 #4082584 (1505 connections now open) 2016-03-13T09:18:29.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60026 #4082585 (1506 connections now open) 2016-03-13T09:18:29.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51404 #4082586 (1507 connections now open) 2016-03-13T09:18:29.311+0000 I NETWORK [conn4082059] end connection 10.0.26.69:42960 (1506 connections now open) 2016-03-13T09:18:29.311+0000 I NETWORK [conn4081557] end connection 10.0.26.69:42708 (1505 connections now open) 2016-03-13T09:18:29.312+0000 I NETWORK [conn4082458] end connection 10.0.20.53:51355 (1504 connections now open) 2016-03-13T09:18:29.312+0000 I NETWORK [conn4082359] end connection 10.0.216.90:57968 (1503 connections now open) 2016-03-13T09:18:29.312+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51405 #4082587 (1504 connections now open) 2016-03-13T09:18:29.312+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 17085 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:18:29.312+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58063 #4082588 (1505 connections now open) 2016-03-13T09:18:29.313+0000 I NETWORK [conn4081233] end connection 10.0.26.69:42544 (1504 connections now open) 2016-03-13T09:18:29.313+0000 I NETWORK [conn4081847] end connection 10.0.27.237:36154 (1503 connections now open) 2016-03-13T09:18:29.314+0000 I NETWORK [conn4081138] end connection 10.0.27.237:35848 (1502 connections now open) 2016-03-13T09:18:29.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36462 #4082589 (1503 connections now open) 2016-03-13T09:18:29.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43160 #4082590 (1504 connections now open) 2016-03-13T09:18:29.316+0000 I NETWORK [conn4081598] end connection 10.0.20.53:50976 (1503 connections now open) 2016-03-13T09:18:29.317+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ActionReports: 4ms sequenceNumber: 17086 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:18:29.317+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43161 #4082591 (1504 connections now open) 2016-03-13T09:18:29.318+0000 I NETWORK [conn4080721] end connection 10.0.7.113:59376 (1503 connections now open) 2016-03-13T09:18:29.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43162 #4082592 (1504 connections now open) 2016-03-13T09:18:29.318+0000 I NETWORK [conn4082417] end connection 10.0.7.113:59976 (1503 connections now open) 2016-03-13T09:18:29.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60031 #4082593 (1504 connections now open) 2016-03-13T09:18:29.318+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17087 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:18:29.319+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60033 #4082594 (1505 connections now open) 2016-03-13T09:18:29.320+0000 I NETWORK [conn4081868] end connection 10.0.179.209:40848 (1504 connections now open) 2016-03-13T09:18:29.320+0000 I NETWORK [conn4081088] end connection 10.0.179.209:40670 (1503 connections now open) 2016-03-13T09:18:29.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41125 #4082595 (1504 connections now open) 2016-03-13T09:18:29.321+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 17088 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:18:29.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51409 #4082596 (1505 connections now open) 2016-03-13T09:18:29.322+0000 I NETWORK [conn4081828] end connection 10.0.216.90:57733 (1504 connections now open) 2016-03-13T09:18:29.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41126 #4082597 (1506 connections now open) 2016-03-13T09:18:29.322+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 17089 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:18:29.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58069 #4082598 (1506 connections now open) 2016-03-13T09:18:29.323+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 17090 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:18:29.324+0000 I NETWORK [conn4080820] end connection 10.0.216.90:57456 (1505 connections now open) 2016-03-13T09:18:29.325+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58070 #4082599 (1506 connections now open) 2016-03-13T09:18:29.325+0000 I NETWORK [conn4081342] end connection 10.0.216.90:57605 (1505 connections now open) 2016-03-13T09:18:29.325+0000 I NETWORK [conn4081607] end connection 10.0.20.53:50980 (1504 connections now open) 2016-03-13T09:18:29.326+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58072 #4082600 (1505 connections now open) 2016-03-13T09:18:29.326+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 3ms sequenceNumber: 17091 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:18:29.326+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51411 #4082601 (1506 connections now open) 2016-03-13T09:18:29.328+0000 I NETWORK [conn4080856] end connection 10.0.179.209:40542 (1505 connections now open) 2016-03-13T09:18:29.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41128 #4082602 (1506 connections now open) 2016-03-13T09:18:29.329+0000 I NETWORK [conn4082258] end connection 10.0.26.69:43035 (1505 connections now open) 2016-03-13T09:18:29.329+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43163 #4082603 (1506 connections now open) 2016-03-13T09:18:29.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36467 #4082604 (1507 connections now open) 2016-03-13T09:18:29.333+0000 I NETWORK [conn4081795] end connection 10.0.216.90:57717 (1506 connections now open) 2016-03-13T09:18:29.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58074 #4082605 (1507 connections now open) 2016-03-13T09:18:29.338+0000 I NETWORK [conn4080888] end connection 10.0.179.209:40572 (1506 connections now open) 2016-03-13T09:18:29.338+0000 I NETWORK [conn4082285] end connection 10.0.27.237:36363 (1506 connections now open) 2016-03-13T09:18:29.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36472 #4082606 (1506 connections now open) 2016-03-13T09:18:29.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41129 #4082607 (1507 connections now open) 2016-03-13T09:18:29.342+0000 I NETWORK [conn4080840] end connection 10.0.179.209:40529 (1506 connections now open) 2016-03-13T09:18:29.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41131 #4082608 (1507 connections now open) 2016-03-13T09:18:29.347+0000 I NETWORK [conn4081509] end connection 10.0.20.53:50933 (1506 connections now open) 2016-03-13T09:18:29.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51412 #4082609 (1507 connections now open) 2016-03-13T09:18:29.349+0000 I NETWORK [conn4081700] end connection 10.0.27.237:36087 (1506 connections now open) 2016-03-13T09:18:29.350+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36476 #4082610 (1507 connections now open) 2016-03-13T09:18:29.352+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.Clients: 25ms sequenceNumber: 17092 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:18:29.352+0000 I NETWORK [conn4081824] end connection 10.0.27.237:36142 (1506 connections now open) 2016-03-13T09:18:29.353+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36477 #4082611 (1507 connections now open) 2016-03-13T09:18:29.355+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 17093 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:18:29.357+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 17094 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:18:29.358+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 17095 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:18:29.359+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 17096 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:18:29.386+0000 I NETWORK [conn4080723] end connection 10.0.7.113:59377 (1506 connections now open) 2016-03-13T09:18:29.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60034 #4082612 (1507 connections now open) 2016-03-13T09:18:29.400+0000 I NETWORK [conn4081332] end connection 10.0.216.90:57601 (1506 connections now open) 2016-03-13T09:18:29.400+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 41ms sequenceNumber: 17097 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:18:29.400+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58075 #4082613 (1507 connections now open) 2016-03-13T09:18:29.405+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CustomEventReports: 1ms sequenceNumber: 17098 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:18:29.406+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 17099 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:18:29.408+0000 I NETWORK [conn4081343] end connection 10.0.26.69:42593 (1506 connections now open) 2016-03-13T09:18:29.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43165 #4082614 (1507 connections now open) 2016-03-13T09:18:29.412+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 17100 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:18:29.416+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 17101 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:18:29.417+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 17102 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:18:29.418+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17103 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:18:29.422+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 17104 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:18:29.433+0000 I NETWORK [conn4082166] end connection 10.0.27.237:36308 (1506 connections now open) 2016-03-13T09:18:29.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36478 #4082615 (1507 connections now open) 2016-03-13T09:18:29.440+0000 I NETWORK [conn4081480] end connection 10.0.20.53:50914 (1506 connections now open) 2016-03-13T09:18:29.441+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51414 #4082616 (1507 connections now open) 2016-03-13T09:18:29.460+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 37ms sequenceNumber: 17105 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:18:29.495+0000 I NETWORK [conn4081009] end connection 10.0.179.209:40637 (1506 connections now open) 2016-03-13T09:18:29.495+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41134 #4082617 (1507 connections now open) 2016-03-13T09:18:29.526+0000 I NETWORK [conn4080843] end connection 10.0.179.209:40531 (1506 connections now open) 2016-03-13T09:18:29.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41135 #4082618 (1507 connections now open) 2016-03-13T09:18:29.577+0000 I NETWORK [conn4082401] end connection 10.0.20.53:51341 (1506 connections now open) 2016-03-13T09:18:29.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51415 #4082619 (1507 connections now open) 2016-03-13T09:18:29.621+0000 I NETWORK [conn4082502] end connection 10.0.7.113:60002 (1506 connections now open) 2016-03-13T09:18:29.622+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60035 #4082620 (1507 connections now open) 2016-03-13T09:18:29.660+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.HeatmapReports: 197ms sequenceNumber: 17106 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:18:29.723+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 50ms sequenceNumber: 17107 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:18:29.736+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.NavigationReports: 7ms sequenceNumber: 17108 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:18:29.738+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 17109 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:18:29.741+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 17110 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:18:29.742+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 17111 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:18:29.743+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 0ms sequenceNumber: 17112 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:18:29.746+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 17113 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:18:29.747+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 17114 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:18:29.748+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17115 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:18:29.871+0000 I NETWORK [conn4081615] end connection 10.0.20.53:50987 (1506 connections now open) 2016-03-13T09:18:29.872+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51416 #4082621 (1507 connections now open) 2016-03-13T09:18:29.902+0000 I NETWORK [conn4082537] end connection 10.0.7.113:60012 (1506 connections now open) 2016-03-13T09:18:29.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60036 #4082622 (1507 connections now open) 2016-03-13T09:18:29.904+0000 I NETWORK [conn4080800] end connection 10.0.216.90:57439 (1506 connections now open) 2016-03-13T09:18:29.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58078 #4082623 (1507 connections now open) 2016-03-13T09:18:30.356+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 607ms sequenceNumber: 17116 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:18:30.677+0000 I NETWORK [conn4081306] end connection 10.0.20.53:50825 (1506 connections now open) 2016-03-13T09:18:30.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51420 #4082624 (1507 connections now open) 2016-03-13T09:18:30.861+0000 I NETWORK [conn4081622] end connection 10.0.20.53:50992 (1506 connections now open) 2016-03-13T09:18:30.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51422 #4082625 (1507 connections now open) 2016-03-13T09:18:31.053+0000 I NETWORK [conn4082329] end connection 10.0.179.209:41042 (1506 connections now open) 2016-03-13T09:18:31.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41139 #4082626 (1507 connections now open) 2016-03-13T09:18:31.054+0000 I NETWORK [conn4082330] end connection 10.0.179.209:41041 (1506 connections now open) 2016-03-13T09:18:31.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41140 #4082627 (1507 connections now open) 2016-03-13T09:18:31.397+0000 I NETWORK [conn4080978] end connection 10.0.179.209:40619 (1506 connections now open) 2016-03-13T09:18:31.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41141 #4082628 (1507 connections now open) 2016-03-13T09:18:31.445+0000 I NETWORK [conn4081836] end connection 10.0.7.113:59751 (1506 connections now open) 2016-03-13T09:18:31.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60039 #4082629 (1507 connections now open) 2016-03-13T09:18:31.452+0000 I NETWORK [conn4081701] end connection 10.0.27.237:36088 (1506 connections now open) 2016-03-13T09:18:31.453+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36481 #4082630 (1507 connections now open) 2016-03-13T09:18:31.750+0000 I NETWORK [conn4082331] end connection 10.0.26.69:43064 (1506 connections now open) 2016-03-13T09:18:31.753+0000 I NETWORK [conn4082332] end connection 10.0.26.69:43065 (1505 connections now open) 2016-03-13T09:18:31.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43182 #4082631 (1506 connections now open) 2016-03-13T09:18:31.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43181 #4082632 (1507 connections now open) 2016-03-13T09:18:32.141+0000 I NETWORK [conn4081585] end connection 10.0.27.237:36027 (1506 connections now open) 2016-03-13T09:18:32.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36483 #4082633 (1507 connections now open) 2016-03-13T09:18:34.518+0000 I NETWORK [conn4082473] end connection 10.0.27.237:36411 (1506 connections now open) 2016-03-13T09:18:34.518+0000 I NETWORK [conn4082472] end connection 10.0.27.237:36412 (1506 connections now open) 2016-03-13T09:18:34.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36485 #4082634 (1506 connections now open) 2016-03-13T09:18:34.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36484 #4082635 (1507 connections now open) 2016-03-13T09:18:34.650+0000 I NETWORK [conn4080866] end connection 10.0.179.209:40550 (1506 connections now open) 2016-03-13T09:18:34.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41143 #4082636 (1507 connections now open) 2016-03-13T09:18:34.653+0000 W SHARDING [conn3430288] inconsistent chunks found when reloading mydomain.Sessions, previous version was 30652|1||563ba74d869758a4542b9075, this should be rare 2016-03-13T09:18:34.725+0000 I NETWORK [conn4082559] end connection 10.0.7.113:60018 (1506 connections now open) 2016-03-13T09:18:34.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60045 #4082637 (1507 connections now open) 2016-03-13T09:18:34.810+0000 W SHARDING [conn3430288] ChunkManager loaded an invalid config for mydomain.Sessions, trying again 2016-03-13T09:18:35.857+0000 I NETWORK [conn4081564] end connection 10.0.26.69:42711 (1506 connections now open) 2016-03-13T09:18:35.857+0000 I NETWORK [conn4081415] end connection 10.0.26.69:42630 (1505 connections now open) 2016-03-13T09:18:35.857+0000 I NETWORK [conn4080894] end connection 10.0.216.90:57491 (1504 connections now open) 2016-03-13T09:18:35.857+0000 I NETWORK [conn4082513] end connection 10.0.7.113:60006 (1504 connections now open) 2016-03-13T09:18:35.858+0000 I NETWORK [conn4081582] end connection 10.0.20.53:50968 (1502 connections now open) 2016-03-13T09:18:35.858+0000 I NETWORK [conn4080886] end connection 10.0.179.209:40571 (1502 connections now open) 2016-03-13T09:18:35.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58089 #4082638 (1502 connections now open) 2016-03-13T09:18:35.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60046 #4082639 (1503 connections now open) 2016-03-13T09:18:35.858+0000 I NETWORK [conn4081216] end connection 10.0.26.69:42538 (1502 connections now open) 2016-03-13T09:18:35.858+0000 I NETWORK [conn4080836] end connection 10.0.216.90:57465 (1501 connections now open) 2016-03-13T09:18:35.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51428 #4082640 (1502 connections now open) 2016-03-13T09:18:35.859+0000 I NETWORK [conn4082241] end connection 10.0.26.69:43033 (1501 connections now open) 2016-03-13T09:18:35.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58092 #4082641 (1502 connections now open) 2016-03-13T09:18:35.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43189 #4082642 (1503 connections now open) 2016-03-13T09:18:35.859+0000 I NETWORK [conn4081239] end connection 10.0.26.69:42547 (1502 connections now open) 2016-03-13T09:18:35.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41147 #4082643 (1503 connections now open) 2016-03-13T09:18:35.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43191 #4082644 (1504 connections now open) 2016-03-13T09:18:35.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43192 #4082645 (1505 connections now open) 2016-03-13T09:18:35.860+0000 I NETWORK [conn4080998] end connection 10.0.27.237:35785 (1503 connections now open) 2016-03-13T09:18:35.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43193 #4082646 (1505 connections now open) 2016-03-13T09:18:35.861+0000 I NETWORK [conn4081894] end connection 10.0.216.90:57770 (1504 connections now open) 2016-03-13T09:18:35.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43194 #4082647 (1506 connections now open) 2016-03-13T09:18:35.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36493 #4082648 (1506 connections now open) 2016-03-13T09:18:35.862+0000 I NETWORK [conn4082495] end connection 10.0.7.113:60000 (1505 connections now open) 2016-03-13T09:18:35.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58093 #4082649 (1506 connections now open) 2016-03-13T09:18:35.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60048 #4082650 (1507 connections now open) 2016-03-13T09:18:35.864+0000 I NETWORK [conn4082528] end connection 10.0.7.113:60009 (1506 connections now open) 2016-03-13T09:18:35.864+0000 I NETWORK [conn4081571] end connection 10.0.26.69:42715 (1505 connections now open) 2016-03-13T09:18:35.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60049 #4082651 (1506 connections now open) 2016-03-13T09:18:35.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43197 #4082652 (1507 connections now open) 2016-03-13T09:18:35.866+0000 I NETWORK [conn4081835] end connection 10.0.179.209:40819 (1506 connections now open) 2016-03-13T09:18:35.866+0000 I NETWORK [conn4081240] end connection 10.0.216.90:57571 (1505 connections now open) 2016-03-13T09:18:35.866+0000 I NETWORK [conn4082270] end connection 10.0.7.113:59924 (1504 connections now open) 2016-03-13T09:18:35.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41150 #4082653 (1505 connections now open) 2016-03-13T09:18:35.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58097 #4082654 (1506 connections now open) 2016-03-13T09:18:35.866+0000 I NETWORK [conn4081466] end connection 10.0.26.69:42659 (1505 connections now open) 2016-03-13T09:18:35.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60050 #4082655 (1507 connections now open) 2016-03-13T09:18:35.867+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43200 #4082656 (1507 connections now open) 2016-03-13T09:18:35.868+0000 I NETWORK [conn4080846] end connection 10.0.179.209:40533 (1506 connections now open) 2016-03-13T09:18:35.868+0000 I NETWORK [conn4081205] end connection 10.0.7.113:59613 (1505 connections now open) 2016-03-13T09:18:35.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41151 #4082657 (1507 connections now open) 2016-03-13T09:18:35.868+0000 I NETWORK [conn4081327] end connection 10.0.179.209:40732 (1505 connections now open) 2016-03-13T09:18:35.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60051 #4082658 (1506 connections now open) 2016-03-13T09:18:35.869+0000 I NETWORK [conn4081386] end connection 10.0.216.90:57618 (1505 connections now open) 2016-03-13T09:18:35.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41152 #4082659 (1506 connections now open) 2016-03-13T09:18:35.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58099 #4082660 (1507 connections now open) 2016-03-13T09:18:35.870+0000 I NETWORK [conn4082146] end connection 10.0.27.237:36297 (1506 connections now open) 2016-03-13T09:18:35.870+0000 I NETWORK [conn4081080] end connection 10.0.7.113:59572 (1505 connections now open) 2016-03-13T09:18:35.870+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36502 #4082661 (1506 connections now open) 2016-03-13T09:18:35.870+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60052 #4082662 (1507 connections now open) 2016-03-13T09:18:35.871+0000 I NETWORK [conn4075955] end connection 10.0.27.237:59141 (1506 connections now open) 2016-03-13T09:18:35.872+0000 I NETWORK [conn4081182] end connection 10.0.7.113:59605 (1505 connections now open) 2016-03-13T09:18:35.872+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36503 #4082663 (1506 connections now open) 2016-03-13T09:18:35.872+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60053 #4082664 (1507 connections now open) 2016-03-13T09:18:35.915+0000 I NETWORK [conn4082021] end connection 10.0.26.69:42932 (1506 connections now open) 2016-03-13T09:18:35.915+0000 I NETWORK [conn4082579] end connection 10.0.216.90:58058 (1506 connections now open) 2016-03-13T09:18:35.915+0000 I NETWORK [conn4081605] end connection 10.0.27.237:36039 (1504 connections now open) 2016-03-13T09:18:35.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43203 #4082665 (1505 connections now open) 2016-03-13T09:18:35.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58101 #4082666 (1506 connections now open) 2016-03-13T09:18:35.916+0000 I NETWORK [conn4080986] end connection 10.0.7.113:59542 (1505 connections now open) 2016-03-13T09:18:35.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36506 #4082667 (1506 connections now open) 2016-03-13T09:18:35.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60054 #4082668 (1507 connections now open) 2016-03-13T09:18:35.917+0000 I NETWORK [conn4081621] end connection 10.0.27.237:36052 (1506 connections now open) 2016-03-13T09:18:35.918+0000 I NETWORK [conn4082172] end connection 10.0.7.113:59880 (1505 connections now open) 2016-03-13T09:18:35.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36508 #4082669 (1506 connections now open) 2016-03-13T09:18:35.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60055 #4082670 (1507 connections now open) 2016-03-13T09:18:35.918+0000 I NETWORK [conn4081382] end connection 10.0.216.90:57617 (1506 connections now open) 2016-03-13T09:18:35.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58102 #4082671 (1507 connections now open) 2016-03-13T09:18:35.919+0000 I NETWORK [conn4081242] end connection 10.0.26.69:42549 (1506 connections now open) 2016-03-13T09:18:35.919+0000 I NETWORK [conn4081024] end connection 10.0.27.237:35800 (1505 connections now open) 2016-03-13T09:18:35.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43205 #4082672 (1506 connections now open) 2016-03-13T09:18:35.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36510 #4082673 (1507 connections now open) 2016-03-13T09:18:35.920+0000 I NETWORK [conn4080858] end connection 10.0.179.209:40543 (1506 connections now open) 2016-03-13T09:18:35.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41156 #4082674 (1507 connections now open) 2016-03-13T09:18:35.922+0000 I NETWORK [conn4081149] end connection 10.0.7.113:59591 (1506 connections now open) 2016-03-13T09:18:35.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60057 #4082675 (1507 connections now open) 2016-03-13T09:18:35.922+0000 I NETWORK [conn4081770] end connection 10.0.216.90:57710 (1506 connections now open) 2016-03-13T09:18:35.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58104 #4082676 (1507 connections now open) 2016-03-13T09:18:35.923+0000 I NETWORK [conn4080870] end connection 10.0.7.113:59476 (1506 connections now open) 2016-03-13T09:18:35.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60058 #4082677 (1507 connections now open) 2016-03-13T09:18:35.924+0000 I NETWORK [conn4081392] end connection 10.0.216.90:57620 (1506 connections now open) 2016-03-13T09:18:35.925+0000 I NETWORK [conn4081574] end connection 10.0.27.237:36023 (1505 connections now open) 2016-03-13T09:18:35.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58105 #4082678 (1506 connections now open) 2016-03-13T09:18:35.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36514 #4082679 (1507 connections now open) 2016-03-13T09:18:35.925+0000 I NETWORK [conn4081407] end connection 10.0.27.237:35942 (1506 connections now open) 2016-03-13T09:18:35.926+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36515 #4082680 (1507 connections now open) 2016-03-13T09:18:35.926+0000 I NETWORK [conn4080859] end connection 10.0.179.209:40545 (1506 connections now open) 2016-03-13T09:18:35.927+0000 I NETWORK [conn4077234] end connection 10.0.27.237:60418 (1505 connections now open) 2016-03-13T09:18:35.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41159 #4082681 (1507 connections now open) 2016-03-13T09:18:35.927+0000 I NETWORK [conn4082136] end connection 10.0.26.69:42985 (1505 connections now open) 2016-03-13T09:18:35.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36516 #4082682 (1506 connections now open) 2016-03-13T09:18:35.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43209 #4082683 (1507 connections now open) 2016-03-13T09:18:35.928+0000 I NETWORK [conn4081217] end connection 10.0.179.209:40707 (1506 connections now open) 2016-03-13T09:18:35.928+0000 I NETWORK [conn4082459] end connection 10.0.216.90:58009 (1505 connections now open) 2016-03-13T09:18:35.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41160 #4082684 (1506 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [conn4082304] end connection 10.0.27.237:36371 (1505 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [conn4081244] end connection 10.0.26.69:42550 (1504 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58107 #4082685 (1505 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [conn4080872] end connection 10.0.7.113:59480 (1504 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36518 #4082686 (1505 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43210 #4082687 (1506 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [conn4081708] end connection 10.0.27.237:36092 (1505 connections now open) 2016-03-13T09:18:35.929+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60061 #4082688 (1507 connections now open) 2016-03-13T09:18:35.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36519 #4082689 (1507 connections now open) 2016-03-13T09:18:35.930+0000 I NETWORK [conn4082180] end connection 10.0.26.69:42998 (1506 connections now open) 2016-03-13T09:18:35.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43211 #4082690 (1507 connections now open) 2016-03-13T09:18:35.932+0000 I NETWORK [conn4082071] end connection 10.0.179.209:40927 (1506 connections now open) 2016-03-13T09:18:35.932+0000 I NETWORK [conn4081273] end connection 10.0.26.69:42562 (1505 connections now open) 2016-03-13T09:18:35.932+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41162 #4082691 (1506 connections now open) 2016-03-13T09:18:35.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43212 #4082692 (1507 connections now open) 2016-03-13T09:18:35.933+0000 I NETWORK [conn4080945] end connection 10.0.27.237:35766 (1506 connections now open) 2016-03-13T09:18:35.933+0000 I NETWORK [conn4082169] end connection 10.0.27.237:36312 (1505 connections now open) 2016-03-13T09:18:35.933+0000 I NETWORK [conn4082379] end connection 10.0.216.90:57975 (1505 connections now open) 2016-03-13T09:18:35.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36521 #4082693 (1505 connections now open) 2016-03-13T09:18:35.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36522 #4082694 (1506 connections now open) 2016-03-13T09:18:35.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58109 #4082695 (1507 connections now open) 2016-03-13T09:18:35.934+0000 I NETWORK [conn4080892] end connection 10.0.179.209:40574 (1506 connections now open) 2016-03-13T09:18:35.934+0000 I NETWORK [conn4081278] end connection 10.0.26.69:42564 (1505 connections now open) 2016-03-13T09:18:35.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41163 #4082696 (1506 connections now open) 2016-03-13T09:18:35.935+0000 I NETWORK [conn4081154] end connection 10.0.7.113:59593 (1504 connections now open) 2016-03-13T09:18:35.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43213 #4082697 (1506 connections now open) 2016-03-13T09:18:35.935+0000 I NETWORK [conn4081937] end connection 10.0.27.237:36198 (1504 connections now open) 2016-03-13T09:18:35.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60064 #4082698 (1507 connections now open) 2016-03-13T09:18:35.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36523 #4082699 (1507 connections now open) 2016-03-13T09:18:35.936+0000 I NETWORK [conn4080882] end connection 10.0.7.113:59491 (1506 connections now open) 2016-03-13T09:18:35.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60065 #4082700 (1507 connections now open) 2016-03-13T09:18:35.937+0000 I NETWORK [conn4081481] end connection 10.0.27.237:35977 (1506 connections now open) 2016-03-13T09:18:35.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36525 #4082701 (1507 connections now open) 2016-03-13T09:18:35.938+0000 I NETWORK [conn4081618] end connection 10.0.26.69:42736 (1506 connections now open) 2016-03-13T09:18:35.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43215 #4082702 (1507 connections now open) 2016-03-13T09:18:35.939+0000 I NETWORK [conn4081529] end connection 10.0.27.237:36002 (1506 connections now open) 2016-03-13T09:18:35.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36527 #4082703 (1507 connections now open) 2016-03-13T09:18:35.939+0000 I NETWORK [conn4081170] end connection 10.0.27.237:35860 (1506 connections now open) 2016-03-13T09:18:35.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36528 #4082704 (1507 connections now open) 2016-03-13T09:18:35.940+0000 I NETWORK [conn4080984] end connection 10.0.7.113:59541 (1506 connections now open) 2016-03-13T09:18:35.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60067 #4082705 (1507 connections now open) 2016-03-13T09:18:35.942+0000 I NETWORK [conn4081035] end connection 10.0.27.237:35805 (1506 connections now open) 2016-03-13T09:18:35.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36530 #4082706 (1507 connections now open) 2016-03-13T09:18:35.943+0000 I NETWORK [conn4081010] end connection 10.0.27.237:35792 (1506 connections now open) 2016-03-13T09:18:35.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36531 #4082707 (1507 connections now open) 2016-03-13T09:18:35.945+0000 I NETWORK [conn4081019] end connection 10.0.26.69:42465 (1506 connections now open) 2016-03-13T09:18:35.945+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43218 #4082708 (1507 connections now open) 2016-03-13T09:18:35.946+0000 I NETWORK [conn4081070] end connection 10.0.27.237:35821 (1506 connections now open) 2016-03-13T09:18:35.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36533 #4082709 (1507 connections now open) 2016-03-13T09:18:35.946+0000 I NETWORK [conn4080896] end connection 10.0.179.209:40575 (1506 connections now open) 2016-03-13T09:18:35.946+0000 I NETWORK [conn4081293] end connection 10.0.26.69:42571 (1505 connections now open) 2016-03-13T09:18:35.947+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41169 #4082710 (1506 connections now open) 2016-03-13T09:18:35.947+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43219 #4082711 (1507 connections now open) 2016-03-13T09:18:36.459+0000 W SHARDING [conn3430288] got invalid chunk version 1|2496||563ba74d869758a4542b9075 in document { _id: "mydomain.Sessions-a_ObjectId('5460a5dd645d051108dffbe9')_id_ObjectId('55b2239e7adcc50aa5443b7e')", ns: "mydomain.Sessions", min: { a: ObjectId('5460a5dd645d051108dffbe9'), _id: ObjectId('55b2239e7adcc50aa5443b7e') }, max: { a: ObjectId('5460a5dd645d051108dffbe9'), _id: ObjectId('55b278fecf4c3027ff8ed02c') }, shard: "shmydomain1", lastmod: Timestamp 1000|2496, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } when trying to load differing chunks at version 0|0||000000000000000000000000 2016-03-13T09:18:36.459+0000 W SHARDING [conn3430288] major change in chunk information found when reloading mydomain.Sessions, previous version was 0|0||000000000000000000000000 2016-03-13T09:18:36.484+0000 I NETWORK [conn4081456] end connection 10.0.216.90:57637 (1506 connections now open) 2016-03-13T09:18:36.484+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58115 #4082712 (1507 connections now open) 2016-03-13T09:18:36.522+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.Sessions: 1701ms sequenceNumber: 17117 version: 0|0||000000000000000000000000 based on: (empty) 2016-03-13T09:18:36.522+0000 W SHARDING [conn3430288] no chunks found for collection mydomain.Sessions, assuming unsharded 2016-03-13T09:18:36.658+0000 I NETWORK [conn4082475] end connection 10.0.7.113:59988 (1506 connections now open) 2016-03-13T09:18:36.658+0000 I NETWORK [conn4082474] end connection 10.0.7.113:59987 (1506 connections now open) 2016-03-13T09:18:36.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60070 #4082713 (1506 connections now open) 2016-03-13T09:18:36.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60071 #4082714 (1507 connections now open) 2016-03-13T09:18:36.688+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ShardTest: 30ms sequenceNumber: 17118 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:18:36.689+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 17119 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:18:36.690+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17120 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:18:36.715+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 24ms sequenceNumber: 17121 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:18:36.718+0000 I NETWORK [conn4081523] end connection 10.0.7.113:59693 (1506 connections now open) 2016-03-13T09:18:36.719+0000 I NETWORK [conn4076589] end connection 10.0.216.90:53139 (1505 connections now open) 2016-03-13T09:18:36.719+0000 I NETWORK [conn4081600] end connection 10.0.26.69:42725 (1504 connections now open) 2016-03-13T09:18:36.719+0000 I NETWORK [conn4082269] end connection 10.0.7.113:59923 (1503 connections now open) 2016-03-13T09:18:36.719+0000 I NETWORK [conn4080873] end connection 10.0.7.113:59481 (1503 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [conn4081662] end connection 10.0.20.53:51022 (1501 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60072 #4082715 (1503 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [conn4081748] end connection 10.0.26.69:42821 (1501 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58116 #4082716 (1503 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43220 #4082717 (1503 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60073 #4082718 (1504 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60074 #4082719 (1505 connections now open) 2016-03-13T09:18:36.720+0000 I NETWORK [conn4074300] end connection 10.0.216.90:50655 (1504 connections now open) 2016-03-13T09:18:36.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58117 #4082720 (1505 connections now open) 2016-03-13T09:18:36.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43221 #4082721 (1506 connections now open) 2016-03-13T09:18:36.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51430 #4082722 (1507 connections now open) 2016-03-13T09:18:36.721+0000 I NETWORK [conn4081339] end connection 10.0.20.53:50838 (1506 connections now open) 2016-03-13T09:18:36.722+0000 I NETWORK [conn4081486] end connection 10.0.27.237:35980 (1505 connections now open) 2016-03-13T09:18:36.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51431 #4082723 (1506 connections now open) 2016-03-13T09:18:36.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36535 #4082724 (1507 connections now open) 2016-03-13T09:18:36.723+0000 I NETWORK [conn4074379] end connection 10.0.216.90:50745 (1506 connections now open) 2016-03-13T09:18:36.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58118 #4082725 (1507 connections now open) 2016-03-13T09:18:36.725+0000 W SHARDING [Balancer] chunk manager not found for mydomain.Sessions :: caused by :: 10181 not sharded:mydomain.Sessions 2016-03-13T09:18:36.726+0000 I NETWORK [conn2756822] end connection 10.0.217.145:54141 (1506 connections now open) 2016-03-13T09:18:36.727+0000 I NETWORK [conn4081249] end connection 10.0.27.237:35888 (1505 connections now open) 2016-03-13T09:18:36.727+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36537 #4082726 (1506 connections now open) 2016-03-13T09:18:36.731+0000 I NETWORK [conn4075531] end connection 10.0.216.90:51995 (1505 connections now open) 2016-03-13T09:18:36.731+0000 I NETWORK [conn4080364] end connection 10.0.27.237:35383 (1504 connections now open) 2016-03-13T09:18:36.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58119 #4082727 (1505 connections now open) 2016-03-13T09:18:36.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36539 #4082728 (1506 connections now open) 2016-03-13T09:18:36.731+0000 I NETWORK [conn4081469] end connection 10.0.27.237:35970 (1505 connections now open) 2016-03-13T09:18:36.732+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36540 #4082729 (1506 connections now open) 2016-03-13T09:18:36.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.217.145:55678 #4082730 (1507 connections now open) 2016-03-13T09:18:36.739+0000 I NETWORK [conn3552645] end connection 10.0.217.145:57902 (1506 connections now open) 2016-03-13T09:18:36.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.217.145:55680 #4082731 (1507 connections now open) 2016-03-13T09:18:36.740+0000 I NETWORK [conn4081449] end connection 10.0.27.237:35961 (1506 connections now open) 2016-03-13T09:18:36.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36545 #4082732 (1507 connections now open) 2016-03-13T09:18:36.742+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-3-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:18:36.752+0000 I NETWORK [conn4080890] end connection 10.0.179.209:40573 (1506 connections now open) 2016-03-13T09:18:36.752+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41177 #4082733 (1507 connections now open) 2016-03-13T09:18:36.754+0000 I NETWORK [conn4081511] end connection 10.0.27.237:35994 (1506 connections now open) 2016-03-13T09:18:36.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36546 #4082734 (1507 connections now open) 2016-03-13T09:18:36.764+0000 I NETWORK [conn4081628] end connection 10.0.20.53:50997 (1506 connections now open) 2016-03-13T09:18:36.765+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51437 #4082735 (1507 connections now open) 2016-03-13T09:18:36.838+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.AccountUsage: 3ms sequenceNumber: 17122 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:18:36.852+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:18:36.852+0000-56e5306c18fa188537623d6b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457860716852), what: "balancer.round", ns: "", details: { executionTimeMillis: 18402, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:18:36.855+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.ActionReports: 17ms sequenceNumber: 17123 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:18:36.868+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.AppReports: 13ms sequenceNumber: 17124 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:18:36.874+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.AppScreenReports: 5ms sequenceNumber: 17125 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:18:36.883+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.AppUsage: 8ms sequenceNumber: 17126 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:18:36.886+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.AppVersionReports: 3ms sequenceNumber: 17127 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:18:36.894+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 8ms sequenceNumber: 17128 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:18:36.987+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.Clients: 92ms sequenceNumber: 17129 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:18:36.990+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 17130 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:18:37.000+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.CountryReports: 9ms sequenceNumber: 17131 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:18:37.001+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 1ms sequenceNumber: 17132 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:18:37.002+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 17133 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:18:37.144+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 142ms sequenceNumber: 17134 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:18:37.145+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:18:37.154+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.CustomEventReports: 6ms sequenceNumber: 17135 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:18:37.163+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 8ms sequenceNumber: 17136 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:18:37.191+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 27ms sequenceNumber: 17137 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:18:37.205+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.DeviceReports: 13ms sequenceNumber: 17138 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:18:37.255+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.DurationReports: 49ms sequenceNumber: 17139 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:18:37.256+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17140 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:18:37.260+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 17141 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:18:37.346+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 85ms sequenceNumber: 17142 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:18:37.674+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.HeatmapReports: 325ms sequenceNumber: 17143 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:18:37.694+0000 I NETWORK [conn4082691] end connection 10.0.179.209:41162 (1506 connections now open) 2016-03-13T09:18:37.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41178 #4082736 (1507 connections now open) 2016-03-13T09:18:37.700+0000 I NETWORK [conn4082476] end connection 10.0.216.90:58017 (1506 connections now open) 2016-03-13T09:18:37.700+0000 I NETWORK [conn4082477] end connection 10.0.216.90:58018 (1506 connections now open) 2016-03-13T09:18:37.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58121 #4082737 (1506 connections now open) 2016-03-13T09:18:37.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58122 #4082738 (1507 connections now open) 2016-03-13T09:18:37.747+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 59ms sequenceNumber: 17144 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:18:37.763+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.NavigationReports: 11ms sequenceNumber: 17145 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:18:37.766+0000 I NETWORK [conn4082482] end connection 10.0.7.113:59994 (1506 connections now open) 2016-03-13T09:18:37.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60078 #4082739 (1507 connections now open) 2016-03-13T09:18:37.768+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.OSVersionReports: 4ms sequenceNumber: 17146 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:18:37.777+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.PendingSessions: 8ms sequenceNumber: 17147 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:18:37.828+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.ResolutionReports: 51ms sequenceNumber: 17148 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:18:37.831+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 2ms sequenceNumber: 17149 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:18:37.835+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.RetentionReports: 3ms sequenceNumber: 17150 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:18:37.849+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 13ms sequenceNumber: 17151 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:18:37.855+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 5ms sequenceNumber: 17152 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:18:38.075+0000 I NETWORK [conn4080972] end connection 10.0.216.90:57511 (1506 connections now open) 2016-03-13T09:18:38.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58123 #4082740 (1507 connections now open) 2016-03-13T09:18:38.735+0000 I NETWORK [conn4082541] end connection 10.0.20.53:51389 (1506 connections now open) 2016-03-13T09:18:38.735+0000 I NETWORK [conn4082542] end connection 10.0.20.53:51390 (1506 connections now open) 2016-03-13T09:18:38.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51450 #4082741 (1506 connections now open) 2016-03-13T09:18:38.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51451 #4082742 (1507 connections now open) 2016-03-13T09:18:39.106+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 1251ms sequenceNumber: 17153 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:18:39.959+0000 I NETWORK [conn4081039] end connection 10.0.27.237:35807 (1506 connections now open) 2016-03-13T09:18:39.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36552 #4082743 (1507 connections now open) 2016-03-13T09:18:40.600+0000 I NETWORK [conn4081737] end connection 10.0.179.209:40774 (1506 connections now open) 2016-03-13T09:18:40.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41183 #4082744 (1507 connections now open) 2016-03-13T09:18:41.066+0000 I NETWORK [conn4082626] end connection 10.0.179.209:41139 (1506 connections now open) 2016-03-13T09:18:41.066+0000 I NETWORK [conn4082627] end connection 10.0.179.209:41140 (1505 connections now open) 2016-03-13T09:18:41.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41185 #4082745 (1506 connections now open) 2016-03-13T09:18:41.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41186 #4082746 (1507 connections now open) 2016-03-13T09:18:41.771+0000 I NETWORK [conn4082631] end connection 10.0.26.69:43182 (1506 connections now open) 2016-03-13T09:18:41.771+0000 I NETWORK [conn4082632] end connection 10.0.26.69:43181 (1505 connections now open) 2016-03-13T09:18:41.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43226 #4082747 (1506 connections now open) 2016-03-13T09:18:41.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43227 #4082748 (1507 connections now open) 2016-03-13T09:18:42.079+0000 I NETWORK [conn4081133] end connection 10.0.20.53:50765 (1506 connections now open) 2016-03-13T09:18:42.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51459 #4082749 (1507 connections now open) 2016-03-13T09:18:42.678+0000 I NETWORK [conn4080725] end connection 10.0.7.113:59378 (1506 connections now open) 2016-03-13T09:18:42.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60084 #4082750 (1507 connections now open) 2016-03-13T09:18:42.695+0000 I NETWORK [conn4078967] end connection 10.0.216.90:55653 (1506 connections now open) 2016-03-13T09:18:42.696+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58131 #4082751 (1507 connections now open) 2016-03-13T09:18:42.847+0000 I NETWORK [conn4082182] end connection 10.0.27.237:36316 (1506 connections now open) 2016-03-13T09:18:42.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36555 #4082752 (1507 connections now open) 2016-03-13T09:18:44.177+0000 I NETWORK [conn4081526] end connection 10.0.20.53:50941 (1506 connections now open) 2016-03-13T09:18:44.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51462 #4082753 (1507 connections now open) 2016-03-13T09:18:44.532+0000 I NETWORK [conn4082634] end connection 10.0.27.237:36485 (1506 connections now open) 2016-03-13T09:18:44.532+0000 I NETWORK [conn4082635] end connection 10.0.27.237:36484 (1506 connections now open) 2016-03-13T09:18:44.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36556 #4082754 (1506 connections now open) 2016-03-13T09:18:44.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36557 #4082755 (1507 connections now open) 2016-03-13T09:18:45.095+0000 I NETWORK [conn4082371] end connection 10.0.20.53:51322 (1506 connections now open) 2016-03-13T09:18:45.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51464 #4082756 (1507 connections now open) 2016-03-13T09:18:46.050+0000 I NETWORK [conn4081876] end connection 10.0.216.90:57763 (1506 connections now open) 2016-03-13T09:18:46.050+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58142 #4082757 (1507 connections now open) 2016-03-13T09:18:46.121+0000 I NETWORK [conn4074860] end connection 10.0.216.90:51266 (1506 connections now open) 2016-03-13T09:18:46.121+0000 I NETWORK [conn4082447] end connection 10.0.26.69:43115 (1506 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [conn4081331] end connection 10.0.27.237:35911 (1506 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [conn4080823] end connection 10.0.7.113:59441 (1505 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [conn4081592] end connection 10.0.179.209:40764 (1506 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41199 #4082758 (1503 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58145 #4082759 (1504 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43233 #4082760 (1505 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36562 #4082761 (1506 connections now open) 2016-03-13T09:18:46.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60098 #4082762 (1507 connections now open) 2016-03-13T09:18:46.672+0000 I NETWORK [conn4082714] end connection 10.0.7.113:60071 (1506 connections now open) 2016-03-13T09:18:46.672+0000 I NETWORK [conn4082713] end connection 10.0.7.113:60070 (1506 connections now open) 2016-03-13T09:18:46.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60100 #4082763 (1506 connections now open) 2016-03-13T09:18:46.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60099 #4082764 (1507 connections now open) 2016-03-13T09:18:47.150+0000 I NETWORK [conn4080741] end connection 10.0.7.113:59387 (1506 connections now open) 2016-03-13T09:18:47.150+0000 I NETWORK [conn4080841] end connection 10.0.216.90:57467 (1505 connections now open) 2016-03-13T09:18:47.150+0000 I NETWORK [conn4081053] end connection 10.0.7.113:59566 (1504 connections now open) 2016-03-13T09:18:47.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58146 #4082765 (1505 connections now open) 2016-03-13T09:18:47.151+0000 I NETWORK [conn4080762] end connection 10.0.7.113:59398 (1504 connections now open) 2016-03-13T09:18:47.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60101 #4082766 (1505 connections now open) 2016-03-13T09:18:47.152+0000 I NETWORK [conn4082066] end connection 10.0.26.69:42965 (1504 connections now open) 2016-03-13T09:18:47.152+0000 I NETWORK [conn4082364] end connection 10.0.7.113:59959 (1503 connections now open) 2016-03-13T09:18:47.152+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43235 #4082767 (1504 connections now open) 2016-03-13T09:18:47.152+0000 I NETWORK [conn4080845] end connection 10.0.216.90:57469 (1503 connections now open) 2016-03-13T09:18:47.152+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60102 #4082768 (1504 connections now open) 2016-03-13T09:18:47.152+0000 I NETWORK [conn4080902] end connection 10.0.179.209:40580 (1503 connections now open) 2016-03-13T09:18:47.152+0000 I NETWORK [conn4076374] end connection 10.0.216.90:52913 (1503 connections now open) 2016-03-13T09:18:47.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60104 #4082769 (1503 connections now open) 2016-03-13T09:18:47.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60105 #4082770 (1504 connections now open) 2016-03-13T09:18:47.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41203 #4082771 (1505 connections now open) 2016-03-13T09:18:47.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58147 #4082772 (1506 connections now open) 2016-03-13T09:18:47.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58148 #4082773 (1507 connections now open) 2016-03-13T09:18:47.156+0000 I NETWORK [conn4080853] end connection 10.0.216.90:57473 (1506 connections now open) 2016-03-13T09:18:47.156+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58149 #4082774 (1507 connections now open) 2016-03-13T09:18:47.157+0000 I NETWORK [conn4081872] end connection 10.0.216.90:57761 (1506 connections now open) 2016-03-13T09:18:47.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58150 #4082775 (1507 connections now open) 2016-03-13T09:18:47.160+0000 I NETWORK [conn4081165] end connection 10.0.27.237:35858 (1506 connections now open) 2016-03-13T09:18:47.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36570 #4082776 (1507 connections now open) 2016-03-13T09:18:47.162+0000 I NETWORK [conn4081798] end connection 10.0.27.237:36135 (1506 connections now open) 2016-03-13T09:18:47.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36571 #4082777 (1507 connections now open) 2016-03-13T09:18:47.162+0000 I NETWORK [conn4082237] end connection 10.0.179.209:41007 (1506 connections now open) 2016-03-13T09:18:47.163+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41205 #4082778 (1507 connections now open) 2016-03-13T09:18:47.164+0000 I NETWORK [conn4080908] end connection 10.0.179.209:40582 (1506 connections now open) 2016-03-13T09:18:47.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41207 #4082779 (1507 connections now open) 2016-03-13T09:18:47.166+0000 I NETWORK [conn4081757] end connection 10.0.179.209:40790 (1506 connections now open) 2016-03-13T09:18:47.167+0000 I NETWORK [conn4080923] end connection 10.0.179.209:40590 (1505 connections now open) 2016-03-13T09:18:47.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41209 #4082780 (1506 connections now open) 2016-03-13T09:18:47.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41210 #4082781 (1507 connections now open) 2016-03-13T09:18:47.168+0000 I NETWORK [conn4080912] end connection 10.0.179.209:40584 (1506 connections now open) 2016-03-13T09:18:47.168+0000 I NETWORK [conn4082543] end connection 10.0.179.209:41107 (1505 connections now open) 2016-03-13T09:18:47.169+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41211 #4082782 (1506 connections now open) 2016-03-13T09:18:47.169+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41212 #4082783 (1507 connections now open) 2016-03-13T09:18:47.170+0000 I NETWORK [conn4082435] end connection 10.0.179.209:41073 (1506 connections now open) 2016-03-13T09:18:47.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41213 #4082784 (1507 connections now open) 2016-03-13T09:18:47.182+0000 I NETWORK [conn4080848] end connection 10.0.179.209:40534 (1506 connections now open) 2016-03-13T09:18:47.182+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41219 #4082785 (1507 connections now open) 2016-03-13T09:18:47.184+0000 I NETWORK [conn4080931] end connection 10.0.179.209:40593 (1506 connections now open) 2016-03-13T09:18:47.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41220 #4082786 (1507 connections now open) 2016-03-13T09:18:47.186+0000 I NETWORK [conn4081508] end connection 10.0.179.209:40751 (1506 connections now open) 2016-03-13T09:18:47.186+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41221 #4082787 (1507 connections now open) 2016-03-13T09:18:47.189+0000 I NETWORK [conn4080940] end connection 10.0.179.209:40597 (1506 connections now open) 2016-03-13T09:18:47.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41223 #4082788 (1507 connections now open) 2016-03-13T09:18:47.191+0000 I NETWORK [conn4080941] end connection 10.0.179.209:40599 (1506 connections now open) 2016-03-13T09:18:47.191+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41224 #4082789 (1507 connections now open) 2016-03-13T09:18:47.193+0000 I NETWORK [conn4081501] end connection 10.0.179.209:40749 (1506 connections now open) 2016-03-13T09:18:47.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41225 #4082790 (1507 connections now open) 2016-03-13T09:18:47.713+0000 I NETWORK [conn4082738] end connection 10.0.216.90:58122 (1506 connections now open) 2016-03-13T09:18:47.713+0000 I NETWORK [conn4082737] end connection 10.0.216.90:58121 (1506 connections now open) 2016-03-13T09:18:47.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58154 #4082791 (1506 connections now open) 2016-03-13T09:18:47.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58155 #4082792 (1507 connections now open) 2016-03-13T09:18:48.748+0000 I NETWORK [conn4082741] end connection 10.0.20.53:51450 (1506 connections now open) 2016-03-13T09:18:48.748+0000 I NETWORK [conn4082742] end connection 10.0.20.53:51451 (1506 connections now open) 2016-03-13T09:18:48.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51471 #4082793 (1506 connections now open) 2016-03-13T09:18:48.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51470 #4082794 (1507 connections now open) 2016-03-13T09:18:49.390+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.Sessions: 10199ms sequenceNumber: 17154 version: 30652|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:18:49.623+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.ShardTest: 18ms sequenceNumber: 17155 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:18:49.639+0000 I NETWORK [conn4080758] end connection 10.0.7.113:59396 (1506 connections now open) 2016-03-13T09:18:49.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60111 #4082795 (1507 connections now open) 2016-03-13T09:18:49.642+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 18ms sequenceNumber: 17156 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:18:49.679+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 36ms sequenceNumber: 17157 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:18:49.744+0000 I SHARDING [conn4082730] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 65ms sequenceNumber: 17158 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:18:49.749+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 17159 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:18:49.753+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ActionReports: 4ms sequenceNumber: 17160 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:18:49.754+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17161 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:18:49.791+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppScreenReports: 36ms sequenceNumber: 17162 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:18:49.793+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppUsage: 2ms sequenceNumber: 17163 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:18:49.802+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.AppVersionReports: 9ms sequenceNumber: 17164 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:18:49.809+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 6ms sequenceNumber: 17165 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:18:49.895+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.Clients: 85ms sequenceNumber: 17166 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:18:49.912+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 15ms sequenceNumber: 17167 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:18:49.967+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CountryReports: 54ms sequenceNumber: 17168 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:18:49.994+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 27ms sequenceNumber: 17169 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:18:50.022+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CrashedSessions: 27ms sequenceNumber: 17170 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:18:50.198+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 175ms sequenceNumber: 17171 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:18:50.207+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.CustomEventReports: 4ms sequenceNumber: 17172 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:18:50.207+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 17173 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:18:50.223+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 15ms sequenceNumber: 17174 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:18:50.238+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DeviceReports: 14ms sequenceNumber: 17175 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:18:50.246+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.DurationReports: 7ms sequenceNumber: 17176 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:18:50.255+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.FunnelReports: 9ms sequenceNumber: 17177 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:18:50.279+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.GestureReports: 23ms sequenceNumber: 17178 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:18:50.410+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 131ms sequenceNumber: 17179 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:18:50.887+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.HeatmapReports: 474ms sequenceNumber: 17180 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:18:51.030+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:18:50.657+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:18:51.030+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:18:51.042+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 142ms sequenceNumber: 17181 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:18:51.079+0000 I NETWORK [conn4082746] end connection 10.0.179.209:41186 (1506 connections now open) 2016-03-13T09:18:51.079+0000 I NETWORK [conn4082745] end connection 10.0.179.209:41185 (1506 connections now open) 2016-03-13T09:18:51.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41231 #4082796 (1506 connections now open) 2016-03-13T09:18:51.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41230 #4082797 (1507 connections now open) 2016-03-13T09:18:51.080+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.NavigationReports: 33ms sequenceNumber: 17182 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:18:51.089+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.OSVersionReports: 8ms sequenceNumber: 17183 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:18:51.123+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.PendingSessions: 33ms sequenceNumber: 17184 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:18:51.143+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ResolutionReports: 19ms sequenceNumber: 17185 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:18:51.162+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 19ms sequenceNumber: 17186 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:18:51.165+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5306e18fa188537623d6c 2016-03-13T09:18:51.190+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionReports: 28ms sequenceNumber: 17187 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:18:51.204+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 12ms sequenceNumber: 17188 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:18:51.214+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 9ms sequenceNumber: 17189 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:18:51.267+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5307818fa188537623d6d 2016-03-13T09:18:51.785+0000 I NETWORK [conn4082747] end connection 10.0.26.69:43226 (1506 connections now open) 2016-03-13T09:18:51.785+0000 I NETWORK [conn4082748] end connection 10.0.26.69:43227 (1505 connections now open) 2016-03-13T09:18:51.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43244 #4082798 (1506 connections now open) 2016-03-13T09:18:51.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43245 #4082799 (1507 connections now open) 2016-03-13T09:18:52.507+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 1292ms sequenceNumber: 17190 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:18:54.545+0000 I NETWORK [conn4082754] end connection 10.0.27.237:36556 (1506 connections now open) 2016-03-13T09:18:54.545+0000 I NETWORK [conn4082755] end connection 10.0.27.237:36557 (1506 connections now open) 2016-03-13T09:18:54.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36575 #4082800 (1506 connections now open) 2016-03-13T09:18:54.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36576 #4082801 (1507 connections now open) 2016-03-13T09:18:56.686+0000 I NETWORK [conn4082763] end connection 10.0.7.113:60100 (1506 connections now open) 2016-03-13T09:18:56.686+0000 I NETWORK [conn4082764] end connection 10.0.7.113:60099 (1506 connections now open) 2016-03-13T09:18:56.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60113 #4082802 (1506 connections now open) 2016-03-13T09:18:56.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60114 #4082803 (1507 connections now open) 2016-03-13T09:18:57.726+0000 I NETWORK [conn4082792] end connection 10.0.216.90:58155 (1506 connections now open) 2016-03-13T09:18:57.726+0000 I NETWORK [conn4082791] end connection 10.0.216.90:58154 (1506 connections now open) 2016-03-13T09:18:57.727+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58162 #4082804 (1506 connections now open) 2016-03-13T09:18:57.727+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58161 #4082805 (1507 connections now open) 2016-03-13T09:18:58.113+0000 I NETWORK [conn4080960] end connection 10.0.179.209:40609 (1506 connections now open) 2016-03-13T09:18:58.113+0000 I NETWORK [conn4081163] end connection 10.0.216.90:57555 (1506 connections now open) 2016-03-13T09:18:58.113+0000 I NETWORK [conn4081500] end connection 10.0.26.69:42677 (1504 connections now open) 2016-03-13T09:18:58.113+0000 I NETWORK [conn4081090] end connection 10.0.7.113:59573 (1503 connections now open) 2016-03-13T09:18:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41238 #4082806 (1505 connections now open) 2016-03-13T09:18:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58164 #4082807 (1505 connections now open) 2016-03-13T09:18:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43253 #4082808 (1506 connections now open) 2016-03-13T09:18:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60119 #4082809 (1507 connections now open) 2016-03-13T09:18:58.762+0000 I NETWORK [conn4082794] end connection 10.0.20.53:51470 (1506 connections now open) 2016-03-13T09:18:58.762+0000 I NETWORK [conn4082793] end connection 10.0.20.53:51471 (1506 connections now open) 2016-03-13T09:18:58.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51487 #4082810 (1506 connections now open) 2016-03-13T09:18:58.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51486 #4082811 (1507 connections now open) 2016-03-13T09:18:59.860+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.Sessions: 7264ms sequenceNumber: 17191 version: 30652|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:19:00.223+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.ShardTest: 24ms sequenceNumber: 17192 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:19:00.255+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 32ms sequenceNumber: 17193 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:19:00.320+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 65ms sequenceNumber: 17194 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:19:00.431+0000 I SHARDING [conn3430288] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 110ms sequenceNumber: 17195 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:19:00.438+0000 I NETWORK [conn4080860] end connection 10.0.216.90:57476 (1506 connections now open) 2016-03-13T09:19:00.438+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58165 #4082812 (1507 connections now open) 2016-03-13T09:19:00.440+0000 I NETWORK [conn4081210] end connection 10.0.7.113:59615 (1506 connections now open) 2016-03-13T09:19:00.441+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60120 #4082813 (1507 connections now open) 2016-03-13T09:19:00.441+0000 I NETWORK [conn4081742] end connection 10.0.216.90:57693 (1506 connections now open) 2016-03-13T09:19:00.441+0000 I NETWORK [conn4080731] end connection 10.0.7.113:59383 (1505 connections now open) 2016-03-13T09:19:00.441+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58166 #4082814 (1506 connections now open) 2016-03-13T09:19:00.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60121 #4082815 (1507 connections now open) 2016-03-13T09:19:00.454+0000 I NETWORK [conn4080862] end connection 10.0.216.90:57477 (1506 connections now open) 2016-03-13T09:19:00.455+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58170 #4082816 (1507 connections now open) 2016-03-13T09:19:00.544+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.AccountUsage: 19ms sequenceNumber: 17197 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:19:00.562+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.ActionReports: 18ms sequenceNumber: 17198 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:19:00.564+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17199 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:19:00.566+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 17200 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:19:00.567+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 17201 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:19:00.568+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 17202 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:19:00.571+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 2ms sequenceNumber: 17203 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:19:00.633+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.Clients: 61ms sequenceNumber: 17204 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:19:00.656+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 20ms sequenceNumber: 17205 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:19:00.699+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.CountryReports: 42ms sequenceNumber: 17206 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:19:00.701+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 2ms sequenceNumber: 17207 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:19:00.702+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 17208 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:19:00.767+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 65ms sequenceNumber: 17209 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:19:00.793+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.CustomEventReports: 22ms sequenceNumber: 17210 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:19:00.815+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 21ms sequenceNumber: 17211 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:19:00.863+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 48ms sequenceNumber: 17212 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:19:00.872+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.DeviceReports: 7ms sequenceNumber: 17213 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:19:00.873+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 17214 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:19:00.874+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17215 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:19:00.878+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 17216 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:19:00.943+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 65ms sequenceNumber: 17217 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:19:01.092+0000 I NETWORK [conn4082797] end connection 10.0.179.209:41230 (1506 connections now open) 2016-03-13T09:19:01.092+0000 I NETWORK [conn4082796] end connection 10.0.179.209:41231 (1506 connections now open) 2016-03-13T09:19:01.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41240 #4082817 (1506 connections now open) 2016-03-13T09:19:01.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41239 #4082818 (1507 connections now open) 2016-03-13T09:19:01.185+0000 I SHARDING [conn3622034] ChunkManager: time to load chunks for mydomain.Sessions: 752ms sequenceNumber: 17196 version: 30653|1||563ba74d869758a4542b9075 based on: 30652|1||563ba74d869758a4542b9075 2016-03-13T09:19:01.358+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.HeatmapReports: 411ms sequenceNumber: 17218 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:19:01.449+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 77ms sequenceNumber: 17219 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:19:01.481+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.NavigationReports: 27ms sequenceNumber: 17221 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:19:01.483+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 17222 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:19:01.486+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 17223 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:19:01.487+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 17224 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:19:01.488+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 1ms sequenceNumber: 17225 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:19:01.491+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 17226 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:19:01.492+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 17227 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:19:01.493+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17228 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:19:01.798+0000 I NETWORK [conn4082798] end connection 10.0.26.69:43244 (1506 connections now open) 2016-03-13T09:19:01.798+0000 I NETWORK [conn4082799] end connection 10.0.26.69:43245 (1506 connections now open) 2016-03-13T09:19:01.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43267 #4082819 (1506 connections now open) 2016-03-13T09:19:01.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43266 #4082820 (1507 connections now open) 2016-03-13T09:19:02.070+0000 I SHARDING [conn3552646] ChunkManager: time to load chunks for mydomain.Sessions: 688ms sequenceNumber: 17220 version: 30653|1||563ba74d869758a4542b9075 based on: 30652|1||563ba74d869758a4542b9075 2016-03-13T09:19:02.144+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 651ms sequenceNumber: 17229 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:19:02.996+0000 I SHARDING [conn2792076] ChunkManager: time to load chunks for mydomain.Sessions: 756ms sequenceNumber: 17231 version: 30653|1||563ba74d869758a4542b9075 based on: 30652|1||563ba74d869758a4542b9075 2016-03-13T09:19:03.893+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 717ms sequenceNumber: 17232 version: 30653|1||563ba74d869758a4542b9075 based on: 30652|1||563ba74d869758a4542b9075 2016-03-13T09:19:04.479+0000 I NETWORK [conn4081991] end connection 10.0.179.209:40895 (1506 connections now open) 2016-03-13T09:19:04.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41244 #4082821 (1507 connections now open) 2016-03-13T09:19:04.558+0000 I NETWORK [conn4082800] end connection 10.0.27.237:36575 (1506 connections now open) 2016-03-13T09:19:04.558+0000 I NETWORK [conn4082801] end connection 10.0.27.237:36576 (1506 connections now open) 2016-03-13T09:19:04.558+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36597 #4082822 (1506 connections now open) 2016-03-13T09:19:04.558+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36598 #4082823 (1507 connections now open) 2016-03-13T09:19:04.776+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 716ms sequenceNumber: 17233 version: 30653|1||563ba74d869758a4542b9075 based on: 30652|1||563ba74d869758a4542b9075 2016-03-13T09:19:05.660+0000 I SHARDING [conn2850105] ChunkManager: time to load chunks for mydomain.Sessions: 721ms sequenceNumber: 17234 version: 30653|1||563ba74d869758a4542b9075 based on: 30652|1||563ba74d869758a4542b9075 2016-03-13T09:19:06.700+0000 I NETWORK [conn4082803] end connection 10.0.7.113:60114 (1506 connections now open) 2016-03-13T09:19:06.700+0000 I NETWORK [conn4082802] end connection 10.0.7.113:60113 (1506 connections now open) 2016-03-13T09:19:06.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60130 #4082824 (1506 connections now open) 2016-03-13T09:19:06.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60131 #4082825 (1507 connections now open) 2016-03-13T09:19:07.739+0000 I NETWORK [conn4082804] end connection 10.0.216.90:58162 (1506 connections now open) 2016-03-13T09:19:07.739+0000 I NETWORK [conn4082805] end connection 10.0.216.90:58161 (1506 connections now open) 2016-03-13T09:19:07.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58182 #4082826 (1506 connections now open) 2016-03-13T09:19:07.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58181 #4082827 (1507 connections now open) 2016-03-13T09:19:08.776+0000 I NETWORK [conn4082811] end connection 10.0.20.53:51486 (1506 connections now open) 2016-03-13T09:19:08.776+0000 I NETWORK [conn4082810] end connection 10.0.20.53:51487 (1506 connections now open) 2016-03-13T09:19:08.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51506 #4082828 (1506 connections now open) 2016-03-13T09:19:08.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51507 #4082829 (1507 connections now open) 2016-03-13T09:19:11.104+0000 I NETWORK [conn4082817] end connection 10.0.179.209:41240 (1506 connections now open) 2016-03-13T09:19:11.105+0000 I NETWORK [conn4082818] end connection 10.0.179.209:41239 (1505 connections now open) 2016-03-13T09:19:11.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41254 #4082830 (1506 connections now open) 2016-03-13T09:19:11.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41253 #4082831 (1507 connections now open) 2016-03-13T09:19:11.520+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.Sessions: 9290ms sequenceNumber: 17230 version: 30653|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:19:11.706+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 17235 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:19:11.707+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 1ms sequenceNumber: 17236 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:19:11.708+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17237 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:19:11.726+0000 I SHARDING [conn3585590] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 17ms sequenceNumber: 17238 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:19:11.727+0000 W SHARDING [conn3622034] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:11.812+0000 I NETWORK [conn4082820] end connection 10.0.26.69:43266 (1506 connections now open) 2016-03-13T09:19:11.812+0000 I NETWORK [conn4082819] end connection 10.0.26.69:43267 (1506 connections now open) 2016-03-13T09:19:11.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43276 #4082832 (1506 connections now open) 2016-03-13T09:19:11.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43277 #4082833 (1507 connections now open) 2016-03-13T09:19:11.828+0000 W SHARDING [conn3552646] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:11.832+0000 W SHARDING [conn2792076] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:11.832+0000 W SHARDING [conn1051657] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:11.832+0000 W SHARDING [conn1051636] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:11.832+0000 W SHARDING [conn2850105] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:11.938+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:12.473+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 644ms sequenceNumber: 17239 version: 30653|1||563ba74d869758a4542b9075 based on: 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:12.641+0000 W SHARDING [conn1051649] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30653|1||563ba74d869758a4542b9075 2016-03-13T09:19:13.273+0000 I NETWORK [conn4081905] end connection 10.0.20.53:51136 (1506 connections now open) 2016-03-13T09:19:13.273+0000 I NETWORK [conn4079433] end connection 10.0.27.237:34397 (1505 connections now open) 2016-03-13T09:19:13.274+0000 I NETWORK [conn4081336] end connection 10.0.179.209:40734 (1504 connections now open) 2016-03-13T09:19:13.274+0000 I NETWORK [conn4081436] end connection 10.0.26.69:42644 (1503 connections now open) 2016-03-13T09:19:13.274+0000 I NETWORK [conn4082391] end connection 10.0.7.113:59968 (1502 connections now open) 2016-03-13T09:19:13.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51513 #4082834 (1503 connections now open) 2016-03-13T09:19:13.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41258 #4082835 (1504 connections now open) 2016-03-13T09:19:13.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43279 #4082836 (1505 connections now open) 2016-03-13T09:19:13.275+0000 I NETWORK [conn4081684] end connection 10.0.27.237:36080 (1504 connections now open) 2016-03-13T09:19:13.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36607 #4082837 (1505 connections now open) 2016-03-13T09:19:13.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36608 #4082838 (1506 connections now open) 2016-03-13T09:19:13.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60141 #4082839 (1507 connections now open) 2016-03-13T09:19:13.276+0000 I NETWORK [conn4080988] end connection 10.0.179.209:40625 (1506 connections now open) 2016-03-13T09:19:13.276+0000 I NETWORK [conn4081670] end connection 10.0.26.69:42769 (1505 connections now open) 2016-03-13T09:19:13.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41260 #4082840 (1506 connections now open) 2016-03-13T09:19:13.278+0000 I NETWORK [conn4081647] end connection 10.0.20.53:51010 (1505 connections now open) 2016-03-13T09:19:13.278+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43280 #4082841 (1506 connections now open) 2016-03-13T09:19:13.278+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51517 #4082842 (1507 connections now open) 2016-03-13T09:19:13.279+0000 I NETWORK [conn4080864] end connection 10.0.216.90:57478 (1506 connections now open) 2016-03-13T09:19:13.279+0000 I NETWORK [conn4081442] end connection 10.0.26.69:42647 (1505 connections now open) 2016-03-13T09:19:13.280+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58193 #4082843 (1506 connections now open) 2016-03-13T09:19:13.280+0000 I NETWORK [conn4081630] end connection 10.0.26.69:42743 (1505 connections now open) 2016-03-13T09:19:13.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43283 #4082844 (1506 connections now open) 2016-03-13T09:19:13.281+0000 I NETWORK [conn4081530] end connection 10.0.20.53:50943 (1505 connections now open) 2016-03-13T09:19:13.282+0000 I NETWORK [conn4080867] end connection 10.0.216.90:57479 (1504 connections now open) 2016-03-13T09:19:13.282+0000 I NETWORK [conn4080869] end connection 10.0.216.90:57480 (1503 connections now open) 2016-03-13T09:19:13.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51519 #4082845 (1504 connections now open) 2016-03-13T09:19:13.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43284 #4082846 (1505 connections now open) 2016-03-13T09:19:13.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58194 #4082847 (1506 connections now open) 2016-03-13T09:19:13.282+0000 I NETWORK [conn4081330] end connection 10.0.26.69:42585 (1505 connections now open) 2016-03-13T09:19:13.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58195 #4082848 (1507 connections now open) 2016-03-13T09:19:13.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43285 #4082849 (1507 connections now open) 2016-03-13T09:19:13.283+0000 I NETWORK [conn4081532] end connection 10.0.20.53:50945 (1506 connections now open) 2016-03-13T09:19:13.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51520 #4082850 (1507 connections now open) 2016-03-13T09:19:13.284+0000 I NETWORK [conn4080817] end connection 10.0.216.90:57454 (1506 connections now open) 2016-03-13T09:19:13.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58196 #4082851 (1507 connections now open) 2016-03-13T09:19:13.285+0000 I NETWORK [conn4081988] end connection 10.0.26.69:42922 (1506 connections now open) 2016-03-13T09:19:13.285+0000 I NETWORK [conn4081656] end connection 10.0.20.53:51019 (1505 connections now open) 2016-03-13T09:19:13.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43287 #4082852 (1506 connections now open) 2016-03-13T09:19:13.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51521 #4082853 (1507 connections now open) 2016-03-13T09:19:13.286+0000 I NETWORK [conn4080879] end connection 10.0.216.90:57486 (1506 connections now open) 2016-03-13T09:19:13.286+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58197 #4082854 (1507 connections now open) 2016-03-13T09:19:13.286+0000 I NETWORK [conn4081633] end connection 10.0.26.69:42745 (1506 connections now open) 2016-03-13T09:19:13.287+0000 I NETWORK [conn4081538] end connection 10.0.20.53:50947 (1505 connections now open) 2016-03-13T09:19:13.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43288 #4082855 (1506 connections now open) 2016-03-13T09:19:13.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51522 #4082856 (1507 connections now open) 2016-03-13T09:19:13.288+0000 I NETWORK [conn4081472] end connection 10.0.26.69:42662 (1506 connections now open) 2016-03-13T09:19:13.288+0000 I NETWORK [conn4081676] end connection 10.0.20.53:51030 (1505 connections now open) 2016-03-13T09:19:13.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43289 #4082857 (1506 connections now open) 2016-03-13T09:19:13.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51523 #4082858 (1507 connections now open) 2016-03-13T09:19:13.289+0000 I NETWORK [conn4080915] end connection 10.0.216.90:57497 (1506 connections now open) 2016-03-13T09:19:13.290+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58199 #4082859 (1507 connections now open) 2016-03-13T09:19:13.291+0000 I NETWORK [conn4080897] end connection 10.0.216.90:57492 (1506 connections now open) 2016-03-13T09:19:13.292+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58200 #4082860 (1507 connections now open) 2016-03-13T09:19:13.292+0000 I NETWORK [conn4081683] end connection 10.0.20.53:51034 (1506 connections now open) 2016-03-13T09:19:13.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51525 #4082861 (1507 connections now open) 2016-03-13T09:19:13.294+0000 I NETWORK [conn4081635] end connection 10.0.26.69:42746 (1506 connections now open) 2016-03-13T09:19:13.294+0000 I NETWORK [conn4081689] end connection 10.0.20.53:51036 (1505 connections now open) 2016-03-13T09:19:13.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43292 #4082862 (1506 connections now open) 2016-03-13T09:19:13.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51526 #4082863 (1507 connections now open) 2016-03-13T09:19:13.295+0000 I NETWORK [conn4080924] end connection 10.0.216.90:57500 (1506 connections now open) 2016-03-13T09:19:13.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58202 #4082864 (1507 connections now open) 2016-03-13T09:19:13.296+0000 I NETWORK [conn4082186] end connection 10.0.26.69:43002 (1506 connections now open) 2016-03-13T09:19:13.296+0000 I NETWORK [conn4081581] end connection 10.0.20.53:50967 (1505 connections now open) 2016-03-13T09:19:13.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43293 #4082865 (1506 connections now open) 2016-03-13T09:19:13.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51527 #4082866 (1507 connections now open) 2016-03-13T09:19:13.298+0000 I NETWORK [conn4081365] end connection 10.0.20.53:50853 (1506 connections now open) 2016-03-13T09:19:13.298+0000 I NETWORK [conn4081653] end connection 10.0.26.69:42757 (1505 connections now open) 2016-03-13T09:19:13.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51528 #4082867 (1506 connections now open) 2016-03-13T09:19:13.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43294 #4082868 (1507 connections now open) 2016-03-13T09:19:13.299+0000 I NETWORK [conn4080900] end connection 10.0.216.90:57493 (1506 connections now open) 2016-03-13T09:19:13.299+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58204 #4082869 (1507 connections now open) 2016-03-13T09:19:13.299+0000 I NETWORK [conn4081147] end connection 10.0.20.53:50772 (1506 connections now open) 2016-03-13T09:19:13.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51529 #4082870 (1507 connections now open) 2016-03-13T09:19:13.302+0000 I NETWORK [conn4081492] end connection 10.0.26.69:42675 (1506 connections now open) 2016-03-13T09:19:13.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43296 #4082871 (1507 connections now open) 2016-03-13T09:19:13.304+0000 I NETWORK [conn4081704] end connection 10.0.26.69:42789 (1506 connections now open) 2016-03-13T09:19:13.304+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43297 #4082872 (1507 connections now open) 2016-03-13T09:19:13.304+0000 I NETWORK [conn4080909] end connection 10.0.216.90:57495 (1506 connections now open) 2016-03-13T09:19:13.305+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58207 #4082873 (1507 connections now open) 2016-03-13T09:19:13.305+0000 I NETWORK [conn4081610] end connection 10.0.20.53:50981 (1506 connections now open) 2016-03-13T09:19:13.305+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51532 #4082874 (1507 connections now open) 2016-03-13T09:19:13.305+0000 I NETWORK [conn4081660] end connection 10.0.26.69:42762 (1506 connections now open) 2016-03-13T09:19:13.306+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43298 #4082875 (1507 connections now open) 2016-03-13T09:19:13.306+0000 I NETWORK [conn4080918] end connection 10.0.216.90:57498 (1506 connections now open) 2016-03-13T09:19:13.306+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58208 #4082876 (1507 connections now open) 2016-03-13T09:19:13.306+0000 I NETWORK [conn4081692] end connection 10.0.20.53:51038 (1506 connections now open) 2016-03-13T09:19:13.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51533 #4082877 (1507 connections now open) 2016-03-13T09:19:13.307+0000 I NETWORK [conn4081586] end connection 10.0.26.69:42718 (1506 connections now open) 2016-03-13T09:19:13.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43299 #4082878 (1507 connections now open) 2016-03-13T09:19:13.308+0000 I NETWORK [conn4081096] end connection 10.0.216.90:57533 (1506 connections now open) 2016-03-13T09:19:13.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58209 #4082879 (1507 connections now open) 2016-03-13T09:19:13.310+0000 I NETWORK [conn4080929] end connection 10.0.216.90:57501 (1506 connections now open) 2016-03-13T09:19:13.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58210 #4082880 (1507 connections now open) 2016-03-13T09:19:13.929+0000 I NETWORK [conn4080989] end connection 10.0.179.209:40626 (1506 connections now open) 2016-03-13T09:19:13.929+0000 I NETWORK [conn4080937] end connection 10.0.216.90:57502 (1505 connections now open) 2016-03-13T09:19:13.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41263 #4082881 (1506 connections now open) 2016-03-13T09:19:13.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58211 #4082882 (1507 connections now open) 2016-03-13T09:19:14.565+0000 I NETWORK [conn4082256] end connection 10.0.20.53:51285 (1506 connections now open) 2016-03-13T09:19:14.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51535 #4082883 (1507 connections now open) 2016-03-13T09:19:14.568+0000 I NETWORK [conn4082707] end connection 10.0.27.237:36531 (1506 connections now open) 2016-03-13T09:19:14.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36613 #4082884 (1507 connections now open) 2016-03-13T09:19:14.570+0000 I NETWORK [conn3466873] end connection 10.0.217.145:57551 (1506 connections now open) 2016-03-13T09:19:14.570+0000 I NETWORK [conn4082823] end connection 10.0.27.237:36598 (1505 connections now open) 2016-03-13T09:19:14.570+0000 I NETWORK [mongosMain] connection accepted from 10.0.217.145:55720 #4082885 (1507 connections now open) 2016-03-13T09:19:14.571+0000 I NETWORK [conn4082822] end connection 10.0.27.237:36597 (1505 connections now open) 2016-03-13T09:19:14.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36614 #4082886 (1506 connections now open) 2016-03-13T09:19:14.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36615 #4082887 (1507 connections now open) 2016-03-13T09:19:14.574+0000 I NETWORK [conn4081190] end connection 10.0.20.53:50786 (1506 connections now open) 2016-03-13T09:19:14.574+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51539 #4082888 (1507 connections now open) 2016-03-13T09:19:14.577+0000 I NETWORK [conn4081810] end connection 10.0.20.53:51099 (1506 connections now open) 2016-03-13T09:19:14.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51541 #4082889 (1507 connections now open) 2016-03-13T09:19:14.578+0000 I NETWORK [conn4081208] end connection 10.0.20.53:50792 (1506 connections now open) 2016-03-13T09:19:14.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51542 #4082890 (1507 connections now open) 2016-03-13T09:19:14.619+0000 I NETWORK [conn4081077] end connection 10.0.20.53:50743 (1506 connections now open) 2016-03-13T09:19:14.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51546 #4082891 (1507 connections now open) 2016-03-13T09:19:14.627+0000 I NETWORK [conn4081393] end connection 10.0.20.53:50870 (1506 connections now open) 2016-03-13T09:19:14.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51550 #4082892 (1507 connections now open) 2016-03-13T09:19:14.629+0000 I NETWORK [conn4081294] end connection 10.0.20.53:50817 (1506 connections now open) 2016-03-13T09:19:14.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51551 #4082893 (1507 connections now open) 2016-03-13T09:19:14.632+0000 I NETWORK [conn4081295] end connection 10.0.20.53:50819 (1506 connections now open) 2016-03-13T09:19:14.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51552 #4082894 (1507 connections now open) 2016-03-13T09:19:14.641+0000 I NETWORK [conn4081752] end connection 10.0.20.53:51065 (1506 connections now open) 2016-03-13T09:19:14.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51556 #4082895 (1507 connections now open) 2016-03-13T09:19:16.118+0000 I NETWORK [conn4082381] end connection 10.0.216.90:57976 (1506 connections now open) 2016-03-13T09:19:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58224 #4082896 (1507 connections now open) 2016-03-13T09:19:16.713+0000 I NETWORK [conn4082825] end connection 10.0.7.113:60131 (1506 connections now open) 2016-03-13T09:19:16.713+0000 I NETWORK [conn4082824] end connection 10.0.7.113:60130 (1506 connections now open) 2016-03-13T09:19:16.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60156 #4082897 (1506 connections now open) 2016-03-13T09:19:16.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60157 #4082898 (1507 connections now open) 2016-03-13T09:19:17.752+0000 I NETWORK [conn4082827] end connection 10.0.216.90:58181 (1506 connections now open) 2016-03-13T09:19:17.752+0000 I NETWORK [conn4082826] end connection 10.0.216.90:58182 (1506 connections now open) 2016-03-13T09:19:17.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58226 #4082899 (1506 connections now open) 2016-03-13T09:19:17.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58225 #4082900 (1507 connections now open) 2016-03-13T09:19:18.789+0000 I NETWORK [conn4082829] end connection 10.0.20.53:51507 (1506 connections now open) 2016-03-13T09:19:18.789+0000 I NETWORK [conn4082828] end connection 10.0.20.53:51506 (1506 connections now open) 2016-03-13T09:19:18.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51559 #4082901 (1506 connections now open) 2016-03-13T09:19:18.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51560 #4082902 (1507 connections now open) 2016-03-13T09:19:18.870+0000 I NETWORK [conn4081250] end connection 10.0.27.237:35889 (1506 connections now open) 2016-03-13T09:19:18.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36627 #4082903 (1507 connections now open) 2016-03-13T09:19:18.889+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 17240 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:19:18.893+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ActionReports: 4ms sequenceNumber: 17241 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:19:18.894+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17242 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:19:18.911+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppScreenReports: 16ms sequenceNumber: 17243 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:19:18.914+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppUsage: 2ms sequenceNumber: 17244 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:19:18.917+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppVersionReports: 3ms sequenceNumber: 17245 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:19:18.928+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 10ms sequenceNumber: 17246 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:19:18.958+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.Clients: 29ms sequenceNumber: 17247 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:19:18.961+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 1ms sequenceNumber: 17248 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:19:18.963+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CountryReports: 2ms sequenceNumber: 17249 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:19:18.964+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 17250 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:19:18.973+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CrashedSessions: 9ms sequenceNumber: 17251 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:19:19.031+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 57ms sequenceNumber: 17252 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:19:19.037+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CustomEventReports: 1ms sequenceNumber: 17253 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:19:19.037+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 17254 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:19:19.043+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 17255 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:19:19.049+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DeviceReports: 5ms sequenceNumber: 17256 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:19:19.062+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DurationReports: 12ms sequenceNumber: 17257 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:19:19.063+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17258 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:19:19.067+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 17259 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:19:19.129+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 61ms sequenceNumber: 17260 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:19:19.368+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.HeatmapReports: 236ms sequenceNumber: 17261 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:19:19.434+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 54ms sequenceNumber: 17262 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:19:19.458+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.NavigationReports: 20ms sequenceNumber: 17263 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:19:19.464+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.OSVersionReports: 4ms sequenceNumber: 17264 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:19:19.473+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.PendingSessions: 9ms sequenceNumber: 17265 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:19:19.477+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ResolutionReports: 3ms sequenceNumber: 17266 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:19:19.478+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 1ms sequenceNumber: 17267 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:19:19.482+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionReports: 3ms sequenceNumber: 17268 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:19:19.488+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 5ms sequenceNumber: 17269 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:19:19.488+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17270 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4081778] end connection 10.0.216.90:57714 (1506 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4077276] end connection 10.0.27.237:60458 (1506 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4080968] end connection 10.0.20.53:50702 (1506 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4080953] end connection 10.0.7.113:59527 (1503 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4080993] end connection 10.0.179.209:40628 (1502 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4080729] end connection 10.0.216.90:57399 (1501 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58229 #4082904 (1503 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4081235] end connection 10.0.179.209:40710 (1500 connections now open) 2016-03-13T09:19:19.743+0000 I NETWORK [conn4080964] end connection 10.0.7.113:59532 (1500 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51562 #4082905 (1503 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [conn4081729] end connection 10.0.26.69:42807 (1500 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36628 #4082906 (1501 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60161 #4082907 (1502 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58230 #4082908 (1503 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60162 #4082909 (1504 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41284 #4082910 (1505 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43306 #4082911 (1506 connections now open) 2016-03-13T09:19:19.744+0000 I NETWORK [conn4080911] end connection 10.0.7.113:59507 (1505 connections now open) 2016-03-13T09:19:19.745+0000 I NETWORK [conn4081264] end connection 10.0.27.237:35893 (1504 connections now open) 2016-03-13T09:19:19.745+0000 I NETWORK [conn4081777] end connection 10.0.26.69:42833 (1504 connections now open) 2016-03-13T09:19:19.745+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60163 #4082912 (1504 connections now open) 2016-03-13T09:19:19.746+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41287 #4082913 (1505 connections now open) 2016-03-13T09:19:19.747+0000 I NETWORK [conn4082133] end connection 10.0.20.53:51233 (1504 connections now open) 2016-03-13T09:19:19.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43308 #4082914 (1505 connections now open) 2016-03-13T09:19:19.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51565 #4082915 (1506 connections now open) 2016-03-13T09:19:19.747+0000 I NETWORK [conn4081247] end connection 10.0.179.209:40715 (1505 connections now open) 2016-03-13T09:19:19.748+0000 I NETWORK [conn4077449] end connection 10.0.216.90:54057 (1504 connections now open) 2016-03-13T09:19:19.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36630 #4082916 (1505 connections now open) 2016-03-13T09:19:19.748+0000 I NETWORK [conn4082636] end connection 10.0.179.209:41143 (1504 connections now open) 2016-03-13T09:19:19.748+0000 I NETWORK [conn4080763] end connection 10.0.7.113:59399 (1503 connections now open) 2016-03-13T09:19:19.748+0000 I NETWORK [conn4079219] end connection 10.0.216.90:55905 (1503 connections now open) 2016-03-13T09:19:19.748+0000 I NETWORK [conn4082757] end connection 10.0.216.90:58142 (1501 connections now open) 2016-03-13T09:19:19.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58232 #4082917 (1502 connections now open) 2016-03-13T09:19:19.749+0000 I NETWORK [conn4082879] end connection 10.0.216.90:58209 (1501 connections now open) 2016-03-13T09:19:19.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58233 #4082918 (1502 connections now open) 2016-03-13T09:19:19.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58234 #4082919 (1503 connections now open) 2016-03-13T09:19:19.751+0000 I NETWORK [conn4081799] end connection 10.0.7.113:59736 (1502 connections now open) 2016-03-13T09:19:19.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58235 #4082920 (1503 connections now open) 2016-03-13T09:19:19.751+0000 I NETWORK [conn4081333] end connection 10.0.26.69:42587 (1502 connections now open) 2016-03-13T09:19:19.752+0000 I NETWORK [conn4080764] end connection 10.0.7.113:59400 (1501 connections now open) 2016-03-13T09:19:19.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60164 #4082921 (1502 connections now open) 2016-03-13T09:19:19.753+0000 I NETWORK [conn4078840] end connection 10.0.216.90:55527 (1501 connections now open) 2016-03-13T09:19:19.753+0000 I NETWORK [conn4081780] end connection 10.0.20.53:51087 (1500 connections now open) 2016-03-13T09:19:19.754+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60165 #4082922 (1501 connections now open) 2016-03-13T09:19:19.754+0000 I NETWORK [conn4080816] end connection 10.0.7.113:59435 (1500 connections now open) 2016-03-13T09:19:19.754+0000 I NETWORK [conn4081385] end connection 10.0.27.237:35932 (1499 connections now open) 2016-03-13T09:19:19.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43314 #4082923 (1500 connections now open) 2016-03-13T09:19:19.755+0000 I NETWORK [conn4082307] end connection 10.0.20.53:51299 (1499 connections now open) 2016-03-13T09:19:19.756+0000 I NETWORK [conn4081276] end connection 10.0.27.237:35897 (1498 connections now open) 2016-03-13T09:19:19.756+0000 I NETWORK [conn4081727] end connection 10.0.26.69:42804 (1497 connections now open) 2016-03-13T09:19:19.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36634 #4082924 (1498 connections now open) 2016-03-13T09:19:19.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60166 #4082925 (1499 connections now open) 2016-03-13T09:19:19.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51566 #4082926 (1500 connections now open) 2016-03-13T09:19:19.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43317 #4082927 (1501 connections now open) 2016-03-13T09:19:19.757+0000 I NETWORK [conn4081368] end connection 10.0.26.69:42605 (1500 connections now open) 2016-03-13T09:19:19.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60168 #4082928 (1501 connections now open) 2016-03-13T09:19:19.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58237 #4082929 (1502 connections now open) 2016-03-13T09:19:19.758+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43318 #4082930 (1503 connections now open) 2016-03-13T09:19:19.758+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36636 #4082931 (1504 connections now open) 2016-03-13T09:19:19.758+0000 I NETWORK [conn4081305] end connection 10.0.27.237:35903 (1502 connections now open) 2016-03-13T09:19:19.758+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41289 #4082932 (1505 connections now open) 2016-03-13T09:19:19.758+0000 I NETWORK [conn4081535] end connection 10.0.26.69:42696 (1502 connections now open) 2016-03-13T09:19:19.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51568 #4082933 (1504 connections now open) 2016-03-13T09:19:19.759+0000 I NETWORK [conn4080979] end connection 10.0.20.53:50705 (1503 connections now open) 2016-03-13T09:19:19.759+0000 I NETWORK [conn4081499] end connection 10.0.20.53:50926 (1502 connections now open) 2016-03-13T09:19:19.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43319 #4082934 (1503 connections now open) 2016-03-13T09:19:19.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51569 #4082935 (1504 connections now open) 2016-03-13T09:19:19.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41290 #4082936 (1505 connections now open) 2016-03-13T09:19:19.760+0000 I NETWORK [conn4080806] end connection 10.0.7.113:59424 (1504 connections now open) 2016-03-13T09:19:19.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36638 #4082937 (1505 connections now open) 2016-03-13T09:19:19.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60171 #4082938 (1506 connections now open) 2016-03-13T09:19:19.760+0000 I NETWORK [conn4081094] end connection 10.0.7.113:59574 (1504 connections now open) 2016-03-13T09:19:19.760+0000 I NETWORK [conn4081362] end connection 10.0.179.209:40739 (1504 connections now open) 2016-03-13T09:19:19.760+0000 I NETWORK [conn4081783] end connection 10.0.26.69:42836 (1503 connections now open) 2016-03-13T09:19:19.761+0000 I NETWORK [conn4081675] end connection 10.0.26.69:42772 (1502 connections now open) 2016-03-13T09:19:19.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43320 #4082939 (1503 connections now open) 2016-03-13T09:19:19.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60172 #4082940 (1504 connections now open) 2016-03-13T09:19:19.762+0000 I NETWORK [conn4082278] end connection 10.0.7.113:59928 (1503 connections now open) 2016-03-13T09:19:19.762+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43322 #4082941 (1504 connections now open) 2016-03-13T09:19:19.762+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41293 #4082942 (1505 connections now open) 2016-03-13T09:19:19.763+0000 I NETWORK [conn4081678] end connection 10.0.26.69:42774 (1504 connections now open) 2016-03-13T09:19:19.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60174 #4082943 (1506 connections now open) 2016-03-13T09:19:19.763+0000 I NETWORK [conn4082525] end connection 10.0.216.90:58039 (1504 connections now open) 2016-03-13T09:19:19.764+0000 I NETWORK [conn4081061] end connection 10.0.27.237:35817 (1503 connections now open) 2016-03-13T09:19:19.764+0000 I NETWORK [conn4080832] end connection 10.0.7.113:59446 (1502 connections now open) 2016-03-13T09:19:19.765+0000 I NETWORK [conn4081100] end connection 10.0.7.113:59575 (1501 connections now open) 2016-03-13T09:19:19.765+0000 I NETWORK [conn4080735] end connection 10.0.179.209:40456 (1500 connections now open) 2016-03-13T09:19:19.765+0000 I NETWORK [conn4081013] end connection 10.0.179.209:40639 (1499 connections now open) 2016-03-13T09:19:19.765+0000 I NETWORK [conn4081103] end connection 10.0.7.113:59576 (1498 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [conn4082734] end connection 10.0.27.237:36546 (1497 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43323 #4082944 (1498 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60175 #4082945 (1499 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41294 #4082946 (1500 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58241 #4082947 (1501 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36639 #4082948 (1502 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36640 #4082949 (1503 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60177 #4082950 (1504 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60176 #4082951 (1505 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51570 #4082952 (1506 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41297 #4082953 (1507 connections now open) 2016-03-13T09:19:19.766+0000 I NETWORK [conn4081054] end connection 10.0.179.209:40657 (1506 connections now open) 2016-03-13T09:19:19.767+0000 I NETWORK [conn4082563] end connection 10.0.216.90:58053 (1505 connections now open) 2016-03-13T09:19:19.767+0000 I NETWORK [conn4081314] end connection 10.0.179.209:40729 (1504 connections now open) 2016-03-13T09:19:19.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41298 #4082954 (1505 connections now open) 2016-03-13T09:19:19.767+0000 I NETWORK [conn4082217] end connection 10.0.7.113:59904 (1504 connections now open) 2016-03-13T09:19:19.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58243 #4082955 (1506 connections now open) 2016-03-13T09:19:19.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41299 #4082956 (1506 connections now open) 2016-03-13T09:19:19.767+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60178 #4082957 (1507 connections now open) 2016-03-13T09:19:19.770+0000 I NETWORK [conn4080739] end connection 10.0.7.113:59386 (1506 connections now open) 2016-03-13T09:19:19.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60180 #4082958 (1507 connections now open) 2016-03-13T09:19:19.772+0000 I NETWORK [conn4081765] end connection 10.0.179.209:40795 (1506 connections now open) 2016-03-13T09:19:19.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41302 #4082959 (1507 connections now open) 2016-03-13T09:19:19.773+0000 I NETWORK [conn4080782] end connection 10.0.7.113:59409 (1506 connections now open) 2016-03-13T09:19:19.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60181 #4082960 (1507 connections now open) 2016-03-13T09:19:19.778+0000 I NETWORK [conn4080835] end connection 10.0.7.113:59449 (1506 connections now open) 2016-03-13T09:19:19.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60184 #4082961 (1507 connections now open) 2016-03-13T09:19:19.785+0000 I NETWORK [conn4082585] end connection 10.0.7.113:60026 (1506 connections now open) 2016-03-13T09:19:19.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60185 #4082962 (1507 connections now open) 2016-03-13T09:19:19.796+0000 I NETWORK [conn4080790] end connection 10.0.7.113:59412 (1506 connections now open) 2016-03-13T09:19:19.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60187 #4082963 (1507 connections now open) 2016-03-13T09:19:19.817+0000 I NETWORK [conn4080792] end connection 10.0.7.113:59413 (1506 connections now open) 2016-03-13T09:19:19.818+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60197 #4082964 (1507 connections now open) 2016-03-13T09:19:20.220+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 731ms sequenceNumber: 17271 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:19:21.126+0000 I NETWORK [conn4082830] end connection 10.0.179.209:41254 (1506 connections now open) 2016-03-13T09:19:21.126+0000 I NETWORK [conn4082831] end connection 10.0.179.209:41253 (1505 connections now open) 2016-03-13T09:19:21.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41304 #4082965 (1506 connections now open) 2016-03-13T09:19:21.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41305 #4082966 (1507 connections now open) 2016-03-13T09:19:21.641+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:19:21.267+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:19:21.642+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:19:21.825+0000 I NETWORK [conn4082832] end connection 10.0.26.69:43276 (1506 connections now open) 2016-03-13T09:19:21.826+0000 I NETWORK [conn4082833] end connection 10.0.26.69:43277 (1505 connections now open) 2016-03-13T09:19:21.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43329 #4082967 (1506 connections now open) 2016-03-13T09:19:21.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43328 #4082968 (1507 connections now open) 2016-03-13T09:19:21.836+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5308218fa188537623d6e 2016-03-13T09:19:22.003+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5308c18fa188537623d6f 2016-03-13T09:19:22.139+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5309718fa188537623d70 2016-03-13T09:19:24.580+0000 W SHARDING [conn3600464] inconsistent chunks found when reloading mydomain.Sessions, previous version was 30654|1||563ba74d869758a4542b9075, this should be rare 2016-03-13T09:19:24.583+0000 I NETWORK [conn4082886] end connection 10.0.27.237:36614 (1506 connections now open) 2016-03-13T09:19:24.584+0000 I NETWORK [conn4082887] end connection 10.0.27.237:36615 (1505 connections now open) 2016-03-13T09:19:24.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36644 #4082969 (1506 connections now open) 2016-03-13T09:19:24.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36643 #4082970 (1507 connections now open) 2016-03-13T09:19:24.754+0000 W SHARDING [conn3600464] ChunkManager loaded an invalid config for mydomain.Sessions, trying again 2016-03-13T09:19:26.727+0000 I NETWORK [conn4082897] end connection 10.0.7.113:60156 (1506 connections now open) 2016-03-13T09:19:26.727+0000 I NETWORK [conn4082898] end connection 10.0.7.113:60157 (1506 connections now open) 2016-03-13T09:19:26.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60201 #4082971 (1506 connections now open) 2016-03-13T09:19:26.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60200 #4082972 (1507 connections now open) 2016-03-13T09:19:27.773+0000 I NETWORK [conn4082900] end connection 10.0.216.90:58225 (1506 connections now open) 2016-03-13T09:19:27.773+0000 I NETWORK [conn4082899] end connection 10.0.216.90:58226 (1506 connections now open) 2016-03-13T09:19:27.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58247 #4082973 (1506 connections now open) 2016-03-13T09:19:27.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58248 #4082974 (1507 connections now open) 2016-03-13T09:19:27.813+0000 W SHARDING [conn3600464] got invalid chunk version 1|2496||563ba74d869758a4542b9075 in document { _id: "mydomain.Sessions-a_ObjectId('5460a5dd645d051108dffbe9')_id_ObjectId('55b2239e7adcc50aa5443b7e')", ns: "mydomain.Sessions", min: { a: ObjectId('5460a5dd645d051108dffbe9'), _id: ObjectId('55b2239e7adcc50aa5443b7e') }, max: { a: ObjectId('5460a5dd645d051108dffbe9'), _id: ObjectId('55b278fecf4c3027ff8ed02c') }, shard: "shmydomain1", lastmod: Timestamp 1000|2496, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } when trying to load differing chunks at version 0|0||000000000000000000000000 2016-03-13T09:19:27.813+0000 W SHARDING [conn3600464] major change in chunk information found when reloading mydomain.Sessions, previous version was 0|0||000000000000000000000000 2016-03-13T09:19:27.882+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.Sessions: 3117ms sequenceNumber: 17272 version: 0|0||000000000000000000000000 based on: (empty) 2016-03-13T09:19:27.882+0000 W SHARDING [conn3600464] no chunks found for collection mydomain.Sessions, assuming unsharded 2016-03-13T09:19:27.944+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 17273 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:19:27.945+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 1ms sequenceNumber: 17274 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:19:27.947+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17275 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:19:27.965+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 17ms sequenceNumber: 17276 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:19:27.968+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 17277 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:19:27.973+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ActionReports: 4ms sequenceNumber: 17278 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:19:27.974+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17279 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:19:27.977+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 17280 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:19:27.978+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 17281 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:19:27.978+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 17282 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:19:27.981+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 2ms sequenceNumber: 17283 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:19:28.007+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.Clients: 24ms sequenceNumber: 17284 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:19:28.010+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 1ms sequenceNumber: 17285 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:19:28.012+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 17286 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:19:28.013+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 17287 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:19:28.013+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 17288 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:19:28.055+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 41ms sequenceNumber: 17289 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:19:28.061+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.CustomEventReports: 1ms sequenceNumber: 17290 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:19:28.061+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 17291 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:19:28.067+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 17292 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:19:28.072+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 17293 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:19:28.073+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 17294 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:19:28.074+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17295 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:19:28.078+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 17296 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:19:28.113+0000 I NETWORK [conn4082859] end connection 10.0.216.90:58199 (1506 connections now open) 2016-03-13T09:19:28.113+0000 I NETWORK [conn4082561] end connection 10.0.26.69:43152 (1506 connections now open) 2016-03-13T09:19:28.114+0000 I NETWORK [conn4081821] end connection 10.0.20.53:51107 (1506 connections now open) 2016-03-13T09:19:28.114+0000 I NETWORK [conn4081008] end connection 10.0.7.113:59550 (1503 connections now open) 2016-03-13T09:19:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58250 #4082975 (1504 connections now open) 2016-03-13T09:19:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43336 #4082976 (1505 connections now open) 2016-03-13T09:19:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51579 #4082977 (1506 connections now open) 2016-03-13T09:19:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60206 #4082978 (1507 connections now open) 2016-03-13T09:19:28.115+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 36ms sequenceNumber: 17297 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:19:28.291+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.HeatmapReports: 172ms sequenceNumber: 17298 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:19:28.346+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 43ms sequenceNumber: 17299 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:19:28.360+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.NavigationReports: 9ms sequenceNumber: 17300 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:19:28.367+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.OSVersionReports: 6ms sequenceNumber: 17301 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:19:28.390+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.PendingSessions: 22ms sequenceNumber: 17302 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:19:28.391+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ResolutionReports: 1ms sequenceNumber: 17303 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:19:28.392+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 1ms sequenceNumber: 17304 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:19:28.395+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 17305 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:19:28.396+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 17306 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:19:28.397+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17307 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:19:28.810+0000 I NETWORK [conn4082901] end connection 10.0.20.53:51559 (1506 connections now open) 2016-03-13T09:19:28.810+0000 I NETWORK [conn4082902] end connection 10.0.20.53:51560 (1506 connections now open) 2016-03-13T09:19:28.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51584 #4082979 (1506 connections now open) 2016-03-13T09:19:28.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51585 #4082980 (1507 connections now open) 2016-03-13T09:19:28.993+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 596ms sequenceNumber: 17308 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:19:31.139+0000 I NETWORK [conn4082966] end connection 10.0.179.209:41305 (1506 connections now open) 2016-03-13T09:19:31.139+0000 I NETWORK [conn4082965] end connection 10.0.179.209:41304 (1506 connections now open) 2016-03-13T09:19:31.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41311 #4082981 (1506 connections now open) 2016-03-13T09:19:31.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41312 #4082982 (1507 connections now open) 2016-03-13T09:19:31.839+0000 I NETWORK [conn4082967] end connection 10.0.26.69:43329 (1506 connections now open) 2016-03-13T09:19:31.839+0000 I NETWORK [conn4082968] end connection 10.0.26.69:43328 (1506 connections now open) 2016-03-13T09:19:31.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43347 #4082983 (1506 connections now open) 2016-03-13T09:19:31.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43348 #4082984 (1507 connections now open) 2016-03-13T09:19:32.744+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.Sessions: 3664ms sequenceNumber: 17309 version: 30654|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:19:32.962+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 17310 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:19:32.963+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 17311 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:19:32.965+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17312 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:19:32.989+0000 I SHARDING [conn3600464] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 24ms sequenceNumber: 17313 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:19:32.998+0000 I ASIO [NetworkInterfaceASIO-TaskExecutorPool-2-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:32.999+0000 I NETWORK [conn4081127] end connection 10.0.20.53:50763 (1506 connections now open) 2016-03-13T09:19:32.999+0000 I NETWORK [conn4080794] end connection 10.0.7.113:59414 (1505 connections now open) 2016-03-13T09:19:33.000+0000 I NETWORK [conn4079346] end connection 10.0.216.90:56032 (1504 connections now open) 2016-03-13T09:19:33.000+0000 I NETWORK [conn4081373] end connection 10.0.20.53:50856 (1503 connections now open) 2016-03-13T09:19:33.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51594 #4082985 (1504 connections now open) 2016-03-13T09:19:33.001+0000 I NETWORK [conn4082885] end connection 10.0.217.145:55720 (1503 connections now open) 2016-03-13T09:19:33.001+0000 I NETWORK [conn4081371] end connection 10.0.20.53:50855 (1502 connections now open) 2016-03-13T09:19:33.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60208 #4082986 (1503 connections now open) 2016-03-13T09:19:33.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.217.145:55728 #4082987 (1504 connections now open) 2016-03-13T09:19:33.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51595 #4082988 (1505 connections now open) 2016-03-13T09:19:33.002+0000 I NETWORK [conn4080795] end connection 10.0.7.113:59415 (1504 connections now open) 2016-03-13T09:19:33.003+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.003+0000 I NETWORK [conn4081120] end connection 10.0.7.113:59581 (1503 connections now open) 2016-03-13T09:19:33.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58259 #4082989 (1504 connections now open) 2016-03-13T09:19:33.004+0000 I NETWORK [conn4080975] end connection 10.0.179.209:40617 (1503 connections now open) 2016-03-13T09:19:33.004+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.004+0000 I NETWORK [conn4082243] end connection 10.0.27.237:36341 (1502 connections now open) 2016-03-13T09:19:33.004+0000 I NETWORK [conn4082400] end connection 10.0.179.209:41067 (1502 connections now open) 2016-03-13T09:19:33.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60209 #4082990 (1502 connections now open) 2016-03-13T09:19:33.004+0000 I NETWORK [conn4082092] end connection 10.0.179.209:40938 (1501 connections now open) 2016-03-13T09:19:33.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36662 #4082991 (1502 connections now open) 2016-03-13T09:19:33.005+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51598 #4082992 (1503 connections now open) 2016-03-13T09:19:33.005+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41314 #4082993 (1504 connections now open) 2016-03-13T09:19:33.005+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41315 #4082994 (1505 connections now open) 2016-03-13T09:19:33.005+0000 I NETWORK [conn4082493] end connection 10.0.216.90:58023 (1504 connections now open) 2016-03-13T09:19:33.005+0000 I NETWORK [conn4081296] end connection 10.0.7.113:59639 (1503 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [conn4082318] end connection 10.0.26.69:43056 (1502 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60210 #4082995 (1503 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41316 #4082996 (1504 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43351 #4082997 (1505 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58261 #4082998 (1506 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [conn4081725] end connection 10.0.7.113:59707 (1505 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [conn4081796] end connection 10.0.7.113:59735 (1504 connections now open) 2016-03-13T09:19:33.006+0000 I NETWORK [conn4081379] end connection 10.0.20.53:50860 (1504 connections now open) 2016-03-13T09:19:33.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60213 #4082999 (1504 connections now open) 2016-03-13T09:19:33.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51600 #4083000 (1505 connections now open) 2016-03-13T09:19:33.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60214 #4083001 (1506 connections now open) 2016-03-13T09:19:33.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60215 #4083002 (1507 connections now open) 2016-03-13T09:19:33.007+0000 I NETWORK [conn4082343] end connection 10.0.216.90:57964 (1505 connections now open) 2016-03-13T09:19:33.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58263 #4083003 (1507 connections now open) 2016-03-13T09:19:33.009+0000 I NETWORK [conn4081599] end connection 10.0.27.237:36037 (1506 connections now open) 2016-03-13T09:19:33.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36666 #4083004 (1507 connections now open) 2016-03-13T09:19:33.010+0000 I NETWORK [conn4081681] end connection 10.0.26.69:42776 (1506 connections now open) 2016-03-13T09:19:33.010+0000 I NETWORK [conn4080881] end connection 10.0.7.113:59490 (1505 connections now open) 2016-03-13T09:19:33.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60217 #4083005 (1506 connections now open) 2016-03-13T09:19:33.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43352 #4083006 (1507 connections now open) 2016-03-13T09:19:33.011+0000 I NETWORK [conn4080994] end connection 10.0.179.209:40629 (1505 connections now open) 2016-03-13T09:19:33.011+0000 I NETWORK [conn4081593] end connection 10.0.26.69:42722 (1505 connections now open) 2016-03-13T09:19:33.011+0000 I NETWORK [conn4080818] end connection 10.0.7.113:59437 (1504 connections now open) 2016-03-13T09:19:33.012+0000 I NETWORK [conn4080996] end connection 10.0.179.209:40631 (1503 connections now open) 2016-03-13T09:19:33.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41324 #4083007 (1504 connections now open) 2016-03-13T09:19:33.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60219 #4083008 (1505 connections now open) 2016-03-13T09:19:33.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41325 #4083009 (1506 connections now open) 2016-03-13T09:19:33.013+0000 I NETWORK [conn4081420] end connection 10.0.216.90:57627 (1505 connections now open) 2016-03-13T09:19:33.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58265 #4083010 (1506 connections now open) 2016-03-13T09:19:33.015+0000 I NETWORK [conn4081587] end connection 10.0.179.209:40761 (1505 connections now open) 2016-03-13T09:19:33.016+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41327 #4083011 (1506 connections now open) 2016-03-13T09:19:33.018+0000 I NETWORK [conn4081173] end connection 10.0.7.113:59602 (1505 connections now open) 2016-03-13T09:19:33.019+0000 I NETWORK [conn4081887] end connection 10.0.26.69:42881 (1504 connections now open) 2016-03-13T09:19:33.019+0000 I NETWORK [conn4081031] end connection 10.0.26.69:42469 (1503 connections now open) 2016-03-13T09:19:33.019+0000 I NETWORK [conn4081281] end connection 10.0.20.53:50814 (1502 connections now open) 2016-03-13T09:19:33.019+0000 I NETWORK [conn4082882] end connection 10.0.216.90:58211 (1502 connections now open) 2016-03-13T09:19:33.020+0000 I NETWORK [conn4081568] end connection 10.0.26.69:42714 (1500 connections now open) 2016-03-13T09:19:33.020+0000 I NETWORK [conn4081087] end connection 10.0.27.237:35827 (1499 connections now open) 2016-03-13T09:19:33.020+0000 I NETWORK [conn4081388] end connection 10.0.179.209:40745 (1498 connections now open) 2016-03-13T09:19:33.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43355 #4083012 (1499 connections now open) 2016-03-13T09:19:33.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51604 #4083013 (1500 connections now open) 2016-03-13T09:19:33.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36667 #4083014 (1501 connections now open) 2016-03-13T09:19:33.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43356 #4083015 (1502 connections now open) 2016-03-13T09:19:33.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43357 #4083016 (1503 connections now open) 2016-03-13T09:19:33.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58267 #4083017 (1504 connections now open) 2016-03-13T09:19:33.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60221 #4083018 (1505 connections now open) 2016-03-13T09:19:33.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41329 #4083019 (1506 connections now open) 2016-03-13T09:19:33.021+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43358 #4083020 (1507 connections now open) 2016-03-13T09:19:33.022+0000 I NETWORK [conn4080970] end connection 10.0.7.113:59535 (1506 connections now open) 2016-03-13T09:19:33.022+0000 I NETWORK [conn4082860] end connection 10.0.216.90:58200 (1505 connections now open) 2016-03-13T09:19:33.022+0000 I NETWORK [conn4081337] end connection 10.0.27.237:35914 (1504 connections now open) 2016-03-13T09:19:33.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58268 #4083021 (1505 connections now open) 2016-03-13T09:19:33.022+0000 I NETWORK [conn4081252] end connection 10.0.7.113:59626 (1504 connections now open) 2016-03-13T09:19:33.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36669 #4083022 (1505 connections now open) 2016-03-13T09:19:33.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60223 #4083023 (1506 connections now open) 2016-03-13T09:19:33.023+0000 I NETWORK [conn4082387] end connection 10.0.216.90:57985 (1505 connections now open) 2016-03-13T09:19:33.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58271 #4083024 (1506 connections now open) 2016-03-13T09:19:33.024+0000 I NETWORK [conn4081037] end connection 10.0.179.209:40649 (1505 connections now open) 2016-03-13T09:19:33.024+0000 I NETWORK [conn4081189] end connection 10.0.27.237:35868 (1504 connections now open) 2016-03-13T09:19:33.024+0000 I NETWORK [conn4081188] end connection 10.0.7.113:59609 (1503 connections now open) 2016-03-13T09:19:33.024+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36670 #4083025 (1504 connections now open) 2016-03-13T09:19:33.024+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41330 #4083026 (1505 connections now open) 2016-03-13T09:19:33.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60224 #4083027 (1506 connections now open) 2016-03-13T09:19:33.025+0000 I NETWORK [conn4080808] end connection 10.0.7.113:59426 (1505 connections now open) 2016-03-13T09:19:33.026+0000 I NETWORK [conn4082448] end connection 10.0.26.69:43116 (1504 connections now open) 2016-03-13T09:19:33.026+0000 I NETWORK [conn4081859] end connection 10.0.27.237:36168 (1503 connections now open) 2016-03-13T09:19:33.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43363 #4083028 (1504 connections now open) 2016-03-13T09:19:33.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60227 #4083029 (1505 connections now open) 2016-03-13T09:19:33.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36671 #4083030 (1506 connections now open) 2016-03-13T09:19:33.028+0000 I NETWORK [conn4082933] end connection 10.0.20.53:51568 (1505 connections now open) 2016-03-13T09:19:33.029+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51610 #4083031 (1506 connections now open) 2016-03-13T09:19:33.029+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60229 #4083032 (1507 connections now open) 2016-03-13T09:19:33.030+0000 I NETWORK [conn4081162] end connection 10.0.7.113:59597 (1506 connections now open) 2016-03-13T09:19:33.032+0000 I NETWORK [conn4082411] end connection 10.0.216.90:57998 (1505 connections now open) 2016-03-13T09:19:33.032+0000 I NETWORK [conn4081121] end connection 10.0.27.237:35842 (1504 connections now open) 2016-03-13T09:19:33.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58277 #4083033 (1505 connections now open) 2016-03-13T09:19:33.033+0000 I NETWORK [conn4082026] end connection 10.0.26.69:42936 (1504 connections now open) 2016-03-13T09:19:33.033+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36674 #4083034 (1505 connections now open) 2016-03-13T09:19:33.033+0000 I NETWORK [conn4081898] end connection 10.0.20.53:51134 (1504 connections now open) 2016-03-13T09:19:33.034+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43364 #4083035 (1505 connections now open) 2016-03-13T09:19:33.034+0000 I NETWORK [conn4082408] end connection 10.0.216.90:57996 (1504 connections now open) 2016-03-13T09:19:33.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58279 #4083036 (1505 connections now open) 2016-03-13T09:19:33.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60230 #4083037 (1506 connections now open) 2016-03-13T09:19:33.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51613 #4083038 (1507 connections now open) 2016-03-13T09:19:33.037+0000 I NETWORK [conn4082504] end connection 10.0.20.53:51374 (1506 connections now open) 2016-03-13T09:19:33.037+0000 I NETWORK [conn4081427] end connection 10.0.20.53:50887 (1505 connections now open) 2016-03-13T09:19:33.037+0000 I NETWORK [conn4082238] end connection 10.0.7.113:59909 (1504 connections now open) 2016-03-13T09:19:33.038+0000 I NETWORK [conn4081228] end connection 10.0.7.113:59620 (1503 connections now open) 2016-03-13T09:19:33.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60231 #4083039 (1504 connections now open) 2016-03-13T09:19:33.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60232 #4083040 (1505 connections now open) 2016-03-13T09:19:33.038+0000 I NETWORK [conn4081234] end connection 10.0.26.69:42545 (1504 connections now open) 2016-03-13T09:19:33.039+0000 I NETWORK [conn4081151] end connection 10.0.7.113:59592 (1504 connections now open) 2016-03-13T09:19:33.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51614 #4083041 (1505 connections now open) 2016-03-13T09:19:33.039+0000 I NETWORK [conn4082101] end connection 10.0.26.69:42973 (1503 connections now open) 2016-03-13T09:19:33.039+0000 I NETWORK [conn4081964] end connection 10.0.20.53:51163 (1502 connections now open) 2016-03-13T09:19:33.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51616 #4083042 (1503 connections now open) 2016-03-13T09:19:33.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51617 #4083043 (1504 connections now open) 2016-03-13T09:19:33.040+0000 I NETWORK [conn4081803] end connection 10.0.7.113:59738 (1503 connections now open) 2016-03-13T09:19:33.040+0000 I NETWORK [conn4082272] end connection 10.0.27.237:36355 (1502 connections now open) 2016-03-13T09:19:33.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60235 #4083044 (1503 connections now open) 2016-03-13T09:19:33.041+0000 I NETWORK [conn4081927] end connection 10.0.179.209:40870 (1502 connections now open) 2016-03-13T09:19:33.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36675 #4083045 (1503 connections now open) 2016-03-13T09:19:33.041+0000 I NETWORK [conn4081439] end connection 10.0.20.53:50896 (1502 connections now open) 2016-03-13T09:19:33.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41335 #4083046 (1503 connections now open) 2016-03-13T09:19:33.042+0000 I NETWORK [conn4081620] end connection 10.0.20.53:50991 (1502 connections now open) 2016-03-13T09:19:33.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43368 #4083047 (1503 connections now open) 2016-03-13T09:19:33.042+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51618 #4083048 (1504 connections now open) 2016-03-13T09:19:33.043+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.043+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43369 #4083049 (1505 connections now open) 2016-03-13T09:19:33.043+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51619 #4083050 (1506 connections now open) 2016-03-13T09:19:33.044+0000 I NETWORK [conn4081487] end connection 10.0.26.69:42670 (1505 connections now open) 2016-03-13T09:19:33.044+0000 I NETWORK [conn4081259] end connection 10.0.179.209:40718 (1504 connections now open) 2016-03-13T09:19:33.044+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60238 #4083051 (1505 connections now open) 2016-03-13T09:19:33.044+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41336 #4083052 (1506 connections now open) 2016-03-13T09:19:33.047+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.048+0000 I NETWORK [conn4081460] end connection 10.0.20.53:50904 (1505 connections now open) 2016-03-13T09:19:33.049+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.049+0000 I NETWORK [conn4080826] end connection 10.0.7.113:59442 (1504 connections now open) 2016-03-13T09:19:33.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51624 #4083053 (1505 connections now open) 2016-03-13T09:19:33.050+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60240 #4083054 (1506 connections now open) 2016-03-13T09:19:33.051+0000 I NETWORK [conn4080773] end connection 10.0.7.113:59405 (1505 connections now open) 2016-03-13T09:19:33.052+0000 I NETWORK [conn4081214] end connection 10.0.7.113:59616 (1504 connections now open) 2016-03-13T09:19:33.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60241 #4083055 (1505 connections now open) 2016-03-13T09:19:33.053+0000 I NETWORK [conn4082893] end connection 10.0.20.53:51551 (1504 connections now open) 2016-03-13T09:19:33.053+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.054+0000 I NETWORK [conn4081286] end connection 10.0.7.113:59636 (1503 connections now open) 2016-03-13T09:19:33.054+0000 I NETWORK [conn4081541] end connection 10.0.7.113:59695 (1502 connections now open) 2016-03-13T09:19:33.055+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60243 #4083056 (1503 connections now open) 2016-03-13T09:19:33.055+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60244 #4083057 (1504 connections now open) 2016-03-13T09:19:33.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51626 #4083058 (1505 connections now open) 2016-03-13T09:19:33.057+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43374 #4083059 (1506 connections now open) 2016-03-13T09:19:33.058+0000 I NETWORK [conn4082368] end connection 10.0.26.69:43081 (1505 connections now open) 2016-03-13T09:19:33.059+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43375 #4083060 (1506 connections now open) 2016-03-13T09:19:33.059+0000 I NETWORK [conn4081467] end connection 10.0.20.53:50907 (1505 connections now open) 2016-03-13T09:19:33.059+0000 I NETWORK [conn4082290] end connection 10.0.20.53:51296 (1504 connections now open) 2016-03-13T09:19:33.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60246 #4083061 (1505 connections now open) 2016-03-13T09:19:33.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51627 #4083062 (1506 connections now open) 2016-03-13T09:19:33.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51628 #4083063 (1507 connections now open) 2016-03-13T09:19:33.063+0000 I NETWORK [conn4081503] end connection 10.0.26.69:42679 (1506 connections now open) 2016-03-13T09:19:33.063+0000 I NETWORK [conn4082963] end connection 10.0.7.113:60187 (1505 connections now open) 2016-03-13T09:19:33.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43377 #4083064 (1506 connections now open) 2016-03-13T09:19:33.063+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60247 #4083065 (1507 connections now open) 2016-03-13T09:19:33.063+0000 I NETWORK [conn4081221] end connection 10.0.7.113:59617 (1506 connections now open) 2016-03-13T09:19:33.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60248 #4083066 (1507 connections now open) 2016-03-13T09:19:33.064+0000 I NETWORK [conn4081900] end connection 10.0.26.69:42882 (1506 connections now open) 2016-03-13T09:19:33.064+0000 I NETWORK [conn4081614] end connection 10.0.26.69:42731 (1506 connections now open) 2016-03-13T09:19:33.064+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43378 #4083067 (1506 connections now open) 2016-03-13T09:19:33.065+0000 I NETWORK [conn4082566] end connection 10.0.26.69:43153 (1505 connections now open) 2016-03-13T09:19:33.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43379 #4083068 (1506 connections now open) 2016-03-13T09:19:33.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43380 #4083069 (1507 connections now open) 2016-03-13T09:19:33.067+0000 I NETWORK [conn4081896] end connection 10.0.179.209:40860 (1506 connections now open) 2016-03-13T09:19:33.067+0000 I NETWORK [conn4080779] end connection 10.0.7.113:59408 (1505 connections now open) 2016-03-13T09:19:33.068+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60249 #4083070 (1506 connections now open) 2016-03-13T09:19:33.068+0000 I NETWORK [conn4082532] end connection 10.0.20.53:51386 (1505 connections now open) 2016-03-13T09:19:33.069+0000 I NETWORK [conn4081672] end connection 10.0.20.53:51026 (1504 connections now open) 2016-03-13T09:19:33.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51629 #4083071 (1505 connections now open) 2016-03-13T09:19:33.069+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51630 #4083072 (1506 connections now open) 2016-03-13T09:19:33.071+0000 I NETWORK [conn4081682] end connection 10.0.20.53:51033 (1505 connections now open) 2016-03-13T09:19:33.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51631 #4083073 (1506 connections now open) 2016-03-13T09:19:33.073+0000 I NETWORK [conn4081813] end connection 10.0.26.69:42843 (1505 connections now open) 2016-03-13T09:19:33.073+0000 I NETWORK [conn4082369] end connection 10.0.7.113:59963 (1504 connections now open) 2016-03-13T09:19:33.073+0000 I NETWORK [conn4081518] end connection 10.0.26.69:42689 (1503 connections now open) 2016-03-13T09:19:33.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60250 #4083074 (1504 connections now open) 2016-03-13T09:19:33.073+0000 I NETWORK [conn4081638] end connection 10.0.26.69:42747 (1503 connections now open) 2016-03-13T09:19:33.074+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43384 #4083075 (1504 connections now open) 2016-03-13T09:19:33.074+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43385 #4083076 (1505 connections now open) 2016-03-13T09:19:33.074+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43386 #4083077 (1506 connections now open) 2016-03-13T09:19:33.078+0000 I NETWORK [conn4081224] end connection 10.0.26.69:42541 (1505 connections now open) 2016-03-13T09:19:33.082+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43388 #4083078 (1506 connections now open) 2016-03-13T09:19:33.082+0000 I NETWORK [conn4081048] end connection 10.0.7.113:59565 (1505 connections now open) 2016-03-13T09:19:33.082+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60251 #4083079 (1506 connections now open) 2016-03-13T09:19:33.089+0000 I NETWORK [conn4082227] end connection 10.0.179.209:41002 (1505 connections now open) 2016-03-13T09:19:33.089+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41341 #4083080 (1506 connections now open) 2016-03-13T09:19:33.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41342 #4083081 (1507 connections now open) 2016-03-13T09:19:33.092+0000 I NETWORK [conn4080857] end connection 10.0.7.113:59465 (1506 connections now open) 2016-03-13T09:19:33.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60254 #4083082 (1507 connections now open) 2016-03-13T09:19:33.094+0000 I NETWORK [conn4082964] end connection 10.0.7.113:60197 (1506 connections now open) 2016-03-13T09:19:33.094+0000 I NETWORK [conn4081104] end connection 10.0.179.209:40676 (1505 connections now open) 2016-03-13T09:19:33.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60255 #4083083 (1506 connections now open) 2016-03-13T09:19:33.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41347 #4083084 (1507 connections now open) 2016-03-13T09:19:33.096+0000 I NETWORK [conn4081566] end connection 10.0.7.113:59697 (1506 connections now open) 2016-03-13T09:19:33.097+0000 I NETWORK [conn4082524] end connection 10.0.179.209:41101 (1505 connections now open) 2016-03-13T09:19:33.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60256 #4083085 (1506 connections now open) 2016-03-13T09:19:33.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41348 #4083086 (1507 connections now open) 2016-03-13T09:19:33.098+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.098+0000 I NETWORK [conn4081114] end connection 10.0.179.209:40683 (1506 connections now open) 2016-03-13T09:19:33.099+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41349 #4083087 (1507 connections now open) 2016-03-13T09:19:33.102+0000 I NETWORK [conn4081573] end connection 10.0.179.209:40758 (1506 connections now open) 2016-03-13T09:19:33.103+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41351 #4083088 (1507 connections now open) 2016-03-13T09:19:33.105+0000 I NETWORK [conn4081989] end connection 10.0.179.209:40894 (1506 connections now open) 2016-03-13T09:19:33.105+0000 I NETWORK [conn4081489] end connection 10.0.7.113:59686 (1505 connections now open) 2016-03-13T09:19:33.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60258 #4083089 (1506 connections now open) 2016-03-13T09:19:33.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41353 #4083090 (1507 connections now open) 2016-03-13T09:19:33.121+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.132+0000 I NETWORK [conn4081301] end connection 10.0.7.113:59640 (1506 connections now open) 2016-03-13T09:19:33.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60259 #4083091 (1507 connections now open) 2016-03-13T09:19:33.143+0000 I NETWORK [conn4081515] end connection 10.0.179.209:40752 (1506 connections now open) 2016-03-13T09:19:33.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41357 #4083092 (1507 connections now open) 2016-03-13T09:19:33.156+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.197+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.202+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db2m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.234+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.237+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.239+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.251+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.256+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.260+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.265+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.300+0000 I ASIO [NetworkInterfaceASIO-ShardRegistry-0] Successfully connected to in.db3m1.mongo323.test.mydomain.com:27017 2016-03-13T09:19:33.322+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.AccountUsage: 0ms sequenceNumber: 17315 version: 16|1||54c675c232c2781032b65b30 based on: (empty) 2016-03-13T09:19:33.326+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.ActionReports: 4ms sequenceNumber: 17316 version: 158|49||54bfd498c21b8c3b0e0546bf based on: (empty) 2016-03-13T09:19:33.328+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.AppReports: 0ms sequenceNumber: 17317 version: 20|1||54bcb8f73ac63cd8b3f3f473 based on: (empty) 2016-03-13T09:19:33.330+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.AppScreenReports: 2ms sequenceNumber: 17318 version: 62|16||54bd28cc3ac63cd8b3f40346 based on: (empty) 2016-03-13T09:19:33.331+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.AppUsage: 0ms sequenceNumber: 17319 version: 10|1||54c675d032c2781032b65b35 based on: (empty) 2016-03-13T09:19:33.332+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.AppVersionReports: 0ms sequenceNumber: 17320 version: 18|0||54be186af5d6feabbc85642c based on: (empty) 2016-03-13T09:19:33.335+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.ClientHistoryReports: 2ms sequenceNumber: 17321 version: 90|52||54bd5af167066c5f3e307c03 based on: (empty) 2016-03-13T09:19:33.360+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.Clients: 25ms sequenceNumber: 17322 version: 681|1||54b68e630db2e6c50b4b6cc0 based on: (empty) 2016-03-13T09:19:33.364+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.ConnectivityReports: 0ms sequenceNumber: 17323 version: 16|1||54be1284f5d6feabbc856426 based on: (empty) 2016-03-13T09:19:33.366+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.CountryReports: 1ms sequenceNumber: 17324 version: 45|16||54bd7cef67066c5f3e307c0d based on: (empty) 2016-03-13T09:19:33.367+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.CrashIssueReports: 0ms sequenceNumber: 17325 version: 8|16||56c0a812f6dbf7ace67b5e89 based on: (empty) 2016-03-13T09:19:33.367+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.CrashedSessions: 0ms sequenceNumber: 17326 version: 13|1||56c0a82df6dbf7ace67b5e8d based on: (empty) 2016-03-13T09:19:33.409+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.CustomEventPropertyReports: 41ms sequenceNumber: 17327 version: 1855|383||54d2a8bc4797494e5a59c148 based on: (empty) 2016-03-13T09:19:33.415+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.CustomEventReports: 1ms sequenceNumber: 17328 version: 55|13||54be63ccf5d6feabbc85643b based on: (empty) 2016-03-13T09:19:33.415+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.DebugSymbolsCache: 0ms sequenceNumber: 17329 version: 3|1||560bdecc3e9033d8062c5b44 based on: (empty) 2016-03-13T09:19:33.421+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.DeviceCodeInternalReports: 5ms sequenceNumber: 17330 version: 240|91||54bf71f0c21b8c3b0e054666 based on: (empty) 2016-03-13T09:19:33.425+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.DeviceReports: 3ms sequenceNumber: 17331 version: 124|55||54be0baaf5d6feabbc856423 based on: (empty) 2016-03-13T09:19:33.426+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.DurationReports: 0ms sequenceNumber: 17332 version: 21|1||54be19c9f5d6feabbc85642f based on: (empty) 2016-03-13T09:19:33.427+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.FunnelReports: 0ms sequenceNumber: 17333 version: 11|1||54bfd0c7c21b8c3b0e0546b9 based on: (empty) 2016-03-13T09:19:33.431+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.GestureReports: 4ms sequenceNumber: 17334 version: 262|43||54be251cf5d6feabbc856432 based on: (empty) 2016-03-13T09:19:33.474+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.HeatmapRectReports: 42ms sequenceNumber: 17335 version: 1192|1||54becdcfb78b87228fe664fc based on: (empty) 2016-03-13T09:19:33.646+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.HeatmapReports: 168ms sequenceNumber: 17336 version: 8892|1468||54be86fcf5d6feabbc856443 based on: (empty) 2016-03-13T09:19:33.706+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.NavigationPathReports: 47ms sequenceNumber: 17337 version: 864|1075||558ab0e73e9033d8061ffc01 based on: (empty) 2016-03-13T09:19:33.719+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.NavigationReports: 7ms sequenceNumber: 17338 version: 416|118||54c0a354edc79a1ce5167cfa based on: (empty) 2016-03-13T09:19:33.721+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.OSVersionReports: 1ms sequenceNumber: 17339 version: 27|4||54bd7ee267066c5f3e307c11 based on: (empty) 2016-03-13T09:19:33.724+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.PendingSessions: 3ms sequenceNumber: 17340 version: 86|1||54c775a4509aea9affefdbe4 based on: (empty) 2016-03-13T09:19:33.726+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.ResolutionReports: 0ms sequenceNumber: 17341 version: 18|4||54be162bf5d6feabbc856429 based on: (empty) 2016-03-13T09:19:33.727+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.RetentionFrequencyReports: 1ms sequenceNumber: 17342 version: 43|1||54be7cb0f5d6feabbc856440 based on: (empty) 2016-03-13T09:19:33.729+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.RetentionReports: 2ms sequenceNumber: 17343 version: 88|22||54bf6061c21b8c3b0e054661 based on: (empty) 2016-03-13T09:19:33.731+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.RetentionTimeReports: 0ms sequenceNumber: 17344 version: 17|1||54bf5c80c21b8c3b0e05465e based on: (empty) 2016-03-13T09:19:33.731+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.RetentionZeroReports: 0ms sequenceNumber: 17345 version: 6|1||54be279af5d6feabbc856435 based on: (empty) 2016-03-13T09:19:33.773+0000 I SHARDING [conn2738625] ChunkManager: time to load chunks for mydomain.Sessions: 781ms sequenceNumber: 17314 version: 30654|1||563ba74d869758a4542b9075 based on: 30654|1||563ba74d869758a4542b9075 2016-03-13T09:19:34.293+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.SessionTouchEvents: 561ms sequenceNumber: 17346 version: 16124|1||5605338c093785687d439f5f based on: (empty) 2016-03-13T09:19:34.606+0000 I NETWORK [conn4082970] end connection 10.0.27.237:36643 (1506 connections now open) 2016-03-13T09:19:34.606+0000 I NETWORK [conn4082969] end connection 10.0.27.237:36644 (1505 connections now open) 2016-03-13T09:19:34.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36677 #4083093 (1506 connections now open) 2016-03-13T09:19:34.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36678 #4083094 (1507 connections now open) 2016-03-13T09:19:34.702+0000 I SHARDING [conn3579100] ChunkManager: time to load chunks for mydomain.Sessions: 709ms sequenceNumber: 17347 version: 30654|1||563ba74d869758a4542b9075 based on: 30654|1||563ba74d869758a4542b9075 2016-03-13T09:19:35.580+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 680ms sequenceNumber: 17349 version: 30654|1||563ba74d869758a4542b9075 based on: 30654|1||563ba74d869758a4542b9075 2016-03-13T09:19:36.742+0000 I NETWORK [conn4082972] end connection 10.0.7.113:60200 (1506 connections now open) 2016-03-13T09:19:36.742+0000 I NETWORK [conn4082971] end connection 10.0.7.113:60201 (1506 connections now open) 2016-03-13T09:19:36.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60266 #4083095 (1506 connections now open) 2016-03-13T09:19:36.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60267 #4083096 (1507 connections now open) 2016-03-13T09:19:37.786+0000 I NETWORK [conn4082974] end connection 10.0.216.90:58248 (1506 connections now open) 2016-03-13T09:19:37.786+0000 I NETWORK [conn4082973] end connection 10.0.216.90:58247 (1506 connections now open) 2016-03-13T09:19:37.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58292 #4083097 (1506 connections now open) 2016-03-13T09:19:37.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58293 #4083098 (1507 connections now open) 2016-03-13T09:19:38.018+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.Sessions: 3642ms sequenceNumber: 17348 version: 30654|1||563ba74d869758a4542b9075 based on: (empty) 2016-03-13T09:19:38.205+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.ShardTest: 1ms sequenceNumber: 17350 version: 7|1||56b72f7ff6dbf7ace67aec4c based on: (empty) 2016-03-13T09:19:38.206+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.UserFunnelDurationReports: 0ms sequenceNumber: 17351 version: 12|4||5523c23a0623e75c580bd19a based on: (empty) 2016-03-13T09:19:38.208+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.UserFunnelReports: 1ms sequenceNumber: 17352 version: 25|4||551cf7140623e75c580af24e based on: (empty) 2016-03-13T09:19:38.225+0000 I SHARDING [conn3585856] ChunkManager: time to load chunks for mydomain.UserFunnelStates: 17ms sequenceNumber: 17353 version: 553|252||551cf67f0623e75c580af236 based on: (empty) 2016-03-13T09:19:38.228+0000 W SHARDING [conn2738625] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30654|1||563ba74d869758a4542b9075 2016-03-13T09:19:38.228+0000 W SHARDING [conn3579100] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30654|1||563ba74d869758a4542b9075 2016-03-13T09:19:38.229+0000 W SHARDING [conn1051637] chunk manager reload forced for collection 'mydomain.Sessions', config version is 30654|1||563ba74d869758a4542b9075 2016-03-13T09:19:38.824+0000 I NETWORK [conn4082980] end connection 10.0.20.53:51585 (1506 connections now open) 2016-03-13T09:19:38.824+0000 I NETWORK [conn4082979] end connection 10.0.20.53:51584 (1506 connections now open) 2016-03-13T09:19:38.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51645 #4083099 (1506 connections now open) 2016-03-13T09:19:38.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51644 #4083100 (1507 connections now open) 2016-03-13T09:19:38.927+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17354 version: 30655|1||563ba74d869758a4542b9075 based on: 30654|1||563ba74d869758a4542b9075 2016-03-13T09:19:41.152+0000 I NETWORK [conn4082981] end connection 10.0.179.209:41311 (1506 connections now open) 2016-03-13T09:19:41.153+0000 I NETWORK [conn4082982] end connection 10.0.179.209:41312 (1505 connections now open) 2016-03-13T09:19:41.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41366 #4083101 (1506 connections now open) 2016-03-13T09:19:41.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41367 #4083102 (1507 connections now open) 2016-03-13T09:19:41.861+0000 I NETWORK [conn4082984] end connection 10.0.26.69:43348 (1506 connections now open) 2016-03-13T09:19:41.861+0000 I NETWORK [conn4082983] end connection 10.0.26.69:43347 (1506 connections now open) 2016-03-13T09:19:41.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43395 #4083103 (1506 connections now open) 2016-03-13T09:19:41.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43394 #4083104 (1507 connections now open) 2016-03-13T09:19:44.619+0000 I NETWORK [conn4083094] end connection 10.0.27.237:36678 (1506 connections now open) 2016-03-13T09:19:44.619+0000 I NETWORK [conn4083093] end connection 10.0.27.237:36677 (1506 connections now open) 2016-03-13T09:19:44.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36684 #4083105 (1506 connections now open) 2016-03-13T09:19:44.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36685 #4083106 (1507 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [conn4079520] end connection 10.0.216.90:56195 (1506 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [conn4081720] end connection 10.0.20.53:51054 (1506 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [conn4081713] end connection 10.0.26.69:42794 (1504 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [conn4081355] end connection 10.0.27.237:35920 (1503 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [conn4080874] end connection 10.0.7.113:59483 (1502 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51653 #4083107 (1503 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58311 #4083108 (1504 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36690 #4083109 (1505 connections now open) 2016-03-13T09:19:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43400 #4083110 (1506 connections now open) 2016-03-13T09:19:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60283 #4083111 (1507 connections now open) 2016-03-13T09:19:46.755+0000 I NETWORK [conn4083095] end connection 10.0.7.113:60266 (1506 connections now open) 2016-03-13T09:19:46.755+0000 I NETWORK [conn4083096] end connection 10.0.7.113:60267 (1506 connections now open) 2016-03-13T09:19:46.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60284 #4083112 (1506 connections now open) 2016-03-13T09:19:46.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60285 #4083113 (1507 connections now open) 2016-03-13T09:19:47.799+0000 I NETWORK [conn4083097] end connection 10.0.216.90:58292 (1506 connections now open) 2016-03-13T09:19:47.799+0000 I NETWORK [conn4083098] end connection 10.0.216.90:58293 (1506 connections now open) 2016-03-13T09:19:47.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58313 #4083114 (1506 connections now open) 2016-03-13T09:19:47.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58312 #4083115 (1507 connections now open) 2016-03-13T09:19:48.838+0000 I NETWORK [conn4083099] end connection 10.0.20.53:51645 (1506 connections now open) 2016-03-13T09:19:48.838+0000 I NETWORK [conn4083100] end connection 10.0.20.53:51644 (1506 connections now open) 2016-03-13T09:19:48.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51656 #4083116 (1506 connections now open) 2016-03-13T09:19:48.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51655 #4083117 (1507 connections now open) 2016-03-13T09:19:51.165+0000 I NETWORK [conn4083101] end connection 10.0.179.209:41366 (1506 connections now open) 2016-03-13T09:19:51.165+0000 I NETWORK [conn4083102] end connection 10.0.179.209:41367 (1505 connections now open) 2016-03-13T09:19:51.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41385 #4083118 (1506 connections now open) 2016-03-13T09:19:51.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41386 #4083119 (1507 connections now open) 2016-03-13T09:19:51.875+0000 I NETWORK [conn4083104] end connection 10.0.26.69:43394 (1506 connections now open) 2016-03-13T09:19:51.875+0000 I NETWORK [conn4083103] end connection 10.0.26.69:43395 (1506 connections now open) 2016-03-13T09:19:51.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43407 #4083120 (1506 connections now open) 2016-03-13T09:19:51.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43406 #4083121 (1507 connections now open) 2016-03-13T09:19:52.295+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:19:52.140+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:19:52.295+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:19:52.363+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530a318fa188537623d71 2016-03-13T09:19:52.431+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530ad18fa188537623d72 2016-03-13T09:19:52.502+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530b718fa188537623d73 2016-03-13T09:19:53.269+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17355 version: 30656|1||563ba74d869758a4542b9075 based on: 30655|1||563ba74d869758a4542b9075 2016-03-13T09:19:54.632+0000 I NETWORK [conn4083105] end connection 10.0.27.237:36684 (1506 connections now open) 2016-03-13T09:19:54.632+0000 I NETWORK [conn4083106] end connection 10.0.27.237:36685 (1506 connections now open) 2016-03-13T09:19:54.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36698 #4083122 (1506 connections now open) 2016-03-13T09:19:54.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36697 #4083123 (1507 connections now open) 2016-03-13T09:19:56.769+0000 I NETWORK [conn4083112] end connection 10.0.7.113:60284 (1506 connections now open) 2016-03-13T09:19:56.769+0000 I NETWORK [conn4083113] end connection 10.0.7.113:60285 (1506 connections now open) 2016-03-13T09:19:56.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60291 #4083124 (1506 connections now open) 2016-03-13T09:19:56.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60290 #4083125 (1507 connections now open) 2016-03-13T09:19:57.812+0000 I NETWORK [conn4083114] end connection 10.0.216.90:58313 (1506 connections now open) 2016-03-13T09:19:57.812+0000 I NETWORK [conn4083115] end connection 10.0.216.90:58312 (1506 connections now open) 2016-03-13T09:19:57.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58322 #4083126 (1506 connections now open) 2016-03-13T09:19:57.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58321 #4083127 (1507 connections now open) 2016-03-13T09:19:58.113+0000 I NETWORK [conn4082415] end connection 10.0.216.90:58001 (1506 connections now open) 2016-03-13T09:19:58.113+0000 I NETWORK [conn4083062] end connection 10.0.20.53:51627 (1506 connections now open) 2016-03-13T09:19:58.113+0000 I NETWORK [conn4081161] end connection 10.0.27.237:35857 (1504 connections now open) 2016-03-13T09:19:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58324 #4083128 (1505 connections now open) 2016-03-13T09:19:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51664 #4083129 (1506 connections now open) 2016-03-13T09:19:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36713 #4083130 (1507 connections now open) 2016-03-13T09:19:58.851+0000 I NETWORK [conn4083116] end connection 10.0.20.53:51656 (1506 connections now open) 2016-03-13T09:19:58.851+0000 I NETWORK [conn4083117] end connection 10.0.20.53:51655 (1506 connections now open) 2016-03-13T09:19:58.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51670 #4083131 (1506 connections now open) 2016-03-13T09:19:58.852+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51671 #4083132 (1507 connections now open) 2016-03-13T09:20:01.178+0000 I NETWORK [conn4083118] end connection 10.0.179.209:41385 (1506 connections now open) 2016-03-13T09:20:01.178+0000 I NETWORK [conn4083119] end connection 10.0.179.209:41386 (1505 connections now open) 2016-03-13T09:20:01.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41394 #4083133 (1506 connections now open) 2016-03-13T09:20:01.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41395 #4083134 (1507 connections now open) 2016-03-13T09:20:01.888+0000 I NETWORK [conn4083121] end connection 10.0.26.69:43406 (1506 connections now open) 2016-03-13T09:20:01.888+0000 I NETWORK [conn4083120] end connection 10.0.26.69:43407 (1506 connections now open) 2016-03-13T09:20:01.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43430 #4083135 (1506 connections now open) 2016-03-13T09:20:01.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43429 #4083136 (1507 connections now open) 2016-03-13T09:20:04.644+0000 I NETWORK [conn4083123] end connection 10.0.27.237:36697 (1506 connections now open) 2016-03-13T09:20:04.645+0000 I NETWORK [conn4083122] end connection 10.0.27.237:36698 (1506 connections now open) 2016-03-13T09:20:04.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36719 #4083137 (1506 connections now open) 2016-03-13T09:20:04.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36720 #4083138 (1507 connections now open) 2016-03-13T09:20:06.782+0000 I NETWORK [conn4083124] end connection 10.0.7.113:60291 (1506 connections now open) 2016-03-13T09:20:06.782+0000 I NETWORK [conn4083125] end connection 10.0.7.113:60290 (1506 connections now open) 2016-03-13T09:20:06.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60306 #4083139 (1506 connections now open) 2016-03-13T09:20:06.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60307 #4083140 (1507 connections now open) 2016-03-13T09:20:07.825+0000 I NETWORK [conn4083127] end connection 10.0.216.90:58321 (1506 connections now open) 2016-03-13T09:20:07.825+0000 I NETWORK [conn4083126] end connection 10.0.216.90:58322 (1506 connections now open) 2016-03-13T09:20:07.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58338 #4083141 (1506 connections now open) 2016-03-13T09:20:07.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58339 #4083142 (1507 connections now open) 2016-03-13T09:20:08.864+0000 I NETWORK [conn4083132] end connection 10.0.20.53:51671 (1506 connections now open) 2016-03-13T09:20:08.864+0000 I NETWORK [conn4083131] end connection 10.0.20.53:51670 (1506 connections now open) 2016-03-13T09:20:08.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51690 #4083143 (1506 connections now open) 2016-03-13T09:20:08.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51691 #4083144 (1507 connections now open) 2016-03-13T09:20:09.137+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17356 version: 30657|1||563ba74d869758a4542b9075 based on: 30656|1||563ba74d869758a4542b9075 2016-03-13T09:20:11.191+0000 I NETWORK [conn4083133] end connection 10.0.179.209:41394 (1506 connections now open) 2016-03-13T09:20:11.191+0000 I NETWORK [conn4083134] end connection 10.0.179.209:41395 (1506 connections now open) 2016-03-13T09:20:11.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41410 #4083145 (1506 connections now open) 2016-03-13T09:20:11.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41409 #4083146 (1507 connections now open) 2016-03-13T09:20:11.901+0000 I NETWORK [conn4083135] end connection 10.0.26.69:43430 (1506 connections now open) 2016-03-13T09:20:11.901+0000 I NETWORK [conn4083136] end connection 10.0.26.69:43429 (1506 connections now open) 2016-03-13T09:20:11.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43440 #4083147 (1507 connections now open) 2016-03-13T09:20:11.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43441 #4083148 (1507 connections now open) 2016-03-13T09:20:14.658+0000 I NETWORK [conn4083137] end connection 10.0.27.237:36719 (1506 connections now open) 2016-03-13T09:20:14.658+0000 I NETWORK [conn4083138] end connection 10.0.27.237:36720 (1506 connections now open) 2016-03-13T09:20:14.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36730 #4083149 (1506 connections now open) 2016-03-13T09:20:14.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36731 #4083150 (1507 connections now open) 2016-03-13T09:20:16.117+0000 I NETWORK [conn4081595] end connection 10.0.26.69:42723 (1506 connections now open) 2016-03-13T09:20:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43447 #4083151 (1507 connections now open) 2016-03-13T09:20:16.795+0000 I NETWORK [conn4083139] end connection 10.0.7.113:60306 (1506 connections now open) 2016-03-13T09:20:16.795+0000 I NETWORK [conn4083140] end connection 10.0.7.113:60307 (1506 connections now open) 2016-03-13T09:20:16.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60328 #4083152 (1506 connections now open) 2016-03-13T09:20:16.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60327 #4083153 (1507 connections now open) 2016-03-13T09:20:17.838+0000 I NETWORK [conn4083142] end connection 10.0.216.90:58339 (1506 connections now open) 2016-03-13T09:20:17.838+0000 I NETWORK [conn4083141] end connection 10.0.216.90:58338 (1506 connections now open) 2016-03-13T09:20:17.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58359 #4083154 (1506 connections now open) 2016-03-13T09:20:17.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58358 #4083155 (1507 connections now open) 2016-03-13T09:20:18.878+0000 I NETWORK [conn4083143] end connection 10.0.20.53:51690 (1506 connections now open) 2016-03-13T09:20:18.878+0000 I NETWORK [conn4083144] end connection 10.0.20.53:51691 (1506 connections now open) 2016-03-13T09:20:18.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51701 #4083156 (1506 connections now open) 2016-03-13T09:20:18.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51700 #4083157 (1507 connections now open) 2016-03-13T09:20:21.267+0000 I NETWORK [conn4083146] end connection 10.0.179.209:41409 (1506 connections now open) 2016-03-13T09:20:21.267+0000 I NETWORK [conn4083145] end connection 10.0.179.209:41410 (1506 connections now open) 2016-03-13T09:20:21.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41433 #4083158 (1506 connections now open) 2016-03-13T09:20:21.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41432 #4083159 (1507 connections now open) 2016-03-13T09:20:21.914+0000 I NETWORK [conn4083147] end connection 10.0.26.69:43440 (1506 connections now open) 2016-03-13T09:20:21.914+0000 I NETWORK [conn4083148] end connection 10.0.26.69:43441 (1506 connections now open) 2016-03-13T09:20:21.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43455 #4083160 (1506 connections now open) 2016-03-13T09:20:21.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43456 #4083161 (1507 connections now open) 2016-03-13T09:20:23.893+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:20:22.502+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:20:23.893+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:20:24.671+0000 I NETWORK [conn4083150] end connection 10.0.27.237:36731 (1506 connections now open) 2016-03-13T09:20:24.671+0000 I NETWORK [conn4083149] end connection 10.0.27.237:36730 (1506 connections now open) 2016-03-13T09:20:24.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36746 #4083162 (1506 connections now open) 2016-03-13T09:20:24.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36745 #4083163 (1507 connections now open) 2016-03-13T09:20:25.358+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530c118fa188537623d74 2016-03-13T09:20:25.493+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530cb18fa188537623d75 2016-03-13T09:20:25.631+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530d618fa188537623d76 2016-03-13T09:20:26.189+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17357 version: 30658|1||563ba74d869758a4542b9075 based on: 30657|1||563ba74d869758a4542b9075 2016-03-13T09:20:26.808+0000 I NETWORK [conn4083153] end connection 10.0.7.113:60327 (1506 connections now open) 2016-03-13T09:20:26.808+0000 I NETWORK [conn4083152] end connection 10.0.7.113:60328 (1506 connections now open) 2016-03-13T09:20:26.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60335 #4083164 (1506 connections now open) 2016-03-13T09:20:26.816+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60336 #4083165 (1507 connections now open) 2016-03-13T09:20:27.860+0000 I NETWORK [conn4083155] end connection 10.0.216.90:58358 (1506 connections now open) 2016-03-13T09:20:27.860+0000 I NETWORK [conn4083154] end connection 10.0.216.90:58359 (1506 connections now open) 2016-03-13T09:20:27.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58366 #4083166 (1506 connections now open) 2016-03-13T09:20:27.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58367 #4083167 (1507 connections now open) 2016-03-13T09:20:28.113+0000 I NETWORK [conn4081623] end connection 10.0.20.53:50994 (1506 connections now open) 2016-03-13T09:20:28.113+0000 I NETWORK [conn4082947] end connection 10.0.216.90:58241 (1506 connections now open) 2016-03-13T09:20:28.113+0000 I NETWORK [conn4081658] end connection 10.0.26.69:42761 (1506 connections now open) 2016-03-13T09:20:28.114+0000 I NETWORK [conn4081953] end connection 10.0.27.237:36202 (1503 connections now open) 2016-03-13T09:20:28.114+0000 I NETWORK [conn4082688] end connection 10.0.7.113:60061 (1503 connections now open) 2016-03-13T09:20:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58370 #4083168 (1503 connections now open) 2016-03-13T09:20:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51709 #4083169 (1504 connections now open) 2016-03-13T09:20:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36763 #4083170 (1505 connections now open) 2016-03-13T09:20:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60341 #4083171 (1506 connections now open) 2016-03-13T09:20:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43461 #4083172 (1507 connections now open) 2016-03-13T09:20:28.900+0000 I NETWORK [conn4083157] end connection 10.0.20.53:51700 (1506 connections now open) 2016-03-13T09:20:28.900+0000 I NETWORK [conn4083156] end connection 10.0.20.53:51701 (1506 connections now open) 2016-03-13T09:20:28.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51715 #4083173 (1507 connections now open) 2016-03-13T09:20:28.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51716 #4083174 (1507 connections now open) 2016-03-13T09:20:31.280+0000 I NETWORK [conn4083158] end connection 10.0.179.209:41433 (1506 connections now open) 2016-03-13T09:20:31.280+0000 I NETWORK [conn4083159] end connection 10.0.179.209:41432 (1506 connections now open) 2016-03-13T09:20:31.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41442 #4083175 (1506 connections now open) 2016-03-13T09:20:31.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41443 #4083176 (1507 connections now open) 2016-03-13T09:20:31.928+0000 I NETWORK [conn4083161] end connection 10.0.26.69:43456 (1506 connections now open) 2016-03-13T09:20:31.928+0000 I NETWORK [conn4083160] end connection 10.0.26.69:43455 (1506 connections now open) 2016-03-13T09:20:31.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43473 #4083177 (1506 connections now open) 2016-03-13T09:20:31.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43474 #4083178 (1507 connections now open) 2016-03-13T09:20:34.684+0000 I NETWORK [conn4083162] end connection 10.0.27.237:36746 (1506 connections now open) 2016-03-13T09:20:34.684+0000 I NETWORK [conn4083163] end connection 10.0.27.237:36745 (1506 connections now open) 2016-03-13T09:20:34.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36766 #4083179 (1506 connections now open) 2016-03-13T09:20:34.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36767 #4083180 (1507 connections now open) 2016-03-13T09:20:36.828+0000 I NETWORK [conn4083164] end connection 10.0.7.113:60335 (1506 connections now open) 2016-03-13T09:20:36.828+0000 I NETWORK [conn4083165] end connection 10.0.7.113:60336 (1506 connections now open) 2016-03-13T09:20:36.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60347 #4083181 (1506 connections now open) 2016-03-13T09:20:36.829+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60348 #4083182 (1507 connections now open) 2016-03-13T09:20:37.874+0000 I NETWORK [conn4083167] end connection 10.0.216.90:58367 (1506 connections now open) 2016-03-13T09:20:37.874+0000 I NETWORK [conn4083166] end connection 10.0.216.90:58366 (1506 connections now open) 2016-03-13T09:20:37.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58383 #4083183 (1506 connections now open) 2016-03-13T09:20:37.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58382 #4083184 (1507 connections now open) 2016-03-13T09:20:38.913+0000 I NETWORK [conn4083174] end connection 10.0.20.53:51716 (1506 connections now open) 2016-03-13T09:20:38.913+0000 I NETWORK [conn4083173] end connection 10.0.20.53:51715 (1506 connections now open) 2016-03-13T09:20:38.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51738 #4083185 (1506 connections now open) 2016-03-13T09:20:38.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51737 #4083186 (1507 connections now open) 2016-03-13T09:20:41.293+0000 I NETWORK [conn4083175] end connection 10.0.179.209:41442 (1506 connections now open) 2016-03-13T09:20:41.293+0000 I NETWORK [conn4083176] end connection 10.0.179.209:41443 (1506 connections now open) 2016-03-13T09:20:41.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41454 #4083187 (1506 connections now open) 2016-03-13T09:20:41.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41453 #4083188 (1507 connections now open) 2016-03-13T09:20:41.918+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17358 version: 30659|1||563ba74d869758a4542b9075 based on: 30658|1||563ba74d869758a4542b9075 2016-03-13T09:20:41.950+0000 I NETWORK [conn4083177] end connection 10.0.26.69:43473 (1506 connections now open) 2016-03-13T09:20:41.950+0000 I NETWORK [conn4083178] end connection 10.0.26.69:43474 (1506 connections now open) 2016-03-13T09:20:41.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43480 #4083189 (1506 connections now open) 2016-03-13T09:20:41.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43481 #4083190 (1507 connections now open) 2016-03-13T09:20:44.697+0000 I NETWORK [conn4083180] end connection 10.0.27.237:36767 (1506 connections now open) 2016-03-13T09:20:44.697+0000 I NETWORK [conn4083179] end connection 10.0.27.237:36766 (1506 connections now open) 2016-03-13T09:20:44.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36775 #4083191 (1506 connections now open) 2016-03-13T09:20:44.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36774 #4083192 (1507 connections now open) 2016-03-13T09:20:46.118+0000 I NETWORK [conn4082116] end connection 10.0.179.209:40950 (1506 connections now open) 2016-03-13T09:20:46.118+0000 I NETWORK [conn4081715] end connection 10.0.26.69:42795 (1506 connections now open) 2016-03-13T09:20:46.118+0000 I NETWORK [conn4076330] end connection 10.0.216.90:52868 (1504 connections now open) 2016-03-13T09:20:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41459 #4083193 (1505 connections now open) 2016-03-13T09:20:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43486 #4083194 (1506 connections now open) 2016-03-13T09:20:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58390 #4083195 (1507 connections now open) 2016-03-13T09:20:46.842+0000 I NETWORK [conn4083182] end connection 10.0.7.113:60348 (1506 connections now open) 2016-03-13T09:20:46.842+0000 I NETWORK [conn4083181] end connection 10.0.7.113:60347 (1506 connections now open) 2016-03-13T09:20:46.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60367 #4083196 (1507 connections now open) 2016-03-13T09:20:46.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60366 #4083197 (1507 connections now open) 2016-03-13T09:20:47.887+0000 I NETWORK [conn4083183] end connection 10.0.216.90:58383 (1506 connections now open) 2016-03-13T09:20:47.887+0000 I NETWORK [conn4083184] end connection 10.0.216.90:58382 (1506 connections now open) 2016-03-13T09:20:47.888+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58401 #4083198 (1506 connections now open) 2016-03-13T09:20:47.888+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58402 #4083199 (1507 connections now open) 2016-03-13T09:20:48.926+0000 I NETWORK [conn4083185] end connection 10.0.20.53:51738 (1506 connections now open) 2016-03-13T09:20:48.926+0000 I NETWORK [conn4083186] end connection 10.0.20.53:51737 (1506 connections now open) 2016-03-13T09:20:48.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51747 #4083200 (1506 connections now open) 2016-03-13T09:20:48.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51748 #4083201 (1507 connections now open) 2016-03-13T09:20:51.306+0000 I NETWORK [conn4083187] end connection 10.0.179.209:41454 (1506 connections now open) 2016-03-13T09:20:51.306+0000 I NETWORK [conn4083188] end connection 10.0.179.209:41453 (1505 connections now open) 2016-03-13T09:20:51.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41474 #4083202 (1506 connections now open) 2016-03-13T09:20:51.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41473 #4083203 (1507 connections now open) 2016-03-13T09:20:51.964+0000 I NETWORK [conn4083190] end connection 10.0.26.69:43481 (1506 connections now open) 2016-03-13T09:20:51.964+0000 I NETWORK [conn4083189] end connection 10.0.26.69:43480 (1506 connections now open) 2016-03-13T09:20:51.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43492 #4083204 (1506 connections now open) 2016-03-13T09:20:51.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43493 #4083205 (1507 connections now open) 2016-03-13T09:20:54.710+0000 I NETWORK [conn4083192] end connection 10.0.27.237:36774 (1506 connections now open) 2016-03-13T09:20:54.711+0000 I NETWORK [conn4083191] end connection 10.0.27.237:36775 (1505 connections now open) 2016-03-13T09:20:54.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36788 #4083206 (1506 connections now open) 2016-03-13T09:20:54.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36789 #4083207 (1507 connections now open) 2016-03-13T09:20:55.761+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:20:55.631+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:20:55.761+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:20:55.830+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530e118fa188537623d77 2016-03-13T09:20:55.898+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530eb18fa188537623d78 2016-03-13T09:20:55.966+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e530f618fa188537623d79 2016-03-13T09:20:56.855+0000 I NETWORK [conn4083196] end connection 10.0.7.113:60367 (1506 connections now open) 2016-03-13T09:20:56.855+0000 I NETWORK [conn4083197] end connection 10.0.7.113:60366 (1506 connections now open) 2016-03-13T09:20:56.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60374 #4083208 (1506 connections now open) 2016-03-13T09:20:56.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60375 #4083209 (1507 connections now open) 2016-03-13T09:20:57.570+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17359 version: 30660|1||563ba74d869758a4542b9075 based on: 30659|1||563ba74d869758a4542b9075 2016-03-13T09:20:57.900+0000 I NETWORK [conn4083198] end connection 10.0.216.90:58401 (1506 connections now open) 2016-03-13T09:20:57.900+0000 I NETWORK [conn4083199] end connection 10.0.216.90:58402 (1506 connections now open) 2016-03-13T09:20:57.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58409 #4083210 (1506 connections now open) 2016-03-13T09:20:57.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58408 #4083211 (1507 connections now open) 2016-03-13T09:20:58.114+0000 I NETWORK [conn4082775] end connection 10.0.216.90:58150 (1506 connections now open) 2016-03-13T09:20:58.114+0000 I NETWORK [conn4081319] end connection 10.0.7.113:59645 (1506 connections now open) 2016-03-13T09:20:58.114+0000 I NETWORK [conn4082497] end connection 10.0.27.237:36439 (1506 connections now open) 2016-03-13T09:20:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36804 #4083212 (1505 connections now open) 2016-03-13T09:20:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58412 #4083213 (1506 connections now open) 2016-03-13T09:20:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60380 #4083214 (1507 connections now open) 2016-03-13T09:20:58.939+0000 I NETWORK [conn4083201] end connection 10.0.20.53:51748 (1506 connections now open) 2016-03-13T09:20:58.939+0000 I NETWORK [conn4083200] end connection 10.0.20.53:51747 (1506 connections now open) 2016-03-13T09:20:58.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51760 #4083215 (1506 connections now open) 2016-03-13T09:20:58.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51761 #4083216 (1507 connections now open) 2016-03-13T09:21:01.319+0000 I NETWORK [conn4083202] end connection 10.0.179.209:41474 (1506 connections now open) 2016-03-13T09:21:01.319+0000 I NETWORK [conn4083203] end connection 10.0.179.209:41473 (1506 connections now open) 2016-03-13T09:21:01.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41483 #4083217 (1506 connections now open) 2016-03-13T09:21:01.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41484 #4083218 (1507 connections now open) 2016-03-13T09:21:01.978+0000 I NETWORK [conn4083204] end connection 10.0.26.69:43492 (1506 connections now open) 2016-03-13T09:21:01.978+0000 I NETWORK [conn4083205] end connection 10.0.26.69:43493 (1506 connections now open) 2016-03-13T09:21:01.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43514 #4083219 (1506 connections now open) 2016-03-13T09:21:01.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43515 #4083220 (1507 connections now open) 2016-03-13T09:21:04.723+0000 I NETWORK [conn4083207] end connection 10.0.27.237:36789 (1506 connections now open) 2016-03-13T09:21:04.723+0000 I NETWORK [conn4083206] end connection 10.0.27.237:36788 (1506 connections now open) 2016-03-13T09:21:04.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36809 #4083221 (1506 connections now open) 2016-03-13T09:21:04.723+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36810 #4083222 (1507 connections now open) 2016-03-13T09:21:06.869+0000 I NETWORK [conn4083208] end connection 10.0.7.113:60374 (1506 connections now open) 2016-03-13T09:21:06.869+0000 I NETWORK [conn4083209] end connection 10.0.7.113:60375 (1506 connections now open) 2016-03-13T09:21:06.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60388 #4083223 (1506 connections now open) 2016-03-13T09:21:06.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60389 #4083224 (1507 connections now open) 2016-03-13T09:21:07.914+0000 I NETWORK [conn4083210] end connection 10.0.216.90:58409 (1506 connections now open) 2016-03-13T09:21:07.914+0000 I NETWORK [conn4083211] end connection 10.0.216.90:58408 (1506 connections now open) 2016-03-13T09:21:07.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58424 #4083225 (1506 connections now open) 2016-03-13T09:21:07.914+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58423 #4083226 (1507 connections now open) 2016-03-13T09:21:08.952+0000 I NETWORK [conn4083215] end connection 10.0.20.53:51760 (1506 connections now open) 2016-03-13T09:21:08.952+0000 I NETWORK [conn4083216] end connection 10.0.20.53:51761 (1505 connections now open) 2016-03-13T09:21:08.953+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51779 #4083227 (1506 connections now open) 2016-03-13T09:21:08.953+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51780 #4083228 (1507 connections now open) 2016-03-13T09:21:11.332+0000 I NETWORK [conn4083217] end connection 10.0.179.209:41483 (1506 connections now open) 2016-03-13T09:21:11.332+0000 I NETWORK [conn4083218] end connection 10.0.179.209:41484 (1505 connections now open) 2016-03-13T09:21:11.339+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41498 #4083229 (1506 connections now open) 2016-03-13T09:21:11.340+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41497 #4083230 (1507 connections now open) 2016-03-13T09:21:11.991+0000 I NETWORK [conn4083219] end connection 10.0.26.69:43514 (1506 connections now open) 2016-03-13T09:21:11.991+0000 I NETWORK [conn4083220] end connection 10.0.26.69:43515 (1506 connections now open) 2016-03-13T09:21:11.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43525 #4083231 (1506 connections now open) 2016-03-13T09:21:11.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43524 #4083232 (1507 connections now open) 2016-03-13T09:21:14.190+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17360 version: 30661|1||563ba74d869758a4542b9075 based on: 30660|1||563ba74d869758a4542b9075 2016-03-13T09:21:14.735+0000 I NETWORK [conn4083221] end connection 10.0.27.237:36809 (1506 connections now open) 2016-03-13T09:21:14.736+0000 I NETWORK [conn4083222] end connection 10.0.27.237:36810 (1505 connections now open) 2016-03-13T09:21:14.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36820 #4083233 (1506 connections now open) 2016-03-13T09:21:14.736+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36821 #4083234 (1507 connections now open) 2016-03-13T09:21:16.117+0000 I NETWORK [conn4081086] end connection 10.0.216.90:57530 (1506 connections now open) 2016-03-13T09:21:16.117+0000 I NETWORK [conn4081350] end connection 10.0.27.237:35919 (1506 connections now open) 2016-03-13T09:21:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36827 #4083235 (1506 connections now open) 2016-03-13T09:21:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58432 #4083236 (1507 connections now open) 2016-03-13T09:21:16.882+0000 I NETWORK [conn4083223] end connection 10.0.7.113:60388 (1506 connections now open) 2016-03-13T09:21:16.882+0000 I NETWORK [conn4083224] end connection 10.0.7.113:60389 (1506 connections now open) 2016-03-13T09:21:16.883+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60411 #4083237 (1506 connections now open) 2016-03-13T09:21:16.883+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60410 #4083238 (1507 connections now open) 2016-03-13T09:21:17.927+0000 I NETWORK [conn4083225] end connection 10.0.216.90:58424 (1506 connections now open) 2016-03-13T09:21:17.927+0000 I NETWORK [conn4083226] end connection 10.0.216.90:58423 (1506 connections now open) 2016-03-13T09:21:17.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58443 #4083239 (1506 connections now open) 2016-03-13T09:21:17.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58442 #4083240 (1507 connections now open) 2016-03-13T09:21:18.965+0000 I NETWORK [conn4083228] end connection 10.0.20.53:51780 (1506 connections now open) 2016-03-13T09:21:18.965+0000 I NETWORK [conn4083227] end connection 10.0.20.53:51779 (1506 connections now open) 2016-03-13T09:21:18.973+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51790 #4083241 (1506 connections now open) 2016-03-13T09:21:18.973+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51789 #4083242 (1507 connections now open) 2016-03-13T09:21:21.352+0000 I NETWORK [conn4083230] end connection 10.0.179.209:41497 (1506 connections now open) 2016-03-13T09:21:21.352+0000 I NETWORK [conn4083229] end connection 10.0.179.209:41498 (1506 connections now open) 2016-03-13T09:21:21.352+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41520 #4083243 (1506 connections now open) 2016-03-13T09:21:21.352+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41521 #4083244 (1507 connections now open) 2016-03-13T09:21:22.005+0000 I NETWORK [conn4083231] end connection 10.0.26.69:43525 (1506 connections now open) 2016-03-13T09:21:22.005+0000 I NETWORK [conn4083232] end connection 10.0.26.69:43524 (1506 connections now open) 2016-03-13T09:21:22.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43538 #4083245 (1506 connections now open) 2016-03-13T09:21:22.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43539 #4083246 (1507 connections now open) 2016-03-13T09:21:24.748+0000 I NETWORK [conn4083233] end connection 10.0.27.237:36820 (1506 connections now open) 2016-03-13T09:21:24.748+0000 I NETWORK [conn4083234] end connection 10.0.27.237:36821 (1506 connections now open) 2016-03-13T09:21:24.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36835 #4083247 (1506 connections now open) 2016-03-13T09:21:24.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36834 #4083248 (1507 connections now open) 2016-03-13T09:21:26.895+0000 I NETWORK [conn4083237] end connection 10.0.7.113:60411 (1506 connections now open) 2016-03-13T09:21:26.895+0000 I NETWORK [conn4083238] end connection 10.0.7.113:60410 (1505 connections now open) 2016-03-13T09:21:26.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60419 #4083249 (1506 connections now open) 2016-03-13T09:21:26.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60418 #4083250 (1507 connections now open) 2016-03-13T09:21:27.907+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:21:25.967+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:21:27.907+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:21:27.940+0000 I NETWORK [conn4083240] end connection 10.0.216.90:58442 (1506 connections now open) 2016-03-13T09:21:27.940+0000 I NETWORK [conn4083239] end connection 10.0.216.90:58443 (1506 connections now open) 2016-03-13T09:21:27.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58451 #4083251 (1506 connections now open) 2016-03-13T09:21:27.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58452 #4083252 (1507 connections now open) 2016-03-13T09:21:28.113+0000 I NETWORK [conn4082366] end connection 10.0.216.90:57970 (1506 connections now open) 2016-03-13T09:21:28.113+0000 I NETWORK [conn4081325] end connection 10.0.7.113:59647 (1505 connections now open) 2016-03-13T09:21:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58454 #4083253 (1506 connections now open) 2016-03-13T09:21:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60425 #4083254 (1507 connections now open) 2016-03-13T09:21:28.986+0000 I NETWORK [conn4083242] end connection 10.0.20.53:51789 (1506 connections now open) 2016-03-13T09:21:28.986+0000 I NETWORK [conn4083241] end connection 10.0.20.53:51790 (1506 connections now open) 2016-03-13T09:21:28.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51798 #4083255 (1506 connections now open) 2016-03-13T09:21:28.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51799 #4083256 (1507 connections now open) 2016-03-13T09:21:29.408+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5310018fa188537623d7a 2016-03-13T09:21:29.619+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5310a18fa188537623d7b 2016-03-13T09:21:29.778+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5311418fa188537623d7c 2016-03-13T09:21:30.256+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 692ms sequenceNumber: 17361 version: 30662|1||563ba74d869758a4542b9075 based on: 30661|1||563ba74d869758a4542b9075 2016-03-13T09:21:31.365+0000 I NETWORK [conn4083243] end connection 10.0.179.209:41520 (1506 connections now open) 2016-03-13T09:21:31.365+0000 I NETWORK [conn4083244] end connection 10.0.179.209:41521 (1506 connections now open) 2016-03-13T09:21:31.365+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41530 #4083257 (1506 connections now open) 2016-03-13T09:21:31.366+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41531 #4083258 (1507 connections now open) 2016-03-13T09:21:32.018+0000 I NETWORK [conn4083245] end connection 10.0.26.69:43538 (1506 connections now open) 2016-03-13T09:21:32.019+0000 I NETWORK [conn4083246] end connection 10.0.26.69:43539 (1505 connections now open) 2016-03-13T09:21:32.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43556 #4083259 (1506 connections now open) 2016-03-13T09:21:32.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43557 #4083260 (1507 connections now open) 2016-03-13T09:21:34.762+0000 I NETWORK [conn4083247] end connection 10.0.27.237:36835 (1506 connections now open) 2016-03-13T09:21:34.762+0000 I NETWORK [conn4083248] end connection 10.0.27.237:36834 (1506 connections now open) 2016-03-13T09:21:34.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36857 #4083261 (1506 connections now open) 2016-03-13T09:21:34.771+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36856 #4083262 (1507 connections now open) 2016-03-13T09:21:36.918+0000 I NETWORK [conn4083249] end connection 10.0.7.113:60419 (1506 connections now open) 2016-03-13T09:21:36.918+0000 I NETWORK [conn4083250] end connection 10.0.7.113:60418 (1505 connections now open) 2016-03-13T09:21:36.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60432 #4083263 (1506 connections now open) 2016-03-13T09:21:36.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60433 #4083264 (1507 connections now open) 2016-03-13T09:21:37.962+0000 I NETWORK [conn4083251] end connection 10.0.216.90:58451 (1506 connections now open) 2016-03-13T09:21:37.962+0000 I NETWORK [conn4083252] end connection 10.0.216.90:58452 (1506 connections now open) 2016-03-13T09:21:37.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58466 #4083265 (1506 connections now open) 2016-03-13T09:21:37.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58467 #4083266 (1507 connections now open) 2016-03-13T09:21:38.999+0000 I NETWORK [conn4083255] end connection 10.0.20.53:51798 (1506 connections now open) 2016-03-13T09:21:38.999+0000 I NETWORK [conn4083256] end connection 10.0.20.53:51799 (1506 connections now open) 2016-03-13T09:21:39.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51814 #4083267 (1506 connections now open) 2016-03-13T09:21:39.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51813 #4083268 (1507 connections now open) 2016-03-13T09:21:41.378+0000 I NETWORK [conn4083257] end connection 10.0.179.209:41530 (1506 connections now open) 2016-03-13T09:21:41.378+0000 I NETWORK [conn4083258] end connection 10.0.179.209:41531 (1506 connections now open) 2016-03-13T09:21:41.379+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41541 #4083269 (1506 connections now open) 2016-03-13T09:21:41.379+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41542 #4083270 (1507 connections now open) 2016-03-13T09:21:42.039+0000 I NETWORK [conn4083259] end connection 10.0.26.69:43556 (1506 connections now open) 2016-03-13T09:21:42.039+0000 I NETWORK [conn4083260] end connection 10.0.26.69:43557 (1506 connections now open) 2016-03-13T09:21:42.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43565 #4083271 (1506 connections now open) 2016-03-13T09:21:42.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43564 #4083272 (1507 connections now open) 2016-03-13T09:21:44.783+0000 I NETWORK [conn4083261] end connection 10.0.27.237:36857 (1506 connections now open) 2016-03-13T09:21:44.783+0000 I NETWORK [conn4083262] end connection 10.0.27.237:36856 (1505 connections now open) 2016-03-13T09:21:44.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36865 #4083273 (1506 connections now open) 2016-03-13T09:21:44.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36866 #4083274 (1507 connections now open) 2016-03-13T09:21:45.474+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 695ms sequenceNumber: 17362 version: 30663|1||563ba74d869758a4542b9075 based on: 30662|1||563ba74d869758a4542b9075 2016-03-13T09:21:46.117+0000 I NETWORK [conn4082786] end connection 10.0.179.209:41220 (1506 connections now open) 2016-03-13T09:21:46.117+0000 I NETWORK [conn4081552] end connection 10.0.27.237:36013 (1505 connections now open) 2016-03-13T09:21:46.117+0000 I NETWORK [conn4081608] end connection 10.0.26.69:42729 (1504 connections now open) 2016-03-13T09:21:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41545 #4083275 (1505 connections now open) 2016-03-13T09:21:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36870 #4083276 (1506 connections now open) 2016-03-13T09:21:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43571 #4083277 (1507 connections now open) 2016-03-13T09:21:46.931+0000 I NETWORK [conn4083263] end connection 10.0.7.113:60432 (1506 connections now open) 2016-03-13T09:21:46.931+0000 I NETWORK [conn4083264] end connection 10.0.7.113:60433 (1506 connections now open) 2016-03-13T09:21:46.932+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60451 #4083278 (1506 connections now open) 2016-03-13T09:21:46.932+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60452 #4083279 (1507 connections now open) 2016-03-13T09:21:47.975+0000 I NETWORK [conn4083266] end connection 10.0.216.90:58467 (1506 connections now open) 2016-03-13T09:21:47.975+0000 I NETWORK [conn4083265] end connection 10.0.216.90:58466 (1506 connections now open) 2016-03-13T09:21:47.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58485 #4083280 (1506 connections now open) 2016-03-13T09:21:47.976+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58484 #4083281 (1507 connections now open) 2016-03-13T09:21:49.013+0000 I NETWORK [conn4083267] end connection 10.0.20.53:51814 (1506 connections now open) 2016-03-13T09:21:49.013+0000 I NETWORK [conn4083268] end connection 10.0.20.53:51813 (1506 connections now open) 2016-03-13T09:21:49.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51833 #4083282 (1506 connections now open) 2016-03-13T09:21:49.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51834 #4083283 (1507 connections now open) 2016-03-13T09:21:51.391+0000 I NETWORK [conn4083269] end connection 10.0.179.209:41541 (1506 connections now open) 2016-03-13T09:21:51.391+0000 I NETWORK [conn4083270] end connection 10.0.179.209:41542 (1506 connections now open) 2016-03-13T09:21:51.392+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41560 #4083284 (1506 connections now open) 2016-03-13T09:21:51.392+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41561 #4083285 (1507 connections now open) 2016-03-13T09:21:52.052+0000 I NETWORK [conn4083272] end connection 10.0.26.69:43564 (1506 connections now open) 2016-03-13T09:21:52.052+0000 I NETWORK [conn4083271] end connection 10.0.26.69:43565 (1506 connections now open) 2016-03-13T09:21:52.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43578 #4083286 (1506 connections now open) 2016-03-13T09:21:52.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43577 #4083287 (1507 connections now open) 2016-03-13T09:21:54.796+0000 I NETWORK [conn4083273] end connection 10.0.27.237:36865 (1506 connections now open) 2016-03-13T09:21:54.796+0000 I NETWORK [conn4083274] end connection 10.0.27.237:36866 (1506 connections now open) 2016-03-13T09:21:54.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36877 #4083288 (1506 connections now open) 2016-03-13T09:21:54.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36878 #4083289 (1507 connections now open) 2016-03-13T09:21:56.945+0000 I NETWORK [conn4083278] end connection 10.0.7.113:60451 (1506 connections now open) 2016-03-13T09:21:56.945+0000 I NETWORK [conn4083279] end connection 10.0.7.113:60452 (1506 connections now open) 2016-03-13T09:21:56.945+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60460 #4083290 (1506 connections now open) 2016-03-13T09:21:56.945+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60459 #4083291 (1507 connections now open) 2016-03-13T09:21:57.988+0000 I NETWORK [conn4083281] end connection 10.0.216.90:58484 (1506 connections now open) 2016-03-13T09:21:57.988+0000 I NETWORK [conn4083280] end connection 10.0.216.90:58485 (1506 connections now open) 2016-03-13T09:21:57.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58491 #4083292 (1506 connections now open) 2016-03-13T09:21:57.989+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58492 #4083293 (1507 connections now open) 2016-03-13T09:21:58.113+0000 I NETWORK [conn4082050] end connection 10.0.26.69:42958 (1506 connections now open) 2016-03-13T09:21:58.113+0000 I NETWORK [conn4080233] end connection 10.0.27.237:35245 (1505 connections now open) 2016-03-13T09:21:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43585 #4083294 (1506 connections now open) 2016-03-13T09:21:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36885 #4083295 (1507 connections now open) 2016-03-13T09:21:59.026+0000 I NETWORK [conn4083283] end connection 10.0.20.53:51834 (1506 connections now open) 2016-03-13T09:21:59.026+0000 I NETWORK [conn4083282] end connection 10.0.20.53:51833 (1506 connections now open) 2016-03-13T09:21:59.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51842 #4083296 (1506 connections now open) 2016-03-13T09:21:59.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51843 #4083297 (1507 connections now open) 2016-03-13T09:21:59.944+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:21:59.778+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:21:59.944+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:22:00.012+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5311e18fa188537623d7d 2016-03-13T09:22:00.080+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5312818fa188537623d7e 2016-03-13T09:22:00.149+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5313218fa188537623d7f 2016-03-13T09:22:00.229+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17363 version: 30664|1||563ba74d869758a4542b9075 based on: 30663|1||563ba74d869758a4542b9075 2016-03-13T09:22:01.404+0000 I NETWORK [conn4083284] end connection 10.0.179.209:41560 (1506 connections now open) 2016-03-13T09:22:01.404+0000 I NETWORK [conn4083285] end connection 10.0.179.209:41561 (1505 connections now open) 2016-03-13T09:22:01.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41570 #4083298 (1506 connections now open) 2016-03-13T09:22:01.405+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41569 #4083299 (1507 connections now open) 2016-03-13T09:22:02.065+0000 I NETWORK [conn4083286] end connection 10.0.26.69:43578 (1506 connections now open) 2016-03-13T09:22:02.065+0000 I NETWORK [conn4083287] end connection 10.0.26.69:43577 (1506 connections now open) 2016-03-13T09:22:02.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43599 #4083300 (1507 connections now open) 2016-03-13T09:22:02.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43598 #4083301 (1507 connections now open) 2016-03-13T09:22:04.809+0000 I NETWORK [conn4083288] end connection 10.0.27.237:36877 (1506 connections now open) 2016-03-13T09:22:04.809+0000 I NETWORK [conn4083289] end connection 10.0.27.237:36878 (1506 connections now open) 2016-03-13T09:22:04.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36901 #4083302 (1506 connections now open) 2016-03-13T09:22:04.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36900 #4083303 (1507 connections now open) 2016-03-13T09:22:06.958+0000 I NETWORK [conn4083290] end connection 10.0.7.113:60460 (1506 connections now open) 2016-03-13T09:22:06.958+0000 I NETWORK [conn4083291] end connection 10.0.7.113:60459 (1506 connections now open) 2016-03-13T09:22:06.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60474 #4083304 (1506 connections now open) 2016-03-13T09:22:06.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60473 #4083305 (1507 connections now open) 2016-03-13T09:22:08.001+0000 I NETWORK [conn4083293] end connection 10.0.216.90:58492 (1506 connections now open) 2016-03-13T09:22:08.001+0000 I NETWORK [conn4083292] end connection 10.0.216.90:58491 (1506 connections now open) 2016-03-13T09:22:08.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58507 #4083306 (1506 connections now open) 2016-03-13T09:22:08.002+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58508 #4083307 (1507 connections now open) 2016-03-13T09:22:09.039+0000 I NETWORK [conn4083297] end connection 10.0.20.53:51843 (1506 connections now open) 2016-03-13T09:22:09.039+0000 I NETWORK [conn4083296] end connection 10.0.20.53:51842 (1506 connections now open) 2016-03-13T09:22:09.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51857 #4083308 (1506 connections now open) 2016-03-13T09:22:09.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51856 #4083309 (1507 connections now open) 2016-03-13T09:22:11.417+0000 I NETWORK [conn4083298] end connection 10.0.179.209:41570 (1506 connections now open) 2016-03-13T09:22:11.417+0000 I NETWORK [conn4083299] end connection 10.0.179.209:41569 (1506 connections now open) 2016-03-13T09:22:11.426+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41582 #4083310 (1506 connections now open) 2016-03-13T09:22:11.426+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41583 #4083311 (1507 connections now open) 2016-03-13T09:22:12.079+0000 I NETWORK [conn4083300] end connection 10.0.26.69:43599 (1506 connections now open) 2016-03-13T09:22:12.079+0000 I NETWORK [conn4083301] end connection 10.0.26.69:43598 (1506 connections now open) 2016-03-13T09:22:12.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43610 #4083312 (1506 connections now open) 2016-03-13T09:22:12.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43611 #4083313 (1507 connections now open) 2016-03-13T09:22:14.822+0000 I NETWORK [conn4083303] end connection 10.0.27.237:36900 (1506 connections now open) 2016-03-13T09:22:14.822+0000 I NETWORK [conn4083302] end connection 10.0.27.237:36901 (1506 connections now open) 2016-03-13T09:22:14.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36911 #4083314 (1506 connections now open) 2016-03-13T09:22:14.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36912 #4083315 (1507 connections now open) 2016-03-13T09:22:16.091+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17364 version: 30665|1||563ba74d869758a4542b9075 based on: 30664|1||563ba74d869758a4542b9075 2016-03-13T09:22:16.117+0000 I NETWORK [conn4081524] end connection 10.0.179.209:40754 (1506 connections now open) 2016-03-13T09:22:16.117+0000 I NETWORK [conn4081446] end connection 10.0.26.69:42650 (1505 connections now open) 2016-03-13T09:22:16.117+0000 I NETWORK [conn4081533] end connection 10.0.27.237:36004 (1504 connections now open) 2016-03-13T09:22:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41587 #4083316 (1505 connections now open) 2016-03-13T09:22:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43617 #4083317 (1506 connections now open) 2016-03-13T09:22:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36917 #4083318 (1507 connections now open) 2016-03-13T09:22:16.971+0000 I NETWORK [conn4083304] end connection 10.0.7.113:60474 (1506 connections now open) 2016-03-13T09:22:16.971+0000 I NETWORK [conn4083305] end connection 10.0.7.113:60473 (1506 connections now open) 2016-03-13T09:22:16.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60494 #4083319 (1506 connections now open) 2016-03-13T09:22:16.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60495 #4083320 (1507 connections now open) 2016-03-13T09:22:18.014+0000 I NETWORK [conn4083306] end connection 10.0.216.90:58507 (1506 connections now open) 2016-03-13T09:22:18.014+0000 I NETWORK [conn4083307] end connection 10.0.216.90:58508 (1505 connections now open) 2016-03-13T09:22:18.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58527 #4083321 (1506 connections now open) 2016-03-13T09:22:18.015+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58526 #4083322 (1507 connections now open) 2016-03-13T09:22:19.053+0000 I NETWORK [conn4083309] end connection 10.0.20.53:51856 (1506 connections now open) 2016-03-13T09:22:19.053+0000 I NETWORK [conn4083308] end connection 10.0.20.53:51857 (1506 connections now open) 2016-03-13T09:22:19.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51877 #4083323 (1506 connections now open) 2016-03-13T09:22:19.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51878 #4083324 (1507 connections now open) 2016-03-13T09:22:21.438+0000 I NETWORK [conn4083310] end connection 10.0.179.209:41582 (1506 connections now open) 2016-03-13T09:22:21.439+0000 I NETWORK [conn4083311] end connection 10.0.179.209:41583 (1505 connections now open) 2016-03-13T09:22:21.439+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41605 #4083325 (1506 connections now open) 2016-03-13T09:22:21.439+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41606 #4083326 (1507 connections now open) 2016-03-13T09:22:22.092+0000 I NETWORK [conn4083313] end connection 10.0.26.69:43611 (1506 connections now open) 2016-03-13T09:22:22.092+0000 I NETWORK [conn4083312] end connection 10.0.26.69:43610 (1506 connections now open) 2016-03-13T09:22:22.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43624 #4083327 (1506 connections now open) 2016-03-13T09:22:22.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43623 #4083328 (1507 connections now open) 2016-03-13T09:22:24.835+0000 I NETWORK [conn4083315] end connection 10.0.27.237:36912 (1506 connections now open) 2016-03-13T09:22:24.835+0000 I NETWORK [conn4083314] end connection 10.0.27.237:36911 (1506 connections now open) 2016-03-13T09:22:24.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36924 #4083329 (1506 connections now open) 2016-03-13T09:22:24.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36925 #4083330 (1507 connections now open) 2016-03-13T09:22:26.984+0000 I NETWORK [conn4083320] end connection 10.0.7.113:60495 (1506 connections now open) 2016-03-13T09:22:26.984+0000 I NETWORK [conn4083319] end connection 10.0.7.113:60494 (1506 connections now open) 2016-03-13T09:22:26.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60502 #4083331 (1506 connections now open) 2016-03-13T09:22:26.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60503 #4083332 (1507 connections now open) 2016-03-13T09:22:28.027+0000 I NETWORK [conn4083321] end connection 10.0.216.90:58527 (1506 connections now open) 2016-03-13T09:22:28.027+0000 I NETWORK [conn4083322] end connection 10.0.216.90:58526 (1506 connections now open) 2016-03-13T09:22:28.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58534 #4083333 (1506 connections now open) 2016-03-13T09:22:28.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58535 #4083334 (1507 connections now open) 2016-03-13T09:22:28.113+0000 I NETWORK [conn4081578] end connection 10.0.26.69:42717 (1506 connections now open) 2016-03-13T09:22:28.114+0000 I NETWORK [conn4081171] end connection 10.0.7.113:59601 (1505 connections now open) 2016-03-13T09:22:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43630 #4083335 (1506 connections now open) 2016-03-13T09:22:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60506 #4083336 (1507 connections now open) 2016-03-13T09:22:29.076+0000 I NETWORK [conn4083324] end connection 10.0.20.53:51878 (1506 connections now open) 2016-03-13T09:22:29.076+0000 I NETWORK [conn4083323] end connection 10.0.20.53:51877 (1506 connections now open) 2016-03-13T09:22:29.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51886 #4083337 (1506 connections now open) 2016-03-13T09:22:29.076+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51887 #4083338 (1507 connections now open) 2016-03-13T09:22:31.452+0000 I NETWORK [conn4083325] end connection 10.0.179.209:41605 (1506 connections now open) 2016-03-13T09:22:31.452+0000 I NETWORK [conn4083326] end connection 10.0.179.209:41606 (1506 connections now open) 2016-03-13T09:22:31.452+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41615 #4083339 (1506 connections now open) 2016-03-13T09:22:31.452+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41616 #4083340 (1507 connections now open) 2016-03-13T09:22:31.974+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:22:30.149+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:22:31.974+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:22:32.104+0000 I NETWORK [conn4083328] end connection 10.0.26.69:43623 (1506 connections now open) 2016-03-13T09:22:32.105+0000 I NETWORK [conn4083327] end connection 10.0.26.69:43624 (1505 connections now open) 2016-03-13T09:22:32.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43643 #4083341 (1506 connections now open) 2016-03-13T09:22:32.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43642 #4083342 (1507 connections now open) 2016-03-13T09:22:34.078+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17365 version: 30666|1||563ba74d869758a4542b9075 based on: 30665|1||563ba74d869758a4542b9075 2016-03-13T09:22:34.848+0000 I NETWORK [conn4083330] end connection 10.0.27.237:36925 (1506 connections now open) 2016-03-13T09:22:34.848+0000 I NETWORK [conn4083329] end connection 10.0.27.237:36924 (1506 connections now open) 2016-03-13T09:22:34.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36946 #4083343 (1506 connections now open) 2016-03-13T09:22:34.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36945 #4083344 (1507 connections now open) 2016-03-13T09:22:35.907+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5313c18fa188537623d80 2016-03-13T09:22:37.007+0000 I NETWORK [conn4083332] end connection 10.0.7.113:60503 (1506 connections now open) 2016-03-13T09:22:37.007+0000 I NETWORK [conn4083331] end connection 10.0.7.113:60502 (1506 connections now open) 2016-03-13T09:22:37.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60514 #4083345 (1506 connections now open) 2016-03-13T09:22:37.008+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60515 #4083346 (1507 connections now open) 2016-03-13T09:22:37.407+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5314618fa188537623d81 2016-03-13T09:22:38.047+0000 I NETWORK [conn4083334] end connection 10.0.216.90:58535 (1506 connections now open) 2016-03-13T09:22:38.048+0000 I NETWORK [conn4083333] end connection 10.0.216.90:58534 (1505 connections now open) 2016-03-13T09:22:38.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58549 #4083347 (1507 connections now open) 2016-03-13T09:22:38.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58550 #4083348 (1507 connections now open) 2016-03-13T09:22:38.738+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5315018fa188537623d82 2016-03-13T09:22:39.089+0000 I NETWORK [conn4083337] end connection 10.0.20.53:51886 (1506 connections now open) 2016-03-13T09:22:39.089+0000 I NETWORK [conn4083338] end connection 10.0.20.53:51887 (1506 connections now open) 2016-03-13T09:22:39.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51903 #4083349 (1506 connections now open) 2016-03-13T09:22:39.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51902 #4083350 (1507 connections now open) 2016-03-13T09:22:41.464+0000 I NETWORK [conn4083339] end connection 10.0.179.209:41615 (1506 connections now open) 2016-03-13T09:22:41.465+0000 I NETWORK [conn4083340] end connection 10.0.179.209:41616 (1505 connections now open) 2016-03-13T09:22:41.465+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41627 #4083351 (1506 connections now open) 2016-03-13T09:22:41.465+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41626 #4083352 (1507 connections now open) 2016-03-13T09:22:42.126+0000 I NETWORK [conn4083341] end connection 10.0.26.69:43643 (1506 connections now open) 2016-03-13T09:22:42.126+0000 I NETWORK [conn4083342] end connection 10.0.26.69:43642 (1506 connections now open) 2016-03-13T09:22:42.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43649 #4083353 (1506 connections now open) 2016-03-13T09:22:42.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43650 #4083354 (1507 connections now open) 2016-03-13T09:22:44.868+0000 I NETWORK [conn4083343] end connection 10.0.27.237:36946 (1506 connections now open) 2016-03-13T09:22:44.868+0000 I NETWORK [conn4083344] end connection 10.0.27.237:36945 (1506 connections now open) 2016-03-13T09:22:44.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36953 #4083355 (1506 connections now open) 2016-03-13T09:22:44.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36954 #4083356 (1507 connections now open) 2016-03-13T09:22:46.118+0000 I NETWORK [conn4081514] end connection 10.0.20.53:50936 (1506 connections now open) 2016-03-13T09:22:46.118+0000 I NETWORK [conn4081143] end connection 10.0.27.237:35850 (1505 connections now open) 2016-03-13T09:22:46.118+0000 I NETWORK [conn4080876] end connection 10.0.7.113:59485 (1504 connections now open) 2016-03-13T09:22:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36960 #4083357 (1505 connections now open) 2016-03-13T09:22:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51920 #4083358 (1506 connections now open) 2016-03-13T09:22:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60523 #4083359 (1507 connections now open) 2016-03-13T09:22:47.021+0000 I NETWORK [conn4083346] end connection 10.0.7.113:60515 (1506 connections now open) 2016-03-13T09:22:47.021+0000 I NETWORK [conn4083345] end connection 10.0.7.113:60514 (1506 connections now open) 2016-03-13T09:22:47.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60534 #4083360 (1506 connections now open) 2016-03-13T09:22:47.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60533 #4083361 (1507 connections now open) 2016-03-13T09:22:48.060+0000 I NETWORK [conn4083347] end connection 10.0.216.90:58549 (1506 connections now open) 2016-03-13T09:22:48.060+0000 I NETWORK [conn4083348] end connection 10.0.216.90:58550 (1505 connections now open) 2016-03-13T09:22:48.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58568 #4083362 (1506 connections now open) 2016-03-13T09:22:48.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58569 #4083363 (1507 connections now open) 2016-03-13T09:22:49.103+0000 I NETWORK [conn4083350] end connection 10.0.20.53:51902 (1506 connections now open) 2016-03-13T09:22:49.103+0000 I NETWORK [conn4083349] end connection 10.0.20.53:51903 (1506 connections now open) 2016-03-13T09:22:49.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51923 #4083364 (1506 connections now open) 2016-03-13T09:22:49.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51922 #4083365 (1507 connections now open) 2016-03-13T09:22:51.477+0000 I NETWORK [conn4083351] end connection 10.0.179.209:41627 (1506 connections now open) 2016-03-13T09:22:51.477+0000 I NETWORK [conn4083352] end connection 10.0.179.209:41626 (1506 connections now open) 2016-03-13T09:22:51.478+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41646 #4083366 (1506 connections now open) 2016-03-13T09:22:51.478+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41647 #4083367 (1507 connections now open) 2016-03-13T09:22:52.139+0000 I NETWORK [conn4083354] end connection 10.0.26.69:43650 (1506 connections now open) 2016-03-13T09:22:52.139+0000 I NETWORK [conn4083353] end connection 10.0.26.69:43649 (1505 connections now open) 2016-03-13T09:22:52.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43658 #4083368 (1506 connections now open) 2016-03-13T09:22:52.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43657 #4083369 (1507 connections now open) 2016-03-13T09:22:54.390+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17366 version: 30667|1||563ba74d869758a4542b9075 based on: 30666|1||563ba74d869758a4542b9075 2016-03-13T09:22:54.881+0000 I NETWORK [conn4083356] end connection 10.0.27.237:36954 (1506 connections now open) 2016-03-13T09:22:54.881+0000 I NETWORK [conn4083355] end connection 10.0.27.237:36953 (1506 connections now open) 2016-03-13T09:22:54.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36969 #4083370 (1506 connections now open) 2016-03-13T09:22:54.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36968 #4083371 (1507 connections now open) 2016-03-13T09:22:57.033+0000 I NETWORK [conn4083360] end connection 10.0.7.113:60534 (1506 connections now open) 2016-03-13T09:22:57.034+0000 I NETWORK [conn4083361] end connection 10.0.7.113:60533 (1505 connections now open) 2016-03-13T09:22:57.034+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60542 #4083372 (1507 connections now open) 2016-03-13T09:22:57.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60543 #4083373 (1507 connections now open) 2016-03-13T09:22:58.073+0000 I NETWORK [conn4083362] end connection 10.0.216.90:58568 (1506 connections now open) 2016-03-13T09:22:58.073+0000 I NETWORK [conn4083363] end connection 10.0.216.90:58569 (1505 connections now open) 2016-03-13T09:22:58.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58574 #4083374 (1506 connections now open) 2016-03-13T09:22:58.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58575 #4083375 (1507 connections now open) 2016-03-13T09:22:58.113+0000 I NETWORK [conn4082184] end connection 10.0.26.69:43001 (1506 connections now open) 2016-03-13T09:22:58.113+0000 I NETWORK [conn4081479] end connection 10.0.7.113:59684 (1505 connections now open) 2016-03-13T09:22:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43669 #4083376 (1506 connections now open) 2016-03-13T09:22:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60545 #4083377 (1507 connections now open) 2016-03-13T09:22:59.117+0000 I NETWORK [conn4083364] end connection 10.0.20.53:51923 (1506 connections now open) 2016-03-13T09:22:59.117+0000 I NETWORK [conn4083365] end connection 10.0.20.53:51922 (1506 connections now open) 2016-03-13T09:22:59.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51932 #4083378 (1506 connections now open) 2016-03-13T09:22:59.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51931 #4083379 (1507 connections now open) 2016-03-13T09:23:01.490+0000 I NETWORK [conn4083367] end connection 10.0.179.209:41647 (1506 connections now open) 2016-03-13T09:23:01.490+0000 I NETWORK [conn4083366] end connection 10.0.179.209:41646 (1506 connections now open) 2016-03-13T09:23:01.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41657 #4083380 (1506 connections now open) 2016-03-13T09:23:01.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41656 #4083381 (1507 connections now open) 2016-03-13T09:23:02.152+0000 I NETWORK [conn4083368] end connection 10.0.26.69:43658 (1506 connections now open) 2016-03-13T09:23:02.152+0000 I NETWORK [conn4083369] end connection 10.0.26.69:43657 (1506 connections now open) 2016-03-13T09:23:02.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43674 #4083382 (1507 connections now open) 2016-03-13T09:23:02.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43673 #4083383 (1507 connections now open) 2016-03-13T09:23:04.894+0000 I NETWORK [conn4083370] end connection 10.0.27.237:36969 (1506 connections now open) 2016-03-13T09:23:04.894+0000 I NETWORK [conn4083371] end connection 10.0.27.237:36968 (1506 connections now open) 2016-03-13T09:23:04.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36990 #4083384 (1506 connections now open) 2016-03-13T09:23:04.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:36991 #4083385 (1507 connections now open) 2016-03-13T09:23:07.048+0000 I NETWORK [conn4083372] end connection 10.0.7.113:60542 (1506 connections now open) 2016-03-13T09:23:07.048+0000 I NETWORK [conn4083373] end connection 10.0.7.113:60543 (1506 connections now open) 2016-03-13T09:23:07.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60557 #4083386 (1506 connections now open) 2016-03-13T09:23:07.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60556 #4083387 (1507 connections now open) 2016-03-13T09:23:08.086+0000 I NETWORK [conn4083374] end connection 10.0.216.90:58574 (1506 connections now open) 2016-03-13T09:23:08.086+0000 I NETWORK [conn4083375] end connection 10.0.216.90:58575 (1506 connections now open) 2016-03-13T09:23:08.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58588 #4083388 (1506 connections now open) 2016-03-13T09:23:08.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58589 #4083389 (1507 connections now open) 2016-03-13T09:23:09.005+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:23:08.738+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:23:09.005+0000 I SHARDING [LockPinger] trying to delete 4 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:23:09.130+0000 I NETWORK [conn4083379] end connection 10.0.20.53:51931 (1506 connections now open) 2016-03-13T09:23:09.130+0000 I NETWORK [conn4083378] end connection 10.0.20.53:51932 (1506 connections now open) 2016-03-13T09:23:09.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51946 #4083390 (1506 connections now open) 2016-03-13T09:23:09.130+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51947 #4083391 (1507 connections now open) 2016-03-13T09:23:09.141+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5315b18fa188537623d83 2016-03-13T09:23:09.276+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5316818fa188537623d84 2016-03-13T09:23:09.413+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5317218fa188537623d85 2016-03-13T09:23:09.549+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5317c18fa188537623d86 2016-03-13T09:23:09.708+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 695ms sequenceNumber: 17367 version: 30668|1||563ba74d869758a4542b9075 based on: 30667|1||563ba74d869758a4542b9075 2016-03-13T09:23:11.502+0000 I NETWORK [conn4083381] end connection 10.0.179.209:41656 (1506 connections now open) 2016-03-13T09:23:11.503+0000 I NETWORK [conn4083380] end connection 10.0.179.209:41657 (1505 connections now open) 2016-03-13T09:23:11.512+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41670 #4083392 (1506 connections now open) 2016-03-13T09:23:11.512+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41671 #4083393 (1507 connections now open) 2016-03-13T09:23:12.166+0000 I NETWORK [conn4083383] end connection 10.0.26.69:43673 (1506 connections now open) 2016-03-13T09:23:12.166+0000 I NETWORK [conn4083382] end connection 10.0.26.69:43674 (1506 connections now open) 2016-03-13T09:23:12.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43693 #4083394 (1506 connections now open) 2016-03-13T09:23:12.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43692 #4083395 (1507 connections now open) 2016-03-13T09:23:14.907+0000 I NETWORK [conn4083384] end connection 10.0.27.237:36990 (1506 connections now open) 2016-03-13T09:23:14.907+0000 I NETWORK [conn4083385] end connection 10.0.27.237:36991 (1506 connections now open) 2016-03-13T09:23:14.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37002 #4083396 (1506 connections now open) 2016-03-13T09:23:14.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37003 #4083397 (1507 connections now open) 2016-03-13T09:23:16.117+0000 I NETWORK [conn4081766] end connection 10.0.20.53:51080 (1506 connections now open) 2016-03-13T09:23:16.117+0000 I NETWORK [conn4082989] end connection 10.0.216.90:58259 (1505 connections now open) 2016-03-13T09:23:16.117+0000 I NETWORK [conn4081895] end connection 10.0.27.237:36181 (1505 connections now open) 2016-03-13T09:23:16.117+0000 I NETWORK [conn4080809] end connection 10.0.7.113:59428 (1503 connections now open) 2016-03-13T09:23:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51964 #4083398 (1504 connections now open) 2016-03-13T09:23:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37009 #4083399 (1505 connections now open) 2016-03-13T09:23:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58596 #4083400 (1506 connections now open) 2016-03-13T09:23:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60569 #4083401 (1507 connections now open) 2016-03-13T09:23:17.061+0000 I NETWORK [conn4083386] end connection 10.0.7.113:60557 (1506 connections now open) 2016-03-13T09:23:17.061+0000 I NETWORK [conn4083387] end connection 10.0.7.113:60556 (1506 connections now open) 2016-03-13T09:23:17.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60580 #4083402 (1506 connections now open) 2016-03-13T09:23:17.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60579 #4083403 (1507 connections now open) 2016-03-13T09:23:18.099+0000 I NETWORK [conn4083388] end connection 10.0.216.90:58588 (1506 connections now open) 2016-03-13T09:23:18.099+0000 I NETWORK [conn4083389] end connection 10.0.216.90:58589 (1506 connections now open) 2016-03-13T09:23:18.100+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58606 #4083404 (1506 connections now open) 2016-03-13T09:23:18.100+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58607 #4083405 (1507 connections now open) 2016-03-13T09:23:19.143+0000 I NETWORK [conn4083390] end connection 10.0.20.53:51946 (1506 connections now open) 2016-03-13T09:23:19.143+0000 I NETWORK [conn4083391] end connection 10.0.20.53:51947 (1506 connections now open) 2016-03-13T09:23:19.152+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51967 #4083406 (1506 connections now open) 2016-03-13T09:23:19.152+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51966 #4083407 (1507 connections now open) 2016-03-13T09:23:21.524+0000 I NETWORK [conn4083393] end connection 10.0.179.209:41671 (1506 connections now open) 2016-03-13T09:23:21.524+0000 I NETWORK [conn4083392] end connection 10.0.179.209:41670 (1506 connections now open) 2016-03-13T09:23:21.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41693 #4083408 (1506 connections now open) 2016-03-13T09:23:21.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41694 #4083409 (1507 connections now open) 2016-03-13T09:23:22.179+0000 I NETWORK [conn4083394] end connection 10.0.26.69:43693 (1506 connections now open) 2016-03-13T09:23:22.179+0000 I NETWORK [conn4083395] end connection 10.0.26.69:43692 (1506 connections now open) 2016-03-13T09:23:22.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43703 #4083410 (1506 connections now open) 2016-03-13T09:23:22.179+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43702 #4083411 (1507 connections now open) 2016-03-13T09:23:24.920+0000 I NETWORK [conn4083397] end connection 10.0.27.237:37003 (1506 connections now open) 2016-03-13T09:23:24.920+0000 I NETWORK [conn4083396] end connection 10.0.27.237:37002 (1506 connections now open) 2016-03-13T09:23:24.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37015 #4083412 (1506 connections now open) 2016-03-13T09:23:24.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37016 #4083413 (1507 connections now open) 2016-03-13T09:23:26.339+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 695ms sequenceNumber: 17368 version: 30669|1||563ba74d869758a4542b9075 based on: 30668|1||563ba74d869758a4542b9075 2016-03-13T09:23:27.075+0000 I NETWORK [conn4083402] end connection 10.0.7.113:60580 (1506 connections now open) 2016-03-13T09:23:27.075+0000 I NETWORK [conn4083403] end connection 10.0.7.113:60579 (1505 connections now open) 2016-03-13T09:23:27.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60589 #4083414 (1506 connections now open) 2016-03-13T09:23:27.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60588 #4083415 (1507 connections now open) 2016-03-13T09:23:28.112+0000 I NETWORK [conn4083405] end connection 10.0.216.90:58607 (1506 connections now open) 2016-03-13T09:23:28.112+0000 I NETWORK [conn4083404] end connection 10.0.216.90:58606 (1506 connections now open) 2016-03-13T09:23:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58615 #4083416 (1506 connections now open) 2016-03-13T09:23:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58616 #4083417 (1507 connections now open) 2016-03-13T09:23:28.114+0000 I NETWORK [conn4081063] end connection 10.0.179.209:40660 (1506 connections now open) 2016-03-13T09:23:28.114+0000 I NETWORK [conn4083169] end connection 10.0.20.53:51709 (1505 connections now open) 2016-03-13T09:23:28.114+0000 I NETWORK [conn4083128] end connection 10.0.216.90:58324 (1504 connections now open) 2016-03-13T09:23:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51975 #4083418 (1505 connections now open) 2016-03-13T09:23:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41698 #4083419 (1506 connections now open) 2016-03-13T09:23:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58617 #4083420 (1507 connections now open) 2016-03-13T09:23:29.164+0000 I NETWORK [conn4083407] end connection 10.0.20.53:51966 (1506 connections now open) 2016-03-13T09:23:29.164+0000 I NETWORK [conn4083406] end connection 10.0.20.53:51967 (1505 connections now open) 2016-03-13T09:23:29.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51976 #4083421 (1506 connections now open) 2016-03-13T09:23:29.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51977 #4083422 (1507 connections now open) 2016-03-13T09:23:31.537+0000 I NETWORK [conn4083408] end connection 10.0.179.209:41693 (1506 connections now open) 2016-03-13T09:23:31.537+0000 I NETWORK [conn4083409] end connection 10.0.179.209:41694 (1506 connections now open) 2016-03-13T09:23:31.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41702 #4083423 (1506 connections now open) 2016-03-13T09:23:31.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41703 #4083424 (1507 connections now open) 2016-03-13T09:23:32.192+0000 I NETWORK [conn4083410] end connection 10.0.26.69:43703 (1506 connections now open) 2016-03-13T09:23:32.192+0000 I NETWORK [conn4083411] end connection 10.0.26.69:43702 (1506 connections now open) 2016-03-13T09:23:32.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43718 #4083425 (1506 connections now open) 2016-03-13T09:23:32.201+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43717 #4083426 (1507 connections now open) 2016-03-13T09:23:34.933+0000 I NETWORK [conn4083413] end connection 10.0.27.237:37016 (1506 connections now open) 2016-03-13T09:23:34.933+0000 I NETWORK [conn4083412] end connection 10.0.27.237:37015 (1506 connections now open) 2016-03-13T09:23:34.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37036 #4083427 (1506 connections now open) 2016-03-13T09:23:34.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37037 #4083428 (1507 connections now open) 2016-03-13T09:23:37.096+0000 I NETWORK [conn4083414] end connection 10.0.7.113:60589 (1506 connections now open) 2016-03-13T09:23:37.096+0000 I NETWORK [conn4083415] end connection 10.0.7.113:60588 (1506 connections now open) 2016-03-13T09:23:37.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60601 #4083429 (1506 connections now open) 2016-03-13T09:23:37.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60602 #4083430 (1507 connections now open) 2016-03-13T09:23:38.060+0000 I NETWORK [conn4061880] end connection 10.0.53.235:60488 (1506 connections now open) 2016-03-13T09:23:38.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.53.235:43311 #4083431 (1507 connections now open) 2016-03-13T09:23:38.125+0000 I NETWORK [conn4083416] end connection 10.0.216.90:58615 (1506 connections now open) 2016-03-13T09:23:38.126+0000 I NETWORK [conn4083417] end connection 10.0.216.90:58616 (1505 connections now open) 2016-03-13T09:23:38.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58629 #4083432 (1506 connections now open) 2016-03-13T09:23:38.134+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58630 #4083433 (1507 connections now open) 2016-03-13T09:23:39.178+0000 I NETWORK [conn4083422] end connection 10.0.20.53:51977 (1506 connections now open) 2016-03-13T09:23:39.178+0000 I NETWORK [conn4083421] end connection 10.0.20.53:51976 (1506 connections now open) 2016-03-13T09:23:39.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51990 #4083434 (1506 connections now open) 2016-03-13T09:23:39.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:51991 #4083435 (1507 connections now open) 2016-03-13T09:23:39.800+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:23:39.550+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:23:39.800+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:23:39.902+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5318718fa188537623d87 2016-03-13T09:23:40.038+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5319118fa188537623d88 2016-03-13T09:23:40.108+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5319b18fa188537623d89 2016-03-13T09:23:40.804+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17369 version: 30670|1||563ba74d869758a4542b9075 based on: 30669|1||563ba74d869758a4542b9075 2016-03-13T09:23:41.550+0000 I NETWORK [conn4083423] end connection 10.0.179.209:41702 (1506 connections now open) 2016-03-13T09:23:41.550+0000 I NETWORK [conn4083424] end connection 10.0.179.209:41703 (1505 connections now open) 2016-03-13T09:23:41.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41714 #4083436 (1506 connections now open) 2016-03-13T09:23:41.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41713 #4083437 (1507 connections now open) 2016-03-13T09:23:42.214+0000 I NETWORK [conn4083426] end connection 10.0.26.69:43717 (1506 connections now open) 2016-03-13T09:23:42.214+0000 I NETWORK [conn4083425] end connection 10.0.26.69:43718 (1506 connections now open) 2016-03-13T09:23:42.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43734 #4083438 (1506 connections now open) 2016-03-13T09:23:42.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43735 #4083439 (1507 connections now open) 2016-03-13T09:23:44.947+0000 I NETWORK [conn4083428] end connection 10.0.27.237:37037 (1506 connections now open) 2016-03-13T09:23:44.947+0000 I NETWORK [conn4083427] end connection 10.0.27.237:37036 (1506 connections now open) 2016-03-13T09:23:44.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37044 #4083440 (1506 connections now open) 2016-03-13T09:23:44.948+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37045 #4083441 (1507 connections now open) 2016-03-13T09:23:46.117+0000 I NETWORK [conn4081517] end connection 10.0.20.53:50938 (1506 connections now open) 2016-03-13T09:23:46.117+0000 I NETWORK [conn4081936] end connection 10.0.7.113:59792 (1505 connections now open) 2016-03-13T09:23:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52009 #4083442 (1506 connections now open) 2016-03-13T09:23:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60608 #4083443 (1507 connections now open) 2016-03-13T09:23:47.109+0000 I NETWORK [conn4083430] end connection 10.0.7.113:60602 (1506 connections now open) 2016-03-13T09:23:47.109+0000 I NETWORK [conn4083429] end connection 10.0.7.113:60601 (1506 connections now open) 2016-03-13T09:23:47.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60620 #4083444 (1506 connections now open) 2016-03-13T09:23:47.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60619 #4083445 (1507 connections now open) 2016-03-13T09:23:48.147+0000 I NETWORK [conn4083432] end connection 10.0.216.90:58629 (1506 connections now open) 2016-03-13T09:23:48.147+0000 I NETWORK [conn4083433] end connection 10.0.216.90:58630 (1506 connections now open) 2016-03-13T09:23:48.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58648 #4083446 (1506 connections now open) 2016-03-13T09:23:48.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58649 #4083447 (1507 connections now open) 2016-03-13T09:23:49.191+0000 I NETWORK [conn4083434] end connection 10.0.20.53:51990 (1506 connections now open) 2016-03-13T09:23:49.191+0000 I NETWORK [conn4083435] end connection 10.0.20.53:51991 (1506 connections now open) 2016-03-13T09:23:49.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52011 #4083448 (1506 connections now open) 2016-03-13T09:23:49.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52012 #4083449 (1507 connections now open) 2016-03-13T09:23:51.563+0000 I NETWORK [conn4083436] end connection 10.0.179.209:41714 (1506 connections now open) 2016-03-13T09:23:51.563+0000 I NETWORK [conn4083437] end connection 10.0.179.209:41713 (1505 connections now open) 2016-03-13T09:23:51.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41734 #4083450 (1506 connections now open) 2016-03-13T09:23:51.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41733 #4083451 (1507 connections now open) 2016-03-13T09:23:52.227+0000 I NETWORK [conn4083439] end connection 10.0.26.69:43735 (1506 connections now open) 2016-03-13T09:23:52.227+0000 I NETWORK [conn4083438] end connection 10.0.26.69:43734 (1506 connections now open) 2016-03-13T09:23:52.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43743 #4083452 (1506 connections now open) 2016-03-13T09:23:52.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43744 #4083453 (1507 connections now open) 2016-03-13T09:23:54.960+0000 I NETWORK [conn4083441] end connection 10.0.27.237:37045 (1506 connections now open) 2016-03-13T09:23:54.960+0000 I NETWORK [conn4083440] end connection 10.0.27.237:37044 (1506 connections now open) 2016-03-13T09:23:54.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37057 #4083454 (1506 connections now open) 2016-03-13T09:23:54.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37058 #4083455 (1507 connections now open) 2016-03-13T09:23:54.994+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17370 version: 30671|1||563ba74d869758a4542b9075 based on: 30670|1||563ba74d869758a4542b9075 2016-03-13T09:23:57.123+0000 I NETWORK [conn4083444] end connection 10.0.7.113:60620 (1506 connections now open) 2016-03-13T09:23:57.123+0000 I NETWORK [conn4083445] end connection 10.0.7.113:60619 (1506 connections now open) 2016-03-13T09:23:57.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60625 #4083456 (1506 connections now open) 2016-03-13T09:23:57.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60626 #4083457 (1507 connections now open) 2016-03-13T09:23:58.113+0000 I NETWORK [conn4081822] end connection 10.0.179.209:40810 (1506 connections now open) 2016-03-13T09:23:58.113+0000 I NETWORK [conn4082640] end connection 10.0.20.53:51428 (1505 connections now open) 2016-03-13T09:23:58.113+0000 I NETWORK [conn4082500] end connection 10.0.216.90:58025 (1505 connections now open) 2016-03-13T09:23:58.113+0000 I NETWORK [conn4081554] end connection 10.0.27.237:36014 (1503 connections now open) 2016-03-13T09:23:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41739 #4083458 (1504 connections now open) 2016-03-13T09:23:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58658 #4083459 (1505 connections now open) 2016-03-13T09:23:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52019 #4083460 (1506 connections now open) 2016-03-13T09:23:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37065 #4083461 (1507 connections now open) 2016-03-13T09:23:58.159+0000 I NETWORK [conn4083446] end connection 10.0.216.90:58648 (1506 connections now open) 2016-03-13T09:23:58.159+0000 I NETWORK [conn4083447] end connection 10.0.216.90:58649 (1506 connections now open) 2016-03-13T09:23:58.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58659 #4083462 (1506 connections now open) 2016-03-13T09:23:58.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58660 #4083463 (1507 connections now open) 2016-03-13T09:23:59.205+0000 I NETWORK [conn4083449] end connection 10.0.20.53:52012 (1506 connections now open) 2016-03-13T09:23:59.205+0000 I NETWORK [conn4083448] end connection 10.0.20.53:52011 (1506 connections now open) 2016-03-13T09:23:59.206+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52020 #4083464 (1506 connections now open) 2016-03-13T09:23:59.206+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52021 #4083465 (1507 connections now open) 2016-03-13T09:24:01.576+0000 I NETWORK [conn4083450] end connection 10.0.179.209:41734 (1506 connections now open) 2016-03-13T09:24:01.576+0000 I NETWORK [conn4083451] end connection 10.0.179.209:41733 (1505 connections now open) 2016-03-13T09:24:01.576+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41743 #4083466 (1506 connections now open) 2016-03-13T09:24:01.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41744 #4083467 (1507 connections now open) 2016-03-13T09:24:02.240+0000 I NETWORK [conn4083453] end connection 10.0.26.69:43744 (1506 connections now open) 2016-03-13T09:24:02.240+0000 I NETWORK [conn4083452] end connection 10.0.26.69:43743 (1506 connections now open) 2016-03-13T09:24:02.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43761 #4083468 (1507 connections now open) 2016-03-13T09:24:02.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43760 #4083469 (1507 connections now open) 2016-03-13T09:24:04.973+0000 I NETWORK [conn4083455] end connection 10.0.27.237:37058 (1506 connections now open) 2016-03-13T09:24:04.974+0000 I NETWORK [conn4083454] end connection 10.0.27.237:37057 (1505 connections now open) 2016-03-13T09:24:04.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37081 #4083470 (1506 connections now open) 2016-03-13T09:24:04.974+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37082 #4083471 (1507 connections now open) 2016-03-13T09:24:07.136+0000 I NETWORK [conn4083456] end connection 10.0.7.113:60625 (1506 connections now open) 2016-03-13T09:24:07.136+0000 I NETWORK [conn4083457] end connection 10.0.7.113:60626 (1506 connections now open) 2016-03-13T09:24:07.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60639 #4083472 (1506 connections now open) 2016-03-13T09:24:07.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60638 #4083473 (1507 connections now open) 2016-03-13T09:24:08.172+0000 I NETWORK [conn4083463] end connection 10.0.216.90:58660 (1506 connections now open) 2016-03-13T09:24:08.172+0000 I NETWORK [conn4083462] end connection 10.0.216.90:58659 (1506 connections now open) 2016-03-13T09:24:08.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58670 #4083474 (1506 connections now open) 2016-03-13T09:24:08.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58671 #4083475 (1507 connections now open) 2016-03-13T09:24:09.219+0000 I NETWORK [conn4083465] end connection 10.0.20.53:52021 (1506 connections now open) 2016-03-13T09:24:09.219+0000 I NETWORK [conn4083464] end connection 10.0.20.53:52020 (1505 connections now open) 2016-03-13T09:24:09.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52033 #4083476 (1506 connections now open) 2016-03-13T09:24:09.220+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52034 #4083477 (1507 connections now open) 2016-03-13T09:24:09.702+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17371 version: 30672|1||563ba74d869758a4542b9075 based on: 30671|1||563ba74d869758a4542b9075 2016-03-13T09:24:10.364+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:24:10.108+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:24:10.364+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:24:10.466+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531a518fa188537623d8a 2016-03-13T09:24:10.603+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531af18fa188537623d8b 2016-03-13T09:24:10.739+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531b918fa188537623d8c 2016-03-13T09:24:11.588+0000 I NETWORK [conn4083467] end connection 10.0.179.209:41744 (1506 connections now open) 2016-03-13T09:24:11.588+0000 I NETWORK [conn4083466] end connection 10.0.179.209:41743 (1506 connections now open) 2016-03-13T09:24:11.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41757 #4083478 (1506 connections now open) 2016-03-13T09:24:11.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41756 #4083479 (1507 connections now open) 2016-03-13T09:24:12.254+0000 I NETWORK [conn4083468] end connection 10.0.26.69:43761 (1506 connections now open) 2016-03-13T09:24:12.254+0000 I NETWORK [conn4083469] end connection 10.0.26.69:43760 (1506 connections now open) 2016-03-13T09:24:12.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43781 #4083480 (1506 connections now open) 2016-03-13T09:24:12.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43782 #4083481 (1507 connections now open) 2016-03-13T09:24:14.986+0000 I NETWORK [conn4083470] end connection 10.0.27.237:37081 (1506 connections now open) 2016-03-13T09:24:14.987+0000 I NETWORK [conn4083471] end connection 10.0.27.237:37082 (1505 connections now open) 2016-03-13T09:24:14.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37092 #4083482 (1506 connections now open) 2016-03-13T09:24:14.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37093 #4083483 (1507 connections now open) 2016-03-13T09:24:16.117+0000 I NETWORK [conn4082296] end connection 10.0.179.209:41024 (1506 connections now open) 2016-03-13T09:24:16.117+0000 I NETWORK [conn4081908] end connection 10.0.20.53:51138 (1505 connections now open) 2016-03-13T09:24:16.117+0000 I NETWORK [conn4075616] end connection 10.0.216.90:52087 (1504 connections now open) 2016-03-13T09:24:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52053 #4083484 (1505 connections now open) 2016-03-13T09:24:16.117+0000 I NETWORK [conn4083359] end connection 10.0.7.113:60523 (1504 connections now open) 2016-03-13T09:24:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58680 #4083485 (1506 connections now open) 2016-03-13T09:24:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41761 #4083486 (1506 connections now open) 2016-03-13T09:24:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60650 #4083487 (1507 connections now open) 2016-03-13T09:24:17.149+0000 I NETWORK [conn4083473] end connection 10.0.7.113:60638 (1506 connections now open) 2016-03-13T09:24:17.149+0000 I NETWORK [conn4083472] end connection 10.0.7.113:60639 (1506 connections now open) 2016-03-13T09:24:17.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60661 #4083488 (1506 connections now open) 2016-03-13T09:24:17.150+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60660 #4083489 (1507 connections now open) 2016-03-13T09:24:18.186+0000 I NETWORK [conn4083475] end connection 10.0.216.90:58671 (1506 connections now open) 2016-03-13T09:24:18.186+0000 I NETWORK [conn4083474] end connection 10.0.216.90:58670 (1506 connections now open) 2016-03-13T09:24:18.186+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58691 #4083490 (1506 connections now open) 2016-03-13T09:24:18.186+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58692 #4083491 (1507 connections now open) 2016-03-13T09:24:19.233+0000 I NETWORK [conn4083476] end connection 10.0.20.53:52033 (1506 connections now open) 2016-03-13T09:24:19.233+0000 I NETWORK [conn4083477] end connection 10.0.20.53:52034 (1506 connections now open) 2016-03-13T09:24:19.242+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52055 #4083492 (1506 connections now open) 2016-03-13T09:24:19.243+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52056 #4083493 (1507 connections now open) 2016-03-13T09:24:21.610+0000 I NETWORK [conn4083478] end connection 10.0.179.209:41757 (1506 connections now open) 2016-03-13T09:24:21.610+0000 I NETWORK [conn4083479] end connection 10.0.179.209:41756 (1505 connections now open) 2016-03-13T09:24:21.611+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41779 #4083494 (1506 connections now open) 2016-03-13T09:24:21.611+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41780 #4083495 (1507 connections now open) 2016-03-13T09:24:22.267+0000 I NETWORK [conn4083480] end connection 10.0.26.69:43781 (1506 connections now open) 2016-03-13T09:24:22.267+0000 I NETWORK [conn4083481] end connection 10.0.26.69:43782 (1506 connections now open) 2016-03-13T09:24:22.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43790 #4083496 (1506 connections now open) 2016-03-13T09:24:22.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43791 #4083497 (1507 connections now open) 2016-03-13T09:24:24.422+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17372 version: 30673|1||563ba74d869758a4542b9075 based on: 30672|1||563ba74d869758a4542b9075 2016-03-13T09:24:24.999+0000 I NETWORK [conn4083482] end connection 10.0.27.237:37092 (1506 connections now open) 2016-03-13T09:24:24.999+0000 I NETWORK [conn4083483] end connection 10.0.27.237:37093 (1505 connections now open) 2016-03-13T09:24:25.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37105 #4083498 (1506 connections now open) 2016-03-13T09:24:25.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37106 #4083499 (1507 connections now open) 2016-03-13T09:24:27.163+0000 I NETWORK [conn4083489] end connection 10.0.7.113:60660 (1506 connections now open) 2016-03-13T09:24:27.163+0000 I NETWORK [conn4083488] end connection 10.0.7.113:60661 (1506 connections now open) 2016-03-13T09:24:27.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60669 #4083500 (1506 connections now open) 2016-03-13T09:24:27.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60668 #4083501 (1507 connections now open) 2016-03-13T09:24:28.113+0000 I NETWORK [conn4081784] end connection 10.0.27.237:36132 (1506 connections now open) 2016-03-13T09:24:28.113+0000 I NETWORK [conn4081502] end connection 10.0.7.113:59690 (1506 connections now open) 2016-03-13T09:24:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37112 #4083502 (1506 connections now open) 2016-03-13T09:24:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60673 #4083503 (1507 connections now open) 2016-03-13T09:24:28.198+0000 I NETWORK [conn4083491] end connection 10.0.216.90:58692 (1506 connections now open) 2016-03-13T09:24:28.198+0000 I NETWORK [conn4083490] end connection 10.0.216.90:58691 (1506 connections now open) 2016-03-13T09:24:28.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58700 #4083504 (1506 connections now open) 2016-03-13T09:24:28.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58701 #4083505 (1507 connections now open) 2016-03-13T09:24:29.255+0000 I NETWORK [conn4083492] end connection 10.0.20.53:52055 (1506 connections now open) 2016-03-13T09:24:29.255+0000 I NETWORK [conn4083493] end connection 10.0.20.53:52056 (1506 connections now open) 2016-03-13T09:24:29.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52064 #4083506 (1506 connections now open) 2016-03-13T09:24:29.256+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52065 #4083507 (1507 connections now open) 2016-03-13T09:24:31.623+0000 I NETWORK [conn4083495] end connection 10.0.179.209:41780 (1506 connections now open) 2016-03-13T09:24:31.623+0000 I NETWORK [conn4083494] end connection 10.0.179.209:41779 (1506 connections now open) 2016-03-13T09:24:31.624+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41788 #4083508 (1506 connections now open) 2016-03-13T09:24:31.624+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41789 #4083509 (1507 connections now open) 2016-03-13T09:24:32.280+0000 I NETWORK [conn4083497] end connection 10.0.26.69:43791 (1506 connections now open) 2016-03-13T09:24:32.280+0000 I NETWORK [conn4083496] end connection 10.0.26.69:43790 (1506 connections now open) 2016-03-13T09:24:32.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43805 #4083510 (1506 connections now open) 2016-03-13T09:24:32.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43806 #4083511 (1507 connections now open) 2016-03-13T09:24:35.012+0000 I NETWORK [conn4083499] end connection 10.0.27.237:37106 (1506 connections now open) 2016-03-13T09:24:35.012+0000 I NETWORK [conn4083498] end connection 10.0.27.237:37105 (1505 connections now open) 2016-03-13T09:24:35.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37126 #4083512 (1506 connections now open) 2016-03-13T09:24:35.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37127 #4083513 (1507 connections now open) 2016-03-13T09:24:37.185+0000 I NETWORK [conn4083500] end connection 10.0.7.113:60669 (1506 connections now open) 2016-03-13T09:24:37.185+0000 I NETWORK [conn4083501] end connection 10.0.7.113:60668 (1506 connections now open) 2016-03-13T09:24:37.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60682 #4083514 (1506 connections now open) 2016-03-13T09:24:37.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60681 #4083515 (1507 connections now open) 2016-03-13T09:24:38.212+0000 I NETWORK [conn4083505] end connection 10.0.216.90:58701 (1506 connections now open) 2016-03-13T09:24:38.212+0000 I NETWORK [conn4083504] end connection 10.0.216.90:58700 (1506 connections now open) 2016-03-13T09:24:38.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58715 #4083516 (1506 connections now open) 2016-03-13T09:24:38.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58714 #4083517 (1507 connections now open) 2016-03-13T09:24:39.269+0000 I NETWORK [conn4083507] end connection 10.0.20.53:52065 (1506 connections now open) 2016-03-13T09:24:39.269+0000 I NETWORK [conn4083506] end connection 10.0.20.53:52064 (1506 connections now open) 2016-03-13T09:24:39.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52079 #4083518 (1506 connections now open) 2016-03-13T09:24:39.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52080 #4083519 (1507 connections now open) 2016-03-13T09:24:40.605+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 699ms sequenceNumber: 17373 version: 30674|1||563ba74d869758a4542b9075 based on: 30673|1||563ba74d869758a4542b9075 2016-03-13T09:24:41.636+0000 I NETWORK [conn4083508] end connection 10.0.179.209:41788 (1506 connections now open) 2016-03-13T09:24:41.636+0000 I NETWORK [conn4083509] end connection 10.0.179.209:41789 (1506 connections now open) 2016-03-13T09:24:41.637+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41799 #4083520 (1506 connections now open) 2016-03-13T09:24:41.637+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41800 #4083521 (1507 connections now open) 2016-03-13T09:24:42.302+0000 I NETWORK [conn4083510] end connection 10.0.26.69:43805 (1506 connections now open) 2016-03-13T09:24:42.302+0000 I NETWORK [conn4083511] end connection 10.0.26.69:43806 (1505 connections now open) 2016-03-13T09:24:42.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43823 #4083522 (1506 connections now open) 2016-03-13T09:24:42.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43822 #4083523 (1507 connections now open) 2016-03-13T09:24:42.734+0000 W SHARDING [thread1] config servers in.dbcfg1.mongo323.test.mydomain.com:27019 and in.dbcfg3.mongo323.test.mydomain.com:27019 differ 2016-03-13T09:24:43.237+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:24:40.740+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:24:43.237+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:24:43.343+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531c318fa188537623d8d 2016-03-13T09:24:43.478+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531cd18fa188537623d8e 2016-03-13T09:24:43.614+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531d818fa188537623d8f 2016-03-13T09:24:45.034+0000 I NETWORK [conn4083513] end connection 10.0.27.237:37127 (1506 connections now open) 2016-03-13T09:24:45.034+0000 I NETWORK [conn4083512] end connection 10.0.27.237:37126 (1506 connections now open) 2016-03-13T09:24:45.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37136 #4083524 (1506 connections now open) 2016-03-13T09:24:45.035+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37135 #4083525 (1507 connections now open) 2016-03-13T09:24:46.118+0000 I NETWORK [conn4081366] end connection 10.0.26.69:42604 (1506 connections now open) 2016-03-13T09:24:46.118+0000 I NETWORK [conn4081735] end connection 10.0.27.237:36108 (1505 connections now open) 2016-03-13T09:24:46.118+0000 I NETWORK [conn4080812] end connection 10.0.7.113:59431 (1504 connections now open) 2016-03-13T09:24:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43828 #4083526 (1505 connections now open) 2016-03-13T09:24:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37139 #4083527 (1506 connections now open) 2016-03-13T09:24:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60691 #4083528 (1507 connections now open) 2016-03-13T09:24:47.197+0000 I NETWORK [conn4083514] end connection 10.0.7.113:60682 (1506 connections now open) 2016-03-13T09:24:47.198+0000 I NETWORK [conn4083515] end connection 10.0.7.113:60681 (1505 connections now open) 2016-03-13T09:24:47.198+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60701 #4083529 (1506 connections now open) 2016-03-13T09:24:47.198+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60702 #4083530 (1507 connections now open) 2016-03-13T09:24:48.233+0000 I NETWORK [conn4083517] end connection 10.0.216.90:58714 (1506 connections now open) 2016-03-13T09:24:48.234+0000 I NETWORK [conn4083516] end connection 10.0.216.90:58715 (1505 connections now open) 2016-03-13T09:24:48.234+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58725 #4083531 (1506 connections now open) 2016-03-13T09:24:48.234+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58726 #4083532 (1507 connections now open) 2016-03-13T09:24:49.282+0000 I NETWORK [conn4083519] end connection 10.0.20.53:52080 (1506 connections now open) 2016-03-13T09:24:49.282+0000 I NETWORK [conn4083518] end connection 10.0.20.53:52079 (1506 connections now open) 2016-03-13T09:24:49.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52100 #4083533 (1506 connections now open) 2016-03-13T09:24:49.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52101 #4083534 (1507 connections now open) 2016-03-13T09:24:51.649+0000 I NETWORK [conn4083520] end connection 10.0.179.209:41799 (1506 connections now open) 2016-03-13T09:24:51.649+0000 I NETWORK [conn4083521] end connection 10.0.179.209:41800 (1506 connections now open) 2016-03-13T09:24:51.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41820 #4083535 (1506 connections now open) 2016-03-13T09:24:51.650+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41819 #4083536 (1507 connections now open) 2016-03-13T09:24:52.315+0000 I NETWORK [conn4083522] end connection 10.0.26.69:43823 (1506 connections now open) 2016-03-13T09:24:52.315+0000 I NETWORK [conn4083523] end connection 10.0.26.69:43822 (1506 connections now open) 2016-03-13T09:24:52.316+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43831 #4083537 (1506 connections now open) 2016-03-13T09:24:52.316+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43830 #4083538 (1507 connections now open) 2016-03-13T09:24:55.047+0000 I NETWORK [conn4083524] end connection 10.0.27.237:37136 (1506 connections now open) 2016-03-13T09:24:55.047+0000 I NETWORK [conn4083525] end connection 10.0.27.237:37135 (1506 connections now open) 2016-03-13T09:24:55.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37148 #4083539 (1506 connections now open) 2016-03-13T09:24:55.048+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37149 #4083540 (1507 connections now open) 2016-03-13T09:24:57.211+0000 I NETWORK [conn4083529] end connection 10.0.7.113:60701 (1506 connections now open) 2016-03-13T09:24:57.211+0000 I NETWORK [conn4083530] end connection 10.0.7.113:60702 (1506 connections now open) 2016-03-13T09:24:57.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60710 #4083541 (1506 connections now open) 2016-03-13T09:24:57.212+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60709 #4083542 (1507 connections now open) 2016-03-13T09:24:57.654+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17374 version: 30675|1||563ba74d869758a4542b9075 based on: 30674|1||563ba74d869758a4542b9075 2016-03-13T09:24:58.113+0000 I NETWORK [conn4081311] end connection 10.0.20.53:50827 (1506 connections now open) 2016-03-13T09:24:58.113+0000 I NETWORK [conn4081012] end connection 10.0.26.69:42462 (1505 connections now open) 2016-03-13T09:24:58.113+0000 I NETWORK [conn4081351] end connection 10.0.7.113:59654 (1504 connections now open) 2016-03-13T09:24:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52107 #4083543 (1505 connections now open) 2016-03-13T09:24:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43842 #4083544 (1506 connections now open) 2016-03-13T09:24:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60712 #4083545 (1507 connections now open) 2016-03-13T09:24:58.246+0000 I NETWORK [conn4083531] end connection 10.0.216.90:58725 (1506 connections now open) 2016-03-13T09:24:58.246+0000 I NETWORK [conn4083532] end connection 10.0.216.90:58726 (1505 connections now open) 2016-03-13T09:24:58.247+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58743 #4083546 (1506 connections now open) 2016-03-13T09:24:58.247+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58744 #4083547 (1507 connections now open) 2016-03-13T09:24:59.295+0000 I NETWORK [conn4083533] end connection 10.0.20.53:52100 (1506 connections now open) 2016-03-13T09:24:59.295+0000 I NETWORK [conn4083534] end connection 10.0.20.53:52101 (1506 connections now open) 2016-03-13T09:24:59.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52108 #4083548 (1506 connections now open) 2016-03-13T09:24:59.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52109 #4083549 (1507 connections now open) 2016-03-13T09:25:01.662+0000 I NETWORK [conn4083535] end connection 10.0.179.209:41820 (1506 connections now open) 2016-03-13T09:25:01.662+0000 I NETWORK [conn4083536] end connection 10.0.179.209:41819 (1506 connections now open) 2016-03-13T09:25:01.663+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41832 #4083550 (1506 connections now open) 2016-03-13T09:25:01.663+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41833 #4083551 (1507 connections now open) 2016-03-13T09:25:02.328+0000 I NETWORK [conn4083538] end connection 10.0.26.69:43830 (1506 connections now open) 2016-03-13T09:25:02.328+0000 I NETWORK [conn4083537] end connection 10.0.26.69:43831 (1506 connections now open) 2016-03-13T09:25:02.329+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43848 #4083552 (1506 connections now open) 2016-03-13T09:25:02.329+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43849 #4083553 (1507 connections now open) 2016-03-13T09:25:05.060+0000 I NETWORK [conn4083540] end connection 10.0.27.237:37149 (1506 connections now open) 2016-03-13T09:25:05.060+0000 I NETWORK [conn4083539] end connection 10.0.27.237:37148 (1506 connections now open) 2016-03-13T09:25:05.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37172 #4083554 (1506 connections now open) 2016-03-13T09:25:05.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37173 #4083555 (1507 connections now open) 2016-03-13T09:25:07.224+0000 I NETWORK [conn4083541] end connection 10.0.7.113:60710 (1506 connections now open) 2016-03-13T09:25:07.224+0000 I NETWORK [conn4083542] end connection 10.0.7.113:60709 (1506 connections now open) 2016-03-13T09:25:07.225+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60720 #4083556 (1507 connections now open) 2016-03-13T09:25:07.225+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60721 #4083557 (1507 connections now open) 2016-03-13T09:25:08.259+0000 I NETWORK [conn4083547] end connection 10.0.216.90:58744 (1506 connections now open) 2016-03-13T09:25:08.259+0000 I NETWORK [conn4083546] end connection 10.0.216.90:58743 (1506 connections now open) 2016-03-13T09:25:08.260+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58754 #4083558 (1506 connections now open) 2016-03-13T09:25:08.260+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58755 #4083559 (1507 connections now open) 2016-03-13T09:25:09.308+0000 I NETWORK [conn4083548] end connection 10.0.20.53:52108 (1506 connections now open) 2016-03-13T09:25:09.308+0000 I NETWORK [conn4083549] end connection 10.0.20.53:52109 (1506 connections now open) 2016-03-13T09:25:09.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52123 #4083560 (1506 connections now open) 2016-03-13T09:25:09.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52124 #4083561 (1507 connections now open) 2016-03-13T09:25:11.675+0000 I NETWORK [conn4083551] end connection 10.0.179.209:41833 (1506 connections now open) 2016-03-13T09:25:11.675+0000 I NETWORK [conn4083550] end connection 10.0.179.209:41832 (1506 connections now open) 2016-03-13T09:25:11.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41844 #4083562 (1506 connections now open) 2016-03-13T09:25:11.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41843 #4083563 (1507 connections now open) 2016-03-13T09:25:12.165+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 692ms sequenceNumber: 17375 version: 30676|1||563ba74d869758a4542b9075 based on: 30675|1||563ba74d869758a4542b9075 2016-03-13T09:25:12.341+0000 I NETWORK [conn4083552] end connection 10.0.26.69:43848 (1506 connections now open) 2016-03-13T09:25:12.342+0000 I NETWORK [conn4083553] end connection 10.0.26.69:43849 (1505 connections now open) 2016-03-13T09:25:12.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43868 #4083564 (1506 connections now open) 2016-03-13T09:25:12.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43869 #4083565 (1507 connections now open) 2016-03-13T09:25:13.656+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e531f918fa188537623d92 2016-03-13T09:25:13.893+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:25:13.614+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:25:13.893+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:25:14.030+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531e518fa188537623d90 2016-03-13T09:25:14.166+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e531ef18fa188537623d91 2016-03-13T09:25:15.073+0000 I NETWORK [conn4083554] end connection 10.0.27.237:37172 (1506 connections now open) 2016-03-13T09:25:15.073+0000 I NETWORK [conn4083555] end connection 10.0.27.237:37173 (1506 connections now open) 2016-03-13T09:25:15.074+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37182 #4083566 (1506 connections now open) 2016-03-13T09:25:15.074+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37181 #4083567 (1507 connections now open) 2016-03-13T09:25:16.117+0000 I NETWORK [conn4081975] end connection 10.0.179.209:40891 (1506 connections now open) 2016-03-13T09:25:16.117+0000 I NETWORK [conn4081639] end connection 10.0.20.53:51004 (1505 connections now open) 2016-03-13T09:25:16.117+0000 I NETWORK [conn4081717] end connection 10.0.26.69:42796 (1505 connections now open) 2016-03-13T09:25:16.117+0000 I NETWORK [conn4081389] end connection 10.0.27.237:35934 (1503 connections now open) 2016-03-13T09:25:16.117+0000 I NETWORK [conn4083051] end connection 10.0.7.113:60238 (1502 connections now open) 2016-03-13T09:25:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41848 #4083568 (1503 connections now open) 2016-03-13T09:25:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43875 #4083569 (1504 connections now open) 2016-03-13T09:25:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52144 #4083570 (1505 connections now open) 2016-03-13T09:25:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60736 #4083571 (1506 connections now open) 2016-03-13T09:25:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37186 #4083572 (1507 connections now open) 2016-03-13T09:25:16.301+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634ca53bcf3ae0ea6c53c47')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ca53bcf3ae0ea6c53c47') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cbfb55418c0ec5a5284a') }, shard: "shmydomain1", lastmod: Timestamp 30676000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:25:16.451+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30676|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ca53bcf3ae0ea6c53c47') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cbfb55418c0ec5a5284a') }) shmydomain1 -> shmydomain2 2016-03-13T09:25:17.237+0000 I NETWORK [conn4083557] end connection 10.0.7.113:60721 (1506 connections now open) 2016-03-13T09:25:17.238+0000 I NETWORK [conn4083556] end connection 10.0.7.113:60720 (1505 connections now open) 2016-03-13T09:25:17.238+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60737 #4083573 (1506 connections now open) 2016-03-13T09:25:17.238+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60738 #4083574 (1507 connections now open) 2016-03-13T09:25:18.272+0000 I NETWORK [conn4083558] end connection 10.0.216.90:58754 (1506 connections now open) 2016-03-13T09:25:18.272+0000 I NETWORK [conn4083559] end connection 10.0.216.90:58755 (1505 connections now open) 2016-03-13T09:25:18.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58768 #4083575 (1506 connections now open) 2016-03-13T09:25:18.273+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58769 #4083576 (1507 connections now open) 2016-03-13T09:25:19.322+0000 I NETWORK [conn4083561] end connection 10.0.20.53:52124 (1506 connections now open) 2016-03-13T09:25:19.322+0000 I NETWORK [conn4083560] end connection 10.0.20.53:52123 (1506 connections now open) 2016-03-13T09:25:19.332+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52147 #4083577 (1506 connections now open) 2016-03-13T09:25:19.332+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52146 #4083578 (1507 connections now open) 2016-03-13T09:25:21.695+0000 I NETWORK [conn4083563] end connection 10.0.179.209:41843 (1506 connections now open) 2016-03-13T09:25:21.695+0000 I NETWORK [conn4083562] end connection 10.0.179.209:41844 (1506 connections now open) 2016-03-13T09:25:21.696+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41866 #4083579 (1506 connections now open) 2016-03-13T09:25:21.696+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41867 #4083580 (1507 connections now open) 2016-03-13T09:25:22.355+0000 I NETWORK [conn4083565] end connection 10.0.26.69:43869 (1506 connections now open) 2016-03-13T09:25:22.355+0000 I NETWORK [conn4083564] end connection 10.0.26.69:43868 (1506 connections now open) 2016-03-13T09:25:22.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43878 #4083581 (1506 connections now open) 2016-03-13T09:25:22.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43879 #4083582 (1507 connections now open) 2016-03-13T09:25:25.086+0000 I NETWORK [conn4083566] end connection 10.0.27.237:37182 (1506 connections now open) 2016-03-13T09:25:25.086+0000 I NETWORK [conn4083567] end connection 10.0.27.237:37181 (1506 connections now open) 2016-03-13T09:25:25.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37196 #4083583 (1506 connections now open) 2016-03-13T09:25:25.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37197 #4083584 (1507 connections now open) 2016-03-13T09:25:26.279+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17376 version: 30677|1||563ba74d869758a4542b9075 based on: 30676|1||563ba74d869758a4542b9075 2016-03-13T09:25:26.571+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:25:26.571+0000-56e5320618fa188537623d93", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861126571), what: "balancer.round", ns: "", details: { executionTimeMillis: 13241, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:25:26.751+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:25:27.251+0000 I NETWORK [conn4083573] end connection 10.0.7.113:60737 (1506 connections now open) 2016-03-13T09:25:27.251+0000 I NETWORK [conn4083574] end connection 10.0.7.113:60738 (1506 connections now open) 2016-03-13T09:25:27.260+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60753 #4083585 (1506 connections now open) 2016-03-13T09:25:27.260+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60754 #4083586 (1507 connections now open) 2016-03-13T09:25:28.047+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5320718fa188537623d94 2016-03-13T09:25:28.113+0000 I NETWORK [conn4081044] end connection 10.0.179.209:40652 (1506 connections now open) 2016-03-13T09:25:28.113+0000 I NETWORK [conn4081378] end connection 10.0.7.113:59662 (1505 connections now open) 2016-03-13T09:25:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41871 #4083587 (1506 connections now open) 2016-03-13T09:25:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60757 #4083588 (1507 connections now open) 2016-03-13T09:25:28.284+0000 I NETWORK [conn4083575] end connection 10.0.216.90:58768 (1506 connections now open) 2016-03-13T09:25:28.284+0000 I NETWORK [conn4083576] end connection 10.0.216.90:58769 (1506 connections now open) 2016-03-13T09:25:28.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58787 #4083589 (1506 connections now open) 2016-03-13T09:25:28.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58786 #4083590 (1507 connections now open) 2016-03-13T09:25:29.345+0000 I NETWORK [conn4083578] end connection 10.0.20.53:52146 (1506 connections now open) 2016-03-13T09:25:29.345+0000 I NETWORK [conn4083577] end connection 10.0.20.53:52147 (1506 connections now open) 2016-03-13T09:25:29.346+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52157 #4083591 (1506 connections now open) 2016-03-13T09:25:29.346+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52156 #4083592 (1507 connections now open) 2016-03-13T09:25:30.534+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634cbfb55418c0ec5a5284a')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cbfb55418c0ec5a5284a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cda5cf7b4e0eb53b226c') }, shard: "shmydomain1", lastmod: Timestamp 30677000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:25:30.684+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30677|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cbfb55418c0ec5a5284a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cda5cf7b4e0eb53b226c') }) shmydomain1 -> shmydomain3 2016-03-13T09:25:31.709+0000 I NETWORK [conn4083579] end connection 10.0.179.209:41866 (1506 connections now open) 2016-03-13T09:25:31.709+0000 I NETWORK [conn4083580] end connection 10.0.179.209:41867 (1506 connections now open) 2016-03-13T09:25:31.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41876 #4083593 (1506 connections now open) 2016-03-13T09:25:31.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41875 #4083594 (1507 connections now open) 2016-03-13T09:25:32.368+0000 I NETWORK [conn4083582] end connection 10.0.26.69:43879 (1506 connections now open) 2016-03-13T09:25:32.368+0000 I NETWORK [conn4083581] end connection 10.0.26.69:43878 (1506 connections now open) 2016-03-13T09:25:32.378+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43892 #4083595 (1506 connections now open) 2016-03-13T09:25:32.378+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43893 #4083596 (1507 connections now open) 2016-03-13T09:25:35.099+0000 I NETWORK [conn4083583] end connection 10.0.27.237:37196 (1506 connections now open) 2016-03-13T09:25:35.099+0000 I NETWORK [conn4083584] end connection 10.0.27.237:37197 (1505 connections now open) 2016-03-13T09:25:35.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37218 #4083597 (1506 connections now open) 2016-03-13T09:25:35.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37217 #4083598 (1507 connections now open) 2016-03-13T09:25:37.273+0000 I NETWORK [conn4083586] end connection 10.0.7.113:60754 (1506 connections now open) 2016-03-13T09:25:37.273+0000 I NETWORK [conn4083585] end connection 10.0.7.113:60753 (1506 connections now open) 2016-03-13T09:25:37.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60761 #4083599 (1506 connections now open) 2016-03-13T09:25:37.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60762 #4083600 (1507 connections now open) 2016-03-13T09:25:38.297+0000 I NETWORK [conn4083590] end connection 10.0.216.90:58786 (1506 connections now open) 2016-03-13T09:25:38.297+0000 I NETWORK [conn4083589] end connection 10.0.216.90:58787 (1506 connections now open) 2016-03-13T09:25:38.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58795 #4083601 (1506 connections now open) 2016-03-13T09:25:38.307+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58796 #4083602 (1507 connections now open) 2016-03-13T09:25:39.359+0000 I NETWORK [conn4083592] end connection 10.0.20.53:52156 (1506 connections now open) 2016-03-13T09:25:39.359+0000 I NETWORK [conn4083591] end connection 10.0.20.53:52157 (1506 connections now open) 2016-03-13T09:25:39.359+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52172 #4083603 (1506 connections now open) 2016-03-13T09:25:39.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52173 #4083604 (1507 connections now open) 2016-03-13T09:25:40.529+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 692ms sequenceNumber: 17377 version: 30678|1||563ba74d869758a4542b9075 based on: 30677|1||563ba74d869758a4542b9075 2016-03-13T09:25:40.727+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:25:40.727+0000-56e5321418fa188537623d95", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861140727), what: "balancer.round", ns: "", details: { executionTimeMillis: 12944, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:25:40.893+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:25:41.721+0000 I NETWORK [conn4083593] end connection 10.0.179.209:41876 (1506 connections now open) 2016-03-13T09:25:41.722+0000 I NETWORK [conn4083594] end connection 10.0.179.209:41875 (1505 connections now open) 2016-03-13T09:25:41.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41887 #4083605 (1506 connections now open) 2016-03-13T09:25:41.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41886 #4083606 (1507 connections now open) 2016-03-13T09:25:42.225+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5321518fa188537623d96 2016-03-13T09:25:42.390+0000 I NETWORK [conn4083596] end connection 10.0.26.69:43893 (1506 connections now open) 2016-03-13T09:25:42.390+0000 I NETWORK [conn4083595] end connection 10.0.26.69:43892 (1506 connections now open) 2016-03-13T09:25:42.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43909 #4083607 (1507 connections now open) 2016-03-13T09:25:42.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43910 #4083608 (1507 connections now open) 2016-03-13T09:25:45.121+0000 I NETWORK [conn4083598] end connection 10.0.27.237:37217 (1506 connections now open) 2016-03-13T09:25:45.121+0000 I NETWORK [conn4083597] end connection 10.0.27.237:37218 (1506 connections now open) 2016-03-13T09:25:45.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37224 #4083609 (1506 connections now open) 2016-03-13T09:25:45.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37223 #4083610 (1507 connections now open) 2016-03-13T09:25:45.686+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634cda5cf7b4e0eb53b226c')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cda5cf7b4e0eb53b226c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cf4455418c0ec5a587c9') }, shard: "shmydomain1", lastmod: Timestamp 30678000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:25:45.836+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30678|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cda5cf7b4e0eb53b226c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cf4455418c0ec5a587c9') }) shmydomain1 -> shmydomain2 2016-03-13T09:25:46.117+0000 I NETWORK [conn4080753] end connection 10.0.179.209:40467 (1506 connections now open) 2016-03-13T09:25:46.117+0000 I NETWORK [conn4079305] end connection 10.0.216.90:55990 (1505 connections now open) 2016-03-13T09:25:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41892 #4083611 (1506 connections now open) 2016-03-13T09:25:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58808 #4083612 (1507 connections now open) 2016-03-13T09:25:46.937+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:25:44.166+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:25:47.287+0000 I NETWORK [conn4083600] end connection 10.0.7.113:60762 (1506 connections now open) 2016-03-13T09:25:47.287+0000 I NETWORK [conn4083599] end connection 10.0.7.113:60761 (1506 connections now open) 2016-03-13T09:25:47.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60776 #4083613 (1506 connections now open) 2016-03-13T09:25:47.287+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60777 #4083614 (1507 connections now open) 2016-03-13T09:25:48.319+0000 I NETWORK [conn4083602] end connection 10.0.216.90:58796 (1506 connections now open) 2016-03-13T09:25:48.319+0000 I NETWORK [conn4083601] end connection 10.0.216.90:58795 (1506 connections now open) 2016-03-13T09:25:48.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58809 #4083615 (1506 connections now open) 2016-03-13T09:25:48.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58810 #4083616 (1507 connections now open) 2016-03-13T09:25:49.372+0000 I NETWORK [conn4083604] end connection 10.0.20.53:52173 (1506 connections now open) 2016-03-13T09:25:49.372+0000 I NETWORK [conn4083603] end connection 10.0.20.53:52172 (1506 connections now open) 2016-03-13T09:25:49.373+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52193 #4083617 (1506 connections now open) 2016-03-13T09:25:49.373+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52194 #4083618 (1507 connections now open) 2016-03-13T09:25:51.734+0000 I NETWORK [conn4083606] end connection 10.0.179.209:41886 (1506 connections now open) 2016-03-13T09:25:51.734+0000 I NETWORK [conn4083605] end connection 10.0.179.209:41887 (1505 connections now open) 2016-03-13T09:25:51.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41907 #4083619 (1506 connections now open) 2016-03-13T09:25:51.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41908 #4083620 (1507 connections now open) 2016-03-13T09:25:52.403+0000 I NETWORK [conn4083608] end connection 10.0.26.69:43910 (1506 connections now open) 2016-03-13T09:25:52.403+0000 I NETWORK [conn4083607] end connection 10.0.26.69:43909 (1506 connections now open) 2016-03-13T09:25:52.404+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43919 #4083621 (1506 connections now open) 2016-03-13T09:25:52.404+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43918 #4083622 (1507 connections now open) 2016-03-13T09:25:55.134+0000 I NETWORK [conn4083609] end connection 10.0.27.237:37224 (1506 connections now open) 2016-03-13T09:25:55.134+0000 I NETWORK [conn4083610] end connection 10.0.27.237:37223 (1506 connections now open) 2016-03-13T09:25:55.135+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37238 #4083623 (1506 connections now open) 2016-03-13T09:25:55.135+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37239 #4083624 (1507 connections now open) 2016-03-13T09:25:57.300+0000 I NETWORK [conn4083614] end connection 10.0.7.113:60777 (1506 connections now open) 2016-03-13T09:25:57.300+0000 I NETWORK [conn4083613] end connection 10.0.7.113:60776 (1505 connections now open) 2016-03-13T09:25:57.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60793 #4083625 (1506 connections now open) 2016-03-13T09:25:57.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60794 #4083626 (1507 connections now open) 2016-03-13T09:25:58.109+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17378 version: 30679|1||563ba74d869758a4542b9075 based on: 30678|1||563ba74d869758a4542b9075 2016-03-13T09:25:58.113+0000 I NETWORK [conn4081823] end connection 10.0.179.209:40811 (1506 connections now open) 2016-03-13T09:25:58.113+0000 I NETWORK [conn4083036] end connection 10.0.216.90:58279 (1505 connections now open) 2016-03-13T09:25:58.113+0000 I NETWORK [conn4080913] end connection 10.0.26.69:42435 (1504 connections now open) 2016-03-13T09:25:58.114+0000 I NETWORK [conn4082945] end connection 10.0.7.113:60175 (1503 connections now open) 2016-03-13T09:25:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41912 #4083627 (1504 connections now open) 2016-03-13T09:25:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58827 #4083628 (1505 connections now open) 2016-03-13T09:25:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43931 #4083629 (1506 connections now open) 2016-03-13T09:25:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60796 #4083630 (1507 connections now open) 2016-03-13T09:25:58.332+0000 I NETWORK [conn4083616] end connection 10.0.216.90:58810 (1506 connections now open) 2016-03-13T09:25:58.332+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58828 #4083631 (1507 connections now open) 2016-03-13T09:25:58.332+0000 I NETWORK [conn4083615] end connection 10.0.216.90:58809 (1506 connections now open) 2016-03-13T09:25:58.333+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58829 #4083632 (1507 connections now open) 2016-03-13T09:25:58.404+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:25:58.404+0000-56e5322618fa188537623d97", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861158404), what: "balancer.round", ns: "", details: { executionTimeMillis: 16477, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:25:58.579+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:25:59.385+0000 I NETWORK [conn4083618] end connection 10.0.20.53:52194 (1506 connections now open) 2016-03-13T09:25:59.385+0000 I NETWORK [conn4083617] end connection 10.0.20.53:52193 (1506 connections now open) 2016-03-13T09:25:59.386+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52203 #4083633 (1506 connections now open) 2016-03-13T09:25:59.386+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52202 #4083634 (1507 connections now open) 2016-03-13T09:25:59.943+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5322718fa188537623d98 2016-03-13T09:26:01.747+0000 I NETWORK [conn4083619] end connection 10.0.179.209:41907 (1506 connections now open) 2016-03-13T09:26:01.747+0000 I NETWORK [conn4083620] end connection 10.0.179.209:41908 (1506 connections now open) 2016-03-13T09:26:01.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41919 #4083635 (1506 connections now open) 2016-03-13T09:26:01.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41918 #4083636 (1507 connections now open) 2016-03-13T09:26:02.417+0000 I NETWORK [conn4083622] end connection 10.0.26.69:43918 (1506 connections now open) 2016-03-13T09:26:02.417+0000 I NETWORK [conn4083621] end connection 10.0.26.69:43919 (1506 connections now open) 2016-03-13T09:26:02.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43935 #4083637 (1506 connections now open) 2016-03-13T09:26:02.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43936 #4083638 (1507 connections now open) 2016-03-13T09:26:02.478+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634cf4455418c0ec5a587c9')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cf4455418c0ec5a587c9') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d0c4cf7b4e0eb3375930') }, shard: "shmydomain1", lastmod: Timestamp 30679000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:26:02.629+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30679|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634cf4455418c0ec5a587c9') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d0c4cf7b4e0eb3375930') }) shmydomain1 -> shmydomain3 2016-03-13T09:26:05.147+0000 I NETWORK [conn4083623] end connection 10.0.27.237:37238 (1506 connections now open) 2016-03-13T09:26:05.147+0000 I NETWORK [conn4083624] end connection 10.0.27.237:37239 (1506 connections now open) 2016-03-13T09:26:05.148+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37262 #4083639 (1506 connections now open) 2016-03-13T09:26:05.148+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37261 #4083640 (1507 connections now open) 2016-03-13T09:26:07.313+0000 I NETWORK [conn4083626] end connection 10.0.7.113:60794 (1506 connections now open) 2016-03-13T09:26:07.313+0000 I NETWORK [conn4083625] end connection 10.0.7.113:60793 (1506 connections now open) 2016-03-13T09:26:07.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60805 #4083641 (1506 connections now open) 2016-03-13T09:26:07.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60804 #4083642 (1507 connections now open) 2016-03-13T09:26:08.344+0000 I NETWORK [conn4083631] end connection 10.0.216.90:58828 (1506 connections now open) 2016-03-13T09:26:08.345+0000 I NETWORK [conn4083632] end connection 10.0.216.90:58829 (1505 connections now open) 2016-03-13T09:26:08.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58836 #4083643 (1506 connections now open) 2016-03-13T09:26:08.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58837 #4083644 (1507 connections now open) 2016-03-13T09:26:09.399+0000 I NETWORK [conn4083634] end connection 10.0.20.53:52202 (1506 connections now open) 2016-03-13T09:26:09.399+0000 I NETWORK [conn4083633] end connection 10.0.20.53:52203 (1506 connections now open) 2016-03-13T09:26:09.408+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52216 #4083645 (1506 connections now open) 2016-03-13T09:26:09.408+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52217 #4083646 (1507 connections now open) 2016-03-13T09:26:11.760+0000 I NETWORK [conn4083635] end connection 10.0.179.209:41919 (1506 connections now open) 2016-03-13T09:26:11.760+0000 I NETWORK [conn4083636] end connection 10.0.179.209:41918 (1506 connections now open) 2016-03-13T09:26:11.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41929 #4083647 (1506 connections now open) 2016-03-13T09:26:11.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41930 #4083648 (1507 connections now open) 2016-03-13T09:26:12.430+0000 I NETWORK [conn4083637] end connection 10.0.26.69:43935 (1506 connections now open) 2016-03-13T09:26:12.430+0000 I NETWORK [conn4083638] end connection 10.0.26.69:43936 (1506 connections now open) 2016-03-13T09:26:12.439+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43957 #4083649 (1506 connections now open) 2016-03-13T09:26:12.439+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43956 #4083650 (1507 connections now open) 2016-03-13T09:26:12.523+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17379 version: 30680|1||563ba74d869758a4542b9075 based on: 30679|1||563ba74d869758a4542b9075 2016-03-13T09:26:12.818+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:26:12.818+0000-56e5323418fa188537623d99", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861172818), what: "balancer.round", ns: "", details: { executionTimeMillis: 13206, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:26:12.986+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:26:14.284+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5323618fa188537623d9a 2016-03-13T09:26:15.160+0000 I NETWORK [conn4083639] end connection 10.0.27.237:37262 (1506 connections now open) 2016-03-13T09:26:15.160+0000 I NETWORK [conn4083640] end connection 10.0.27.237:37261 (1506 connections now open) 2016-03-13T09:26:15.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37272 #4083651 (1506 connections now open) 2016-03-13T09:26:15.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37273 #4083652 (1507 connections now open) 2016-03-13T09:26:16.117+0000 I NETWORK [conn4082780] end connection 10.0.179.209:41209 (1506 connections now open) 2016-03-13T09:26:16.117+0000 I NETWORK [conn4079979] end connection 10.0.216.90:56660 (1505 connections now open) 2016-03-13T09:26:16.117+0000 I NETWORK [conn4082287] end connection 10.0.27.237:36365 (1504 connections now open) 2016-03-13T09:26:16.117+0000 I NETWORK [conn4081131] end connection 10.0.7.113:59584 (1504 connections now open) 2016-03-13T09:26:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41935 #4083653 (1504 connections now open) 2016-03-13T09:26:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58848 #4083654 (1505 connections now open) 2016-03-13T09:26:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37276 #4083655 (1506 connections now open) 2016-03-13T09:26:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60821 #4083656 (1507 connections now open) 2016-03-13T09:26:17.239+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:26:16.937+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:26:17.327+0000 I NETWORK [conn4083641] end connection 10.0.7.113:60805 (1506 connections now open) 2016-03-13T09:26:17.327+0000 I NETWORK [conn4083642] end connection 10.0.7.113:60804 (1506 connections now open) 2016-03-13T09:26:17.327+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60823 #4083657 (1506 connections now open) 2016-03-13T09:26:17.327+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60822 #4083658 (1507 connections now open) 2016-03-13T09:26:17.386+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634d0c4cf7b4e0eb3375930')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d0c4cf7b4e0eb3375930') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d23f55418c0ec30577d1') }, shard: "shmydomain1", lastmod: Timestamp 30680000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:26:17.537+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30680|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d0c4cf7b4e0eb3375930') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d23f55418c0ec30577d1') }) shmydomain1 -> shmydomain2 2016-03-13T09:26:18.358+0000 I NETWORK [conn4083643] end connection 10.0.216.90:58836 (1506 connections now open) 2016-03-13T09:26:18.358+0000 I NETWORK [conn4083644] end connection 10.0.216.90:58837 (1506 connections now open) 2016-03-13T09:26:18.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58850 #4083659 (1506 connections now open) 2016-03-13T09:26:18.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58851 #4083660 (1507 connections now open) 2016-03-13T09:26:19.420+0000 I NETWORK [conn4083646] end connection 10.0.20.53:52217 (1506 connections now open) 2016-03-13T09:26:19.420+0000 I NETWORK [conn4083645] end connection 10.0.20.53:52216 (1506 connections now open) 2016-03-13T09:26:19.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52236 #4083661 (1506 connections now open) 2016-03-13T09:26:19.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52237 #4083662 (1507 connections now open) 2016-03-13T09:26:21.782+0000 I NETWORK [conn4083647] end connection 10.0.179.209:41929 (1506 connections now open) 2016-03-13T09:26:21.782+0000 I NETWORK [conn4083648] end connection 10.0.179.209:41930 (1506 connections now open) 2016-03-13T09:26:21.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41954 #4083663 (1506 connections now open) 2016-03-13T09:26:21.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41953 #4083664 (1507 connections now open) 2016-03-13T09:26:22.452+0000 I NETWORK [conn4083650] end connection 10.0.26.69:43956 (1506 connections now open) 2016-03-13T09:26:22.452+0000 I NETWORK [conn4083649] end connection 10.0.26.69:43957 (1506 connections now open) 2016-03-13T09:26:22.452+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43964 #4083665 (1506 connections now open) 2016-03-13T09:26:22.453+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43965 #4083666 (1507 connections now open) 2016-03-13T09:26:25.174+0000 I NETWORK [conn4083652] end connection 10.0.27.237:37273 (1506 connections now open) 2016-03-13T09:26:25.174+0000 I NETWORK [conn4083651] end connection 10.0.27.237:37272 (1506 connections now open) 2016-03-13T09:26:25.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37286 #4083667 (1506 connections now open) 2016-03-13T09:26:25.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37285 #4083668 (1507 connections now open) 2016-03-13T09:26:27.340+0000 I NETWORK [conn4083657] end connection 10.0.7.113:60823 (1506 connections now open) 2016-03-13T09:26:27.340+0000 I NETWORK [conn4083658] end connection 10.0.7.113:60822 (1506 connections now open) 2016-03-13T09:26:27.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60839 #4083669 (1506 connections now open) 2016-03-13T09:26:27.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60838 #4083670 (1507 connections now open) 2016-03-13T09:26:27.468+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17380 version: 30681|1||563ba74d869758a4542b9075 based on: 30680|1||563ba74d869758a4542b9075 2016-03-13T09:26:27.670+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:26:27.670+0000-56e5324318fa188537623d9b", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861187670), what: "balancer.round", ns: "", details: { executionTimeMillis: 13650, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:26:27.776+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:26:28.115+0000 I NETWORK [conn4081767] end connection 10.0.20.53:51081 (1506 connections now open) 2016-03-13T09:26:28.115+0000 I NETWORK [conn4082864] end connection 10.0.216.90:58202 (1506 connections now open) 2016-03-13T09:26:28.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58868 #4083671 (1506 connections now open) 2016-03-13T09:26:28.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52246 #4083672 (1507 connections now open) 2016-03-13T09:26:28.370+0000 I NETWORK [conn4083660] end connection 10.0.216.90:58851 (1506 connections now open) 2016-03-13T09:26:28.371+0000 I NETWORK [conn4083659] end connection 10.0.216.90:58850 (1505 connections now open) 2016-03-13T09:26:28.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58870 #4083673 (1506 connections now open) 2016-03-13T09:26:28.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58869 #4083674 (1507 connections now open) 2016-03-13T09:26:28.971+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5324418fa188537623d9c 2016-03-13T09:26:29.442+0000 I NETWORK [conn4083661] end connection 10.0.20.53:52236 (1506 connections now open) 2016-03-13T09:26:29.442+0000 I NETWORK [conn4083662] end connection 10.0.20.53:52237 (1506 connections now open) 2016-03-13T09:26:29.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52247 #4083675 (1506 connections now open) 2016-03-13T09:26:29.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52248 #4083676 (1507 connections now open) 2016-03-13T09:26:31.738+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634d23f55418c0ec30577d1')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d23f55418c0ec30577d1') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d3b655418c0ec5a611a0') }, shard: "shmydomain1", lastmod: Timestamp 30681000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:26:31.795+0000 I NETWORK [conn4083664] end connection 10.0.179.209:41953 (1506 connections now open) 2016-03-13T09:26:31.795+0000 I NETWORK [conn4083663] end connection 10.0.179.209:41954 (1505 connections now open) 2016-03-13T09:26:31.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41963 #4083677 (1506 connections now open) 2016-03-13T09:26:31.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41962 #4083678 (1507 connections now open) 2016-03-13T09:26:31.886+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30681|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d23f55418c0ec30577d1') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d3b655418c0ec5a611a0') }) shmydomain1 -> shmydomain3 2016-03-13T09:26:32.465+0000 I NETWORK [conn4083666] end connection 10.0.26.69:43965 (1506 connections now open) 2016-03-13T09:26:32.465+0000 I NETWORK [conn4083665] end connection 10.0.26.69:43964 (1506 connections now open) 2016-03-13T09:26:32.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43976 #4083679 (1506 connections now open) 2016-03-13T09:26:32.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43977 #4083680 (1507 connections now open) 2016-03-13T09:26:35.187+0000 I NETWORK [conn4083668] end connection 10.0.27.237:37285 (1506 connections now open) 2016-03-13T09:26:35.187+0000 I NETWORK [conn4083667] end connection 10.0.27.237:37286 (1506 connections now open) 2016-03-13T09:26:35.188+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37305 #4083681 (1506 connections now open) 2016-03-13T09:26:35.188+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37306 #4083682 (1507 connections now open) 2016-03-13T09:26:37.362+0000 I NETWORK [conn4083670] end connection 10.0.7.113:60838 (1506 connections now open) 2016-03-13T09:26:37.362+0000 I NETWORK [conn4083669] end connection 10.0.7.113:60839 (1505 connections now open) 2016-03-13T09:26:37.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60849 #4083683 (1506 connections now open) 2016-03-13T09:26:37.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60848 #4083684 (1507 connections now open) 2016-03-13T09:26:38.383+0000 I NETWORK [conn4083673] end connection 10.0.216.90:58870 (1506 connections now open) 2016-03-13T09:26:38.383+0000 I NETWORK [conn4083674] end connection 10.0.216.90:58869 (1506 connections now open) 2016-03-13T09:26:38.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58880 #4083685 (1506 connections now open) 2016-03-13T09:26:38.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58879 #4083686 (1507 connections now open) 2016-03-13T09:26:39.456+0000 I NETWORK [conn4083676] end connection 10.0.20.53:52248 (1506 connections now open) 2016-03-13T09:26:39.456+0000 I NETWORK [conn4083675] end connection 10.0.20.53:52247 (1506 connections now open) 2016-03-13T09:26:39.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52263 #4083687 (1506 connections now open) 2016-03-13T09:26:39.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52264 #4083688 (1507 connections now open) 2016-03-13T09:26:41.725+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17381 version: 30682|1||563ba74d869758a4542b9075 based on: 30681|1||563ba74d869758a4542b9075 2016-03-13T09:26:41.808+0000 I NETWORK [conn4083678] end connection 10.0.179.209:41962 (1506 connections now open) 2016-03-13T09:26:41.808+0000 I NETWORK [conn4083677] end connection 10.0.179.209:41963 (1506 connections now open) 2016-03-13T09:26:41.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41974 #4083689 (1506 connections now open) 2016-03-13T09:26:41.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41973 #4083690 (1507 connections now open) 2016-03-13T09:26:42.016+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:26:42.016+0000-56e5325218fa188537623d9d", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861202016), what: "balancer.round", ns: "", details: { executionTimeMillis: 13207, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:26:42.166+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:26:42.486+0000 I NETWORK [conn4083680] end connection 10.0.26.69:43977 (1506 connections now open) 2016-03-13T09:26:42.486+0000 I NETWORK [conn4083679] end connection 10.0.26.69:43976 (1506 connections now open) 2016-03-13T09:26:42.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43993 #4083691 (1506 connections now open) 2016-03-13T09:26:42.487+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:43994 #4083692 (1507 connections now open) 2016-03-13T09:26:43.492+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5325318fa188537623d9e 2016-03-13T09:26:45.201+0000 I NETWORK [conn4083681] end connection 10.0.27.237:37305 (1506 connections now open) 2016-03-13T09:26:45.201+0000 I NETWORK [conn4083682] end connection 10.0.27.237:37306 (1506 connections now open) 2016-03-13T09:26:45.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37312 #4083693 (1506 connections now open) 2016-03-13T09:26:45.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37313 #4083694 (1507 connections now open) 2016-03-13T09:26:46.052+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634d3b655418c0ec5a611a0')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d3b655418c0ec5a611a0') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d522bcf3ae0ea4e63fda') }, shard: "shmydomain1", lastmod: Timestamp 30682000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:26:46.117+0000 I NETWORK [conn4082487] end connection 10.0.179.209:41093 (1506 connections now open) 2016-03-13T09:26:46.117+0000 I NETWORK [conn4082396] end connection 10.0.20.53:51336 (1506 connections now open) 2016-03-13T09:26:46.117+0000 I NETWORK [conn4081949] end connection 10.0.26.69:42905 (1504 connections now open) 2016-03-13T09:26:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41979 #4083695 (1505 connections now open) 2016-03-13T09:26:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52281 #4083696 (1506 connections now open) 2016-03-13T09:26:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44000 #4083697 (1507 connections now open) 2016-03-13T09:26:46.206+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30682|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d3b655418c0ec5a611a0') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634d522bcf3ae0ea4e63fda') }) shmydomain1 -> shmydomain2 2016-03-13T09:26:47.374+0000 I NETWORK [conn4083684] end connection 10.0.7.113:60848 (1506 connections now open) 2016-03-13T09:26:47.374+0000 I NETWORK [conn4083683] end connection 10.0.7.113:60849 (1506 connections now open) 2016-03-13T09:26:47.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60863 #4083698 (1506 connections now open) 2016-03-13T09:26:47.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60864 #4083699 (1507 connections now open) 2016-03-13T09:26:47.618+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:26:47.239+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:26:48.406+0000 I NETWORK [conn4083685] end connection 10.0.216.90:58880 (1506 connections now open) 2016-03-13T09:26:48.406+0000 I NETWORK [conn4083686] end connection 10.0.216.90:58879 (1506 connections now open) 2016-03-13T09:26:48.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58893 #4083700 (1506 connections now open) 2016-03-13T09:26:48.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58894 #4083701 (1507 connections now open) 2016-03-13T09:26:49.469+0000 I NETWORK [conn4083687] end connection 10.0.20.53:52263 (1506 connections now open) 2016-03-13T09:26:49.469+0000 I NETWORK [conn4083688] end connection 10.0.20.53:52264 (1506 connections now open) 2016-03-13T09:26:49.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52283 #4083702 (1506 connections now open) 2016-03-13T09:26:49.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52284 #4083703 (1507 connections now open) 2016-03-13T09:26:51.821+0000 I NETWORK [conn4083689] end connection 10.0.179.209:41974 (1506 connections now open) 2016-03-13T09:26:51.821+0000 I NETWORK [conn4083690] end connection 10.0.179.209:41973 (1506 connections now open) 2016-03-13T09:26:51.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41994 #4083704 (1506 connections now open) 2016-03-13T09:26:51.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:41995 #4083705 (1507 connections now open) 2016-03-13T09:26:52.499+0000 I NETWORK [conn4083692] end connection 10.0.26.69:43994 (1506 connections now open) 2016-03-13T09:26:52.499+0000 I NETWORK [conn4083691] end connection 10.0.26.69:43993 (1506 connections now open) 2016-03-13T09:26:52.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44001 #4083706 (1506 connections now open) 2016-03-13T09:26:52.500+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44002 #4083707 (1507 connections now open) 2016-03-13T09:26:55.222+0000 I NETWORK [conn4083694] end connection 10.0.27.237:37313 (1506 connections now open) 2016-03-13T09:26:55.222+0000 I NETWORK [conn4083693] end connection 10.0.27.237:37312 (1506 connections now open) 2016-03-13T09:26:55.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37326 #4083708 (1506 connections now open) 2016-03-13T09:26:55.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37327 #4083709 (1507 connections now open) 2016-03-13T09:26:56.339+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17382 version: 30683|1||563ba74d869758a4542b9075 based on: 30682|1||563ba74d869758a4542b9075 2016-03-13T09:26:56.629+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:26:56.629+0000-56e5326018fa188537623d9f", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861216629), what: "balancer.round", ns: "", details: { executionTimeMillis: 13435, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:26:56.776+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:26:57.388+0000 I NETWORK [conn4083698] end connection 10.0.7.113:60863 (1506 connections now open) 2016-03-13T09:26:57.388+0000 I NETWORK [conn4083699] end connection 10.0.7.113:60864 (1506 connections now open) 2016-03-13T09:26:57.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60879 #4083710 (1506 connections now open) 2016-03-13T09:26:57.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60880 #4083711 (1507 connections now open) 2016-03-13T09:26:58.113+0000 I NETWORK [conn4082908] end connection 10.0.216.90:58230 (1506 connections now open) 2016-03-13T09:26:58.113+0000 I NETWORK [conn4082176] end connection 10.0.7.113:59885 (1505 connections now open) 2016-03-13T09:26:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58910 #4083712 (1506 connections now open) 2016-03-13T09:26:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60883 #4083713 (1507 connections now open) 2016-03-13T09:26:58.419+0000 I NETWORK [conn4083700] end connection 10.0.216.90:58893 (1506 connections now open) 2016-03-13T09:26:58.419+0000 I NETWORK [conn4083701] end connection 10.0.216.90:58894 (1506 connections now open) 2016-03-13T09:26:58.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58912 #4083714 (1506 connections now open) 2016-03-13T09:26:58.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58911 #4083715 (1507 connections now open) 2016-03-13T09:26:59.482+0000 I NETWORK [conn4083703] end connection 10.0.20.53:52284 (1506 connections now open) 2016-03-13T09:26:59.482+0000 I NETWORK [conn4083702] end connection 10.0.20.53:52283 (1506 connections now open) 2016-03-13T09:26:59.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52295 #4083716 (1506 connections now open) 2016-03-13T09:26:59.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52294 #4083717 (1507 connections now open) 2016-03-13T09:27:01.833+0000 I NETWORK [conn4083704] end connection 10.0.179.209:41994 (1506 connections now open) 2016-03-13T09:27:01.834+0000 I NETWORK [conn4083705] end connection 10.0.179.209:41995 (1506 connections now open) 2016-03-13T09:27:01.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42005 #4083718 (1506 connections now open) 2016-03-13T09:27:01.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42006 #4083719 (1507 connections now open) 2016-03-13T09:27:02.513+0000 I NETWORK [conn4083706] end connection 10.0.26.69:44001 (1506 connections now open) 2016-03-13T09:27:02.513+0000 I NETWORK [conn4083707] end connection 10.0.26.69:44002 (1506 connections now open) 2016-03-13T09:27:02.513+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44019 #4083720 (1506 connections now open) 2016-03-13T09:27:02.513+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44020 #4083721 (1507 connections now open) 2016-03-13T09:27:05.235+0000 I NETWORK [conn4083708] end connection 10.0.27.237:37326 (1506 connections now open) 2016-03-13T09:27:05.235+0000 I NETWORK [conn4083709] end connection 10.0.27.237:37327 (1505 connections now open) 2016-03-13T09:27:05.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37347 #4083722 (1506 connections now open) 2016-03-13T09:27:05.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37348 #4083723 (1507 connections now open) 2016-03-13T09:27:07.401+0000 I NETWORK [conn4083710] end connection 10.0.7.113:60879 (1506 connections now open) 2016-03-13T09:27:07.401+0000 I NETWORK [conn4083711] end connection 10.0.7.113:60880 (1506 connections now open) 2016-03-13T09:27:07.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60889 #4083724 (1506 connections now open) 2016-03-13T09:27:07.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60890 #4083725 (1507 connections now open) 2016-03-13T09:27:08.432+0000 I NETWORK [conn4083715] end connection 10.0.216.90:58911 (1506 connections now open) 2016-03-13T09:27:08.432+0000 I NETWORK [conn4083714] end connection 10.0.216.90:58912 (1506 connections now open) 2016-03-13T09:27:08.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58920 #4083726 (1506 connections now open) 2016-03-13T09:27:08.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58921 #4083727 (1507 connections now open) 2016-03-13T09:27:09.495+0000 I NETWORK [conn4083717] end connection 10.0.20.53:52294 (1506 connections now open) 2016-03-13T09:27:09.495+0000 I NETWORK [conn4083716] end connection 10.0.20.53:52295 (1506 connections now open) 2016-03-13T09:27:09.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52308 #4083728 (1506 connections now open) 2016-03-13T09:27:09.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52309 #4083729 (1507 connections now open) 2016-03-13T09:27:10.454+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17383 version: 30684|1||563ba74d869758a4542b9075 based on: 30683|1||563ba74d869758a4542b9075 2016-03-13T09:27:11.847+0000 I NETWORK [conn4083719] end connection 10.0.179.209:42006 (1506 connections now open) 2016-03-13T09:27:11.847+0000 I NETWORK [conn4083718] end connection 10.0.179.209:42005 (1506 connections now open) 2016-03-13T09:27:11.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42015 #4083730 (1506 connections now open) 2016-03-13T09:27:11.856+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42016 #4083731 (1507 connections now open) 2016-03-13T09:27:12.526+0000 I NETWORK [conn4083720] end connection 10.0.26.69:44019 (1506 connections now open) 2016-03-13T09:27:12.526+0000 I NETWORK [conn4083721] end connection 10.0.26.69:44020 (1506 connections now open) 2016-03-13T09:27:12.527+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44037 #4083732 (1507 connections now open) 2016-03-13T09:27:12.527+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44038 #4083733 (1507 connections now open) 2016-03-13T09:27:15.247+0000 I NETWORK [conn4083722] end connection 10.0.27.237:37347 (1506 connections now open) 2016-03-13T09:27:15.248+0000 I NETWORK [conn4083723] end connection 10.0.27.237:37348 (1505 connections now open) 2016-03-13T09:27:15.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37357 #4083734 (1506 connections now open) 2016-03-13T09:27:15.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37358 #4083735 (1507 connections now open) 2016-03-13T09:27:16.117+0000 I NETWORK [conn4083418] end connection 10.0.20.53:51975 (1506 connections now open) 2016-03-13T09:27:16.117+0000 I NETWORK [conn4082622] end connection 10.0.7.113:60036 (1505 connections now open) 2016-03-13T09:27:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52327 #4083736 (1506 connections now open) 2016-03-13T09:27:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60905 #4083737 (1507 connections now open) 2016-03-13T09:27:17.414+0000 I NETWORK [conn4083725] end connection 10.0.7.113:60890 (1506 connections now open) 2016-03-13T09:27:17.414+0000 I NETWORK [conn4083724] end connection 10.0.7.113:60889 (1506 connections now open) 2016-03-13T09:27:17.415+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60907 #4083738 (1506 connections now open) 2016-03-13T09:27:17.415+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60906 #4083739 (1507 connections now open) 2016-03-13T09:27:17.829+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:27:17.618+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:27:17.829+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:27:17.979+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5326118fa188537623da0 2016-03-13T09:27:18.129+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5326b18fa188537623da1 2016-03-13T09:27:18.445+0000 I NETWORK [conn4083727] end connection 10.0.216.90:58921 (1506 connections now open) 2016-03-13T09:27:18.445+0000 I NETWORK [conn4083726] end connection 10.0.216.90:58920 (1506 connections now open) 2016-03-13T09:27:18.446+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58934 #4083740 (1506 connections now open) 2016-03-13T09:27:18.446+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58935 #4083741 (1507 connections now open) 2016-03-13T09:27:19.508+0000 I NETWORK [conn4083729] end connection 10.0.20.53:52309 (1506 connections now open) 2016-03-13T09:27:19.508+0000 I NETWORK [conn4083728] end connection 10.0.20.53:52308 (1506 connections now open) 2016-03-13T09:27:19.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52330 #4083742 (1506 connections now open) 2016-03-13T09:27:19.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52329 #4083743 (1507 connections now open) 2016-03-13T09:27:21.869+0000 I NETWORK [conn4083731] end connection 10.0.179.209:42016 (1506 connections now open) 2016-03-13T09:27:21.869+0000 I NETWORK [conn4083730] end connection 10.0.179.209:42015 (1505 connections now open) 2016-03-13T09:27:21.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42040 #4083744 (1506 connections now open) 2016-03-13T09:27:21.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42039 #4083745 (1507 connections now open) 2016-03-13T09:27:22.540+0000 I NETWORK [conn4083733] end connection 10.0.26.69:44038 (1506 connections now open) 2016-03-13T09:27:22.540+0000 I NETWORK [conn4083732] end connection 10.0.26.69:44037 (1506 connections now open) 2016-03-13T09:27:22.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44046 #4083746 (1506 connections now open) 2016-03-13T09:27:22.541+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44047 #4083747 (1507 connections now open) 2016-03-13T09:27:25.174+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17384 version: 30685|1||563ba74d869758a4542b9075 based on: 30684|1||563ba74d869758a4542b9075 2016-03-13T09:27:25.260+0000 I NETWORK [conn4083734] end connection 10.0.27.237:37357 (1506 connections now open) 2016-03-13T09:27:25.261+0000 I NETWORK [conn4083735] end connection 10.0.27.237:37358 (1505 connections now open) 2016-03-13T09:27:25.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37370 #4083748 (1506 connections now open) 2016-03-13T09:27:25.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37371 #4083749 (1507 connections now open) 2016-03-13T09:27:27.427+0000 I NETWORK [conn4083738] end connection 10.0.7.113:60907 (1506 connections now open) 2016-03-13T09:27:27.427+0000 I NETWORK [conn4083739] end connection 10.0.7.113:60906 (1506 connections now open) 2016-03-13T09:27:27.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60925 #4083750 (1506 connections now open) 2016-03-13T09:27:27.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60924 #4083751 (1507 connections now open) 2016-03-13T09:27:28.114+0000 I NETWORK [conn4083007] end connection 10.0.179.209:41324 (1506 connections now open) 2016-03-13T09:27:28.114+0000 I NETWORK [conn4082412] end connection 10.0.26.69:43102 (1505 connections now open) 2016-03-13T09:27:28.114+0000 I NETWORK [conn4080613] end connection 10.0.27.237:35655 (1504 connections now open) 2016-03-13T09:27:28.114+0000 I NETWORK [conn4082515] end connection 10.0.216.90:58035 (1504 connections now open) 2016-03-13T09:27:28.114+0000 I NETWORK [conn4081364] end connection 10.0.7.113:59659 (1502 connections now open) 2016-03-13T09:27:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42044 #4083752 (1504 connections now open) 2016-03-13T09:27:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44056 #4083753 (1504 connections now open) 2016-03-13T09:27:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37378 #4083754 (1505 connections now open) 2016-03-13T09:27:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58952 #4083755 (1506 connections now open) 2016-03-13T09:27:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60928 #4083756 (1507 connections now open) 2016-03-13T09:27:28.458+0000 I NETWORK [conn4083741] end connection 10.0.216.90:58935 (1506 connections now open) 2016-03-13T09:27:28.458+0000 I NETWORK [conn4083740] end connection 10.0.216.90:58934 (1506 connections now open) 2016-03-13T09:27:28.459+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58953 #4083757 (1506 connections now open) 2016-03-13T09:27:28.459+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58954 #4083758 (1507 connections now open) 2016-03-13T09:27:29.531+0000 I NETWORK [conn4083743] end connection 10.0.20.53:52329 (1506 connections now open) 2016-03-13T09:27:29.531+0000 I NETWORK [conn4083742] end connection 10.0.20.53:52330 (1506 connections now open) 2016-03-13T09:27:29.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52340 #4083759 (1506 connections now open) 2016-03-13T09:27:29.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52339 #4083760 (1507 connections now open) 2016-03-13T09:27:31.882+0000 I NETWORK [conn4083745] end connection 10.0.179.209:42039 (1506 connections now open) 2016-03-13T09:27:31.882+0000 I NETWORK [conn4083744] end connection 10.0.179.209:42040 (1506 connections now open) 2016-03-13T09:27:31.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42049 #4083761 (1506 connections now open) 2016-03-13T09:27:31.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42048 #4083762 (1507 connections now open) 2016-03-13T09:27:32.554+0000 I NETWORK [conn4083747] end connection 10.0.26.69:44047 (1506 connections now open) 2016-03-13T09:27:32.554+0000 I NETWORK [conn4083746] end connection 10.0.26.69:44046 (1506 connections now open) 2016-03-13T09:27:32.563+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44059 #4083763 (1506 connections now open) 2016-03-13T09:27:32.563+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44060 #4083764 (1507 connections now open) 2016-03-13T09:27:35.274+0000 I NETWORK [conn4083748] end connection 10.0.27.237:37370 (1506 connections now open) 2016-03-13T09:27:35.274+0000 I NETWORK [conn4083749] end connection 10.0.27.237:37371 (1505 connections now open) 2016-03-13T09:27:35.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37391 #4083765 (1506 connections now open) 2016-03-13T09:27:35.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37392 #4083766 (1507 connections now open) 2016-03-13T09:27:37.449+0000 I NETWORK [conn4083750] end connection 10.0.7.113:60925 (1506 connections now open) 2016-03-13T09:27:37.449+0000 I NETWORK [conn4083751] end connection 10.0.7.113:60924 (1506 connections now open) 2016-03-13T09:27:37.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60933 #4083767 (1506 connections now open) 2016-03-13T09:27:37.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60934 #4083768 (1507 connections now open) 2016-03-13T09:27:38.471+0000 I NETWORK [conn4083757] end connection 10.0.216.90:58953 (1506 connections now open) 2016-03-13T09:27:38.471+0000 I NETWORK [conn4083758] end connection 10.0.216.90:58954 (1506 connections now open) 2016-03-13T09:27:38.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58965 #4083769 (1506 connections now open) 2016-03-13T09:27:38.480+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58964 #4083770 (1507 connections now open) 2016-03-13T09:27:39.544+0000 I NETWORK [conn4083759] end connection 10.0.20.53:52340 (1506 connections now open) 2016-03-13T09:27:39.544+0000 I NETWORK [conn4083760] end connection 10.0.20.53:52339 (1505 connections now open) 2016-03-13T09:27:39.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52354 #4083771 (1506 connections now open) 2016-03-13T09:27:39.545+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52355 #4083772 (1507 connections now open) 2016-03-13T09:27:39.725+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 691ms sequenceNumber: 17385 version: 30686|1||563ba74d869758a4542b9075 based on: 30685|1||563ba74d869758a4542b9075 2016-03-13T09:27:41.895+0000 I NETWORK [conn4083762] end connection 10.0.179.209:42048 (1506 connections now open) 2016-03-13T09:27:41.895+0000 I NETWORK [conn4083761] end connection 10.0.179.209:42049 (1506 connections now open) 2016-03-13T09:27:41.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42059 #4083773 (1506 connections now open) 2016-03-13T09:27:41.895+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42060 #4083774 (1507 connections now open) 2016-03-13T09:27:42.576+0000 I NETWORK [conn4083763] end connection 10.0.26.69:44059 (1506 connections now open) 2016-03-13T09:27:42.576+0000 I NETWORK [conn4083764] end connection 10.0.26.69:44060 (1506 connections now open) 2016-03-13T09:27:42.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44076 #4083775 (1506 connections now open) 2016-03-13T09:27:42.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44075 #4083776 (1507 connections now open) 2016-03-13T09:27:45.287+0000 I NETWORK [conn4083765] end connection 10.0.27.237:37391 (1506 connections now open) 2016-03-13T09:27:45.287+0000 I NETWORK [conn4083766] end connection 10.0.27.237:37392 (1506 connections now open) 2016-03-13T09:27:45.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37398 #4083777 (1506 connections now open) 2016-03-13T09:27:45.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37399 #4083778 (1507 connections now open) 2016-03-13T09:27:46.118+0000 I NETWORK [conn4076080] end connection 10.0.216.90:52591 (1506 connections now open) 2016-03-13T09:27:46.118+0000 I NETWORK [conn4080851] end connection 10.0.7.113:59459 (1506 connections now open) 2016-03-13T09:27:46.118+0000 I NETWORK [conn4081211] end connection 10.0.27.237:35874 (1506 connections now open) 2016-03-13T09:27:46.118+0000 I NETWORK [conn4081722] end connection 10.0.26.69:42801 (1506 connections now open) 2016-03-13T09:27:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58977 #4083779 (1504 connections now open) 2016-03-13T09:27:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44083 #4083780 (1505 connections now open) 2016-03-13T09:27:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37402 #4083781 (1506 connections now open) 2016-03-13T09:27:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60947 #4083782 (1507 connections now open) 2016-03-13T09:27:47.462+0000 I NETWORK [conn4083768] end connection 10.0.7.113:60934 (1506 connections now open) 2016-03-13T09:27:47.462+0000 I NETWORK [conn4083767] end connection 10.0.7.113:60933 (1506 connections now open) 2016-03-13T09:27:47.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60949 #4083783 (1506 connections now open) 2016-03-13T09:27:47.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60950 #4083784 (1507 connections now open) 2016-03-13T09:27:48.384+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:27:48.129+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:27:48.384+0000 I SHARDING [LockPinger] trying to delete 4 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:27:48.492+0000 I NETWORK [conn4083770] end connection 10.0.216.90:58964 (1506 connections now open) 2016-03-13T09:27:48.492+0000 I NETWORK [conn4083769] end connection 10.0.216.90:58965 (1506 connections now open) 2016-03-13T09:27:48.493+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58979 #4083785 (1506 connections now open) 2016-03-13T09:27:48.493+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58980 #4083786 (1507 connections now open) 2016-03-13T09:27:48.519+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5327618fa188537623da2 2016-03-13T09:27:48.656+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5328018fa188537623da3 2016-03-13T09:27:48.800+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5328a18fa188537623da4 2016-03-13T09:27:48.938+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5329418fa188537623da5 2016-03-13T09:27:49.557+0000 I NETWORK [conn4083772] end connection 10.0.20.53:52355 (1506 connections now open) 2016-03-13T09:27:49.557+0000 I NETWORK [conn4083771] end connection 10.0.20.53:52354 (1506 connections now open) 2016-03-13T09:27:49.557+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52374 #4083787 (1506 connections now open) 2016-03-13T09:27:49.557+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52373 #4083788 (1507 connections now open) 2016-03-13T09:27:51.908+0000 I NETWORK [conn4083774] end connection 10.0.179.209:42060 (1506 connections now open) 2016-03-13T09:27:51.908+0000 I NETWORK [conn4083773] end connection 10.0.179.209:42059 (1506 connections now open) 2016-03-13T09:27:51.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42078 #4083789 (1506 connections now open) 2016-03-13T09:27:51.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42079 #4083790 (1507 connections now open) 2016-03-13T09:27:52.589+0000 I NETWORK [conn4083775] end connection 10.0.26.69:44076 (1506 connections now open) 2016-03-13T09:27:52.589+0000 I NETWORK [conn4083776] end connection 10.0.26.69:44075 (1506 connections now open) 2016-03-13T09:27:52.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44085 #4083791 (1506 connections now open) 2016-03-13T09:27:52.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44086 #4083792 (1507 connections now open) 2016-03-13T09:27:55.310+0000 I NETWORK [conn4083778] end connection 10.0.27.237:37399 (1506 connections now open) 2016-03-13T09:27:55.310+0000 I NETWORK [conn4083777] end connection 10.0.27.237:37398 (1506 connections now open) 2016-03-13T09:27:55.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37413 #4083793 (1506 connections now open) 2016-03-13T09:27:55.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37412 #4083794 (1507 connections now open) 2016-03-13T09:27:57.180+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17386 version: 30687|1||563ba74d869758a4542b9075 based on: 30686|1||563ba74d869758a4542b9075 2016-03-13T09:27:57.475+0000 I NETWORK [conn4083783] end connection 10.0.7.113:60949 (1506 connections now open) 2016-03-13T09:27:57.475+0000 I NETWORK [conn4083784] end connection 10.0.7.113:60950 (1506 connections now open) 2016-03-13T09:27:57.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60965 #4083795 (1506 connections now open) 2016-03-13T09:27:57.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60966 #4083796 (1507 connections now open) 2016-03-13T09:27:58.113+0000 I NETWORK [conn4082322] end connection 10.0.20.53:51305 (1506 connections now open) 2016-03-13T09:27:58.113+0000 I NETWORK [conn4082854] end connection 10.0.216.90:58197 (1505 connections now open) 2016-03-13T09:27:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52383 #4083797 (1506 connections now open) 2016-03-13T09:27:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58997 #4083798 (1507 connections now open) 2016-03-13T09:27:58.505+0000 I NETWORK [conn4083786] end connection 10.0.216.90:58980 (1506 connections now open) 2016-03-13T09:27:58.505+0000 I NETWORK [conn4083785] end connection 10.0.216.90:58979 (1506 connections now open) 2016-03-13T09:27:58.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:58999 #4083799 (1506 connections now open) 2016-03-13T09:27:58.506+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59000 #4083800 (1507 connections now open) 2016-03-13T09:27:58.634+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5329e18fa188537623da6 2016-03-13T09:27:59.570+0000 I NETWORK [conn4083787] end connection 10.0.20.53:52374 (1506 connections now open) 2016-03-13T09:27:59.570+0000 I NETWORK [conn4083788] end connection 10.0.20.53:52373 (1505 connections now open) 2016-03-13T09:27:59.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52384 #4083801 (1506 connections now open) 2016-03-13T09:27:59.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52385 #4083802 (1507 connections now open) 2016-03-13T09:28:01.386+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634dab0f1ad750eb46ed0c4')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dab0f1ad750eb46ed0c4') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dc09584ad10f21451662') }, shard: "shmydomain1", lastmod: Timestamp 30687000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:28:01.539+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30687|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dab0f1ad750eb46ed0c4') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dc09584ad10f21451662') }) shmydomain1 -> shmydomain3 2016-03-13T09:28:01.921+0000 I NETWORK [conn4083790] end connection 10.0.179.209:42079 (1506 connections now open) 2016-03-13T09:28:01.921+0000 I NETWORK [conn4083789] end connection 10.0.179.209:42078 (1506 connections now open) 2016-03-13T09:28:01.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42090 #4083803 (1506 connections now open) 2016-03-13T09:28:01.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42091 #4083804 (1507 connections now open) 2016-03-13T09:28:02.602+0000 I NETWORK [conn4083791] end connection 10.0.26.69:44085 (1506 connections now open) 2016-03-13T09:28:02.602+0000 I NETWORK [conn4083792] end connection 10.0.26.69:44086 (1506 connections now open) 2016-03-13T09:28:02.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44101 #4083805 (1506 connections now open) 2016-03-13T09:28:02.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44102 #4083806 (1507 connections now open) 2016-03-13T09:28:05.323+0000 I NETWORK [conn4083794] end connection 10.0.27.237:37412 (1506 connections now open) 2016-03-13T09:28:05.323+0000 I NETWORK [conn4083793] end connection 10.0.27.237:37413 (1506 connections now open) 2016-03-13T09:28:05.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37435 #4083807 (1506 connections now open) 2016-03-13T09:28:05.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37434 #4083808 (1507 connections now open) 2016-03-13T09:28:07.488+0000 I NETWORK [conn4083795] end connection 10.0.7.113:60965 (1506 connections now open) 2016-03-13T09:28:07.488+0000 I NETWORK [conn4083796] end connection 10.0.7.113:60966 (1506 connections now open) 2016-03-13T09:28:07.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60975 #4083809 (1506 connections now open) 2016-03-13T09:28:07.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60976 #4083810 (1507 connections now open) 2016-03-13T09:28:08.518+0000 I NETWORK [conn4083800] end connection 10.0.216.90:59000 (1506 connections now open) 2016-03-13T09:28:08.518+0000 I NETWORK [conn4083799] end connection 10.0.216.90:58999 (1506 connections now open) 2016-03-13T09:28:08.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59009 #4083811 (1506 connections now open) 2016-03-13T09:28:08.519+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59008 #4083812 (1507 connections now open) 2016-03-13T09:28:09.583+0000 I NETWORK [conn4083801] end connection 10.0.20.53:52384 (1506 connections now open) 2016-03-13T09:28:09.583+0000 I NETWORK [conn4083802] end connection 10.0.20.53:52385 (1505 connections now open) 2016-03-13T09:28:09.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52399 #4083813 (1506 connections now open) 2016-03-13T09:28:09.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52398 #4083814 (1507 connections now open) 2016-03-13T09:28:11.657+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 692ms sequenceNumber: 17387 version: 30688|1||563ba74d869758a4542b9075 based on: 30687|1||563ba74d869758a4542b9075 2016-03-13T09:28:11.855+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:28:11.855+0000-56e532ab18fa188537623da7", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861291855), what: "balancer.round", ns: "", details: { executionTimeMillis: 13503, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:28:11.934+0000 I NETWORK [conn4083804] end connection 10.0.179.209:42091 (1506 connections now open) 2016-03-13T09:28:11.934+0000 I NETWORK [conn4083803] end connection 10.0.179.209:42090 (1506 connections now open) 2016-03-13T09:28:11.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42100 #4083815 (1506 connections now open) 2016-03-13T09:28:11.942+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42101 #4083816 (1507 connections now open) 2016-03-13T09:28:12.088+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:28:12.616+0000 I NETWORK [conn4083805] end connection 10.0.26.69:44101 (1506 connections now open) 2016-03-13T09:28:12.616+0000 I NETWORK [conn4083806] end connection 10.0.26.69:44102 (1506 connections now open) 2016-03-13T09:28:12.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44121 #4083817 (1507 connections now open) 2016-03-13T09:28:12.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44120 #4083818 (1507 connections now open) 2016-03-13T09:28:13.509+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e532ad18fa188537623da8 2016-03-13T09:28:15.335+0000 I NETWORK [conn4083808] end connection 10.0.27.237:37434 (1506 connections now open) 2016-03-13T09:28:15.336+0000 I NETWORK [conn4083807] end connection 10.0.27.237:37435 (1505 connections now open) 2016-03-13T09:28:15.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37446 #4083819 (1506 connections now open) 2016-03-13T09:28:15.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37447 #4083820 (1507 connections now open) 2016-03-13T09:28:16.061+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634dc09584ad10f21451662')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dc09584ad10f21451662') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dd62a2a4330ea430f8bf') }, shard: "shmydomain1", lastmod: Timestamp 30688000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:28:16.117+0000 I NETWORK [conn4083046] end connection 10.0.179.209:41335 (1506 connections now open) 2016-03-13T09:28:16.117+0000 I NETWORK [conn4081107] end connection 10.0.20.53:50754 (1505 connections now open) 2016-03-13T09:28:16.117+0000 I NETWORK [conn4077364] end connection 10.0.216.90:53964 (1504 connections now open) 2016-03-13T09:28:16.118+0000 I NETWORK [conn4081724] end connection 10.0.26.69:42802 (1503 connections now open) 2016-03-13T09:28:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42105 #4083821 (1504 connections now open) 2016-03-13T09:28:16.118+0000 I NETWORK [conn4080895] end connection 10.0.7.113:59498 (1503 connections now open) 2016-03-13T09:28:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59020 #4083822 (1504 connections now open) 2016-03-13T09:28:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44128 #4083823 (1505 connections now open) 2016-03-13T09:28:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52416 #4083824 (1506 connections now open) 2016-03-13T09:28:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60990 #4083825 (1507 connections now open) 2016-03-13T09:28:16.210+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30688|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dc09584ad10f21451662') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dd62a2a4330ea430f8bf') }) shmydomain1 -> shmydomain2 2016-03-13T09:28:17.501+0000 I NETWORK [conn4083810] end connection 10.0.7.113:60976 (1506 connections now open) 2016-03-13T09:28:17.502+0000 I NETWORK [conn4083809] end connection 10.0.7.113:60975 (1505 connections now open) 2016-03-13T09:28:17.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60992 #4083826 (1506 connections now open) 2016-03-13T09:28:17.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:60993 #4083827 (1507 connections now open) 2016-03-13T09:28:18.531+0000 I NETWORK [conn4083812] end connection 10.0.216.90:59008 (1506 connections now open) 2016-03-13T09:28:18.531+0000 I NETWORK [conn4083811] end connection 10.0.216.90:59009 (1506 connections now open) 2016-03-13T09:28:18.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59021 #4083828 (1506 connections now open) 2016-03-13T09:28:18.532+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59022 #4083829 (1507 connections now open) 2016-03-13T09:28:19.220+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:28:18.938+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:28:19.596+0000 I NETWORK [conn4083813] end connection 10.0.20.53:52399 (1506 connections now open) 2016-03-13T09:28:19.597+0000 I NETWORK [conn4083814] end connection 10.0.20.53:52398 (1505 connections now open) 2016-03-13T09:28:19.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52418 #4083830 (1506 connections now open) 2016-03-13T09:28:19.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52419 #4083831 (1507 connections now open) 2016-03-13T09:28:21.954+0000 I NETWORK [conn4083816] end connection 10.0.179.209:42101 (1506 connections now open) 2016-03-13T09:28:21.954+0000 I NETWORK [conn4083815] end connection 10.0.179.209:42100 (1506 connections now open) 2016-03-13T09:28:21.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42123 #4083832 (1506 connections now open) 2016-03-13T09:28:21.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42124 #4083833 (1507 connections now open) 2016-03-13T09:28:22.630+0000 I NETWORK [conn4083817] end connection 10.0.26.69:44121 (1506 connections now open) 2016-03-13T09:28:22.630+0000 I NETWORK [conn4083818] end connection 10.0.26.69:44120 (1506 connections now open) 2016-03-13T09:28:22.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44131 #4083834 (1506 connections now open) 2016-03-13T09:28:22.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44130 #4083835 (1507 connections now open) 2016-03-13T09:28:25.348+0000 I NETWORK [conn4083819] end connection 10.0.27.237:37446 (1506 connections now open) 2016-03-13T09:28:25.348+0000 I NETWORK [conn4083820] end connection 10.0.27.237:37447 (1506 connections now open) 2016-03-13T09:28:25.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37461 #4083836 (1506 connections now open) 2016-03-13T09:28:25.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37460 #4083837 (1507 connections now open) 2016-03-13T09:28:27.263+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17388 version: 30689|1||563ba74d869758a4542b9075 based on: 30688|1||563ba74d869758a4542b9075 2016-03-13T09:28:27.468+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:28:27.467+0000-56e532bb18fa188537623da9", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861307467), what: "balancer.round", ns: "", details: { executionTimeMillis: 14292, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:28:27.515+0000 I NETWORK [conn4083826] end connection 10.0.7.113:60992 (1506 connections now open) 2016-03-13T09:28:27.515+0000 I NETWORK [conn4083827] end connection 10.0.7.113:60993 (1506 connections now open) 2016-03-13T09:28:27.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32776 #4083838 (1506 connections now open) 2016-03-13T09:28:27.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32775 #4083839 (1507 connections now open) 2016-03-13T09:28:27.701+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:28:28.113+0000 I NETWORK [conn4082007] end connection 10.0.179.209:40900 (1506 connections now open) 2016-03-13T09:28:28.114+0000 I NETWORK [conn4081370] end connection 10.0.7.113:59660 (1505 connections now open) 2016-03-13T09:28:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42128 #4083840 (1506 connections now open) 2016-03-13T09:28:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32780 #4083841 (1507 connections now open) 2016-03-13T09:28:28.543+0000 I NETWORK [conn4083828] end connection 10.0.216.90:59021 (1506 connections now open) 2016-03-13T09:28:28.544+0000 I NETWORK [conn4083829] end connection 10.0.216.90:59022 (1506 connections now open) 2016-03-13T09:28:28.544+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59040 #4083842 (1506 connections now open) 2016-03-13T09:28:28.544+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59041 #4083843 (1507 connections now open) 2016-03-13T09:28:28.931+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e532bc18fa188537623daa 2016-03-13T09:28:29.618+0000 I NETWORK [conn4083831] end connection 10.0.20.53:52419 (1506 connections now open) 2016-03-13T09:28:29.618+0000 I NETWORK [conn4083830] end connection 10.0.20.53:52418 (1505 connections now open) 2016-03-13T09:28:29.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52429 #4083844 (1506 connections now open) 2016-03-13T09:28:29.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52428 #4083845 (1507 connections now open) 2016-03-13T09:28:31.739+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634dd62a2a4330ea430f8bf')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dd62a2a4330ea430f8bf') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634deb5a2a4330ea2a54b81') }, shard: "shmydomain1", lastmod: Timestamp 30689000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:28:31.891+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30689|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634dd62a2a4330ea430f8bf') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634deb5a2a4330ea2a54b81') }) shmydomain1 -> shmydomain3 2016-03-13T09:28:31.968+0000 I NETWORK [conn4083833] end connection 10.0.179.209:42124 (1506 connections now open) 2016-03-13T09:28:31.968+0000 I NETWORK [conn4083832] end connection 10.0.179.209:42123 (1506 connections now open) 2016-03-13T09:28:31.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42133 #4083846 (1506 connections now open) 2016-03-13T09:28:31.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42134 #4083847 (1507 connections now open) 2016-03-13T09:28:32.643+0000 I NETWORK [conn4083834] end connection 10.0.26.69:44131 (1506 connections now open) 2016-03-13T09:28:32.643+0000 I NETWORK [conn4083835] end connection 10.0.26.69:44130 (1506 connections now open) 2016-03-13T09:28:32.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44144 #4083848 (1506 connections now open) 2016-03-13T09:28:32.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44145 #4083849 (1507 connections now open) 2016-03-13T09:28:35.362+0000 I NETWORK [conn4083836] end connection 10.0.27.237:37461 (1506 connections now open) 2016-03-13T09:28:35.362+0000 I NETWORK [conn4083837] end connection 10.0.27.237:37460 (1506 connections now open) 2016-03-13T09:28:35.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37482 #4083850 (1506 connections now open) 2016-03-13T09:28:35.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37481 #4083851 (1507 connections now open) 2016-03-13T09:28:37.592+0000 I NETWORK [conn4083839] end connection 10.0.7.113:32775 (1506 connections now open) 2016-03-13T09:28:37.592+0000 I NETWORK [conn4083838] end connection 10.0.7.113:32776 (1506 connections now open) 2016-03-13T09:28:37.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32785 #4083852 (1506 connections now open) 2016-03-13T09:28:37.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32786 #4083853 (1507 connections now open) 2016-03-13T09:28:38.556+0000 I NETWORK [conn4083843] end connection 10.0.216.90:59041 (1506 connections now open) 2016-03-13T09:28:38.556+0000 I NETWORK [conn4083842] end connection 10.0.216.90:59040 (1505 connections now open) 2016-03-13T09:28:38.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59050 #4083854 (1506 connections now open) 2016-03-13T09:28:38.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59049 #4083855 (1507 connections now open) 2016-03-13T09:28:39.631+0000 I NETWORK [conn4083845] end connection 10.0.20.53:52428 (1506 connections now open) 2016-03-13T09:28:39.631+0000 I NETWORK [conn4083844] end connection 10.0.20.53:52429 (1506 connections now open) 2016-03-13T09:28:39.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52443 #4083856 (1506 connections now open) 2016-03-13T09:28:39.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52442 #4083857 (1507 connections now open) 2016-03-13T09:28:41.718+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 693ms sequenceNumber: 17389 version: 30690|1||563ba74d869758a4542b9075 based on: 30689|1||563ba74d869758a4542b9075 2016-03-13T09:28:41.916+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:28:41.916+0000-56e532c918fa188537623dab", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861321916), what: "balancer.round", ns: "", details: { executionTimeMillis: 13180, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:28:41.980+0000 I NETWORK [conn4083846] end connection 10.0.179.209:42133 (1506 connections now open) 2016-03-13T09:28:41.980+0000 I NETWORK [conn4083847] end connection 10.0.179.209:42134 (1506 connections now open) 2016-03-13T09:28:41.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42145 #4083858 (1506 connections now open) 2016-03-13T09:28:41.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42144 #4083859 (1507 connections now open) 2016-03-13T09:28:42.120+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:28:42.665+0000 I NETWORK [conn4083848] end connection 10.0.26.69:44144 (1506 connections now open) 2016-03-13T09:28:42.665+0000 I NETWORK [conn4083849] end connection 10.0.26.69:44145 (1506 connections now open) 2016-03-13T09:28:42.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44163 #4083860 (1506 connections now open) 2016-03-13T09:28:42.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44162 #4083861 (1507 connections now open) 2016-03-13T09:28:43.452+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e532cb18fa188537623dac 2016-03-13T09:28:45.375+0000 I NETWORK [conn4083851] end connection 10.0.27.237:37481 (1506 connections now open) 2016-03-13T09:28:45.375+0000 I NETWORK [conn4083850] end connection 10.0.27.237:37482 (1505 connections now open) 2016-03-13T09:28:45.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37491 #4083862 (1506 connections now open) 2016-03-13T09:28:45.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37490 #4083863 (1507 connections now open) 2016-03-13T09:28:46.117+0000 I NETWORK [conn4083041] end connection 10.0.20.53:51614 (1506 connections now open) 2016-03-13T09:28:46.117+0000 I NETWORK [conn4081279] end connection 10.0.216.90:57582 (1506 connections now open) 2016-03-13T09:28:46.117+0000 I NETWORK [conn4082215] end connection 10.0.179.209:40996 (1506 connections now open) 2016-03-13T09:28:46.117+0000 I NETWORK [conn4082012] end connection 10.0.26.69:42929 (1503 connections now open) 2016-03-13T09:28:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52461 #4083864 (1504 connections now open) 2016-03-13T09:28:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59061 #4083865 (1505 connections now open) 2016-03-13T09:28:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42149 #4083866 (1506 connections now open) 2016-03-13T09:28:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44169 #4083867 (1507 connections now open) 2016-03-13T09:28:46.119+0000 I NETWORK [conn4081018] end connection 10.0.27.237:35797 (1506 connections now open) 2016-03-13T09:28:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37494 #4083868 (1507 connections now open) 2016-03-13T09:28:46.182+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634deb5a2a4330ea2a54b81')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634deb5a2a4330ea2a54b81') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e001584ad10f21457d60') }, shard: "shmydomain1", lastmod: Timestamp 30690000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:28:46.331+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30690|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634deb5a2a4330ea2a54b81') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e001584ad10f21457d60') }) shmydomain1 -> shmydomain2 2016-03-13T09:28:47.605+0000 I NETWORK [conn4083852] end connection 10.0.7.113:32785 (1506 connections now open) 2016-03-13T09:28:47.605+0000 I NETWORK [conn4083853] end connection 10.0.7.113:32786 (1506 connections now open) 2016-03-13T09:28:47.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32800 #4083869 (1506 connections now open) 2016-03-13T09:28:47.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32801 #4083870 (1507 connections now open) 2016-03-13T09:28:48.577+0000 I NETWORK [conn4083855] end connection 10.0.216.90:59049 (1506 connections now open) 2016-03-13T09:28:48.577+0000 I NETWORK [conn4083854] end connection 10.0.216.90:59050 (1506 connections now open) 2016-03-13T09:28:48.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59064 #4083871 (1506 connections now open) 2016-03-13T09:28:48.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59063 #4083872 (1507 connections now open) 2016-03-13T09:28:49.428+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:28:49.220+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:28:49.644+0000 I NETWORK [conn4083857] end connection 10.0.20.53:52442 (1506 connections now open) 2016-03-13T09:28:49.644+0000 I NETWORK [conn4083856] end connection 10.0.20.53:52443 (1506 connections now open) 2016-03-13T09:28:49.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52463 #4083873 (1506 connections now open) 2016-03-13T09:28:49.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52464 #4083874 (1507 connections now open) 2016-03-13T09:28:51.993+0000 I NETWORK [conn4083859] end connection 10.0.179.209:42144 (1506 connections now open) 2016-03-13T09:28:51.993+0000 I NETWORK [conn4083858] end connection 10.0.179.209:42145 (1506 connections now open) 2016-03-13T09:28:51.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42164 #4083875 (1506 connections now open) 2016-03-13T09:28:51.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42165 #4083876 (1507 connections now open) 2016-03-13T09:28:52.679+0000 I NETWORK [conn4083860] end connection 10.0.26.69:44163 (1506 connections now open) 2016-03-13T09:28:52.679+0000 I NETWORK [conn4083861] end connection 10.0.26.69:44162 (1506 connections now open) 2016-03-13T09:28:52.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44170 #4083877 (1506 connections now open) 2016-03-13T09:28:52.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44171 #4083878 (1507 connections now open) 2016-03-13T09:28:55.388+0000 I NETWORK [conn4083863] end connection 10.0.27.237:37490 (1506 connections now open) 2016-03-13T09:28:55.389+0000 I NETWORK [conn4083862] end connection 10.0.27.237:37491 (1505 connections now open) 2016-03-13T09:28:55.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37505 #4083879 (1506 connections now open) 2016-03-13T09:28:55.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37504 #4083880 (1507 connections now open) 2016-03-13T09:28:57.618+0000 I NETWORK [conn4083869] end connection 10.0.7.113:32800 (1506 connections now open) 2016-03-13T09:28:57.618+0000 I NETWORK [conn4083870] end connection 10.0.7.113:32801 (1506 connections now open) 2016-03-13T09:28:57.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32817 #4083881 (1506 connections now open) 2016-03-13T09:28:57.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32818 #4083882 (1507 connections now open) 2016-03-13T09:28:58.114+0000 I NETWORK [conn4082534] end connection 10.0.216.90:58042 (1506 connections now open) 2016-03-13T09:28:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59079 #4083883 (1507 connections now open) 2016-03-13T09:28:58.114+0000 I NETWORK [conn4082837] end connection 10.0.27.237:36607 (1506 connections now open) 2016-03-13T09:28:58.114+0000 I NETWORK [conn4082698] end connection 10.0.7.113:60064 (1505 connections now open) 2016-03-13T09:28:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37512 #4083884 (1506 connections now open) 2016-03-13T09:28:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32821 #4083885 (1507 connections now open) 2016-03-13T09:28:58.590+0000 I NETWORK [conn4083872] end connection 10.0.216.90:59063 (1506 connections now open) 2016-03-13T09:28:58.590+0000 I NETWORK [conn4083871] end connection 10.0.216.90:59064 (1506 connections now open) 2016-03-13T09:28:58.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59080 #4083886 (1506 connections now open) 2016-03-13T09:28:58.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59081 #4083887 (1507 connections now open) 2016-03-13T09:28:59.657+0000 I NETWORK [conn4083874] end connection 10.0.20.53:52464 (1506 connections now open) 2016-03-13T09:28:59.657+0000 I NETWORK [conn4083873] end connection 10.0.20.53:52463 (1506 connections now open) 2016-03-13T09:28:59.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52473 #4083888 (1506 connections now open) 2016-03-13T09:28:59.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52474 #4083889 (1507 connections now open) 2016-03-13T09:29:01.269+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 706ms sequenceNumber: 17390 version: 30691|1||563ba74d869758a4542b9075 based on: 30690|1||563ba74d869758a4542b9075 2016-03-13T09:29:02.006+0000 I NETWORK [conn4083875] end connection 10.0.179.209:42164 (1506 connections now open) 2016-03-13T09:29:02.006+0000 I NETWORK [conn4083876] end connection 10.0.179.209:42165 (1506 connections now open) 2016-03-13T09:29:02.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42175 #4083890 (1506 connections now open) 2016-03-13T09:29:02.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42176 #4083891 (1507 connections now open) 2016-03-13T09:29:02.693+0000 I NETWORK [conn4083877] end connection 10.0.26.69:44170 (1506 connections now open) 2016-03-13T09:29:02.693+0000 I NETWORK [conn4083878] end connection 10.0.26.69:44171 (1506 connections now open) 2016-03-13T09:29:02.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44186 #4083892 (1506 connections now open) 2016-03-13T09:29:02.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44187 #4083893 (1507 connections now open) 2016-03-13T09:29:04.476+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:29:04.476+0000-56e532e018fa188537623dad", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861344476), what: "balancer.round", ns: "", details: { executionTimeMillis: 21322, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:29:04.690+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:29:05.402+0000 I NETWORK [conn4083880] end connection 10.0.27.237:37504 (1506 connections now open) 2016-03-13T09:29:05.402+0000 I NETWORK [conn4083879] end connection 10.0.27.237:37505 (1506 connections now open) 2016-03-13T09:29:05.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37527 #4083894 (1506 connections now open) 2016-03-13T09:29:05.402+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37528 #4083895 (1507 connections now open) 2016-03-13T09:29:06.091+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e532e118fa188537623dae 2016-03-13T09:29:07.631+0000 I NETWORK [conn4083882] end connection 10.0.7.113:32818 (1506 connections now open) 2016-03-13T09:29:07.631+0000 I NETWORK [conn4083881] end connection 10.0.7.113:32817 (1506 connections now open) 2016-03-13T09:29:07.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32828 #4083896 (1506 connections now open) 2016-03-13T09:29:07.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32827 #4083897 (1507 connections now open) 2016-03-13T09:29:08.604+0000 I NETWORK [conn4083887] end connection 10.0.216.90:59081 (1506 connections now open) 2016-03-13T09:29:08.604+0000 I NETWORK [conn4083886] end connection 10.0.216.90:59080 (1506 connections now open) 2016-03-13T09:29:08.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59088 #4083898 (1506 connections now open) 2016-03-13T09:29:08.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59089 #4083899 (1507 connections now open) 2016-03-13T09:29:08.834+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634e001584ad10f21457d60')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e001584ad10f21457d60') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e154f1ad750eb6b2581c') }, shard: "shmydomain1", lastmod: Timestamp 30691000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:29:08.983+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30691|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e001584ad10f21457d60') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e154f1ad750eb6b2581c') }) shmydomain1 -> shmydomain3 2016-03-13T09:29:09.671+0000 I NETWORK [conn4083888] end connection 10.0.20.53:52473 (1506 connections now open) 2016-03-13T09:29:09.671+0000 I NETWORK [conn4083889] end connection 10.0.20.53:52474 (1506 connections now open) 2016-03-13T09:29:09.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52486 #4083900 (1506 connections now open) 2016-03-13T09:29:09.672+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52485 #4083901 (1507 connections now open) 2016-03-13T09:29:12.020+0000 I NETWORK [conn4083891] end connection 10.0.179.209:42176 (1506 connections now open) 2016-03-13T09:29:12.020+0000 I NETWORK [conn4083890] end connection 10.0.179.209:42175 (1506 connections now open) 2016-03-13T09:29:12.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42188 #4083902 (1506 connections now open) 2016-03-13T09:29:12.085+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42187 #4083903 (1507 connections now open) 2016-03-13T09:29:12.706+0000 I NETWORK [conn4083893] end connection 10.0.26.69:44187 (1506 connections now open) 2016-03-13T09:29:12.706+0000 I NETWORK [conn4083892] end connection 10.0.26.69:44186 (1506 connections now open) 2016-03-13T09:29:12.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44207 #4083904 (1507 connections now open) 2016-03-13T09:29:12.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44206 #4083905 (1507 connections now open) 2016-03-13T09:29:15.415+0000 I NETWORK [conn4083895] end connection 10.0.27.237:37528 (1506 connections now open) 2016-03-13T09:29:15.415+0000 I NETWORK [conn4083894] end connection 10.0.27.237:37527 (1506 connections now open) 2016-03-13T09:29:15.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37539 #4083906 (1506 connections now open) 2016-03-13T09:29:15.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37538 #4083907 (1507 connections now open) 2016-03-13T09:29:16.117+0000 I NETWORK [conn4081117] end connection 10.0.20.53:50760 (1506 connections now open) 2016-03-13T09:29:16.117+0000 I NETWORK [conn4082416] end connection 10.0.26.69:43106 (1506 connections now open) 2016-03-13T09:29:16.117+0000 I NETWORK [conn4077658] end connection 10.0.216.90:54284 (1506 connections now open) 2016-03-13T09:29:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59100 #4083908 (1505 connections now open) 2016-03-13T09:29:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44214 #4083909 (1506 connections now open) 2016-03-13T09:29:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52503 #4083910 (1507 connections now open) 2016-03-13T09:29:17.645+0000 I NETWORK [conn4083897] end connection 10.0.7.113:32827 (1506 connections now open) 2016-03-13T09:29:17.645+0000 I NETWORK [conn4083896] end connection 10.0.7.113:32828 (1506 connections now open) 2016-03-13T09:29:17.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32844 #4083911 (1506 connections now open) 2016-03-13T09:29:17.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32843 #4083912 (1507 connections now open) 2016-03-13T09:29:18.617+0000 I NETWORK [conn4083899] end connection 10.0.216.90:59089 (1506 connections now open) 2016-03-13T09:29:18.617+0000 I NETWORK [conn4083898] end connection 10.0.216.90:59088 (1506 connections now open) 2016-03-13T09:29:18.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59102 #4083913 (1506 connections now open) 2016-03-13T09:29:18.617+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59103 #4083914 (1507 connections now open) 2016-03-13T09:29:19.047+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 694ms sequenceNumber: 17391 version: 30692|1||563ba74d869758a4542b9075 based on: 30691|1||563ba74d869758a4542b9075 2016-03-13T09:29:19.339+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:29:19.339+0000-56e532ef18fa188537623daf", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861359339), what: "balancer.round", ns: "", details: { executionTimeMillis: 13615, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:29:19.514+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:29:19.684+0000 I NETWORK [conn4083901] end connection 10.0.20.53:52485 (1506 connections now open) 2016-03-13T09:29:19.684+0000 I NETWORK [conn4083900] end connection 10.0.20.53:52486 (1506 connections now open) 2016-03-13T09:29:19.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52505 #4083915 (1506 connections now open) 2016-03-13T09:29:19.686+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52506 #4083916 (1507 connections now open) 2016-03-13T09:29:19.717+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:29:19.428+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:29:20.845+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e532f018fa188537623db0 2016-03-13T09:29:22.097+0000 I NETWORK [conn4083902] end connection 10.0.179.209:42188 (1506 connections now open) 2016-03-13T09:29:22.097+0000 I NETWORK [conn4083903] end connection 10.0.179.209:42187 (1506 connections now open) 2016-03-13T09:29:22.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42210 #4083917 (1506 connections now open) 2016-03-13T09:29:22.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42211 #4083918 (1507 connections now open) 2016-03-13T09:29:22.720+0000 I NETWORK [conn4083904] end connection 10.0.26.69:44207 (1506 connections now open) 2016-03-13T09:29:22.720+0000 I NETWORK [conn4083905] end connection 10.0.26.69:44206 (1506 connections now open) 2016-03-13T09:29:22.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44216 #4083919 (1506 connections now open) 2016-03-13T09:29:22.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44217 #4083920 (1507 connections now open) 2016-03-13T09:29:23.662+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634e154f1ad750eb6b2581c')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e154f1ad750eb6b2581c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e2a03694e50ec42a5c72') }, shard: "shmydomain1", lastmod: Timestamp 30692000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:29:23.814+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30692|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e154f1ad750eb6b2581c') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e2a03694e50ec42a5c72') }) shmydomain1 -> shmydomain2 2016-03-13T09:29:25.428+0000 I NETWORK [conn4083906] end connection 10.0.27.237:37539 (1506 connections now open) 2016-03-13T09:29:25.428+0000 I NETWORK [conn4083907] end connection 10.0.27.237:37538 (1506 connections now open) 2016-03-13T09:29:25.438+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37553 #4083921 (1506 connections now open) 2016-03-13T09:29:25.438+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37552 #4083922 (1507 connections now open) 2016-03-13T09:29:27.658+0000 I NETWORK [conn4083912] end connection 10.0.7.113:32843 (1506 connections now open) 2016-03-13T09:29:27.658+0000 I NETWORK [conn4083911] end connection 10.0.7.113:32844 (1506 connections now open) 2016-03-13T09:29:27.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32860 #4083923 (1506 connections now open) 2016-03-13T09:29:27.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32861 #4083924 (1507 connections now open) 2016-03-13T09:29:28.113+0000 I NETWORK [conn4082027] end connection 10.0.179.209:40910 (1506 connections now open) 2016-03-13T09:29:28.114+0000 I NETWORK [conn4079899] end connection 10.0.27.237:34888 (1505 connections now open) 2016-03-13T09:29:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42215 #4083925 (1506 connections now open) 2016-03-13T09:29:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37558 #4083926 (1507 connections now open) 2016-03-13T09:29:28.629+0000 I NETWORK [conn4083913] end connection 10.0.216.90:59102 (1506 connections now open) 2016-03-13T09:29:28.629+0000 I NETWORK [conn4083914] end connection 10.0.216.90:59103 (1506 connections now open) 2016-03-13T09:29:28.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59122 #4083927 (1506 connections now open) 2016-03-13T09:29:28.630+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59121 #4083928 (1507 connections now open) 2016-03-13T09:29:29.698+0000 I NETWORK [conn4083915] end connection 10.0.20.53:52505 (1506 connections now open) 2016-03-13T09:29:29.698+0000 I NETWORK [conn4083916] end connection 10.0.20.53:52506 (1506 connections now open) 2016-03-13T09:29:29.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52515 #4083929 (1506 connections now open) 2016-03-13T09:29:29.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52514 #4083930 (1507 connections now open) 2016-03-13T09:29:32.110+0000 I NETWORK [conn4083917] end connection 10.0.179.209:42210 (1506 connections now open) 2016-03-13T09:29:32.110+0000 I NETWORK [conn4083918] end connection 10.0.179.209:42211 (1505 connections now open) 2016-03-13T09:29:32.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42219 #4083931 (1506 connections now open) 2016-03-13T09:29:32.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42220 #4083932 (1507 connections now open) 2016-03-13T09:29:32.733+0000 I NETWORK [conn4083919] end connection 10.0.26.69:44216 (1506 connections now open) 2016-03-13T09:29:32.733+0000 I NETWORK [conn4083920] end connection 10.0.26.69:44217 (1506 connections now open) 2016-03-13T09:29:32.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44231 #4083933 (1506 connections now open) 2016-03-13T09:29:32.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44230 #4083934 (1507 connections now open) 2016-03-13T09:29:33.829+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 695ms sequenceNumber: 17392 version: 30693|1||563ba74d869758a4542b9075 based on: 30692|1||563ba74d869758a4542b9075 2016-03-13T09:29:34.121+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:29:34.121+0000-56e532fe18fa188537623db1", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861374121), what: "balancer.round", ns: "", details: { executionTimeMillis: 13570, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:29:34.226+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:29:35.450+0000 I NETWORK [conn4083922] end connection 10.0.27.237:37552 (1506 connections now open) 2016-03-13T09:29:35.450+0000 I NETWORK [conn4083921] end connection 10.0.27.237:37553 (1506 connections now open) 2016-03-13T09:29:35.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37572 #4083935 (1506 connections now open) 2016-03-13T09:29:35.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37573 #4083936 (1507 connections now open) 2016-03-13T09:29:35.456+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e532ff18fa188537623db2 2016-03-13T09:29:37.672+0000 I NETWORK [conn4083924] end connection 10.0.7.113:32861 (1506 connections now open) 2016-03-13T09:29:37.672+0000 I NETWORK [conn4083923] end connection 10.0.7.113:32860 (1506 connections now open) 2016-03-13T09:29:37.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32868 #4083937 (1506 connections now open) 2016-03-13T09:29:37.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32869 #4083938 (1507 connections now open) 2016-03-13T09:29:37.955+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634e2a03694e50ec42a5c72')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e2a03694e50ec42a5c72') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e7daa2a4330ea4321b02') }, shard: "shmydomain1", lastmod: Timestamp 30693000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:29:38.105+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30693|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e2a03694e50ec42a5c72') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e7daa2a4330ea4321b02') }) shmydomain1 -> shmydomain3 2016-03-13T09:29:38.642+0000 I NETWORK [conn4083928] end connection 10.0.216.90:59121 (1506 connections now open) 2016-03-13T09:29:38.642+0000 I NETWORK [conn4083927] end connection 10.0.216.90:59122 (1506 connections now open) 2016-03-13T09:29:38.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59133 #4083939 (1506 connections now open) 2016-03-13T09:29:38.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59132 #4083940 (1507 connections now open) 2016-03-13T09:29:39.110+0000 I SHARDING [Balancer] moveChunk result: { chunkTooBig: true, estimatedChunkSize: 131123576, ok: 0.0, errmsg: "chunk too big to move" } 2016-03-13T09:29:39.111+0000 I SHARDING [Balancer] balancer move failed: { chunkTooBig: true, estimatedChunkSize: 131123576, ok: 0.0, errmsg: "chunk too big to move" } from: shmydomain1 to: shmydomain3 chunk: min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e2a03694e50ec42a5c72') } max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e7daa2a4330ea4321b02') } 2016-03-13T09:29:39.111+0000 I SHARDING [Balancer] performing a split because migrate failed for size reasons 2016-03-13T09:29:39.720+0000 I NETWORK [conn4083929] end connection 10.0.20.53:52515 (1506 connections now open) 2016-03-13T09:29:39.720+0000 I NETWORK [conn4083930] end connection 10.0.20.53:52514 (1506 connections now open) 2016-03-13T09:29:39.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52529 #4083941 (1506 connections now open) 2016-03-13T09:29:39.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52530 #4083942 (1507 connections now open) 2016-03-13T09:29:41.210+0000 I SHARDING [Balancer] ChunkManager: time to load chunks for mydomain.Sessions: 730ms sequenceNumber: 17393 version: 30693|5||563ba74d869758a4542b9075 based on: 30693|1||563ba74d869758a4542b9075 2016-03-13T09:29:41.415+0000 I SHARDING [Balancer] split results: OK 2016-03-13T09:29:41.415+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:29:41.415+0000-56e5330518fa188537623db3", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861381415), what: "balancer.round", ns: "", details: { executionTimeMillis: 6154, errorOccured: false, candidateChunks: 1, chunksMoved: 0 } } 2016-03-13T09:29:41.607+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:29:42.123+0000 I NETWORK [conn4083932] end connection 10.0.179.209:42220 (1506 connections now open) 2016-03-13T09:29:42.123+0000 I NETWORK [conn4083931] end connection 10.0.179.209:42219 (1506 connections now open) 2016-03-13T09:29:42.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42231 #4083943 (1506 connections now open) 2016-03-13T09:29:42.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42230 #4083944 (1507 connections now open) 2016-03-13T09:29:42.756+0000 I NETWORK [conn4083934] end connection 10.0.26.69:44230 (1506 connections now open) 2016-03-13T09:29:42.756+0000 I NETWORK [conn4083933] end connection 10.0.26.69:44231 (1506 connections now open) 2016-03-13T09:29:42.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44248 #4083945 (1506 connections now open) 2016-03-13T09:29:42.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44247 #4083946 (1507 connections now open) 2016-03-13T09:29:45.463+0000 I NETWORK [conn4083936] end connection 10.0.27.237:37573 (1506 connections now open) 2016-03-13T09:29:45.463+0000 I NETWORK [conn4083935] end connection 10.0.27.237:37572 (1506 connections now open) 2016-03-13T09:29:45.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37580 #4083947 (1506 connections now open) 2016-03-13T09:29:45.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37579 #4083948 (1507 connections now open) 2016-03-13T09:29:46.117+0000 I NETWORK [conn4076243] end connection 10.0.216.90:52779 (1506 connections now open) 2016-03-13T09:29:46.117+0000 I NETWORK [conn4082732] end connection 10.0.27.237:36545 (1506 connections now open) 2016-03-13T09:29:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59144 #4083949 (1506 connections now open) 2016-03-13T09:29:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37583 #4083950 (1507 connections now open) 2016-03-13T09:29:46.118+0000 I NETWORK [conn4083443] end connection 10.0.7.113:60608 (1506 connections now open) 2016-03-13T09:29:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32882 #4083951 (1507 connections now open) 2016-03-13T09:29:47.692+0000 I NETWORK [conn4083937] end connection 10.0.7.113:32868 (1506 connections now open) 2016-03-13T09:29:47.693+0000 I NETWORK [conn4083938] end connection 10.0.7.113:32869 (1505 connections now open) 2016-03-13T09:29:47.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32884 #4083952 (1506 connections now open) 2016-03-13T09:29:47.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32885 #4083953 (1507 connections now open) 2016-03-13T09:29:48.664+0000 I NETWORK [conn4083939] end connection 10.0.216.90:59133 (1506 connections now open) 2016-03-13T09:29:48.664+0000 I NETWORK [conn4083940] end connection 10.0.216.90:59132 (1506 connections now open) 2016-03-13T09:29:48.664+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59146 #4083954 (1506 connections now open) 2016-03-13T09:29:48.664+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59147 #4083955 (1507 connections now open) 2016-03-13T09:29:49.733+0000 I NETWORK [conn4083942] end connection 10.0.20.53:52530 (1506 connections now open) 2016-03-13T09:29:49.733+0000 I NETWORK [conn4083941] end connection 10.0.20.53:52529 (1506 connections now open) 2016-03-13T09:29:49.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52549 #4083956 (1506 connections now open) 2016-03-13T09:29:49.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52550 #4083957 (1507 connections now open) 2016-03-13T09:29:49.969+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:29:49.717+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:29:52.136+0000 I NETWORK [conn4083943] end connection 10.0.179.209:42231 (1506 connections now open) 2016-03-13T09:29:52.136+0000 I NETWORK [conn4083944] end connection 10.0.179.209:42230 (1506 connections now open) 2016-03-13T09:29:52.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42250 #4083958 (1506 connections now open) 2016-03-13T09:29:52.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42251 #4083959 (1507 connections now open) 2016-03-13T09:29:52.769+0000 I NETWORK [conn4083945] end connection 10.0.26.69:44248 (1506 connections now open) 2016-03-13T09:29:52.769+0000 I NETWORK [conn4083946] end connection 10.0.26.69:44247 (1506 connections now open) 2016-03-13T09:29:52.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44257 #4083960 (1507 connections now open) 2016-03-13T09:29:52.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44258 #4083961 (1507 connections now open) 2016-03-13T09:29:55.475+0000 I NETWORK [conn4083947] end connection 10.0.27.237:37580 (1506 connections now open) 2016-03-13T09:29:55.475+0000 I NETWORK [conn4083948] end connection 10.0.27.237:37579 (1506 connections now open) 2016-03-13T09:29:55.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37594 #4083962 (1506 connections now open) 2016-03-13T09:29:55.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37595 #4083963 (1507 connections now open) 2016-03-13T09:29:57.683+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17394 version: 30694|1||563ba74d869758a4542b9075 based on: 30693|5||563ba74d869758a4542b9075 2016-03-13T09:29:57.706+0000 I NETWORK [conn4083952] end connection 10.0.7.113:32884 (1506 connections now open) 2016-03-13T09:29:57.706+0000 I NETWORK [conn4083953] end connection 10.0.7.113:32885 (1506 connections now open) 2016-03-13T09:29:57.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32901 #4083964 (1506 connections now open) 2016-03-13T09:29:57.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32900 #4083965 (1507 connections now open) 2016-03-13T09:29:58.114+0000 I NETWORK [conn4081648] end connection 10.0.20.53:51011 (1506 connections now open) 2016-03-13T09:29:58.114+0000 I NETWORK [conn4081588] end connection 10.0.27.237:36029 (1505 connections now open) 2016-03-13T09:29:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52558 #4083966 (1507 connections now open) 2016-03-13T09:29:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37601 #4083967 (1507 connections now open) 2016-03-13T09:29:58.677+0000 I NETWORK [conn4083954] end connection 10.0.216.90:59146 (1506 connections now open) 2016-03-13T09:29:58.677+0000 I NETWORK [conn4083955] end connection 10.0.216.90:59147 (1506 connections now open) 2016-03-13T09:29:58.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59166 #4083968 (1506 connections now open) 2016-03-13T09:29:58.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59165 #4083969 (1507 connections now open) 2016-03-13T09:29:59.746+0000 I NETWORK [conn4083957] end connection 10.0.20.53:52550 (1506 connections now open) 2016-03-13T09:29:59.746+0000 I NETWORK [conn4083956] end connection 10.0.20.53:52549 (1506 connections now open) 2016-03-13T09:29:59.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52560 #4083970 (1506 connections now open) 2016-03-13T09:29:59.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52561 #4083971 (1507 connections now open) 2016-03-13T09:30:02.149+0000 I NETWORK [conn4083959] end connection 10.0.179.209:42251 (1506 connections now open) 2016-03-13T09:30:02.149+0000 I NETWORK [conn4083958] end connection 10.0.179.209:42250 (1506 connections now open) 2016-03-13T09:30:02.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42263 #4083972 (1506 connections now open) 2016-03-13T09:30:02.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42264 #4083973 (1507 connections now open) 2016-03-13T09:30:02.782+0000 I NETWORK [conn4083961] end connection 10.0.26.69:44258 (1506 connections now open) 2016-03-13T09:30:02.782+0000 I NETWORK [conn4083960] end connection 10.0.26.69:44257 (1506 connections now open) 2016-03-13T09:30:02.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44274 #4083974 (1506 connections now open) 2016-03-13T09:30:02.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44275 #4083975 (1507 connections now open) 2016-03-13T09:30:05.489+0000 I NETWORK [conn4083963] end connection 10.0.27.237:37595 (1506 connections now open) 2016-03-13T09:30:05.489+0000 I NETWORK [conn4083962] end connection 10.0.27.237:37594 (1506 connections now open) 2016-03-13T09:30:05.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37619 #4083976 (1506 connections now open) 2016-03-13T09:30:05.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37618 #4083977 (1507 connections now open) 2016-03-13T09:30:07.719+0000 I NETWORK [conn4083965] end connection 10.0.7.113:32900 (1506 connections now open) 2016-03-13T09:30:07.719+0000 I NETWORK [conn4083964] end connection 10.0.7.113:32901 (1506 connections now open) 2016-03-13T09:30:07.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32913 #4083978 (1506 connections now open) 2016-03-13T09:30:07.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32914 #4083979 (1507 connections now open) 2016-03-13T09:30:08.690+0000 I NETWORK [conn4083969] end connection 10.0.216.90:59165 (1506 connections now open) 2016-03-13T09:30:08.690+0000 I NETWORK [conn4083968] end connection 10.0.216.90:59166 (1506 connections now open) 2016-03-13T09:30:08.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59174 #4083980 (1506 connections now open) 2016-03-13T09:30:08.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59175 #4083981 (1507 connections now open) 2016-03-13T09:30:09.759+0000 I NETWORK [conn4083970] end connection 10.0.20.53:52560 (1506 connections now open) 2016-03-13T09:30:09.759+0000 I NETWORK [conn4083971] end connection 10.0.20.53:52561 (1506 connections now open) 2016-03-13T09:30:09.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52575 #4083982 (1506 connections now open) 2016-03-13T09:30:09.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52576 #4083983 (1507 connections now open) 2016-03-13T09:30:12.161+0000 I NETWORK [conn4083972] end connection 10.0.179.209:42263 (1506 connections now open) 2016-03-13T09:30:12.162+0000 I NETWORK [conn4083973] end connection 10.0.179.209:42264 (1505 connections now open) 2016-03-13T09:30:12.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42275 #4083984 (1506 connections now open) 2016-03-13T09:30:12.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42276 #4083985 (1507 connections now open) 2016-03-13T09:30:12.796+0000 I NETWORK [conn4083975] end connection 10.0.26.69:44275 (1506 connections now open) 2016-03-13T09:30:12.796+0000 I NETWORK [conn4083974] end connection 10.0.26.69:44274 (1506 connections now open) 2016-03-13T09:30:12.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44296 #4083986 (1507 connections now open) 2016-03-13T09:30:12.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44297 #4083987 (1507 connections now open) 2016-03-13T09:30:15.023+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17395 version: 30695|1||563ba74d869758a4542b9075 based on: 30694|1||563ba74d869758a4542b9075 2016-03-13T09:30:15.502+0000 I NETWORK [conn4083976] end connection 10.0.27.237:37619 (1506 connections now open) 2016-03-13T09:30:15.502+0000 I NETWORK [conn4083977] end connection 10.0.27.237:37618 (1506 connections now open) 2016-03-13T09:30:15.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37631 #4083988 (1506 connections now open) 2016-03-13T09:30:15.502+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37632 #4083989 (1507 connections now open) 2016-03-13T09:30:16.117+0000 I NETWORK [conn4081567] end connection 10.0.20.53:50962 (1506 connections now open) 2016-03-13T09:30:16.117+0000 I NETWORK [conn4077406] end connection 10.0.216.90:54010 (1506 connections now open) 2016-03-13T09:30:16.117+0000 I NETWORK [conn4081728] end connection 10.0.26.69:42805 (1504 connections now open) 2016-03-13T09:30:16.117+0000 I NETWORK [conn4081345] end connection 10.0.27.237:35916 (1504 connections now open) 2016-03-13T09:30:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52593 #4083990 (1504 connections now open) 2016-03-13T09:30:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59188 #4083991 (1505 connections now open) 2016-03-13T09:30:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37635 #4083992 (1506 connections now open) 2016-03-13T09:30:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44302 #4083993 (1507 connections now open) 2016-03-13T09:30:17.733+0000 I NETWORK [conn4083978] end connection 10.0.7.113:32913 (1506 connections now open) 2016-03-13T09:30:17.733+0000 I NETWORK [conn4083979] end connection 10.0.7.113:32914 (1506 connections now open) 2016-03-13T09:30:17.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32932 #4083994 (1506 connections now open) 2016-03-13T09:30:17.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32933 #4083995 (1507 connections now open) 2016-03-13T09:30:18.703+0000 I NETWORK [conn4083980] end connection 10.0.216.90:59174 (1506 connections now open) 2016-03-13T09:30:18.703+0000 I NETWORK [conn4083981] end connection 10.0.216.90:59175 (1506 connections now open) 2016-03-13T09:30:18.703+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59189 #4083996 (1506 connections now open) 2016-03-13T09:30:18.703+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59190 #4083997 (1507 connections now open) 2016-03-13T09:30:19.772+0000 I NETWORK [conn4083982] end connection 10.0.20.53:52575 (1506 connections now open) 2016-03-13T09:30:19.772+0000 I NETWORK [conn4083983] end connection 10.0.20.53:52576 (1506 connections now open) 2016-03-13T09:30:19.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52596 #4083998 (1506 connections now open) 2016-03-13T09:30:19.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52595 #4083999 (1507 connections now open) 2016-03-13T09:30:20.250+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:30:19.969+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:30:20.250+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:30:20.382+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5330f18fa188537623db4 2016-03-13T09:30:20.519+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5331a18fa188537623db5 2016-03-13T09:30:20.655+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5332618fa188537623db6 2016-03-13T09:30:22.184+0000 I NETWORK [conn4083984] end connection 10.0.179.209:42275 (1506 connections now open) 2016-03-13T09:30:22.184+0000 I NETWORK [conn4083985] end connection 10.0.179.209:42276 (1505 connections now open) 2016-03-13T09:30:22.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42299 #4084000 (1506 connections now open) 2016-03-13T09:30:22.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42300 #4084001 (1507 connections now open) 2016-03-13T09:30:22.809+0000 I NETWORK [conn4083987] end connection 10.0.26.69:44297 (1506 connections now open) 2016-03-13T09:30:22.809+0000 I NETWORK [conn4083986] end connection 10.0.26.69:44296 (1506 connections now open) 2016-03-13T09:30:22.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44304 #4084002 (1506 connections now open) 2016-03-13T09:30:22.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44305 #4084003 (1507 connections now open) 2016-03-13T09:30:25.515+0000 I NETWORK [conn4083989] end connection 10.0.27.237:37632 (1506 connections now open) 2016-03-13T09:30:25.515+0000 I NETWORK [conn4083988] end connection 10.0.27.237:37631 (1506 connections now open) 2016-03-13T09:30:25.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37645 #4084004 (1506 connections now open) 2016-03-13T09:30:25.516+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37646 #4084005 (1507 connections now open) 2016-03-13T09:30:27.746+0000 I NETWORK [conn4083994] end connection 10.0.7.113:32932 (1506 connections now open) 2016-03-13T09:30:27.746+0000 I NETWORK [conn4083995] end connection 10.0.7.113:32933 (1506 connections now open) 2016-03-13T09:30:27.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32949 #4084006 (1506 connections now open) 2016-03-13T09:30:27.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32948 #4084007 (1507 connections now open) 2016-03-13T09:30:28.113+0000 I NETWORK [conn4082533] end connection 10.0.216.90:58041 (1506 connections now open) 2016-03-13T09:30:28.113+0000 I NETWORK [conn4082547] end connection 10.0.26.69:43146 (1506 connections now open) 2016-03-13T09:30:28.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59208 #4084008 (1506 connections now open) 2016-03-13T09:30:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44316 #4084009 (1507 connections now open) 2016-03-13T09:30:28.715+0000 I NETWORK [conn4083997] end connection 10.0.216.90:59190 (1506 connections now open) 2016-03-13T09:30:28.716+0000 I NETWORK [conn4083996] end connection 10.0.216.90:59189 (1505 connections now open) 2016-03-13T09:30:28.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59210 #4084010 (1506 connections now open) 2016-03-13T09:30:28.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59209 #4084011 (1507 connections now open) 2016-03-13T09:30:29.680+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17396 version: 30696|1||563ba74d869758a4542b9075 based on: 30695|1||563ba74d869758a4542b9075 2016-03-13T09:30:29.786+0000 I NETWORK [conn4083998] end connection 10.0.20.53:52596 (1506 connections now open) 2016-03-13T09:30:29.786+0000 I NETWORK [conn4083999] end connection 10.0.20.53:52595 (1506 connections now open) 2016-03-13T09:30:29.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52606 #4084012 (1506 connections now open) 2016-03-13T09:30:29.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52605 #4084013 (1507 connections now open) 2016-03-13T09:30:32.197+0000 I NETWORK [conn4084000] end connection 10.0.179.209:42299 (1506 connections now open) 2016-03-13T09:30:32.197+0000 I NETWORK [conn4084001] end connection 10.0.179.209:42300 (1506 connections now open) 2016-03-13T09:30:32.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42309 #4084014 (1506 connections now open) 2016-03-13T09:30:32.198+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42308 #4084015 (1507 connections now open) 2016-03-13T09:30:32.822+0000 I NETWORK [conn4084002] end connection 10.0.26.69:44304 (1506 connections now open) 2016-03-13T09:30:32.822+0000 I NETWORK [conn4084003] end connection 10.0.26.69:44305 (1506 connections now open) 2016-03-13T09:30:32.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44318 #4084016 (1506 connections now open) 2016-03-13T09:30:32.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44319 #4084017 (1507 connections now open) 2016-03-13T09:30:35.528+0000 I NETWORK [conn4084004] end connection 10.0.27.237:37645 (1506 connections now open) 2016-03-13T09:30:35.528+0000 I NETWORK [conn4084005] end connection 10.0.27.237:37646 (1506 connections now open) 2016-03-13T09:30:35.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37667 #4084018 (1506 connections now open) 2016-03-13T09:30:35.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37668 #4084019 (1507 connections now open) 2016-03-13T09:30:37.759+0000 I NETWORK [conn4084006] end connection 10.0.7.113:32949 (1506 connections now open) 2016-03-13T09:30:37.759+0000 I NETWORK [conn4084007] end connection 10.0.7.113:32948 (1506 connections now open) 2016-03-13T09:30:37.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32959 #4084020 (1506 connections now open) 2016-03-13T09:30:37.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32958 #4084021 (1507 connections now open) 2016-03-13T09:30:38.729+0000 I NETWORK [conn4084010] end connection 10.0.216.90:59210 (1506 connections now open) 2016-03-13T09:30:38.729+0000 I NETWORK [conn4084011] end connection 10.0.216.90:59209 (1506 connections now open) 2016-03-13T09:30:38.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59218 #4084022 (1506 connections now open) 2016-03-13T09:30:38.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59219 #4084023 (1507 connections now open) 2016-03-13T09:30:39.808+0000 I NETWORK [conn4084013] end connection 10.0.20.53:52605 (1506 connections now open) 2016-03-13T09:30:39.808+0000 I NETWORK [conn4084012] end connection 10.0.20.53:52606 (1506 connections now open) 2016-03-13T09:30:39.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52622 #4084024 (1506 connections now open) 2016-03-13T09:30:39.809+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52621 #4084025 (1507 connections now open) 2016-03-13T09:30:42.209+0000 I NETWORK [conn4084014] end connection 10.0.179.209:42309 (1506 connections now open) 2016-03-13T09:30:42.209+0000 I NETWORK [conn4084015] end connection 10.0.179.209:42308 (1506 connections now open) 2016-03-13T09:30:42.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42321 #4084026 (1506 connections now open) 2016-03-13T09:30:42.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42320 #4084027 (1507 connections now open) 2016-03-13T09:30:42.845+0000 I NETWORK [conn4084016] end connection 10.0.26.69:44318 (1506 connections now open) 2016-03-13T09:30:42.845+0000 I NETWORK [conn4084017] end connection 10.0.26.69:44319 (1506 connections now open) 2016-03-13T09:30:42.846+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44336 #4084028 (1506 connections now open) 2016-03-13T09:30:42.846+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44335 #4084029 (1507 connections now open) 2016-03-13T09:30:43.430+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17397 version: 30697|1||563ba74d869758a4542b9075 based on: 30696|1||563ba74d869758a4542b9075 2016-03-13T09:30:43.575+0000 I NETWORK [conn4080668] end connection 10.0.91.115:55150 (1506 connections now open) 2016-03-13T09:30:43.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:55845 #4084030 (1507 connections now open) 2016-03-13T09:30:44.640+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5334418fa188537623db9 2016-03-13T09:30:45.551+0000 I NETWORK [conn4084018] end connection 10.0.27.237:37667 (1506 connections now open) 2016-03-13T09:30:45.551+0000 I NETWORK [conn4084019] end connection 10.0.27.237:37668 (1506 connections now open) 2016-03-13T09:30:45.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37677 #4084031 (1506 connections now open) 2016-03-13T09:30:45.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37676 #4084032 (1507 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [conn4082628] end connection 10.0.179.209:41141 (1506 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [conn4082992] end connection 10.0.20.53:51598 (1506 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [conn4081468] end connection 10.0.216.90:57639 (1504 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [conn4082841] end connection 10.0.26.69:43280 (1503 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52639 #4084033 (1504 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59231 #4084034 (1505 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42325 #4084035 (1506 connections now open) 2016-03-13T09:30:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44341 #4084036 (1507 connections now open) 2016-03-13T09:30:47.413+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634e7daa2a4330ea4321b02')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e7daa2a4330ea4321b02') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e923bcf3ae0ea4e85fbf') }, shard: "shmydomain1", lastmod: Timestamp 30697000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:30:47.563+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30697|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e7daa2a4330ea4321b02') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e923bcf3ae0ea4e85fbf') }) shmydomain1 -> shmydomain3 2016-03-13T09:30:47.781+0000 I NETWORK [conn4084020] end connection 10.0.7.113:32959 (1506 connections now open) 2016-03-13T09:30:47.781+0000 I NETWORK [conn4084021] end connection 10.0.7.113:32958 (1506 connections now open) 2016-03-13T09:30:47.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32974 #4084037 (1506 connections now open) 2016-03-13T09:30:47.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32973 #4084038 (1507 connections now open) 2016-03-13T09:30:48.751+0000 I NETWORK [conn4084022] end connection 10.0.216.90:59218 (1506 connections now open) 2016-03-13T09:30:48.751+0000 I NETWORK [conn4084023] end connection 10.0.216.90:59219 (1506 connections now open) 2016-03-13T09:30:48.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59233 #4084039 (1506 connections now open) 2016-03-13T09:30:48.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59234 #4084040 (1507 connections now open) 2016-03-13T09:30:49.821+0000 I NETWORK [conn4084025] end connection 10.0.20.53:52621 (1506 connections now open) 2016-03-13T09:30:49.821+0000 I NETWORK [conn4084024] end connection 10.0.20.53:52622 (1506 connections now open) 2016-03-13T09:30:49.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52642 #4084041 (1507 connections now open) 2016-03-13T09:30:49.822+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52641 #4084042 (1507 connections now open) 2016-03-13T09:30:50.786+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:30:50.655+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:30:50.786+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:30:50.854+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5333018fa188537623db7 2016-03-13T09:30:50.922+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5333a18fa188537623db8 2016-03-13T09:30:52.223+0000 I NETWORK [conn4084027] end connection 10.0.179.209:42320 (1506 connections now open) 2016-03-13T09:30:52.223+0000 I NETWORK [conn4084026] end connection 10.0.179.209:42321 (1506 connections now open) 2016-03-13T09:30:52.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42341 #4084043 (1506 connections now open) 2016-03-13T09:30:52.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42340 #4084044 (1507 connections now open) 2016-03-13T09:30:52.859+0000 I NETWORK [conn4084029] end connection 10.0.26.69:44335 (1506 connections now open) 2016-03-13T09:30:52.859+0000 I NETWORK [conn4084028] end connection 10.0.26.69:44336 (1506 connections now open) 2016-03-13T09:30:52.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44342 #4084045 (1506 connections now open) 2016-03-13T09:30:52.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44343 #4084046 (1507 connections now open) 2016-03-13T09:30:55.564+0000 I NETWORK [conn4084032] end connection 10.0.27.237:37676 (1506 connections now open) 2016-03-13T09:30:55.564+0000 I NETWORK [conn4084031] end connection 10.0.27.237:37677 (1506 connections now open) 2016-03-13T09:30:55.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37688 #4084047 (1506 connections now open) 2016-03-13T09:30:55.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37689 #4084048 (1507 connections now open) 2016-03-13T09:30:57.474+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17398 version: 30698|1||563ba74d869758a4542b9075 based on: 30697|1||563ba74d869758a4542b9075 2016-03-13T09:30:57.767+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:30:57.767+0000-56e5335118fa188537623dba", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861457767), what: "balancer.round", ns: "", details: { executionTimeMillis: 13465, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:30:57.795+0000 I NETWORK [conn4084038] end connection 10.0.7.113:32973 (1506 connections now open) 2016-03-13T09:30:57.795+0000 I NETWORK [conn4084037] end connection 10.0.7.113:32974 (1506 connections now open) 2016-03-13T09:30:57.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32990 #4084049 (1506 connections now open) 2016-03-13T09:30:57.795+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32991 #4084050 (1507 connections now open) 2016-03-13T09:30:57.988+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:30:58.114+0000 I NETWORK [conn4080884] end connection 10.0.179.209:40570 (1506 connections now open) 2016-03-13T09:30:58.114+0000 I NETWORK [conn4083966] end connection 10.0.20.53:52558 (1505 connections now open) 2016-03-13T09:30:58.114+0000 I NETWORK [conn4081132] end connection 10.0.27.237:35845 (1504 connections now open) 2016-03-13T09:30:58.114+0000 I NETWORK [conn4081374] end connection 10.0.7.113:59661 (1503 connections now open) 2016-03-13T09:30:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42346 #4084051 (1504 connections now open) 2016-03-13T09:30:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52650 #4084052 (1505 connections now open) 2016-03-13T09:30:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37695 #4084053 (1506 connections now open) 2016-03-13T09:30:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:32994 #4084054 (1507 connections now open) 2016-03-13T09:30:58.763+0000 I NETWORK [conn4084039] end connection 10.0.216.90:59233 (1506 connections now open) 2016-03-13T09:30:58.764+0000 I NETWORK [conn4084040] end connection 10.0.216.90:59234 (1506 connections now open) 2016-03-13T09:30:58.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59251 #4084055 (1506 connections now open) 2016-03-13T09:30:58.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59252 #4084056 (1507 connections now open) 2016-03-13T09:30:59.320+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5335318fa188537623dbb 2016-03-13T09:30:59.835+0000 I NETWORK [conn4084042] end connection 10.0.20.53:52641 (1506 connections now open) 2016-03-13T09:30:59.835+0000 I NETWORK [conn4084041] end connection 10.0.20.53:52642 (1506 connections now open) 2016-03-13T09:30:59.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52652 #4084057 (1506 connections now open) 2016-03-13T09:30:59.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52653 #4084058 (1507 connections now open) 2016-03-13T09:31:02.035+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634e923bcf3ae0ea4e85fbf')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e923bcf3ae0ea4e85fbf') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ea6ba2a4330ea4325d3a') }, shard: "shmydomain1", lastmod: Timestamp 30698000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:31:02.191+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30698|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634e923bcf3ae0ea4e85fbf') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ea6ba2a4330ea4325d3a') }) shmydomain1 -> shmydomain2 2016-03-13T09:31:02.236+0000 I NETWORK [conn4084044] end connection 10.0.179.209:42340 (1506 connections now open) 2016-03-13T09:31:02.236+0000 I NETWORK [conn4084043] end connection 10.0.179.209:42341 (1506 connections now open) 2016-03-13T09:31:02.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42352 #4084059 (1506 connections now open) 2016-03-13T09:31:02.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42353 #4084060 (1507 connections now open) 2016-03-13T09:31:02.873+0000 I NETWORK [conn4084045] end connection 10.0.26.69:44342 (1506 connections now open) 2016-03-13T09:31:02.873+0000 I NETWORK [conn4084046] end connection 10.0.26.69:44343 (1506 connections now open) 2016-03-13T09:31:02.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44359 #4084061 (1506 connections now open) 2016-03-13T09:31:02.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44358 #4084062 (1507 connections now open) 2016-03-13T09:31:05.578+0000 I NETWORK [conn4084048] end connection 10.0.27.237:37689 (1506 connections now open) 2016-03-13T09:31:05.578+0000 I NETWORK [conn4084047] end connection 10.0.27.237:37688 (1506 connections now open) 2016-03-13T09:31:05.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37711 #4084063 (1506 connections now open) 2016-03-13T09:31:05.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37712 #4084064 (1507 connections now open) 2016-03-13T09:31:07.809+0000 I NETWORK [conn4084050] end connection 10.0.7.113:32991 (1506 connections now open) 2016-03-13T09:31:07.809+0000 I NETWORK [conn4084049] end connection 10.0.7.113:32990 (1506 connections now open) 2016-03-13T09:31:07.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33001 #4084065 (1506 connections now open) 2016-03-13T09:31:07.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33002 #4084066 (1507 connections now open) 2016-03-13T09:31:08.777+0000 I NETWORK [conn4084056] end connection 10.0.216.90:59252 (1506 connections now open) 2016-03-13T09:31:08.777+0000 I NETWORK [conn4084055] end connection 10.0.216.90:59251 (1506 connections now open) 2016-03-13T09:31:08.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59260 #4084067 (1506 connections now open) 2016-03-13T09:31:08.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59261 #4084068 (1507 connections now open) 2016-03-13T09:31:09.847+0000 I NETWORK [conn4084057] end connection 10.0.20.53:52652 (1506 connections now open) 2016-03-13T09:31:09.848+0000 I NETWORK [conn4084058] end connection 10.0.20.53:52653 (1505 connections now open) 2016-03-13T09:31:09.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52666 #4084069 (1506 connections now open) 2016-03-13T09:31:09.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52667 #4084070 (1507 connections now open) 2016-03-13T09:31:12.208+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17399 version: 30699|1||563ba74d869758a4542b9075 based on: 30698|1||563ba74d869758a4542b9075 2016-03-13T09:31:12.249+0000 I NETWORK [conn4084059] end connection 10.0.179.209:42352 (1506 connections now open) 2016-03-13T09:31:12.249+0000 I NETWORK [conn4084060] end connection 10.0.179.209:42353 (1506 connections now open) 2016-03-13T09:31:12.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42364 #4084071 (1506 connections now open) 2016-03-13T09:31:12.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42363 #4084072 (1507 connections now open) 2016-03-13T09:31:12.500+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:31:12.500+0000-56e5336018fa188537623dbc", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861472500), what: "balancer.round", ns: "", details: { executionTimeMillis: 13476, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:31:12.731+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:31:12.886+0000 I NETWORK [conn4084061] end connection 10.0.26.69:44359 (1506 connections now open) 2016-03-13T09:31:12.886+0000 I NETWORK [conn4084062] end connection 10.0.26.69:44358 (1506 connections now open) 2016-03-13T09:31:12.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44379 #4084073 (1507 connections now open) 2016-03-13T09:31:12.887+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44380 #4084074 (1507 connections now open) 2016-03-13T09:31:13.963+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5336118fa188537623dbd 2016-03-13T09:31:15.591+0000 I NETWORK [conn4084063] end connection 10.0.27.237:37711 (1506 connections now open) 2016-03-13T09:31:15.591+0000 I NETWORK [conn4084064] end connection 10.0.27.237:37712 (1506 connections now open) 2016-03-13T09:31:15.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37723 #4084075 (1506 connections now open) 2016-03-13T09:31:15.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37724 #4084076 (1507 connections now open) 2016-03-13T09:31:16.118+0000 I NETWORK [conn4080705] end connection 10.0.216.90:57379 (1506 connections now open) 2016-03-13T09:31:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59270 #4084077 (1507 connections now open) 2016-03-13T09:31:16.461+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634ea6ba2a4330ea4325d3a')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ea6ba2a4330ea4325d3a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ebb4a2a4330ea4327f2e') }, shard: "shmydomain1", lastmod: Timestamp 30699000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:31:16.610+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30699|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ea6ba2a4330ea4325d3a') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ebb4a2a4330ea4327f2e') }) shmydomain1 -> shmydomain3 2016-03-13T09:31:17.823+0000 I NETWORK [conn4084066] end connection 10.0.7.113:33002 (1506 connections now open) 2016-03-13T09:31:17.823+0000 I NETWORK [conn4084065] end connection 10.0.7.113:33001 (1506 connections now open) 2016-03-13T09:31:17.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33018 #4084078 (1506 connections now open) 2016-03-13T09:31:17.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33019 #4084079 (1507 connections now open) 2016-03-13T09:31:18.790+0000 I NETWORK [conn4084068] end connection 10.0.216.90:59261 (1506 connections now open) 2016-03-13T09:31:18.790+0000 I NETWORK [conn4084067] end connection 10.0.216.90:59260 (1506 connections now open) 2016-03-13T09:31:18.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59273 #4084080 (1506 connections now open) 2016-03-13T09:31:18.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59272 #4084081 (1507 connections now open) 2016-03-13T09:31:19.860+0000 I NETWORK [conn4084069] end connection 10.0.20.53:52666 (1506 connections now open) 2016-03-13T09:31:19.860+0000 I NETWORK [conn4084070] end connection 10.0.20.53:52667 (1505 connections now open) 2016-03-13T09:31:19.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52686 #4084082 (1506 connections now open) 2016-03-13T09:31:19.861+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52687 #4084083 (1507 connections now open) 2016-03-13T09:31:21.179+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:31:20.923+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:31:22.271+0000 I NETWORK [conn4084071] end connection 10.0.179.209:42364 (1506 connections now open) 2016-03-13T09:31:22.271+0000 I NETWORK [conn4084072] end connection 10.0.179.209:42363 (1506 connections now open) 2016-03-13T09:31:22.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42387 #4084084 (1506 connections now open) 2016-03-13T09:31:22.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42386 #4084085 (1507 connections now open) 2016-03-13T09:31:22.900+0000 I NETWORK [conn4084073] end connection 10.0.26.69:44379 (1506 connections now open) 2016-03-13T09:31:22.900+0000 I NETWORK [conn4084074] end connection 10.0.26.69:44380 (1506 connections now open) 2016-03-13T09:31:22.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44387 #4084086 (1506 connections now open) 2016-03-13T09:31:22.901+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44386 #4084087 (1507 connections now open) 2016-03-13T09:31:25.604+0000 I NETWORK [conn4084076] end connection 10.0.27.237:37724 (1506 connections now open) 2016-03-13T09:31:25.604+0000 I NETWORK [conn4084075] end connection 10.0.27.237:37723 (1506 connections now open) 2016-03-13T09:31:25.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37736 #4084088 (1506 connections now open) 2016-03-13T09:31:25.604+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37735 #4084089 (1507 connections now open) 2016-03-13T09:31:26.528+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 699ms sequenceNumber: 17400 version: 30700|1||563ba74d869758a4542b9075 based on: 30699|1||563ba74d869758a4542b9075 2016-03-13T09:31:26.832+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:31:26.832+0000-56e5336e18fa188537623dbe", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861486832), what: "balancer.round", ns: "", details: { executionTimeMillis: 13065, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:31:26.973+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:31:27.836+0000 I NETWORK [conn4084079] end connection 10.0.7.113:33019 (1506 connections now open) 2016-03-13T09:31:27.836+0000 I NETWORK [conn4084078] end connection 10.0.7.113:33018 (1506 connections now open) 2016-03-13T09:31:27.837+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33036 #4084090 (1506 connections now open) 2016-03-13T09:31:27.837+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33035 #4084091 (1507 connections now open) 2016-03-13T09:31:28.114+0000 I NETWORK [conn4080956] end connection 10.0.179.209:40606 (1506 connections now open) 2016-03-13T09:31:28.114+0000 I NETWORK [conn4081659] end connection 10.0.20.53:51021 (1505 connections now open) 2016-03-13T09:31:28.114+0000 I NETWORK [conn4082551] end connection 10.0.27.237:36453 (1504 connections now open) 2016-03-13T09:31:28.114+0000 I NETWORK [conn4082705] end connection 10.0.7.113:60067 (1503 connections now open) 2016-03-13T09:31:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42392 #4084092 (1504 connections now open) 2016-03-13T09:31:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52694 #4084093 (1505 connections now open) 2016-03-13T09:31:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37743 #4084094 (1506 connections now open) 2016-03-13T09:31:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33038 #4084095 (1507 connections now open) 2016-03-13T09:31:28.474+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5337018fa188537623dbf 2016-03-13T09:31:28.802+0000 I NETWORK [conn4084080] end connection 10.0.216.90:59273 (1506 connections now open) 2016-03-13T09:31:28.802+0000 I NETWORK [conn4084081] end connection 10.0.216.90:59272 (1506 connections now open) 2016-03-13T09:31:28.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59294 #4084096 (1506 connections now open) 2016-03-13T09:31:28.803+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59295 #4084097 (1507 connections now open) 2016-03-13T09:31:29.874+0000 I NETWORK [conn4084083] end connection 10.0.20.53:52687 (1506 connections now open) 2016-03-13T09:31:29.874+0000 I NETWORK [conn4084082] end connection 10.0.20.53:52686 (1506 connections now open) 2016-03-13T09:31:29.883+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52696 #4084098 (1506 connections now open) 2016-03-13T09:31:29.883+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52697 #4084099 (1507 connections now open) 2016-03-13T09:31:31.204+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634ebb4a2a4330ea4327f2e')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ebb4a2a4330ea4327f2e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ecfacf7b4e0eb53e990d') }, shard: "shmydomain1", lastmod: Timestamp 30700000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:31:31.355+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30700|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ebb4a2a4330ea4327f2e') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ecfacf7b4e0eb53e990d') }) shmydomain1 -> shmydomain2 2016-03-13T09:31:32.284+0000 I NETWORK [conn4084085] end connection 10.0.179.209:42386 (1506 connections now open) 2016-03-13T09:31:32.284+0000 I NETWORK [conn4084084] end connection 10.0.179.209:42387 (1506 connections now open) 2016-03-13T09:31:32.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42397 #4084100 (1506 connections now open) 2016-03-13T09:31:32.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42396 #4084101 (1507 connections now open) 2016-03-13T09:31:32.914+0000 I NETWORK [conn4084086] end connection 10.0.26.69:44387 (1506 connections now open) 2016-03-13T09:31:32.914+0000 I NETWORK [conn4084087] end connection 10.0.26.69:44386 (1506 connections now open) 2016-03-13T09:31:32.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44402 #4084102 (1506 connections now open) 2016-03-13T09:31:32.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44401 #4084103 (1507 connections now open) 2016-03-13T09:31:35.617+0000 I NETWORK [conn4084089] end connection 10.0.27.237:37735 (1506 connections now open) 2016-03-13T09:31:35.617+0000 I NETWORK [conn4084088] end connection 10.0.27.237:37736 (1506 connections now open) 2016-03-13T09:31:35.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37758 #4084104 (1506 connections now open) 2016-03-13T09:31:35.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37757 #4084105 (1507 connections now open) 2016-03-13T09:31:37.850+0000 I NETWORK [conn4084091] end connection 10.0.7.113:33035 (1506 connections now open) 2016-03-13T09:31:37.850+0000 I NETWORK [conn4084090] end connection 10.0.7.113:33036 (1506 connections now open) 2016-03-13T09:31:37.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33043 #4084106 (1506 connections now open) 2016-03-13T09:31:37.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33044 #4084107 (1507 connections now open) 2016-03-13T09:31:38.815+0000 I NETWORK [conn4084096] end connection 10.0.216.90:59294 (1506 connections now open) 2016-03-13T09:31:38.815+0000 I NETWORK [conn4084097] end connection 10.0.216.90:59295 (1506 connections now open) 2016-03-13T09:31:38.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59301 #4084108 (1506 connections now open) 2016-03-13T09:31:38.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59302 #4084109 (1507 connections now open) 2016-03-13T09:31:39.896+0000 I NETWORK [conn4084098] end connection 10.0.20.53:52696 (1506 connections now open) 2016-03-13T09:31:39.896+0000 I NETWORK [conn4084099] end connection 10.0.20.53:52697 (1506 connections now open) 2016-03-13T09:31:39.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52710 #4084110 (1506 connections now open) 2016-03-13T09:31:39.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52711 #4084111 (1507 connections now open) 2016-03-13T09:31:42.159+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17401 version: 30701|1||563ba74d869758a4542b9075 based on: 30700|1||563ba74d869758a4542b9075 2016-03-13T09:31:42.297+0000 I NETWORK [conn4084100] end connection 10.0.179.209:42397 (1506 connections now open) 2016-03-13T09:31:42.297+0000 I NETWORK [conn4084101] end connection 10.0.179.209:42396 (1506 connections now open) 2016-03-13T09:31:42.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42408 #4084112 (1506 connections now open) 2016-03-13T09:31:42.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42409 #4084113 (1507 connections now open) 2016-03-13T09:31:42.453+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:31:42.453+0000-56e5337e18fa188537623dc0", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861502453), what: "balancer.round", ns: "", details: { executionTimeMillis: 14335, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:31:42.627+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:31:42.936+0000 I NETWORK [conn4084103] end connection 10.0.26.69:44401 (1506 connections now open) 2016-03-13T09:31:42.936+0000 I NETWORK [conn4084102] end connection 10.0.26.69:44402 (1506 connections now open) 2016-03-13T09:31:42.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44417 #4084114 (1507 connections now open) 2016-03-13T09:31:42.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44416 #4084115 (1507 connections now open) 2016-03-13T09:31:43.994+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5337f18fa188537623dc1 2016-03-13T09:31:45.638+0000 I NETWORK [conn4084105] end connection 10.0.27.237:37757 (1506 connections now open) 2016-03-13T09:31:45.638+0000 I NETWORK [conn4084104] end connection 10.0.27.237:37758 (1506 connections now open) 2016-03-13T09:31:45.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37765 #4084116 (1506 connections now open) 2016-03-13T09:31:45.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37766 #4084117 (1507 connections now open) 2016-03-13T09:31:46.117+0000 I NETWORK [conn4082744] end connection 10.0.179.209:41183 (1506 connections now open) 2016-03-13T09:31:46.117+0000 I NETWORK [conn4081575] end connection 10.0.20.53:50965 (1505 connections now open) 2016-03-13T09:31:46.117+0000 I NETWORK [conn4079770] end connection 10.0.216.90:56445 (1504 connections now open) 2016-03-13T09:31:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42413 #4084118 (1505 connections now open) 2016-03-13T09:31:46.117+0000 I NETWORK [conn4080899] end connection 10.0.7.113:59500 (1504 connections now open) 2016-03-13T09:31:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52729 #4084119 (1505 connections now open) 2016-03-13T09:31:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59313 #4084120 (1506 connections now open) 2016-03-13T09:31:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33057 #4084121 (1507 connections now open) 2016-03-13T09:31:46.579+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634ecfacf7b4e0eb53e990d')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ecfacf7b4e0eb53e990d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ee473694e50ec42b9b32') }, shard: "shmydomain1", lastmod: Timestamp 30701000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:31:46.732+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30701|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ecfacf7b4e0eb53e990d') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ee473694e50ec42b9b32') }) shmydomain1 -> shmydomain3 2016-03-13T09:31:47.873+0000 I NETWORK [conn4084106] end connection 10.0.7.113:33043 (1506 connections now open) 2016-03-13T09:31:47.873+0000 I NETWORK [conn4084107] end connection 10.0.7.113:33044 (1506 connections now open) 2016-03-13T09:31:47.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33059 #4084122 (1506 connections now open) 2016-03-13T09:31:47.874+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33060 #4084123 (1507 connections now open) 2016-03-13T09:31:48.837+0000 I NETWORK [conn4084109] end connection 10.0.216.90:59302 (1506 connections now open) 2016-03-13T09:31:48.837+0000 I NETWORK [conn4084108] end connection 10.0.216.90:59301 (1506 connections now open) 2016-03-13T09:31:48.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59315 #4084124 (1506 connections now open) 2016-03-13T09:31:48.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59316 #4084125 (1507 connections now open) 2016-03-13T09:31:49.909+0000 I NETWORK [conn4084111] end connection 10.0.20.53:52711 (1506 connections now open) 2016-03-13T09:31:49.909+0000 I NETWORK [conn4084110] end connection 10.0.20.53:52710 (1506 connections now open) 2016-03-13T09:31:49.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52731 #4084126 (1506 connections now open) 2016-03-13T09:31:49.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52732 #4084127 (1507 connections now open) 2016-03-13T09:31:51.468+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:31:51.179+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:31:52.309+0000 I NETWORK [conn4084112] end connection 10.0.179.209:42408 (1506 connections now open) 2016-03-13T09:31:52.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42428 #4084128 (1507 connections now open) 2016-03-13T09:31:52.310+0000 I NETWORK [conn4084113] end connection 10.0.179.209:42409 (1506 connections now open) 2016-03-13T09:31:52.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42429 #4084129 (1507 connections now open) 2016-03-13T09:31:52.950+0000 I NETWORK [conn4084114] end connection 10.0.26.69:44417 (1506 connections now open) 2016-03-13T09:31:52.950+0000 I NETWORK [conn4084115] end connection 10.0.26.69:44416 (1506 connections now open) 2016-03-13T09:31:52.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44426 #4084130 (1506 connections now open) 2016-03-13T09:31:52.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44425 #4084131 (1507 connections now open) 2016-03-13T09:31:55.650+0000 I NETWORK [conn4084117] end connection 10.0.27.237:37766 (1506 connections now open) 2016-03-13T09:31:55.650+0000 I NETWORK [conn4084116] end connection 10.0.27.237:37765 (1506 connections now open) 2016-03-13T09:31:55.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37777 #4084132 (1506 connections now open) 2016-03-13T09:31:55.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37778 #4084133 (1507 connections now open) 2016-03-13T09:31:56.597+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17402 version: 30702|1||563ba74d869758a4542b9075 based on: 30701|1||563ba74d869758a4542b9075 2016-03-13T09:31:56.888+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:31:56.888+0000-56e5338c18fa188537623dc2", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861516888), what: "balancer.round", ns: "", details: { executionTimeMillis: 13203, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:31:57.061+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:31:57.887+0000 I NETWORK [conn4084122] end connection 10.0.7.113:33059 (1506 connections now open) 2016-03-13T09:31:57.887+0000 I NETWORK [conn4084123] end connection 10.0.7.113:33060 (1506 connections now open) 2016-03-13T09:31:57.888+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33074 #4084134 (1506 connections now open) 2016-03-13T09:31:57.888+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33075 #4084135 (1507 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [conn4081665] end connection 10.0.20.53:51023 (1506 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [conn4082843] end connection 10.0.216.90:58193 (1505 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [conn4081067] end connection 10.0.27.237:35820 (1504 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [conn4081619] end connection 10.0.26.69:42738 (1504 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [conn4081454] end connection 10.0.7.113:59681 (1502 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52738 #4084136 (1503 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59332 #4084137 (1504 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44435 #4084138 (1505 connections now open) 2016-03-13T09:31:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37785 #4084139 (1506 connections now open) 2016-03-13T09:31:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33078 #4084140 (1507 connections now open) 2016-03-13T09:31:58.437+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5338e18fa188537623dc3 2016-03-13T09:31:58.850+0000 I NETWORK [conn4084124] end connection 10.0.216.90:59315 (1506 connections now open) 2016-03-13T09:31:58.850+0000 I NETWORK [conn4084125] end connection 10.0.216.90:59316 (1506 connections now open) 2016-03-13T09:31:58.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59334 #4084141 (1506 connections now open) 2016-03-13T09:31:58.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59333 #4084142 (1507 connections now open) 2016-03-13T09:31:59.923+0000 I NETWORK [conn4084126] end connection 10.0.20.53:52731 (1506 connections now open) 2016-03-13T09:31:59.923+0000 I NETWORK [conn4084127] end connection 10.0.20.53:52732 (1506 connections now open) 2016-03-13T09:31:59.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52740 #4084143 (1506 connections now open) 2016-03-13T09:31:59.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52741 #4084144 (1507 connections now open) 2016-03-13T09:32:01.462+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634ee473694e50ec42b9b32')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ee473694e50ec42b9b32') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ef7bcf7b4e0eb53edfe3') }, shard: "shmydomain1", lastmod: Timestamp 30702000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:32:01.613+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30702|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ee473694e50ec42b9b32') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ef7bcf7b4e0eb53edfe3') }) shmydomain1 -> shmydomain2 2016-03-13T09:32:02.322+0000 I NETWORK [conn4084128] end connection 10.0.179.209:42428 (1506 connections now open) 2016-03-13T09:32:02.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42439 #4084145 (1507 connections now open) 2016-03-13T09:32:02.323+0000 I NETWORK [conn4084129] end connection 10.0.179.209:42429 (1506 connections now open) 2016-03-13T09:32:02.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42440 #4084146 (1507 connections now open) 2016-03-13T09:32:02.963+0000 I NETWORK [conn4084130] end connection 10.0.26.69:44426 (1506 connections now open) 2016-03-13T09:32:02.963+0000 I NETWORK [conn4084131] end connection 10.0.26.69:44425 (1506 connections now open) 2016-03-13T09:32:02.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44440 #4084147 (1506 connections now open) 2016-03-13T09:32:02.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44441 #4084148 (1507 connections now open) 2016-03-13T09:32:05.664+0000 I NETWORK [conn4084132] end connection 10.0.27.237:37777 (1506 connections now open) 2016-03-13T09:32:05.664+0000 I NETWORK [conn4084133] end connection 10.0.27.237:37778 (1506 connections now open) 2016-03-13T09:32:05.664+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37801 #4084149 (1506 connections now open) 2016-03-13T09:32:05.664+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37802 #4084150 (1507 connections now open) 2016-03-13T09:32:07.901+0000 I NETWORK [conn4084135] end connection 10.0.7.113:33075 (1506 connections now open) 2016-03-13T09:32:07.901+0000 I NETWORK [conn4084134] end connection 10.0.7.113:33074 (1506 connections now open) 2016-03-13T09:32:07.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33085 #4084151 (1506 connections now open) 2016-03-13T09:32:07.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33086 #4084152 (1507 connections now open) 2016-03-13T09:32:08.864+0000 I NETWORK [conn4084142] end connection 10.0.216.90:59333 (1506 connections now open) 2016-03-13T09:32:08.864+0000 I NETWORK [conn4084141] end connection 10.0.216.90:59334 (1506 connections now open) 2016-03-13T09:32:08.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59344 #4084153 (1506 connections now open) 2016-03-13T09:32:08.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59343 #4084154 (1507 connections now open) 2016-03-13T09:32:09.936+0000 I NETWORK [conn4084143] end connection 10.0.20.53:52740 (1506 connections now open) 2016-03-13T09:32:09.936+0000 I NETWORK [conn4084144] end connection 10.0.20.53:52741 (1506 connections now open) 2016-03-13T09:32:09.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52754 #4084155 (1506 connections now open) 2016-03-13T09:32:09.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52753 #4084156 (1507 connections now open) 2016-03-13T09:32:12.335+0000 I NETWORK [conn4084146] end connection 10.0.179.209:42440 (1506 connections now open) 2016-03-13T09:32:12.335+0000 I NETWORK [conn4084145] end connection 10.0.179.209:42439 (1506 connections now open) 2016-03-13T09:32:12.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42450 #4084157 (1506 connections now open) 2016-03-13T09:32:12.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42449 #4084158 (1507 connections now open) 2016-03-13T09:32:12.977+0000 I NETWORK [conn4084147] end connection 10.0.26.69:44440 (1506 connections now open) 2016-03-13T09:32:12.977+0000 I NETWORK [conn4084148] end connection 10.0.26.69:44441 (1506 connections now open) 2016-03-13T09:32:12.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44460 #4084159 (1507 connections now open) 2016-03-13T09:32:12.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44461 #4084160 (1507 connections now open) 2016-03-13T09:32:14.351+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17403 version: 30703|1||563ba74d869758a4542b9075 based on: 30702|1||563ba74d869758a4542b9075 2016-03-13T09:32:14.643+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:32:14.643+0000-56e5339e18fa188537623dc4", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861534643), what: "balancer.round", ns: "", details: { executionTimeMillis: 16530, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:32:14.890+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:32:15.677+0000 I NETWORK [conn4084149] end connection 10.0.27.237:37801 (1506 connections now open) 2016-03-13T09:32:15.677+0000 I NETWORK [conn4084150] end connection 10.0.27.237:37802 (1506 connections now open) 2016-03-13T09:32:15.677+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37811 #4084161 (1506 connections now open) 2016-03-13T09:32:15.678+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37812 #4084162 (1507 connections now open) 2016-03-13T09:32:16.117+0000 I NETWORK [conn4082113] end connection 10.0.20.53:51230 (1506 connections now open) 2016-03-13T09:32:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52771 #4084163 (1507 connections now open) 2016-03-13T09:32:17.914+0000 I NETWORK [conn4084152] end connection 10.0.7.113:33086 (1506 connections now open) 2016-03-13T09:32:17.914+0000 I NETWORK [conn4084151] end connection 10.0.7.113:33085 (1506 connections now open) 2016-03-13T09:32:17.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33102 #4084164 (1506 connections now open) 2016-03-13T09:32:17.915+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33103 #4084165 (1507 connections now open) 2016-03-13T09:32:18.877+0000 I NETWORK [conn4084153] end connection 10.0.216.90:59344 (1506 connections now open) 2016-03-13T09:32:18.877+0000 I NETWORK [conn4084154] end connection 10.0.216.90:59343 (1506 connections now open) 2016-03-13T09:32:18.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59357 #4084166 (1506 connections now open) 2016-03-13T09:32:18.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59358 #4084167 (1507 connections now open) 2016-03-13T09:32:19.949+0000 I NETWORK [conn4084155] end connection 10.0.20.53:52754 (1506 connections now open) 2016-03-13T09:32:19.949+0000 I NETWORK [conn4084156] end connection 10.0.20.53:52753 (1506 connections now open) 2016-03-13T09:32:19.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52774 #4084168 (1506 connections now open) 2016-03-13T09:32:19.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52775 #4084169 (1507 connections now open) 2016-03-13T09:32:21.739+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:32:21.468+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:32:21.739+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:32:21.876+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5339f18fa188537623dc5 2016-03-13T09:32:22.356+0000 I NETWORK [conn4084158] end connection 10.0.179.209:42449 (1506 connections now open) 2016-03-13T09:32:22.356+0000 I NETWORK [conn4084157] end connection 10.0.179.209:42450 (1506 connections now open) 2016-03-13T09:32:22.357+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42472 #4084170 (1506 connections now open) 2016-03-13T09:32:22.357+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42473 #4084171 (1507 connections now open) 2016-03-13T09:32:22.990+0000 I NETWORK [conn4084160] end connection 10.0.26.69:44461 (1506 connections now open) 2016-03-13T09:32:22.990+0000 I NETWORK [conn4084159] end connection 10.0.26.69:44460 (1506 connections now open) 2016-03-13T09:32:22.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44468 #4084172 (1506 connections now open) 2016-03-13T09:32:22.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44469 #4084173 (1507 connections now open) 2016-03-13T09:32:25.689+0000 I NETWORK [conn4084161] end connection 10.0.27.237:37811 (1506 connections now open) 2016-03-13T09:32:25.690+0000 I NETWORK [conn4084162] end connection 10.0.27.237:37812 (1505 connections now open) 2016-03-13T09:32:25.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37823 #4084174 (1506 connections now open) 2016-03-13T09:32:25.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37824 #4084175 (1507 connections now open) 2016-03-13T09:32:27.927+0000 I NETWORK [conn4084165] end connection 10.0.7.113:33103 (1506 connections now open) 2016-03-13T09:32:27.927+0000 I NETWORK [conn4084164] end connection 10.0.7.113:33102 (1506 connections now open) 2016-03-13T09:32:27.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33120 #4084176 (1506 connections now open) 2016-03-13T09:32:27.928+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33121 #4084177 (1507 connections now open) 2016-03-13T09:32:27.942+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17404 version: 30704|1||563ba74d869758a4542b9075 based on: 30703|1||563ba74d869758a4542b9075 2016-03-13T09:32:28.113+0000 I NETWORK [conn4081928] end connection 10.0.179.209:40873 (1506 connections now open) 2016-03-13T09:32:28.113+0000 I NETWORK [conn4082562] end connection 10.0.216.90:58052 (1505 connections now open) 2016-03-13T09:32:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42477 #4084178 (1506 connections now open) 2016-03-13T09:32:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59375 #4084179 (1507 connections now open) 2016-03-13T09:32:28.890+0000 I NETWORK [conn4084166] end connection 10.0.216.90:59357 (1506 connections now open) 2016-03-13T09:32:28.890+0000 I NETWORK [conn4084167] end connection 10.0.216.90:59358 (1506 connections now open) 2016-03-13T09:32:28.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59380 #4084180 (1506 connections now open) 2016-03-13T09:32:28.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59379 #4084181 (1507 connections now open) 2016-03-13T09:32:29.962+0000 I NETWORK [conn4084168] end connection 10.0.20.53:52774 (1506 connections now open) 2016-03-13T09:32:29.962+0000 I NETWORK [conn4084169] end connection 10.0.20.53:52775 (1506 connections now open) 2016-03-13T09:32:29.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52787 #4084182 (1506 connections now open) 2016-03-13T09:32:29.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52786 #4084183 (1507 connections now open) 2016-03-13T09:32:32.369+0000 I NETWORK [conn4084171] end connection 10.0.179.209:42473 (1506 connections now open) 2016-03-13T09:32:32.369+0000 I NETWORK [conn4084170] end connection 10.0.179.209:42472 (1506 connections now open) 2016-03-13T09:32:32.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42482 #4084184 (1506 connections now open) 2016-03-13T09:32:32.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42481 #4084185 (1507 connections now open) 2016-03-13T09:32:33.003+0000 I NETWORK [conn4084173] end connection 10.0.26.69:44469 (1506 connections now open) 2016-03-13T09:32:33.004+0000 I NETWORK [conn4084172] end connection 10.0.26.69:44468 (1505 connections now open) 2016-03-13T09:32:33.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44483 #4084186 (1506 connections now open) 2016-03-13T09:32:33.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44482 #4084187 (1507 connections now open) 2016-03-13T09:32:35.702+0000 I NETWORK [conn4084174] end connection 10.0.27.237:37823 (1506 connections now open) 2016-03-13T09:32:35.702+0000 I NETWORK [conn4084175] end connection 10.0.27.237:37824 (1505 connections now open) 2016-03-13T09:32:35.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37844 #4084188 (1506 connections now open) 2016-03-13T09:32:35.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37845 #4084189 (1507 connections now open) 2016-03-13T09:32:37.941+0000 I NETWORK [conn4084176] end connection 10.0.7.113:33120 (1506 connections now open) 2016-03-13T09:32:37.941+0000 I NETWORK [conn4084177] end connection 10.0.7.113:33121 (1506 connections now open) 2016-03-13T09:32:37.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33129 #4084190 (1506 connections now open) 2016-03-13T09:32:37.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33130 #4084191 (1507 connections now open) 2016-03-13T09:32:38.903+0000 I NETWORK [conn4084180] end connection 10.0.216.90:59380 (1506 connections now open) 2016-03-13T09:32:38.903+0000 I NETWORK [conn4084181] end connection 10.0.216.90:59379 (1506 connections now open) 2016-03-13T09:32:38.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59386 #4084192 (1506 connections now open) 2016-03-13T09:32:38.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59385 #4084193 (1507 connections now open) 2016-03-13T09:32:39.983+0000 I NETWORK [conn4084183] end connection 10.0.20.53:52786 (1506 connections now open) 2016-03-13T09:32:39.983+0000 I NETWORK [conn4084182] end connection 10.0.20.53:52787 (1506 connections now open) 2016-03-13T09:32:39.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52798 #4084194 (1506 connections now open) 2016-03-13T09:32:39.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52799 #4084195 (1507 connections now open) 2016-03-13T09:32:42.383+0000 I NETWORK [conn4084184] end connection 10.0.179.209:42482 (1506 connections now open) 2016-03-13T09:32:42.383+0000 I NETWORK [conn4084185] end connection 10.0.179.209:42481 (1506 connections now open) 2016-03-13T09:32:42.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42492 #4084196 (1506 connections now open) 2016-03-13T09:32:42.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42491 #4084197 (1507 connections now open) 2016-03-13T09:32:43.025+0000 I NETWORK [conn4084187] end connection 10.0.26.69:44482 (1506 connections now open) 2016-03-13T09:32:43.025+0000 I NETWORK [conn4084186] end connection 10.0.26.69:44483 (1506 connections now open) 2016-03-13T09:32:43.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44498 #4084198 (1507 connections now open) 2016-03-13T09:32:43.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44499 #4084199 (1507 connections now open) 2016-03-13T09:32:44.195+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17405 version: 30705|1||563ba74d869758a4542b9075 based on: 30704|1||563ba74d869758a4542b9075 2016-03-13T09:32:45.723+0000 I NETWORK [conn4084188] end connection 10.0.27.237:37844 (1506 connections now open) 2016-03-13T09:32:45.723+0000 I NETWORK [conn4084189] end connection 10.0.27.237:37845 (1506 connections now open) 2016-03-13T09:32:45.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37851 #4084200 (1506 connections now open) 2016-03-13T09:32:45.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37852 #4084201 (1507 connections now open) 2016-03-13T09:32:46.117+0000 I NETWORK [conn4081584] end connection 10.0.20.53:50970 (1506 connections now open) 2016-03-13T09:32:46.117+0000 I NETWORK [conn4082491] end connection 10.0.216.90:58022 (1506 connections now open) 2016-03-13T09:32:46.117+0000 I NETWORK [conn4081873] end connection 10.0.26.69:42880 (1504 connections now open) 2016-03-13T09:32:46.117+0000 I NETWORK [conn4082122] end connection 10.0.7.113:59861 (1503 connections now open) 2016-03-13T09:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52816 #4084202 (1504 connections now open) 2016-03-13T09:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59396 #4084203 (1505 connections now open) 2016-03-13T09:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44502 #4084204 (1506 connections now open) 2016-03-13T09:32:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33142 #4084205 (1507 connections now open) 2016-03-13T09:32:47.963+0000 I NETWORK [conn4084190] end connection 10.0.7.113:33129 (1506 connections now open) 2016-03-13T09:32:47.963+0000 I NETWORK [conn4084191] end connection 10.0.7.113:33130 (1506 connections now open) 2016-03-13T09:32:47.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33144 #4084206 (1506 connections now open) 2016-03-13T09:32:47.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33145 #4084207 (1507 connections now open) 2016-03-13T09:32:48.924+0000 I NETWORK [conn4084193] end connection 10.0.216.90:59385 (1506 connections now open) 2016-03-13T09:32:48.924+0000 I NETWORK [conn4084192] end connection 10.0.216.90:59386 (1506 connections now open) 2016-03-13T09:32:48.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59398 #4084208 (1506 connections now open) 2016-03-13T09:32:48.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59399 #4084209 (1507 connections now open) 2016-03-13T09:32:49.996+0000 I NETWORK [conn4084195] end connection 10.0.20.53:52799 (1506 connections now open) 2016-03-13T09:32:49.996+0000 I NETWORK [conn4084194] end connection 10.0.20.53:52798 (1506 connections now open) 2016-03-13T09:32:49.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52820 #4084210 (1506 connections now open) 2016-03-13T09:32:49.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52819 #4084211 (1507 connections now open) 2016-03-13T09:32:52.033+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:32:51.876+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:32:52.033+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:32:52.101+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533aa18fa188537623dc6 2016-03-13T09:32:52.170+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533b418fa188537623dc7 2016-03-13T09:32:52.238+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533be18fa188537623dc8 2016-03-13T09:32:52.396+0000 I NETWORK [conn4084196] end connection 10.0.179.209:42492 (1506 connections now open) 2016-03-13T09:32:52.396+0000 I NETWORK [conn4084197] end connection 10.0.179.209:42491 (1505 connections now open) 2016-03-13T09:32:52.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42501 #4084212 (1506 connections now open) 2016-03-13T09:32:52.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42502 #4084213 (1507 connections now open) 2016-03-13T09:32:53.039+0000 I NETWORK [conn4084198] end connection 10.0.26.69:44498 (1506 connections now open) 2016-03-13T09:32:53.039+0000 I NETWORK [conn4084199] end connection 10.0.26.69:44499 (1506 connections now open) 2016-03-13T09:32:53.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44506 #4084214 (1506 connections now open) 2016-03-13T09:32:53.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44507 #4084215 (1507 connections now open) 2016-03-13T09:32:55.736+0000 I NETWORK [conn4084201] end connection 10.0.27.237:37852 (1506 connections now open) 2016-03-13T09:32:55.736+0000 I NETWORK [conn4084200] end connection 10.0.27.237:37851 (1506 connections now open) 2016-03-13T09:32:55.737+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37866 #4084216 (1506 connections now open) 2016-03-13T09:32:55.737+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37865 #4084217 (1507 connections now open) 2016-03-13T09:32:57.977+0000 I NETWORK [conn4084207] end connection 10.0.7.113:33145 (1506 connections now open) 2016-03-13T09:32:57.977+0000 I NETWORK [conn4084206] end connection 10.0.7.113:33144 (1506 connections now open) 2016-03-13T09:32:57.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33160 #4084218 (1506 connections now open) 2016-03-13T09:32:57.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33161 #4084219 (1507 connections now open) 2016-03-13T09:32:58.114+0000 I NETWORK [conn4081785] end connection 10.0.7.113:59731 (1506 connections now open) 2016-03-13T09:32:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33164 #4084220 (1507 connections now open) 2016-03-13T09:32:58.587+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17406 version: 30706|1||563ba74d869758a4542b9075 based on: 30705|1||563ba74d869758a4542b9075 2016-03-13T09:32:58.937+0000 I NETWORK [conn4084209] end connection 10.0.216.90:59399 (1506 connections now open) 2016-03-13T09:32:58.937+0000 I NETWORK [conn4084208] end connection 10.0.216.90:59398 (1506 connections now open) 2016-03-13T09:32:58.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59416 #4084221 (1506 connections now open) 2016-03-13T09:32:58.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59417 #4084222 (1507 connections now open) 2016-03-13T09:33:00.009+0000 I NETWORK [conn4084211] end connection 10.0.20.53:52819 (1506 connections now open) 2016-03-13T09:33:00.009+0000 I NETWORK [conn4084210] end connection 10.0.20.53:52820 (1506 connections now open) 2016-03-13T09:33:00.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52828 #4084223 (1506 connections now open) 2016-03-13T09:33:00.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52829 #4084224 (1507 connections now open) 2016-03-13T09:33:02.409+0000 I NETWORK [conn4084213] end connection 10.0.179.209:42502 (1506 connections now open) 2016-03-13T09:33:02.409+0000 I NETWORK [conn4084212] end connection 10.0.179.209:42501 (1506 connections now open) 2016-03-13T09:33:02.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42522 #4084225 (1506 connections now open) 2016-03-13T09:33:02.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42523 #4084226 (1507 connections now open) 2016-03-13T09:33:03.052+0000 I NETWORK [conn4084215] end connection 10.0.26.69:44507 (1506 connections now open) 2016-03-13T09:33:03.052+0000 I NETWORK [conn4084214] end connection 10.0.26.69:44506 (1506 connections now open) 2016-03-13T09:33:03.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44523 #4084227 (1506 connections now open) 2016-03-13T09:33:03.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44524 #4084228 (1507 connections now open) 2016-03-13T09:33:05.749+0000 I NETWORK [conn4084217] end connection 10.0.27.237:37865 (1506 connections now open) 2016-03-13T09:33:05.749+0000 I NETWORK [conn4084216] end connection 10.0.27.237:37866 (1506 connections now open) 2016-03-13T09:33:05.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37887 #4084229 (1506 connections now open) 2016-03-13T09:33:05.750+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37888 #4084230 (1507 connections now open) 2016-03-13T09:33:07.990+0000 I NETWORK [conn4084219] end connection 10.0.7.113:33161 (1506 connections now open) 2016-03-13T09:33:07.990+0000 I NETWORK [conn4084218] end connection 10.0.7.113:33160 (1506 connections now open) 2016-03-13T09:33:07.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33171 #4084231 (1507 connections now open) 2016-03-13T09:33:07.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33172 #4084232 (1507 connections now open) 2016-03-13T09:33:08.950+0000 I NETWORK [conn4084222] end connection 10.0.216.90:59417 (1506 connections now open) 2016-03-13T09:33:08.950+0000 I NETWORK [conn4084221] end connection 10.0.216.90:59416 (1506 connections now open) 2016-03-13T09:33:08.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59425 #4084233 (1506 connections now open) 2016-03-13T09:33:08.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59426 #4084234 (1507 connections now open) 2016-03-13T09:33:10.022+0000 I NETWORK [conn4084223] end connection 10.0.20.53:52828 (1506 connections now open) 2016-03-13T09:33:10.022+0000 I NETWORK [conn4084224] end connection 10.0.20.53:52829 (1506 connections now open) 2016-03-13T09:33:10.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52843 #4084235 (1506 connections now open) 2016-03-13T09:33:10.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52842 #4084236 (1507 connections now open) 2016-03-13T09:33:12.422+0000 I NETWORK [conn4084226] end connection 10.0.179.209:42523 (1506 connections now open) 2016-03-13T09:33:12.422+0000 I NETWORK [conn4084225] end connection 10.0.179.209:42522 (1506 connections now open) 2016-03-13T09:33:12.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42528 #4084237 (1506 connections now open) 2016-03-13T09:33:12.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42529 #4084238 (1507 connections now open) 2016-03-13T09:33:12.489+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17407 version: 30707|1||563ba74d869758a4542b9075 based on: 30706|1||563ba74d869758a4542b9075 2016-03-13T09:33:13.066+0000 I NETWORK [conn4084227] end connection 10.0.26.69:44523 (1506 connections now open) 2016-03-13T09:33:13.066+0000 I NETWORK [conn4084228] end connection 10.0.26.69:44524 (1506 connections now open) 2016-03-13T09:33:13.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44544 #4084239 (1507 connections now open) 2016-03-13T09:33:13.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44543 #4084240 (1507 connections now open) 2016-03-13T09:33:15.763+0000 I NETWORK [conn4084230] end connection 10.0.27.237:37888 (1506 connections now open) 2016-03-13T09:33:15.763+0000 I NETWORK [conn4084229] end connection 10.0.27.237:37887 (1506 connections now open) 2016-03-13T09:33:15.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37898 #4084241 (1506 connections now open) 2016-03-13T09:33:15.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37899 #4084242 (1507 connections now open) 2016-03-13T09:33:16.117+0000 I NETWORK [conn4083028] end connection 10.0.26.69:43363 (1506 connections now open) 2016-03-13T09:33:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44548 #4084243 (1507 connections now open) 2016-03-13T09:33:18.003+0000 I NETWORK [conn4084232] end connection 10.0.7.113:33172 (1506 connections now open) 2016-03-13T09:33:18.004+0000 I NETWORK [conn4084231] end connection 10.0.7.113:33171 (1505 connections now open) 2016-03-13T09:33:18.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33188 #4084244 (1506 connections now open) 2016-03-13T09:33:18.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33189 #4084245 (1507 connections now open) 2016-03-13T09:33:18.963+0000 I NETWORK [conn4084233] end connection 10.0.216.90:59425 (1506 connections now open) 2016-03-13T09:33:18.963+0000 I NETWORK [conn4084234] end connection 10.0.216.90:59426 (1506 connections now open) 2016-03-13T09:33:18.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59439 #4084246 (1506 connections now open) 2016-03-13T09:33:18.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59440 #4084247 (1507 connections now open) 2016-03-13T09:33:20.035+0000 I NETWORK [conn4084236] end connection 10.0.20.53:52842 (1506 connections now open) 2016-03-13T09:33:20.035+0000 I NETWORK [conn4084235] end connection 10.0.20.53:52843 (1506 connections now open) 2016-03-13T09:33:20.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52863 #4084248 (1506 connections now open) 2016-03-13T09:33:20.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52864 #4084249 (1507 connections now open) 2016-03-13T09:33:22.443+0000 I NETWORK [conn4084237] end connection 10.0.179.209:42528 (1506 connections now open) 2016-03-13T09:33:22.443+0000 I NETWORK [conn4084238] end connection 10.0.179.209:42529 (1506 connections now open) 2016-03-13T09:33:22.444+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42546 #4084250 (1506 connections now open) 2016-03-13T09:33:22.444+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42545 #4084251 (1507 connections now open) 2016-03-13T09:33:22.504+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:33:22.238+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:33:22.504+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:33:22.640+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533c818fa188537623dc9 2016-03-13T09:33:22.777+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533d218fa188537623dca 2016-03-13T09:33:22.913+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533dd18fa188537623dcb 2016-03-13T09:33:23.079+0000 I NETWORK [conn4084240] end connection 10.0.26.69:44543 (1506 connections now open) 2016-03-13T09:33:23.079+0000 I NETWORK [conn4084239] end connection 10.0.26.69:44544 (1506 connections now open) 2016-03-13T09:33:23.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44551 #4084252 (1506 connections now open) 2016-03-13T09:33:23.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44552 #4084253 (1507 connections now open) 2016-03-13T09:33:25.775+0000 I NETWORK [conn4084241] end connection 10.0.27.237:37898 (1506 connections now open) 2016-03-13T09:33:25.775+0000 I NETWORK [conn4084242] end connection 10.0.27.237:37899 (1505 connections now open) 2016-03-13T09:33:25.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37910 #4084254 (1506 connections now open) 2016-03-13T09:33:25.776+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37911 #4084255 (1507 connections now open) 2016-03-13T09:33:28.016+0000 I NETWORK [conn4084245] end connection 10.0.7.113:33189 (1506 connections now open) 2016-03-13T09:33:28.016+0000 I NETWORK [conn4084244] end connection 10.0.7.113:33188 (1506 connections now open) 2016-03-13T09:33:28.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33205 #4084256 (1506 connections now open) 2016-03-13T09:33:28.017+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33206 #4084257 (1507 connections now open) 2016-03-13T09:33:28.113+0000 I NETWORK [conn4081197] end connection 10.0.179.209:40701 (1506 connections now open) 2016-03-13T09:33:28.113+0000 I NETWORK [conn4081625] end connection 10.0.26.69:42740 (1505 connections now open) 2016-03-13T09:33:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42560 #4084258 (1506 connections now open) 2016-03-13T09:33:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44562 #4084259 (1507 connections now open) 2016-03-13T09:33:28.976+0000 I NETWORK [conn4084246] end connection 10.0.216.90:59439 (1506 connections now open) 2016-03-13T09:33:28.976+0000 I NETWORK [conn4084247] end connection 10.0.216.90:59440 (1506 connections now open) 2016-03-13T09:33:28.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59462 #4084260 (1506 connections now open) 2016-03-13T09:33:28.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59463 #4084261 (1507 connections now open) 2016-03-13T09:33:30.048+0000 I NETWORK [conn4084248] end connection 10.0.20.53:52863 (1506 connections now open) 2016-03-13T09:33:30.048+0000 I NETWORK [conn4084249] end connection 10.0.20.53:52864 (1505 connections now open) 2016-03-13T09:33:30.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52876 #4084262 (1506 connections now open) 2016-03-13T09:33:30.056+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52875 #4084263 (1507 connections now open) 2016-03-13T09:33:30.750+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17408 version: 30708|1||563ba74d869758a4542b9075 based on: 30707|1||563ba74d869758a4542b9075 2016-03-13T09:33:32.456+0000 I NETWORK [conn4084251] end connection 10.0.179.209:42545 (1506 connections now open) 2016-03-13T09:33:32.456+0000 I NETWORK [conn4084250] end connection 10.0.179.209:42546 (1506 connections now open) 2016-03-13T09:33:32.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42565 #4084264 (1506 connections now open) 2016-03-13T09:33:32.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42564 #4084265 (1507 connections now open) 2016-03-13T09:33:33.093+0000 I NETWORK [conn4084252] end connection 10.0.26.69:44551 (1506 connections now open) 2016-03-13T09:33:33.093+0000 I NETWORK [conn4084253] end connection 10.0.26.69:44552 (1506 connections now open) 2016-03-13T09:33:33.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44566 #4084266 (1506 connections now open) 2016-03-13T09:33:33.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44565 #4084267 (1507 connections now open) 2016-03-13T09:33:35.788+0000 I NETWORK [conn4084254] end connection 10.0.27.237:37910 (1506 connections now open) 2016-03-13T09:33:35.788+0000 I NETWORK [conn4084255] end connection 10.0.27.237:37911 (1506 connections now open) 2016-03-13T09:33:35.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37930 #4084268 (1506 connections now open) 2016-03-13T09:33:35.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37931 #4084269 (1507 connections now open) 2016-03-13T09:33:38.029+0000 I NETWORK [conn4084257] end connection 10.0.7.113:33206 (1506 connections now open) 2016-03-13T09:33:38.030+0000 I NETWORK [conn4084256] end connection 10.0.7.113:33205 (1505 connections now open) 2016-03-13T09:33:38.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33214 #4084270 (1506 connections now open) 2016-03-13T09:33:38.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33215 #4084271 (1507 connections now open) 2016-03-13T09:33:38.989+0000 I NETWORK [conn4084260] end connection 10.0.216.90:59462 (1506 connections now open) 2016-03-13T09:33:38.989+0000 I NETWORK [conn4084261] end connection 10.0.216.90:59463 (1506 connections now open) 2016-03-13T09:33:38.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59469 #4084272 (1506 connections now open) 2016-03-13T09:33:38.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59470 #4084273 (1507 connections now open) 2016-03-13T09:33:40.069+0000 I NETWORK [conn4084262] end connection 10.0.20.53:52876 (1506 connections now open) 2016-03-13T09:33:40.069+0000 I NETWORK [conn4084263] end connection 10.0.20.53:52875 (1506 connections now open) 2016-03-13T09:33:40.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52889 #4084274 (1506 connections now open) 2016-03-13T09:33:40.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52888 #4084275 (1507 connections now open) 2016-03-13T09:33:42.469+0000 I NETWORK [conn4084265] end connection 10.0.179.209:42564 (1506 connections now open) 2016-03-13T09:33:42.469+0000 I NETWORK [conn4084264] end connection 10.0.179.209:42565 (1505 connections now open) 2016-03-13T09:33:42.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42572 #4084276 (1506 connections now open) 2016-03-13T09:33:42.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42571 #4084277 (1507 connections now open) 2016-03-13T09:33:43.114+0000 I NETWORK [conn4084266] end connection 10.0.26.69:44566 (1506 connections now open) 2016-03-13T09:33:43.114+0000 I NETWORK [conn4084267] end connection 10.0.26.69:44565 (1506 connections now open) 2016-03-13T09:33:43.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44581 #4084278 (1506 connections now open) 2016-03-13T09:33:43.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44582 #4084279 (1507 connections now open) 2016-03-13T09:33:44.974+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 695ms sequenceNumber: 17409 version: 30709|1||563ba74d869758a4542b9075 based on: 30708|1||563ba74d869758a4542b9075 2016-03-13T09:33:45.810+0000 I NETWORK [conn4084269] end connection 10.0.27.237:37931 (1506 connections now open) 2016-03-13T09:33:45.810+0000 I NETWORK [conn4084268] end connection 10.0.27.237:37930 (1506 connections now open) 2016-03-13T09:33:45.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37939 #4084280 (1506 connections now open) 2016-03-13T09:33:45.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37940 #4084281 (1507 connections now open) 2016-03-13T09:33:46.117+0000 I NETWORK [conn4081280] end connection 10.0.216.90:57583 (1506 connections now open) 2016-03-13T09:33:46.117+0000 I NETWORK [conn4081448] end connection 10.0.26.69:42651 (1505 connections now open) 2016-03-13T09:33:46.117+0000 I NETWORK [conn4080987] end connection 10.0.27.237:35780 (1504 connections now open) 2016-03-13T09:33:46.117+0000 I NETWORK [conn4081271] end connection 10.0.7.113:59631 (1503 connections now open) 2016-03-13T09:33:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59481 #4084282 (1504 connections now open) 2016-03-13T09:33:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37942 #4084283 (1505 connections now open) 2016-03-13T09:33:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44585 #4084284 (1506 connections now open) 2016-03-13T09:33:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33228 #4084285 (1507 connections now open) 2016-03-13T09:33:48.051+0000 I NETWORK [conn4084271] end connection 10.0.7.113:33215 (1506 connections now open) 2016-03-13T09:33:48.051+0000 I NETWORK [conn4084270] end connection 10.0.7.113:33214 (1506 connections now open) 2016-03-13T09:33:48.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33229 #4084286 (1506 connections now open) 2016-03-13T09:33:48.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33230 #4084287 (1507 connections now open) 2016-03-13T09:33:49.011+0000 I NETWORK [conn4084272] end connection 10.0.216.90:59469 (1506 connections now open) 2016-03-13T09:33:49.011+0000 I NETWORK [conn4084273] end connection 10.0.216.90:59470 (1506 connections now open) 2016-03-13T09:33:49.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59483 #4084288 (1506 connections now open) 2016-03-13T09:33:49.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59484 #4084289 (1507 connections now open) 2016-03-13T09:33:50.082+0000 I NETWORK [conn4084275] end connection 10.0.20.53:52888 (1506 connections now open) 2016-03-13T09:33:50.082+0000 I NETWORK [conn4084274] end connection 10.0.20.53:52889 (1506 connections now open) 2016-03-13T09:33:50.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52910 #4084290 (1506 connections now open) 2016-03-13T09:33:50.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52909 #4084291 (1507 connections now open) 2016-03-13T09:33:52.482+0000 I NETWORK [conn4084277] end connection 10.0.179.209:42571 (1506 connections now open) 2016-03-13T09:33:52.482+0000 I NETWORK [conn4084276] end connection 10.0.179.209:42572 (1506 connections now open) 2016-03-13T09:33:52.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42584 #4084292 (1506 connections now open) 2016-03-13T09:33:52.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42585 #4084293 (1507 connections now open) 2016-03-13T09:33:53.127+0000 I NETWORK [conn4084279] end connection 10.0.26.69:44582 (1506 connections now open) 2016-03-13T09:33:53.127+0000 I NETWORK [conn4084278] end connection 10.0.26.69:44581 (1506 connections now open) 2016-03-13T09:33:53.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44591 #4084294 (1506 connections now open) 2016-03-13T09:33:53.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44590 #4084295 (1507 connections now open) 2016-03-13T09:33:53.182+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:33:52.913+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:33:53.182+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:33:53.319+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533e718fa188537623dcc 2016-03-13T09:33:53.458+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533f118fa188537623dcd 2016-03-13T09:33:53.612+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e533fb18fa188537623dce 2016-03-13T09:33:55.823+0000 I NETWORK [conn4084281] end connection 10.0.27.237:37940 (1506 connections now open) 2016-03-13T09:33:55.823+0000 I NETWORK [conn4084280] end connection 10.0.27.237:37939 (1506 connections now open) 2016-03-13T09:33:55.824+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37951 #4084296 (1506 connections now open) 2016-03-13T09:33:55.824+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37952 #4084297 (1507 connections now open) 2016-03-13T09:33:58.065+0000 I NETWORK [conn4084287] end connection 10.0.7.113:33230 (1506 connections now open) 2016-03-13T09:33:58.065+0000 I NETWORK [conn4084286] end connection 10.0.7.113:33229 (1506 connections now open) 2016-03-13T09:33:58.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33247 #4084298 (1506 connections now open) 2016-03-13T09:33:58.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33246 #4084299 (1507 connections now open) 2016-03-13T09:33:58.113+0000 I NETWORK [conn4082550] end connection 10.0.216.90:58050 (1506 connections now open) 2016-03-13T09:33:58.113+0000 I NETWORK [conn4080917] end connection 10.0.7.113:59510 (1505 connections now open) 2016-03-13T09:33:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59501 #4084300 (1506 connections now open) 2016-03-13T09:33:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33250 #4084301 (1507 connections now open) 2016-03-13T09:33:59.024+0000 I NETWORK [conn4084289] end connection 10.0.216.90:59484 (1506 connections now open) 2016-03-13T09:33:59.024+0000 I NETWORK [conn4084288] end connection 10.0.216.90:59483 (1506 connections now open) 2016-03-13T09:33:59.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59503 #4084302 (1506 connections now open) 2016-03-13T09:33:59.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59502 #4084303 (1507 connections now open) 2016-03-13T09:33:59.785+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17410 version: 30710|1||563ba74d869758a4542b9075 based on: 30709|1||563ba74d869758a4542b9075 2016-03-13T09:34:00.095+0000 I NETWORK [conn4084291] end connection 10.0.20.53:52909 (1506 connections now open) 2016-03-13T09:34:00.095+0000 I NETWORK [conn4084290] end connection 10.0.20.53:52910 (1506 connections now open) 2016-03-13T09:34:00.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52920 #4084304 (1506 connections now open) 2016-03-13T09:34:00.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52919 #4084305 (1507 connections now open) 2016-03-13T09:34:02.496+0000 I NETWORK [conn4084292] end connection 10.0.179.209:42584 (1506 connections now open) 2016-03-13T09:34:02.496+0000 I NETWORK [conn4084293] end connection 10.0.179.209:42585 (1506 connections now open) 2016-03-13T09:34:02.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42606 #4084306 (1506 connections now open) 2016-03-13T09:34:02.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42607 #4084307 (1507 connections now open) 2016-03-13T09:34:03.140+0000 I NETWORK [conn4084294] end connection 10.0.26.69:44591 (1506 connections now open) 2016-03-13T09:34:03.140+0000 I NETWORK [conn4084295] end connection 10.0.26.69:44590 (1506 connections now open) 2016-03-13T09:34:03.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44608 #4084308 (1506 connections now open) 2016-03-13T09:34:03.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44607 #4084309 (1507 connections now open) 2016-03-13T09:34:05.836+0000 I NETWORK [conn4084297] end connection 10.0.27.237:37952 (1506 connections now open) 2016-03-13T09:34:05.836+0000 I NETWORK [conn4084296] end connection 10.0.27.237:37951 (1506 connections now open) 2016-03-13T09:34:05.837+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37973 #4084310 (1506 connections now open) 2016-03-13T09:34:05.837+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37972 #4084311 (1507 connections now open) 2016-03-13T09:34:08.078+0000 I NETWORK [conn4084299] end connection 10.0.7.113:33246 (1506 connections now open) 2016-03-13T09:34:08.078+0000 I NETWORK [conn4084298] end connection 10.0.7.113:33247 (1506 connections now open) 2016-03-13T09:34:08.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33259 #4084312 (1506 connections now open) 2016-03-13T09:34:08.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33258 #4084313 (1507 connections now open) 2016-03-13T09:34:09.037+0000 I NETWORK [conn4084302] end connection 10.0.216.90:59503 (1506 connections now open) 2016-03-13T09:34:09.037+0000 I NETWORK [conn4084303] end connection 10.0.216.90:59502 (1506 connections now open) 2016-03-13T09:34:09.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59511 #4084314 (1506 connections now open) 2016-03-13T09:34:09.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59512 #4084315 (1507 connections now open) 2016-03-13T09:34:10.108+0000 I NETWORK [conn4084305] end connection 10.0.20.53:52919 (1506 connections now open) 2016-03-13T09:34:10.108+0000 I NETWORK [conn4084304] end connection 10.0.20.53:52920 (1506 connections now open) 2016-03-13T09:34:10.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52933 #4084316 (1506 connections now open) 2016-03-13T09:34:10.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52934 #4084317 (1507 connections now open) 2016-03-13T09:34:12.509+0000 I NETWORK [conn4084306] end connection 10.0.179.209:42606 (1506 connections now open) 2016-03-13T09:34:12.509+0000 I NETWORK [conn4084307] end connection 10.0.179.209:42607 (1506 connections now open) 2016-03-13T09:34:12.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42614 #4084318 (1506 connections now open) 2016-03-13T09:34:12.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42613 #4084319 (1507 connections now open) 2016-03-13T09:34:13.153+0000 I NETWORK [conn4084308] end connection 10.0.26.69:44608 (1506 connections now open) 2016-03-13T09:34:13.153+0000 I NETWORK [conn4084309] end connection 10.0.26.69:44607 (1506 connections now open) 2016-03-13T09:34:13.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44629 #4084320 (1507 connections now open) 2016-03-13T09:34:13.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44628 #4084321 (1507 connections now open) 2016-03-13T09:34:14.594+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17411 version: 30711|1||563ba74d869758a4542b9075 based on: 30710|1||563ba74d869758a4542b9075 2016-03-13T09:34:15.849+0000 I NETWORK [conn4084311] end connection 10.0.27.237:37972 (1506 connections now open) 2016-03-13T09:34:15.849+0000 I NETWORK [conn4084310] end connection 10.0.27.237:37973 (1506 connections now open) 2016-03-13T09:34:15.850+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37982 #4084322 (1506 connections now open) 2016-03-13T09:34:15.850+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37983 #4084323 (1507 connections now open) 2016-03-13T09:34:16.117+0000 I NETWORK [conn4082429] end connection 10.0.27.237:36404 (1506 connections now open) 2016-03-13T09:34:16.117+0000 I NETWORK [conn4080914] end connection 10.0.7.113:59509 (1505 connections now open) 2016-03-13T09:34:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37987 #4084324 (1506 connections now open) 2016-03-13T09:34:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33273 #4084325 (1507 connections now open) 2016-03-13T09:34:18.091+0000 I NETWORK [conn4084313] end connection 10.0.7.113:33258 (1506 connections now open) 2016-03-13T09:34:18.091+0000 I NETWORK [conn4084312] end connection 10.0.7.113:33259 (1505 connections now open) 2016-03-13T09:34:18.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33275 #4084326 (1506 connections now open) 2016-03-13T09:34:18.091+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33276 #4084327 (1507 connections now open) 2016-03-13T09:34:19.050+0000 I NETWORK [conn4084314] end connection 10.0.216.90:59511 (1506 connections now open) 2016-03-13T09:34:19.050+0000 I NETWORK [conn4084315] end connection 10.0.216.90:59512 (1506 connections now open) 2016-03-13T09:34:19.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59526 #4084328 (1506 connections now open) 2016-03-13T09:34:19.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59525 #4084329 (1507 connections now open) 2016-03-13T09:34:20.121+0000 I NETWORK [conn4084317] end connection 10.0.20.53:52934 (1506 connections now open) 2016-03-13T09:34:20.121+0000 I NETWORK [conn4084316] end connection 10.0.20.53:52933 (1505 connections now open) 2016-03-13T09:34:20.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52952 #4084330 (1506 connections now open) 2016-03-13T09:34:20.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52953 #4084331 (1507 connections now open) 2016-03-13T09:34:22.522+0000 I NETWORK [conn4084318] end connection 10.0.179.209:42614 (1506 connections now open) 2016-03-13T09:34:22.522+0000 I NETWORK [conn4084319] end connection 10.0.179.209:42613 (1506 connections now open) 2016-03-13T09:34:22.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42631 #4084332 (1506 connections now open) 2016-03-13T09:34:22.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42632 #4084333 (1507 connections now open) 2016-03-13T09:34:23.167+0000 I NETWORK [conn4084320] end connection 10.0.26.69:44629 (1506 connections now open) 2016-03-13T09:34:23.167+0000 I NETWORK [conn4084321] end connection 10.0.26.69:44628 (1506 connections now open) 2016-03-13T09:34:23.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44637 #4084334 (1506 connections now open) 2016-03-13T09:34:23.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44638 #4084335 (1507 connections now open) 2016-03-13T09:34:23.870+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:34:23.612+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:34:23.870+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:34:24.005+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5340518fa188537623dcf 2016-03-13T09:34:24.141+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5340f18fa188537623dd0 2016-03-13T09:34:24.278+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5341a18fa188537623dd1 2016-03-13T09:34:25.862+0000 I NETWORK [conn4084322] end connection 10.0.27.237:37982 (1506 connections now open) 2016-03-13T09:34:25.862+0000 I NETWORK [conn4084323] end connection 10.0.27.237:37983 (1506 connections now open) 2016-03-13T09:34:25.863+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37997 #4084336 (1506 connections now open) 2016-03-13T09:34:25.863+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:37998 #4084337 (1507 connections now open) 2016-03-13T09:34:28.104+0000 I NETWORK [conn4084326] end connection 10.0.7.113:33275 (1506 connections now open) 2016-03-13T09:34:28.104+0000 I NETWORK [conn4084327] end connection 10.0.7.113:33276 (1506 connections now open) 2016-03-13T09:34:28.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33292 #4084338 (1506 connections now open) 2016-03-13T09:34:28.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33293 #4084339 (1507 connections now open) 2016-03-13T09:34:28.113+0000 I NETWORK [conn4081204] end connection 10.0.179.209:40704 (1506 connections now open) 2016-03-13T09:34:28.114+0000 I NETWORK [conn4081238] end connection 10.0.7.113:59624 (1505 connections now open) 2016-03-13T09:34:28.114+0000 I NETWORK [conn4081381] end connection 10.0.27.237:35929 (1504 connections now open) 2016-03-13T09:34:28.114+0000 I NETWORK [conn4084009] end connection 10.0.26.69:44316 (1504 connections now open) 2016-03-13T09:34:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42645 #4084340 (1504 connections now open) 2016-03-13T09:34:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33295 #4084341 (1505 connections now open) 2016-03-13T09:34:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44648 #4084342 (1506 connections now open) 2016-03-13T09:34:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38004 #4084343 (1507 connections now open) 2016-03-13T09:34:29.063+0000 I NETWORK [conn4084329] end connection 10.0.216.90:59525 (1506 connections now open) 2016-03-13T09:34:29.063+0000 I NETWORK [conn4084328] end connection 10.0.216.90:59526 (1506 connections now open) 2016-03-13T09:34:29.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59549 #4084344 (1506 connections now open) 2016-03-13T09:34:29.063+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59548 #4084345 (1507 connections now open) 2016-03-13T09:34:30.134+0000 I NETWORK [conn4084331] end connection 10.0.20.53:52953 (1506 connections now open) 2016-03-13T09:34:30.134+0000 I NETWORK [conn4084330] end connection 10.0.20.53:52952 (1506 connections now open) 2016-03-13T09:34:30.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52965 #4084346 (1506 connections now open) 2016-03-13T09:34:30.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52964 #4084347 (1507 connections now open) 2016-03-13T09:34:31.252+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17412 version: 30712|1||563ba74d869758a4542b9075 based on: 30711|1||563ba74d869758a4542b9075 2016-03-13T09:34:32.535+0000 I NETWORK [conn4084332] end connection 10.0.179.209:42631 (1506 connections now open) 2016-03-13T09:34:32.536+0000 I NETWORK [conn4084333] end connection 10.0.179.209:42632 (1506 connections now open) 2016-03-13T09:34:32.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42651 #4084348 (1506 connections now open) 2016-03-13T09:34:32.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42652 #4084349 (1507 connections now open) 2016-03-13T09:34:33.180+0000 I NETWORK [conn4084334] end connection 10.0.26.69:44637 (1506 connections now open) 2016-03-13T09:34:33.180+0000 I NETWORK [conn4084335] end connection 10.0.26.69:44638 (1506 connections now open) 2016-03-13T09:34:33.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44652 #4084350 (1506 connections now open) 2016-03-13T09:34:33.188+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44651 #4084351 (1507 connections now open) 2016-03-13T09:34:35.875+0000 I NETWORK [conn4084337] end connection 10.0.27.237:37998 (1506 connections now open) 2016-03-13T09:34:35.875+0000 I NETWORK [conn4084336] end connection 10.0.27.237:37997 (1506 connections now open) 2016-03-13T09:34:35.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38018 #4084352 (1506 connections now open) 2016-03-13T09:34:35.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38017 #4084353 (1507 connections now open) 2016-03-13T09:34:38.117+0000 I NETWORK [conn4084339] end connection 10.0.7.113:33293 (1506 connections now open) 2016-03-13T09:34:38.117+0000 I NETWORK [conn4084338] end connection 10.0.7.113:33292 (1506 connections now open) 2016-03-13T09:34:38.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33301 #4084354 (1506 connections now open) 2016-03-13T09:34:38.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33300 #4084355 (1507 connections now open) 2016-03-13T09:34:39.076+0000 I NETWORK [conn4084345] end connection 10.0.216.90:59548 (1506 connections now open) 2016-03-13T09:34:39.076+0000 I NETWORK [conn4084344] end connection 10.0.216.90:59549 (1506 connections now open) 2016-03-13T09:34:39.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59555 #4084356 (1506 connections now open) 2016-03-13T09:34:39.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59556 #4084357 (1507 connections now open) 2016-03-13T09:34:40.154+0000 I NETWORK [conn4084347] end connection 10.0.20.53:52964 (1506 connections now open) 2016-03-13T09:34:40.154+0000 I NETWORK [conn4084346] end connection 10.0.20.53:52965 (1506 connections now open) 2016-03-13T09:34:40.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52975 #4084358 (1506 connections now open) 2016-03-13T09:34:40.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52976 #4084359 (1507 connections now open) 2016-03-13T09:34:42.548+0000 I NETWORK [conn4084348] end connection 10.0.179.209:42651 (1506 connections now open) 2016-03-13T09:34:42.549+0000 I NETWORK [conn4084349] end connection 10.0.179.209:42652 (1506 connections now open) 2016-03-13T09:34:42.558+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42659 #4084360 (1506 connections now open) 2016-03-13T09:34:42.558+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42658 #4084361 (1507 connections now open) 2016-03-13T09:34:43.200+0000 I NETWORK [conn4084351] end connection 10.0.26.69:44651 (1506 connections now open) 2016-03-13T09:34:43.200+0000 I NETWORK [conn4084350] end connection 10.0.26.69:44652 (1506 connections now open) 2016-03-13T09:34:43.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44668 #4084362 (1506 connections now open) 2016-03-13T09:34:43.200+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44669 #4084363 (1507 connections now open) 2016-03-13T09:34:44.942+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17413 version: 30713|1||563ba74d869758a4542b9075 based on: 30712|1||563ba74d869758a4542b9075 2016-03-13T09:34:45.889+0000 I NETWORK [conn4084353] end connection 10.0.27.237:38017 (1506 connections now open) 2016-03-13T09:34:45.889+0000 I NETWORK [conn4084352] end connection 10.0.27.237:38018 (1506 connections now open) 2016-03-13T09:34:45.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38026 #4084364 (1506 connections now open) 2016-03-13T09:34:45.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38025 #4084365 (1507 connections now open) 2016-03-13T09:34:46.118+0000 I NETWORK [conn4082499] end connection 10.0.27.237:36440 (1506 connections now open) 2016-03-13T09:34:46.118+0000 I NETWORK [conn4081773] end connection 10.0.26.69:42831 (1505 connections now open) 2016-03-13T09:34:46.118+0000 I NETWORK [conn4080919] end connection 10.0.7.113:59511 (1504 connections now open) 2016-03-13T09:34:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38029 #4084366 (1505 connections now open) 2016-03-13T09:34:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44672 #4084367 (1506 connections now open) 2016-03-13T09:34:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33313 #4084368 (1507 connections now open) 2016-03-13T09:34:48.139+0000 I NETWORK [conn4084355] end connection 10.0.7.113:33300 (1506 connections now open) 2016-03-13T09:34:48.139+0000 I NETWORK [conn4084354] end connection 10.0.7.113:33301 (1506 connections now open) 2016-03-13T09:34:48.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33315 #4084369 (1506 connections now open) 2016-03-13T09:34:48.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33316 #4084370 (1507 connections now open) 2016-03-13T09:34:49.096+0000 I NETWORK [conn4084356] end connection 10.0.216.90:59555 (1506 connections now open) 2016-03-13T09:34:49.096+0000 I NETWORK [conn4084357] end connection 10.0.216.90:59556 (1506 connections now open) 2016-03-13T09:34:49.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59569 #4084371 (1506 connections now open) 2016-03-13T09:34:49.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59570 #4084372 (1507 connections now open) 2016-03-13T09:34:50.167+0000 I NETWORK [conn4084358] end connection 10.0.20.53:52975 (1506 connections now open) 2016-03-13T09:34:50.167+0000 I NETWORK [conn4084359] end connection 10.0.20.53:52976 (1506 connections now open) 2016-03-13T09:34:50.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52996 #4084373 (1506 connections now open) 2016-03-13T09:34:50.168+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:52995 #4084374 (1507 connections now open) 2016-03-13T09:34:52.570+0000 I NETWORK [conn4084360] end connection 10.0.179.209:42659 (1506 connections now open) 2016-03-13T09:34:52.570+0000 I NETWORK [conn4084361] end connection 10.0.179.209:42658 (1505 connections now open) 2016-03-13T09:34:52.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42673 #4084375 (1506 connections now open) 2016-03-13T09:34:52.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42672 #4084376 (1507 connections now open) 2016-03-13T09:34:53.213+0000 I NETWORK [conn4084363] end connection 10.0.26.69:44669 (1506 connections now open) 2016-03-13T09:34:53.213+0000 I NETWORK [conn4084362] end connection 10.0.26.69:44668 (1506 connections now open) 2016-03-13T09:34:53.213+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44676 #4084377 (1507 connections now open) 2016-03-13T09:34:53.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44677 #4084378 (1507 connections now open) 2016-03-13T09:34:54.444+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:34:54.278+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:34:54.444+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:34:54.546+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5342618fa188537623dd2 2016-03-13T09:34:54.683+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5343018fa188537623dd3 2016-03-13T09:34:54.819+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5343a18fa188537623dd4 2016-03-13T09:34:55.910+0000 I NETWORK [conn4084364] end connection 10.0.27.237:38026 (1506 connections now open) 2016-03-13T09:34:55.910+0000 I NETWORK [conn4084365] end connection 10.0.27.237:38025 (1505 connections now open) 2016-03-13T09:34:55.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38039 #4084379 (1506 connections now open) 2016-03-13T09:34:55.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38040 #4084380 (1507 connections now open) 2016-03-13T09:34:58.113+0000 I NETWORK [conn4081645] end connection 10.0.26.69:42751 (1506 connections now open) 2016-03-13T09:34:58.113+0000 I NETWORK [conn4082466] end connection 10.0.20.53:51359 (1506 connections now open) 2016-03-13T09:34:58.113+0000 I NETWORK [conn4080526] end connection 10.0.27.237:35562 (1504 connections now open) 2016-03-13T09:34:58.113+0000 I NETWORK [conn4082812] end connection 10.0.216.90:58165 (1506 connections now open) 2016-03-13T09:34:58.113+0000 I NETWORK [conn4082565] end connection 10.0.179.209:41117 (1506 connections now open) 2016-03-13T09:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42689 #4084381 (1503 connections now open) 2016-03-13T09:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59587 #4084382 (1504 connections now open) 2016-03-13T09:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44689 #4084383 (1505 connections now open) 2016-03-13T09:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53005 #4084384 (1506 connections now open) 2016-03-13T09:34:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38045 #4084385 (1507 connections now open) 2016-03-13T09:34:58.152+0000 I NETWORK [conn4084369] end connection 10.0.7.113:33315 (1506 connections now open) 2016-03-13T09:34:58.152+0000 I NETWORK [conn4084370] end connection 10.0.7.113:33316 (1506 connections now open) 2016-03-13T09:34:58.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33333 #4084386 (1506 connections now open) 2016-03-13T09:34:58.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33334 #4084387 (1507 connections now open) 2016-03-13T09:34:59.033+0000 I SHARDING [conn1051649] ChunkManager: time to load chunks for mydomain.Sessions: 695ms sequenceNumber: 17414 version: 30714|1||563ba74d869758a4542b9075 based on: 30713|1||563ba74d869758a4542b9075 2016-03-13T09:34:59.109+0000 I NETWORK [conn4084371] end connection 10.0.216.90:59569 (1506 connections now open) 2016-03-13T09:34:59.109+0000 I NETWORK [conn4084372] end connection 10.0.216.90:59570 (1506 connections now open) 2016-03-13T09:34:59.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59588 #4084388 (1506 connections now open) 2016-03-13T09:34:59.110+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59589 #4084389 (1507 connections now open) 2016-03-13T09:35:00.180+0000 I NETWORK [conn4084374] end connection 10.0.20.53:52995 (1506 connections now open) 2016-03-13T09:35:00.180+0000 I NETWORK [conn4084373] end connection 10.0.20.53:52996 (1506 connections now open) 2016-03-13T09:35:00.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53007 #4084390 (1506 connections now open) 2016-03-13T09:35:00.181+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53006 #4084391 (1507 connections now open) 2016-03-13T09:35:00.627+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5344418fa188537623dd5 2016-03-13T09:35:02.583+0000 I NETWORK [conn4084375] end connection 10.0.179.209:42673 (1506 connections now open) 2016-03-13T09:35:02.583+0000 I NETWORK [conn4084376] end connection 10.0.179.209:42672 (1505 connections now open) 2016-03-13T09:35:02.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42699 #4084392 (1506 connections now open) 2016-03-13T09:35:02.584+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42698 #4084393 (1507 connections now open) 2016-03-13T09:35:03.226+0000 I NETWORK [conn4084378] end connection 10.0.26.69:44677 (1506 connections now open) 2016-03-13T09:35:03.226+0000 I NETWORK [conn4084377] end connection 10.0.26.69:44676 (1506 connections now open) 2016-03-13T09:35:03.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44695 #4084394 (1506 connections now open) 2016-03-13T09:35:03.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44694 #4084395 (1507 connections now open) 2016-03-13T09:35:04.139+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634fce8f1ad750eb6b55d57')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634fce8f1ad750eb6b55d57') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634fe2655418c0ec5aaa151') }, shard: "shmydomain1", lastmod: Timestamp 30714000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:35:04.295+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30714|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634fce8f1ad750eb6b55d57') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634fe2655418c0ec5aaa151') }) shmydomain1 -> shmydomain2 2016-03-13T09:35:05.923+0000 I NETWORK [conn4084379] end connection 10.0.27.237:38039 (1506 connections now open) 2016-03-13T09:35:05.923+0000 I NETWORK [conn4084380] end connection 10.0.27.237:38040 (1506 connections now open) 2016-03-13T09:35:05.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38062 #4084396 (1506 connections now open) 2016-03-13T09:35:05.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38061 #4084397 (1507 connections now open) 2016-03-13T09:35:08.166+0000 I NETWORK [conn4084387] end connection 10.0.7.113:33334 (1506 connections now open) 2016-03-13T09:35:08.166+0000 I NETWORK [conn4084386] end connection 10.0.7.113:33333 (1506 connections now open) 2016-03-13T09:35:08.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33346 #4084398 (1506 connections now open) 2016-03-13T09:35:08.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33345 #4084399 (1507 connections now open) 2016-03-13T09:35:09.122+0000 I NETWORK [conn4084389] end connection 10.0.216.90:59589 (1506 connections now open) 2016-03-13T09:35:09.122+0000 I NETWORK [conn4084388] end connection 10.0.216.90:59588 (1506 connections now open) 2016-03-13T09:35:09.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59601 #4084400 (1506 connections now open) 2016-03-13T09:35:09.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59600 #4084401 (1507 connections now open) 2016-03-13T09:35:10.193+0000 I NETWORK [conn4084390] end connection 10.0.20.53:53007 (1506 connections now open) 2016-03-13T09:35:10.193+0000 I NETWORK [conn4084391] end connection 10.0.20.53:53006 (1506 connections now open) 2016-03-13T09:35:10.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53019 #4084402 (1506 connections now open) 2016-03-13T09:35:10.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53020 #4084403 (1507 connections now open) 2016-03-13T09:35:12.596+0000 I NETWORK [conn4084393] end connection 10.0.179.209:42698 (1506 connections now open) 2016-03-13T09:35:12.596+0000 I NETWORK [conn4084392] end connection 10.0.179.209:42699 (1506 connections now open) 2016-03-13T09:35:12.596+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42706 #4084404 (1506 connections now open) 2016-03-13T09:35:12.597+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42707 #4084405 (1507 connections now open) 2016-03-13T09:35:13.239+0000 I NETWORK [conn4084394] end connection 10.0.26.69:44695 (1506 connections now open) 2016-03-13T09:35:13.239+0000 I NETWORK [conn4084395] end connection 10.0.26.69:44694 (1506 connections now open) 2016-03-13T09:35:13.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44716 #4084406 (1506 connections now open) 2016-03-13T09:35:13.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44717 #4084407 (1507 connections now open) 2016-03-13T09:35:14.124+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17415 version: 30715|1||563ba74d869758a4542b9075 based on: 30714|1||563ba74d869758a4542b9075 2016-03-13T09:35:14.324+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:35:14.324+0000-56e5345218fa188537623dd6", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861714324), what: "balancer.round", ns: "", details: { executionTimeMillis: 14032, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:35:14.484+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:35:15.757+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5345318fa188537623dd7 2016-03-13T09:35:15.936+0000 I NETWORK [conn4084396] end connection 10.0.27.237:38062 (1506 connections now open) 2016-03-13T09:35:15.936+0000 I NETWORK [conn4084397] end connection 10.0.27.237:38061 (1506 connections now open) 2016-03-13T09:35:15.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38074 #4084408 (1506 connections now open) 2016-03-13T09:35:15.937+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38073 #4084409 (1507 connections now open) 2016-03-13T09:35:16.117+0000 I NETWORK [conn4081790] end connection 10.0.216.90:57716 (1506 connections now open) 2016-03-13T09:35:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59612 #4084410 (1507 connections now open) 2016-03-13T09:35:18.179+0000 I NETWORK [conn4084398] end connection 10.0.7.113:33346 (1506 connections now open) 2016-03-13T09:35:18.179+0000 I NETWORK [conn4084399] end connection 10.0.7.113:33345 (1505 connections now open) 2016-03-13T09:35:18.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33359 #4084411 (1506 connections now open) 2016-03-13T09:35:18.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33360 #4084412 (1507 connections now open) 2016-03-13T09:35:18.557+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634fe2655418c0ec5aaa151')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634fe2655418c0ec5aaa151') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ff5acf7b4e0eb5406725') }, shard: "shmydomain1", lastmod: Timestamp 30715000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:35:18.708+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30715|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634fe2655418c0ec5aaa151') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ff5acf7b4e0eb5406725') }) shmydomain1 -> shmydomain3 2016-03-13T09:35:19.135+0000 I NETWORK [conn4084401] end connection 10.0.216.90:59600 (1506 connections now open) 2016-03-13T09:35:19.135+0000 I NETWORK [conn4084400] end connection 10.0.216.90:59601 (1506 connections now open) 2016-03-13T09:35:19.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59614 #4084413 (1506 connections now open) 2016-03-13T09:35:19.136+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59615 #4084414 (1507 connections now open) 2016-03-13T09:35:20.206+0000 I NETWORK [conn4084402] end connection 10.0.20.53:53019 (1506 connections now open) 2016-03-13T09:35:20.206+0000 I NETWORK [conn4084403] end connection 10.0.20.53:53020 (1506 connections now open) 2016-03-13T09:35:20.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53041 #4084415 (1506 connections now open) 2016-03-13T09:35:20.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53040 #4084416 (1507 connections now open) 2016-03-13T09:35:22.609+0000 I NETWORK [conn4084404] end connection 10.0.179.209:42706 (1506 connections now open) 2016-03-13T09:35:22.609+0000 I NETWORK [conn4084405] end connection 10.0.179.209:42707 (1506 connections now open) 2016-03-13T09:35:22.610+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42722 #4084417 (1506 connections now open) 2016-03-13T09:35:22.610+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42723 #4084418 (1507 connections now open) 2016-03-13T09:35:23.252+0000 I NETWORK [conn4084407] end connection 10.0.26.69:44717 (1506 connections now open) 2016-03-13T09:35:23.252+0000 I NETWORK [conn4084406] end connection 10.0.26.69:44716 (1506 connections now open) 2016-03-13T09:35:23.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44725 #4084419 (1506 connections now open) 2016-03-13T09:35:23.253+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44724 #4084420 (1507 connections now open) 2016-03-13T09:35:25.652+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:35:24.819+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:35:25.950+0000 I NETWORK [conn4084409] end connection 10.0.27.237:38073 (1506 connections now open) 2016-03-13T09:35:25.950+0000 I NETWORK [conn4084408] end connection 10.0.27.237:38074 (1506 connections now open) 2016-03-13T09:35:25.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38090 #4084421 (1506 connections now open) 2016-03-13T09:35:25.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38089 #4084422 (1507 connections now open) 2016-03-13T09:35:27.541+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17416 version: 30716|1||563ba74d869758a4542b9075 based on: 30715|1||563ba74d869758a4542b9075 2016-03-13T09:35:27.833+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:35:27.833+0000-56e5345f18fa188537623dd8", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861727833), what: "balancer.round", ns: "", details: { executionTimeMillis: 12315, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:35:28.003+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:35:28.114+0000 I NETWORK [conn4081998] end connection 10.0.179.209:40897 (1506 connections now open) 2016-03-13T09:35:28.114+0000 I NETWORK [conn4082599] end connection 10.0.216.90:58070 (1505 connections now open) 2016-03-13T09:35:28.114+0000 I NETWORK [conn4081110] end connection 10.0.27.237:35836 (1504 connections now open) 2016-03-13T09:35:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42737 #4084423 (1506 connections now open) 2016-03-13T09:35:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59632 #4084424 (1506 connections now open) 2016-03-13T09:35:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38095 #4084425 (1507 connections now open) 2016-03-13T09:35:28.192+0000 I NETWORK [conn4084412] end connection 10.0.7.113:33360 (1506 connections now open) 2016-03-13T09:35:28.193+0000 I NETWORK [conn4084411] end connection 10.0.7.113:33359 (1505 connections now open) 2016-03-13T09:35:28.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33381 #4084426 (1506 connections now open) 2016-03-13T09:35:28.193+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33380 #4084427 (1507 connections now open) 2016-03-13T09:35:29.148+0000 I NETWORK [conn4084414] end connection 10.0.216.90:59615 (1506 connections now open) 2016-03-13T09:35:29.148+0000 I NETWORK [conn4084413] end connection 10.0.216.90:59614 (1506 connections now open) 2016-03-13T09:35:29.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59638 #4084428 (1506 connections now open) 2016-03-13T09:35:29.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59637 #4084429 (1507 connections now open) 2016-03-13T09:35:29.334+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5346118fa188537623dd9 2016-03-13T09:35:30.220+0000 I NETWORK [conn4084415] end connection 10.0.20.53:53041 (1506 connections now open) 2016-03-13T09:35:30.220+0000 I NETWORK [conn4084416] end connection 10.0.20.53:53040 (1505 connections now open) 2016-03-13T09:35:30.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53053 #4084430 (1506 connections now open) 2016-03-13T09:35:30.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53054 #4084431 (1507 connections now open) 2016-03-13T09:35:31.836+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5634ff5acf7b4e0eb5406725')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ff5acf7b4e0eb5406725') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5635008bbcf3ae0ea6caddc1') }, shard: "shmydomain1", lastmod: Timestamp 30716000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:35:31.991+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30716|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5634ff5acf7b4e0eb5406725') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5635008bbcf3ae0ea6caddc1') }) shmydomain1 -> shmydomain2 2016-03-13T09:35:32.622+0000 I NETWORK [conn4084417] end connection 10.0.179.209:42722 (1506 connections now open) 2016-03-13T09:35:32.622+0000 I NETWORK [conn4084418] end connection 10.0.179.209:42723 (1505 connections now open) 2016-03-13T09:35:32.623+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42742 #4084432 (1506 connections now open) 2016-03-13T09:35:32.623+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42743 #4084433 (1507 connections now open) 2016-03-13T09:35:33.266+0000 I NETWORK [conn4084419] end connection 10.0.26.69:44725 (1506 connections now open) 2016-03-13T09:35:33.266+0000 I NETWORK [conn4084420] end connection 10.0.26.69:44724 (1506 connections now open) 2016-03-13T09:35:33.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44737 #4084434 (1506 connections now open) 2016-03-13T09:35:33.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44738 #4084435 (1507 connections now open) 2016-03-13T09:35:35.963+0000 I NETWORK [conn4084421] end connection 10.0.27.237:38090 (1506 connections now open) 2016-03-13T09:35:35.963+0000 I NETWORK [conn4084422] end connection 10.0.27.237:38089 (1506 connections now open) 2016-03-13T09:35:35.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38109 #4084436 (1506 connections now open) 2016-03-13T09:35:35.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38110 #4084437 (1507 connections now open) 2016-03-13T09:35:38.206+0000 I NETWORK [conn4084427] end connection 10.0.7.113:33380 (1506 connections now open) 2016-03-13T09:35:38.206+0000 I NETWORK [conn4084426] end connection 10.0.7.113:33381 (1506 connections now open) 2016-03-13T09:35:38.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33387 #4084438 (1506 connections now open) 2016-03-13T09:35:38.207+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33386 #4084439 (1507 connections now open) 2016-03-13T09:35:39.161+0000 I NETWORK [conn4084429] end connection 10.0.216.90:59637 (1506 connections now open) 2016-03-13T09:35:39.161+0000 I NETWORK [conn4084428] end connection 10.0.216.90:59638 (1506 connections now open) 2016-03-13T09:35:39.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59644 #4084440 (1506 connections now open) 2016-03-13T09:35:39.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59643 #4084441 (1507 connections now open) 2016-03-13T09:35:40.241+0000 I NETWORK [conn4084430] end connection 10.0.20.53:53053 (1506 connections now open) 2016-03-13T09:35:40.241+0000 I NETWORK [conn4084431] end connection 10.0.20.53:53054 (1506 connections now open) 2016-03-13T09:35:40.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53066 #4084442 (1506 connections now open) 2016-03-13T09:35:40.242+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53067 #4084443 (1507 connections now open) 2016-03-13T09:35:41.901+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17417 version: 30717|1||563ba74d869758a4542b9075 based on: 30716|1||563ba74d869758a4542b9075 2016-03-13T09:35:42.099+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:35:42.099+0000-56e5346e18fa188537623dda", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861742099), what: "balancer.round", ns: "", details: { executionTimeMillis: 13060, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:35:42.294+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:35:42.635+0000 I NETWORK [conn4084433] end connection 10.0.179.209:42743 (1506 connections now open) 2016-03-13T09:35:42.635+0000 I NETWORK [conn4084432] end connection 10.0.179.209:42742 (1505 connections now open) 2016-03-13T09:35:42.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42749 #4084444 (1506 connections now open) 2016-03-13T09:35:42.636+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42748 #4084445 (1507 connections now open) 2016-03-13T09:35:43.287+0000 I NETWORK [conn4084435] end connection 10.0.26.69:44738 (1506 connections now open) 2016-03-13T09:35:43.287+0000 I NETWORK [conn4084434] end connection 10.0.26.69:44737 (1506 connections now open) 2016-03-13T09:35:43.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44755 #4084446 (1506 connections now open) 2016-03-13T09:35:43.288+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44756 #4084447 (1507 connections now open) 2016-03-13T09:35:43.526+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5346f18fa188537623ddb 2016-03-13T09:35:45.975+0000 I NETWORK [conn4084437] end connection 10.0.27.237:38110 (1506 connections now open) 2016-03-13T09:35:45.975+0000 I NETWORK [conn4084436] end connection 10.0.27.237:38109 (1506 connections now open) 2016-03-13T09:35:45.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38117 #4084448 (1506 connections now open) 2016-03-13T09:35:45.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38118 #4084449 (1507 connections now open) 2016-03-13T09:35:46.046+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('5635008bbcf3ae0ea6caddc1')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5635008bbcf3ae0ea6caddc1') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563501be584ad10f1fabcac4') }, shard: "shmydomain1", lastmod: Timestamp 30717000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:35:46.117+0000 I NETWORK [conn4082586] end connection 10.0.20.53:51404 (1506 connections now open) 2016-03-13T09:35:46.117+0000 I NETWORK [conn4081650] end connection 10.0.26.69:42755 (1505 connections now open) 2016-03-13T09:35:46.117+0000 I NETWORK [conn4081926] end connection 10.0.27.237:36195 (1505 connections now open) 2016-03-13T09:35:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53074 #4084450 (1505 connections now open) 2016-03-13T09:35:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38121 #4084451 (1506 connections now open) 2016-03-13T09:35:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44759 #4084452 (1507 connections now open) 2016-03-13T09:35:46.122+0000 I NETWORK [conn4081806] end connection 10.0.7.113:59739 (1506 connections now open) 2016-03-13T09:35:46.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33399 #4084453 (1507 connections now open) 2016-03-13T09:35:46.200+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30717|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('5635008bbcf3ae0ea6caddc1') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563501be584ad10f1fabcac4') }) shmydomain1 -> shmydomain3 2016-03-13T09:35:48.220+0000 I NETWORK [conn4084438] end connection 10.0.7.113:33387 (1506 connections now open) 2016-03-13T09:35:48.220+0000 I NETWORK [conn4084439] end connection 10.0.7.113:33386 (1506 connections now open) 2016-03-13T09:35:48.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33400 #4084454 (1506 connections now open) 2016-03-13T09:35:48.221+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33401 #4084455 (1507 connections now open) 2016-03-13T09:35:49.182+0000 I NETWORK [conn4084440] end connection 10.0.216.90:59644 (1506 connections now open) 2016-03-13T09:35:49.182+0000 I NETWORK [conn4084441] end connection 10.0.216.90:59643 (1506 connections now open) 2016-03-13T09:35:49.183+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59656 #4084456 (1506 connections now open) 2016-03-13T09:35:49.183+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59657 #4084457 (1507 connections now open) 2016-03-13T09:35:50.254+0000 I NETWORK [conn4084443] end connection 10.0.20.53:53067 (1506 connections now open) 2016-03-13T09:35:50.254+0000 I NETWORK [conn4084442] end connection 10.0.20.53:53066 (1506 connections now open) 2016-03-13T09:35:50.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53086 #4084458 (1506 connections now open) 2016-03-13T09:35:50.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53087 #4084459 (1507 connections now open) 2016-03-13T09:35:52.649+0000 I NETWORK [conn4084445] end connection 10.0.179.209:42748 (1506 connections now open) 2016-03-13T09:35:52.649+0000 I NETWORK [conn4084444] end connection 10.0.179.209:42749 (1506 connections now open) 2016-03-13T09:35:52.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42762 #4084460 (1506 connections now open) 2016-03-13T09:35:52.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42761 #4084461 (1507 connections now open) 2016-03-13T09:35:53.301+0000 I NETWORK [conn4084447] end connection 10.0.26.69:44756 (1506 connections now open) 2016-03-13T09:35:53.301+0000 I NETWORK [conn4084446] end connection 10.0.26.69:44755 (1506 connections now open) 2016-03-13T09:35:53.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44764 #4084462 (1506 connections now open) 2016-03-13T09:35:53.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44765 #4084463 (1507 connections now open) 2016-03-13T09:35:55.096+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17418 version: 30718|1||563ba74d869758a4542b9075 based on: 30717|1||563ba74d869758a4542b9075 2016-03-13T09:35:55.298+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:35:55.298+0000-56e5347b18fa188537623ddc", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861755298), what: "balancer.round", ns: "", details: { executionTimeMillis: 11970, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:35:55.488+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:35:55.930+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:35:55.653+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:35:55.996+0000 I NETWORK [conn4084448] end connection 10.0.27.237:38117 (1506 connections now open) 2016-03-13T09:35:55.996+0000 I NETWORK [conn4084449] end connection 10.0.27.237:38118 (1506 connections now open) 2016-03-13T09:35:55.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38131 #4084464 (1506 connections now open) 2016-03-13T09:35:55.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38132 #4084465 (1507 connections now open) 2016-03-13T09:35:56.823+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e5347c18fa188537623ddd 2016-03-13T09:35:58.113+0000 I NETWORK [conn4081209] end connection 10.0.179.209:40705 (1506 connections now open) 2016-03-13T09:35:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42777 #4084466 (1507 connections now open) 2016-03-13T09:35:58.234+0000 I NETWORK [conn4084454] end connection 10.0.7.113:33400 (1506 connections now open) 2016-03-13T09:35:58.234+0000 I NETWORK [conn4084455] end connection 10.0.7.113:33401 (1506 connections now open) 2016-03-13T09:35:58.234+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33420 #4084467 (1506 connections now open) 2016-03-13T09:35:58.234+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33421 #4084468 (1507 connections now open) 2016-03-13T09:35:59.195+0000 I NETWORK [conn4084457] end connection 10.0.216.90:59657 (1506 connections now open) 2016-03-13T09:35:59.196+0000 I NETWORK [conn4084456] end connection 10.0.216.90:59656 (1505 connections now open) 2016-03-13T09:35:59.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59677 #4084469 (1506 connections now open) 2016-03-13T09:35:59.196+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59676 #4084470 (1507 connections now open) 2016-03-13T09:35:59.365+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563501be584ad10f1fabcac4')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563501be584ad10f1fabcac4') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563502f843465f0e92b85e22') }, shard: "shmydomain1", lastmod: Timestamp 30718000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:35:59.515+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30718|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563501be584ad10f1fabcac4') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563502f843465f0e92b85e22') }) shmydomain1 -> shmydomain2 2016-03-13T09:36:00.267+0000 I NETWORK [conn4084458] end connection 10.0.20.53:53086 (1506 connections now open) 2016-03-13T09:36:00.267+0000 I NETWORK [conn4084459] end connection 10.0.20.53:53087 (1505 connections now open) 2016-03-13T09:36:00.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53096 #4084471 (1506 connections now open) 2016-03-13T09:36:00.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53097 #4084472 (1507 connections now open) 2016-03-13T09:36:02.671+0000 I NETWORK [conn4084460] end connection 10.0.179.209:42762 (1506 connections now open) 2016-03-13T09:36:02.671+0000 I NETWORK [conn4084461] end connection 10.0.179.209:42761 (1506 connections now open) 2016-03-13T09:36:02.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42784 #4084473 (1506 connections now open) 2016-03-13T09:36:02.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42785 #4084474 (1507 connections now open) 2016-03-13T09:36:03.314+0000 I NETWORK [conn4084462] end connection 10.0.26.69:44764 (1506 connections now open) 2016-03-13T09:36:03.314+0000 I NETWORK [conn4084463] end connection 10.0.26.69:44765 (1506 connections now open) 2016-03-13T09:36:03.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44782 #4084475 (1506 connections now open) 2016-03-13T09:36:03.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44781 #4084476 (1507 connections now open) 2016-03-13T09:36:06.009+0000 I NETWORK [conn4084464] end connection 10.0.27.237:38131 (1506 connections now open) 2016-03-13T09:36:06.009+0000 I NETWORK [conn4084465] end connection 10.0.27.237:38132 (1506 connections now open) 2016-03-13T09:36:06.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38153 #4084477 (1506 connections now open) 2016-03-13T09:36:06.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38154 #4084478 (1507 connections now open) 2016-03-13T09:36:08.247+0000 I NETWORK [conn4084468] end connection 10.0.7.113:33421 (1506 connections now open) 2016-03-13T09:36:08.247+0000 I NETWORK [conn4084467] end connection 10.0.7.113:33420 (1506 connections now open) 2016-03-13T09:36:08.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33432 #4084479 (1506 connections now open) 2016-03-13T09:36:08.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33433 #4084480 (1507 connections now open) 2016-03-13T09:36:09.209+0000 I NETWORK [conn4084470] end connection 10.0.216.90:59676 (1506 connections now open) 2016-03-13T09:36:09.209+0000 I NETWORK [conn4084469] end connection 10.0.216.90:59677 (1506 connections now open) 2016-03-13T09:36:09.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59685 #4084481 (1507 connections now open) 2016-03-13T09:36:09.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59684 #4084482 (1507 connections now open) 2016-03-13T09:36:09.503+0000 I SHARDING [conn1051636] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17419 version: 30719|1||563ba74d869758a4542b9075 based on: 30718|1||563ba74d869758a4542b9075 2016-03-13T09:36:09.796+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:36:09.796+0000-56e5348918fa188537623dde", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861769796), what: "balancer.round", ns: "", details: { executionTimeMillis: 13274, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:36:10.214+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:36:10.280+0000 I NETWORK [conn4084471] end connection 10.0.20.53:53096 (1506 connections now open) 2016-03-13T09:36:10.280+0000 I NETWORK [conn4084472] end connection 10.0.20.53:53097 (1506 connections now open) 2016-03-13T09:36:10.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53109 #4084483 (1506 connections now open) 2016-03-13T09:36:10.281+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53110 #4084484 (1507 connections now open) 2016-03-13T09:36:12.684+0000 I NETWORK [conn4084473] end connection 10.0.179.209:42784 (1506 connections now open) 2016-03-13T09:36:12.684+0000 I NETWORK [conn4084474] end connection 10.0.179.209:42785 (1506 connections now open) 2016-03-13T09:36:12.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42790 #4084485 (1506 connections now open) 2016-03-13T09:36:12.685+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42791 #4084486 (1507 connections now open) 2016-03-13T09:36:13.328+0000 I NETWORK [conn4084476] end connection 10.0.26.69:44781 (1506 connections now open) 2016-03-13T09:36:13.328+0000 I NETWORK [conn4084475] end connection 10.0.26.69:44782 (1506 connections now open) 2016-03-13T09:36:13.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44802 #4084487 (1506 connections now open) 2016-03-13T09:36:13.329+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44803 #4084488 (1507 connections now open) 2016-03-13T09:36:16.022+0000 I NETWORK [conn4084478] end connection 10.0.27.237:38154 (1506 connections now open) 2016-03-13T09:36:16.022+0000 I NETWORK [conn4084477] end connection 10.0.27.237:38153 (1506 connections now open) 2016-03-13T09:36:16.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38165 #4084489 (1506 connections now open) 2016-03-13T09:36:16.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38164 #4084490 (1507 connections now open) 2016-03-13T09:36:16.117+0000 I NETWORK [conn4082231] end connection 10.0.20.53:51279 (1506 connections now open) 2016-03-13T09:36:16.117+0000 I NETWORK [conn4082245] end connection 10.0.179.209:41009 (1506 connections now open) 2016-03-13T09:36:16.117+0000 I NETWORK [conn4081940] end connection 10.0.216.90:57789 (1506 connections now open) 2016-03-13T09:36:16.117+0000 I NETWORK [conn4080935] end connection 10.0.7.113:59519 (1503 connections now open) 2016-03-13T09:36:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53117 #4084491 (1504 connections now open) 2016-03-13T09:36:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42797 #4084492 (1505 connections now open) 2016-03-13T09:36:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59695 #4084493 (1506 connections now open) 2016-03-13T09:36:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33443 #4084494 (1507 connections now open) 2016-03-13T09:36:18.261+0000 I NETWORK [conn4084479] end connection 10.0.7.113:33432 (1506 connections now open) 2016-03-13T09:36:18.261+0000 I NETWORK [conn4084480] end connection 10.0.7.113:33433 (1506 connections now open) 2016-03-13T09:36:18.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33445 #4084495 (1506 connections now open) 2016-03-13T09:36:18.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33446 #4084496 (1507 connections now open) 2016-03-13T09:36:19.223+0000 I NETWORK [conn4084482] end connection 10.0.216.90:59684 (1506 connections now open) 2016-03-13T09:36:19.223+0000 I NETWORK [conn4084481] end connection 10.0.216.90:59685 (1506 connections now open) 2016-03-13T09:36:19.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59697 #4084497 (1506 connections now open) 2016-03-13T09:36:19.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59698 #4084498 (1507 connections now open) 2016-03-13T09:36:20.293+0000 I NETWORK [conn4084483] end connection 10.0.20.53:53109 (1506 connections now open) 2016-03-13T09:36:20.293+0000 I NETWORK [conn4084484] end connection 10.0.20.53:53110 (1506 connections now open) 2016-03-13T09:36:20.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53130 #4084499 (1506 connections now open) 2016-03-13T09:36:20.294+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53129 #4084500 (1507 connections now open) 2016-03-13T09:36:22.697+0000 I NETWORK [conn4084485] end connection 10.0.179.209:42790 (1506 connections now open) 2016-03-13T09:36:22.697+0000 I NETWORK [conn4084486] end connection 10.0.179.209:42791 (1506 connections now open) 2016-03-13T09:36:22.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42805 #4084501 (1506 connections now open) 2016-03-13T09:36:22.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42806 #4084502 (1507 connections now open) 2016-03-13T09:36:23.341+0000 I NETWORK [conn4084487] end connection 10.0.26.69:44802 (1506 connections now open) 2016-03-13T09:36:23.341+0000 I NETWORK [conn4084488] end connection 10.0.26.69:44803 (1506 connections now open) 2016-03-13T09:36:23.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44811 #4084503 (1506 connections now open) 2016-03-13T09:36:23.342+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44812 #4084504 (1507 connections now open) 2016-03-13T09:36:24.046+0000 I SHARDING [conn1051659] ChunkManager: time to load chunks for mydomain.Sessions: 699ms sequenceNumber: 17420 version: 30720|1||563ba74d869758a4542b9075 based on: 30719|1||563ba74d869758a4542b9075 2016-03-13T09:36:26.035+0000 I NETWORK [conn4084490] end connection 10.0.27.237:38164 (1506 connections now open) 2016-03-13T09:36:26.035+0000 I NETWORK [conn4084489] end connection 10.0.27.237:38165 (1506 connections now open) 2016-03-13T09:36:26.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38179 #4084505 (1506 connections now open) 2016-03-13T09:36:26.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38178 #4084506 (1507 connections now open) 2016-03-13T09:36:28.065+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:36:25.930+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:36:28.065+0000 I SHARDING [LockPinger] trying to delete 2 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:36:28.113+0000 I NETWORK [conn4082341] end connection 10.0.179.209:41045 (1506 connections now open) 2016-03-13T09:36:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42820 #4084507 (1507 connections now open) 2016-03-13T09:36:28.114+0000 I NETWORK [conn4080928] end connection 10.0.26.69:42439 (1506 connections now open) 2016-03-13T09:36:28.114+0000 I NETWORK [conn4082955] end connection 10.0.216.90:58243 (1506 connections now open) 2016-03-13T09:36:28.114+0000 I NETWORK [conn4079012] end connection 10.0.27.237:33958 (1506 connections now open) 2016-03-13T09:36:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59716 #4084508 (1505 connections now open) 2016-03-13T09:36:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38185 #4084509 (1506 connections now open) 2016-03-13T09:36:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44817 #4084510 (1507 connections now open) 2016-03-13T09:36:28.273+0000 I NETWORK [conn4084496] end connection 10.0.7.113:33446 (1506 connections now open) 2016-03-13T09:36:28.274+0000 I NETWORK [conn4084495] end connection 10.0.7.113:33445 (1505 connections now open) 2016-03-13T09:36:28.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33465 #4084511 (1506 connections now open) 2016-03-13T09:36:28.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33466 #4084512 (1507 connections now open) 2016-03-13T09:36:29.236+0000 I NETWORK [conn4084497] end connection 10.0.216.90:59697 (1506 connections now open) 2016-03-13T09:36:29.236+0000 I NETWORK [conn4084498] end connection 10.0.216.90:59698 (1506 connections now open) 2016-03-13T09:36:29.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59720 #4084513 (1506 connections now open) 2016-03-13T09:36:29.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59721 #4084514 (1507 connections now open) 2016-03-13T09:36:29.533+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5348b18fa188537623ddf 2016-03-13T09:36:29.670+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5349518fa188537623de0 2016-03-13T09:36:30.306+0000 I NETWORK [conn4084499] end connection 10.0.20.53:53130 (1506 connections now open) 2016-03-13T09:36:30.306+0000 I NETWORK [conn4084500] end connection 10.0.20.53:53129 (1506 connections now open) 2016-03-13T09:36:30.316+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53143 #4084515 (1506 connections now open) 2016-03-13T09:36:30.316+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53144 #4084516 (1507 connections now open) 2016-03-13T09:36:32.710+0000 I NETWORK [conn4084502] end connection 10.0.179.209:42806 (1506 connections now open) 2016-03-13T09:36:32.710+0000 I NETWORK [conn4084501] end connection 10.0.179.209:42805 (1505 connections now open) 2016-03-13T09:36:32.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42826 #4084517 (1506 connections now open) 2016-03-13T09:36:32.711+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42825 #4084518 (1507 connections now open) 2016-03-13T09:36:33.354+0000 I NETWORK [conn4084504] end connection 10.0.26.69:44812 (1506 connections now open) 2016-03-13T09:36:33.354+0000 I NETWORK [conn4084503] end connection 10.0.26.69:44811 (1506 connections now open) 2016-03-13T09:36:33.364+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44824 #4084519 (1506 connections now open) 2016-03-13T09:36:33.364+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44825 #4084520 (1507 connections now open) 2016-03-13T09:36:36.049+0000 I NETWORK [conn4084506] end connection 10.0.27.237:38178 (1506 connections now open) 2016-03-13T09:36:36.049+0000 I NETWORK [conn4084505] end connection 10.0.27.237:38179 (1506 connections now open) 2016-03-13T09:36:36.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38199 #4084521 (1506 connections now open) 2016-03-13T09:36:36.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38200 #4084522 (1507 connections now open) 2016-03-13T09:36:38.286+0000 I NETWORK [conn4084512] end connection 10.0.7.113:33466 (1506 connections now open) 2016-03-13T09:36:38.286+0000 I NETWORK [conn4084511] end connection 10.0.7.113:33465 (1505 connections now open) 2016-03-13T09:36:38.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33472 #4084523 (1506 connections now open) 2016-03-13T09:36:38.295+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33471 #4084524 (1507 connections now open) 2016-03-13T09:36:39.249+0000 I NETWORK [conn4084513] end connection 10.0.216.90:59720 (1506 connections now open) 2016-03-13T09:36:39.249+0000 I NETWORK [conn4084514] end connection 10.0.216.90:59721 (1506 connections now open) 2016-03-13T09:36:39.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59727 #4084525 (1506 connections now open) 2016-03-13T09:36:39.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59728 #4084526 (1507 connections now open) 2016-03-13T09:36:39.503+0000 I SHARDING [conn1051653] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17421 version: 30721|1||563ba74d869758a4542b9075 based on: 30720|1||563ba74d869758a4542b9075 2016-03-13T09:36:40.329+0000 I NETWORK [conn4084515] end connection 10.0.20.53:53143 (1506 connections now open) 2016-03-13T09:36:40.329+0000 I NETWORK [conn4084516] end connection 10.0.20.53:53144 (1506 connections now open) 2016-03-13T09:36:40.329+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53157 #4084527 (1506 connections now open) 2016-03-13T09:36:40.329+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53156 #4084528 (1507 connections now open) 2016-03-13T09:36:42.723+0000 I NETWORK [conn4084518] end connection 10.0.179.209:42825 (1506 connections now open) 2016-03-13T09:36:42.723+0000 I NETWORK [conn4084517] end connection 10.0.179.209:42826 (1506 connections now open) 2016-03-13T09:36:42.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42831 #4084529 (1506 connections now open) 2016-03-13T09:36:42.724+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42832 #4084530 (1507 connections now open) 2016-03-13T09:36:43.377+0000 I NETWORK [conn4084520] end connection 10.0.26.69:44825 (1506 connections now open) 2016-03-13T09:36:43.377+0000 I NETWORK [conn4084519] end connection 10.0.26.69:44824 (1506 connections now open) 2016-03-13T09:36:43.377+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44843 #4084531 (1506 connections now open) 2016-03-13T09:36:43.377+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44842 #4084532 (1507 connections now open) 2016-03-13T09:36:46.062+0000 I NETWORK [conn4084522] end connection 10.0.27.237:38200 (1506 connections now open) 2016-03-13T09:36:46.062+0000 I NETWORK [conn4084521] end connection 10.0.27.237:38199 (1506 connections now open) 2016-03-13T09:36:46.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38207 #4084533 (1506 connections now open) 2016-03-13T09:36:46.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38208 #4084534 (1507 connections now open) 2016-03-13T09:36:46.117+0000 I NETWORK [conn4081907] end connection 10.0.27.237:36185 (1506 connections now open) 2016-03-13T09:36:46.117+0000 I NETWORK [conn4083032] end connection 10.0.7.113:60229 (1505 connections now open) 2016-03-13T09:36:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38211 #4084535 (1506 connections now open) 2016-03-13T09:36:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33483 #4084536 (1507 connections now open) 2016-03-13T09:36:48.307+0000 I NETWORK [conn4084523] end connection 10.0.7.113:33472 (1506 connections now open) 2016-03-13T09:36:48.307+0000 I NETWORK [conn4084524] end connection 10.0.7.113:33471 (1506 connections now open) 2016-03-13T09:36:48.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33485 #4084537 (1506 connections now open) 2016-03-13T09:36:48.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33486 #4084538 (1507 connections now open) 2016-03-13T09:36:49.270+0000 I NETWORK [conn4084526] end connection 10.0.216.90:59728 (1506 connections now open) 2016-03-13T09:36:49.270+0000 I NETWORK [conn4084525] end connection 10.0.216.90:59727 (1506 connections now open) 2016-03-13T09:36:49.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59742 #4084539 (1506 connections now open) 2016-03-13T09:36:49.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59743 #4084540 (1507 connections now open) 2016-03-13T09:36:50.342+0000 I NETWORK [conn4084528] end connection 10.0.20.53:53156 (1506 connections now open) 2016-03-13T09:36:50.342+0000 I NETWORK [conn4084527] end connection 10.0.20.53:53157 (1506 connections now open) 2016-03-13T09:36:50.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53176 #4084541 (1506 connections now open) 2016-03-13T09:36:50.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53177 #4084542 (1507 connections now open) 2016-03-13T09:36:52.737+0000 I NETWORK [conn4084530] end connection 10.0.179.209:42832 (1506 connections now open) 2016-03-13T09:36:52.737+0000 I NETWORK [conn4084529] end connection 10.0.179.209:42831 (1505 connections now open) 2016-03-13T09:36:52.746+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42845 #4084543 (1506 connections now open) 2016-03-13T09:36:52.746+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42844 #4084544 (1507 connections now open) 2016-03-13T09:36:53.390+0000 I NETWORK [conn4084531] end connection 10.0.26.69:44843 (1506 connections now open) 2016-03-13T09:36:53.390+0000 I NETWORK [conn4084532] end connection 10.0.26.69:44842 (1506 connections now open) 2016-03-13T09:36:53.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44850 #4084545 (1506 connections now open) 2016-03-13T09:36:53.391+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44851 #4084546 (1507 connections now open) 2016-03-13T09:36:55.633+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 697ms sequenceNumber: 17422 version: 30722|1||563ba74d869758a4542b9075 based on: 30721|1||563ba74d869758a4542b9075 2016-03-13T09:36:56.084+0000 I NETWORK [conn4084533] end connection 10.0.27.237:38207 (1506 connections now open) 2016-03-13T09:36:56.084+0000 I NETWORK [conn4084534] end connection 10.0.27.237:38208 (1506 connections now open) 2016-03-13T09:36:56.085+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38221 #4084547 (1506 connections now open) 2016-03-13T09:36:56.085+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38222 #4084548 (1507 connections now open) 2016-03-13T09:36:58.113+0000 I NETWORK [conn4081212] end connection 10.0.179.209:40706 (1506 connections now open) 2016-03-13T09:36:58.113+0000 I NETWORK [conn4081287] end connection 10.0.20.53:50816 (1505 connections now open) 2016-03-13T09:36:58.113+0000 I NETWORK [conn4082918] end connection 10.0.216.90:58233 (1505 connections now open) 2016-03-13T09:36:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42860 #4084549 (1507 connections now open) 2016-03-13T09:36:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59760 #4084550 (1506 connections now open) 2016-03-13T09:36:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53185 #4084551 (1507 connections now open) 2016-03-13T09:36:58.320+0000 I NETWORK [conn4084537] end connection 10.0.7.113:33485 (1506 connections now open) 2016-03-13T09:36:58.320+0000 I NETWORK [conn4084538] end connection 10.0.7.113:33486 (1506 connections now open) 2016-03-13T09:36:58.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33505 #4084552 (1506 connections now open) 2016-03-13T09:36:58.321+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33504 #4084553 (1507 connections now open) 2016-03-13T09:36:59.283+0000 I NETWORK [conn4084539] end connection 10.0.216.90:59742 (1506 connections now open) 2016-03-13T09:36:59.283+0000 I NETWORK [conn4084540] end connection 10.0.216.90:59743 (1506 connections now open) 2016-03-13T09:36:59.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59762 #4084554 (1506 connections now open) 2016-03-13T09:36:59.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59761 #4084555 (1507 connections now open) 2016-03-13T09:36:59.952+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:36:59.670+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:36:59.952+0000 I SHARDING [LockPinger] trying to delete 3 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:37:00.088+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e5349f18fa188537623de1 2016-03-13T09:37:00.225+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e534a918fa188537623de2 2016-03-13T09:37:00.356+0000 I NETWORK [conn4084542] end connection 10.0.20.53:53177 (1506 connections now open) 2016-03-13T09:37:00.356+0000 I NETWORK [conn4084541] end connection 10.0.20.53:53176 (1506 connections now open) 2016-03-13T09:37:00.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53188 #4084556 (1506 connections now open) 2016-03-13T09:37:00.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53187 #4084557 (1507 connections now open) 2016-03-13T09:37:00.361+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e534b318fa188537623de3 2016-03-13T09:37:02.759+0000 I NETWORK [conn4084544] end connection 10.0.179.209:42844 (1506 connections now open) 2016-03-13T09:37:02.759+0000 I NETWORK [conn4084543] end connection 10.0.179.209:42845 (1506 connections now open) 2016-03-13T09:37:02.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42867 #4084558 (1506 connections now open) 2016-03-13T09:37:02.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42868 #4084559 (1507 connections now open) 2016-03-13T09:37:03.403+0000 I NETWORK [conn4084545] end connection 10.0.26.69:44850 (1506 connections now open) 2016-03-13T09:37:03.403+0000 I NETWORK [conn4084546] end connection 10.0.26.69:44851 (1506 connections now open) 2016-03-13T09:37:03.404+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44865 #4084560 (1506 connections now open) 2016-03-13T09:37:03.404+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44864 #4084561 (1507 connections now open) 2016-03-13T09:37:06.097+0000 I NETWORK [conn4084547] end connection 10.0.27.237:38221 (1506 connections now open) 2016-03-13T09:37:06.097+0000 I NETWORK [conn4084548] end connection 10.0.27.237:38222 (1506 connections now open) 2016-03-13T09:37:06.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38235 #4084562 (1506 connections now open) 2016-03-13T09:37:06.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38236 #4084563 (1507 connections now open) 2016-03-13T09:37:08.334+0000 I NETWORK [conn4084552] end connection 10.0.7.113:33505 (1506 connections now open) 2016-03-13T09:37:08.334+0000 I NETWORK [conn4084553] end connection 10.0.7.113:33504 (1506 connections now open) 2016-03-13T09:37:08.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33517 #4084564 (1506 connections now open) 2016-03-13T09:37:08.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33516 #4084565 (1507 connections now open) 2016-03-13T09:37:09.296+0000 I NETWORK [conn4084554] end connection 10.0.216.90:59762 (1506 connections now open) 2016-03-13T09:37:09.296+0000 I NETWORK [conn4084555] end connection 10.0.216.90:59761 (1506 connections now open) 2016-03-13T09:37:09.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59770 #4084566 (1507 connections now open) 2016-03-13T09:37:09.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59771 #4084567 (1507 connections now open) 2016-03-13T09:37:10.369+0000 I NETWORK [conn4084556] end connection 10.0.20.53:53188 (1506 connections now open) 2016-03-13T09:37:10.369+0000 I NETWORK [conn4084557] end connection 10.0.20.53:53187 (1506 connections now open) 2016-03-13T09:37:10.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53200 #4084568 (1506 connections now open) 2016-03-13T09:37:10.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53201 #4084569 (1507 connections now open) 2016-03-13T09:37:10.863+0000 I SHARDING [conn1051657] ChunkManager: time to load chunks for mydomain.Sessions: 698ms sequenceNumber: 17423 version: 30723|1||563ba74d869758a4542b9075 based on: 30722|1||563ba74d869758a4542b9075 2016-03-13T09:37:12.075+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e534c718fa188537623de5 2016-03-13T09:37:12.772+0000 I NETWORK [conn4084559] end connection 10.0.179.209:42868 (1506 connections now open) 2016-03-13T09:37:12.772+0000 I NETWORK [conn4084558] end connection 10.0.179.209:42867 (1506 connections now open) 2016-03-13T09:37:12.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42874 #4084570 (1506 connections now open) 2016-03-13T09:37:12.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42875 #4084571 (1507 connections now open) 2016-03-13T09:37:13.417+0000 I NETWORK [conn4084560] end connection 10.0.26.69:44865 (1506 connections now open) 2016-03-13T09:37:13.417+0000 I NETWORK [conn4084561] end connection 10.0.26.69:44864 (1506 connections now open) 2016-03-13T09:37:13.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44885 #4084572 (1506 connections now open) 2016-03-13T09:37:13.418+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44886 #4084573 (1507 connections now open) 2016-03-13T09:37:15.128+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563507cc3694e50ec2bd5ce7')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563507cc3694e50ec2bd5ce7') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563508faf1ad750eb4739b6f') }, shard: "shmydomain1", lastmod: Timestamp 30723000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain3 tag [mydomainSessions] 2016-03-13T09:37:15.278+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30723|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563507cc3694e50ec2bd5ce7') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563508faf1ad750eb4739b6f') }) shmydomain1 -> shmydomain3 2016-03-13T09:37:16.110+0000 I NETWORK [conn4084562] end connection 10.0.27.237:38235 (1506 connections now open) 2016-03-13T09:37:16.110+0000 I NETWORK [conn4084563] end connection 10.0.27.237:38236 (1506 connections now open) 2016-03-13T09:37:16.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38254 #4084574 (1506 connections now open) 2016-03-13T09:37:16.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38255 #4084575 (1507 connections now open) 2016-03-13T09:37:16.116+0000 I NETWORK [conn4081317] end connection 10.0.20.53:50831 (1506 connections now open) 2016-03-13T09:37:16.117+0000 I NETWORK [conn4078036] end connection 10.0.216.90:54677 (1505 connections now open) 2016-03-13T09:37:16.117+0000 I NETWORK [conn4082612] end connection 10.0.7.113:60034 (1504 connections now open) 2016-03-13T09:37:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53207 #4084576 (1505 connections now open) 2016-03-13T09:37:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59782 #4084577 (1506 connections now open) 2016-03-13T09:37:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33529 #4084578 (1507 connections now open) 2016-03-13T09:37:18.347+0000 I NETWORK [conn4084564] end connection 10.0.7.113:33517 (1506 connections now open) 2016-03-13T09:37:18.347+0000 I NETWORK [conn4084565] end connection 10.0.7.113:33516 (1506 connections now open) 2016-03-13T09:37:18.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33531 #4084579 (1506 connections now open) 2016-03-13T09:37:18.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33532 #4084580 (1507 connections now open) 2016-03-13T09:37:19.310+0000 I NETWORK [conn4084567] end connection 10.0.216.90:59771 (1506 connections now open) 2016-03-13T09:37:19.310+0000 I NETWORK [conn4084566] end connection 10.0.216.90:59770 (1506 connections now open) 2016-03-13T09:37:19.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59785 #4084581 (1506 connections now open) 2016-03-13T09:37:19.311+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59784 #4084582 (1507 connections now open) 2016-03-13T09:37:20.382+0000 I NETWORK [conn4084569] end connection 10.0.20.53:53201 (1506 connections now open) 2016-03-13T09:37:20.382+0000 I NETWORK [conn4084568] end connection 10.0.20.53:53200 (1506 connections now open) 2016-03-13T09:37:20.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53221 #4084583 (1506 connections now open) 2016-03-13T09:37:20.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53220 #4084584 (1507 connections now open) 2016-03-13T09:37:22.785+0000 I NETWORK [conn4084571] end connection 10.0.179.209:42875 (1506 connections now open) 2016-03-13T09:37:22.785+0000 I NETWORK [conn4084570] end connection 10.0.179.209:42874 (1506 connections now open) 2016-03-13T09:37:22.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42889 #4084585 (1506 connections now open) 2016-03-13T09:37:22.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42890 #4084586 (1507 connections now open) 2016-03-13T09:37:23.430+0000 I NETWORK [conn4084572] end connection 10.0.26.69:44885 (1506 connections now open) 2016-03-13T09:37:23.430+0000 I NETWORK [conn4084573] end connection 10.0.26.69:44886 (1506 connections now open) 2016-03-13T09:37:23.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44896 #4084587 (1506 connections now open) 2016-03-13T09:37:23.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44895 #4084588 (1507 connections now open) 2016-03-13T09:37:25.285+0000 I SHARDING [conn1051647] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17424 version: 30724|1||563ba74d869758a4542b9075 based on: 30723|1||563ba74d869758a4542b9075 2016-03-13T09:37:25.486+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:37:25.486+0000-56e534d518fa188537623de6", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861845486), what: "balancer.round", ns: "", details: { executionTimeMillis: 13750, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:37:25.645+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:37:26.123+0000 I NETWORK [conn4084574] end connection 10.0.27.237:38254 (1506 connections now open) 2016-03-13T09:37:26.123+0000 I NETWORK [conn4084575] end connection 10.0.27.237:38255 (1506 connections now open) 2016-03-13T09:37:26.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38265 #4084589 (1506 connections now open) 2016-03-13T09:37:26.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38264 #4084590 (1507 connections now open) 2016-03-13T09:37:27.034+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' acquired for 'doing balance round', ts : 56e534d618fa188537623de7 2016-03-13T09:37:28.113+0000 I NETWORK [conn4082869] end connection 10.0.216.90:58204 (1506 connections now open) 2016-03-13T09:37:28.114+0000 I NETWORK [conn4081874] end connection 10.0.7.113:59776 (1505 connections now open) 2016-03-13T09:37:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59801 #4084591 (1506 connections now open) 2016-03-13T09:37:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33550 #4084592 (1507 connections now open) 2016-03-13T09:37:28.360+0000 I NETWORK [conn4084579] end connection 10.0.7.113:33531 (1506 connections now open) 2016-03-13T09:37:28.360+0000 I NETWORK [conn4084580] end connection 10.0.7.113:33532 (1506 connections now open) 2016-03-13T09:37:28.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33551 #4084593 (1506 connections now open) 2016-03-13T09:37:28.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33552 #4084594 (1507 connections now open) 2016-03-13T09:37:29.323+0000 I NETWORK [conn4084582] end connection 10.0.216.90:59784 (1506 connections now open) 2016-03-13T09:37:29.323+0000 I NETWORK [conn4084581] end connection 10.0.216.90:59785 (1506 connections now open) 2016-03-13T09:37:29.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59807 #4084595 (1506 connections now open) 2016-03-13T09:37:29.324+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59806 #4084596 (1507 connections now open) 2016-03-13T09:37:29.805+0000 I SHARDING [Balancer] ns: mydomain.Sessions going to move { _id: "mydomain.Sessions-a_ObjectId('533efb4e645cff1f6c911e5a')_id_ObjectId('563508faf1ad750eb4739b6f')", ns: "mydomain.Sessions", min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563508faf1ad750eb4739b6f') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56350a2da2a4330ea43589b7') }, shard: "shmydomain1", lastmod: Timestamp 30724000|1, lastmodEpoch: ObjectId('563ba74d869758a4542b9075') } from: shmydomain1 to: shmydomain2 tag [mydomainSessions] 2016-03-13T09:37:29.957+0000 I SHARDING [Balancer] moving chunk ns: mydomain.Sessions moving ( ns: mydomain.Sessions, shard: shmydomain1, lastmod: 30724|1||563ba74d869758a4542b9075, min: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('563508faf1ad750eb4739b6f') }, max: { a: ObjectId('533efb4e645cff1f6c911e5a'), _id: ObjectId('56350a2da2a4330ea43589b7') }) shmydomain1 -> shmydomain2 2016-03-13T09:37:30.396+0000 I NETWORK [conn4084583] end connection 10.0.20.53:53221 (1506 connections now open) 2016-03-13T09:37:30.396+0000 I NETWORK [conn4084584] end connection 10.0.20.53:53220 (1506 connections now open) 2016-03-13T09:37:30.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53235 #4084597 (1506 connections now open) 2016-03-13T09:37:30.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53234 #4084598 (1507 connections now open) 2016-03-13T09:37:32.087+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:37:30.362+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:37:32.087+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:37:32.798+0000 I NETWORK [conn4084586] end connection 10.0.179.209:42890 (1506 connections now open) 2016-03-13T09:37:32.798+0000 I NETWORK [conn4084585] end connection 10.0.179.209:42889 (1506 connections now open) 2016-03-13T09:37:32.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42909 #4084599 (1506 connections now open) 2016-03-13T09:37:32.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42910 #4084600 (1507 connections now open) 2016-03-13T09:37:33.443+0000 I NETWORK [conn4084587] end connection 10.0.26.69:44896 (1506 connections now open) 2016-03-13T09:37:33.443+0000 I NETWORK [conn4084588] end connection 10.0.26.69:44895 (1506 connections now open) 2016-03-13T09:37:33.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44910 #4084601 (1506 connections now open) 2016-03-13T09:37:33.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44909 #4084602 (1507 connections now open) 2016-03-13T09:37:33.493+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e534bd18fa188537623de4 2016-03-13T09:37:36.137+0000 I NETWORK [conn4084589] end connection 10.0.27.237:38265 (1506 connections now open) 2016-03-13T09:37:36.137+0000 I NETWORK [conn4084590] end connection 10.0.27.237:38264 (1506 connections now open) 2016-03-13T09:37:36.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38278 #4084603 (1506 connections now open) 2016-03-13T09:37:36.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38279 #4084604 (1507 connections now open) 2016-03-13T09:37:38.373+0000 I NETWORK [conn4084594] end connection 10.0.7.113:33552 (1506 connections now open) 2016-03-13T09:37:38.373+0000 I NETWORK [conn4084593] end connection 10.0.7.113:33551 (1506 connections now open) 2016-03-13T09:37:38.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33557 #4084605 (1506 connections now open) 2016-03-13T09:37:38.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33556 #4084606 (1507 connections now open) 2016-03-13T09:37:39.336+0000 I NETWORK [conn4084596] end connection 10.0.216.90:59806 (1506 connections now open) 2016-03-13T09:37:39.336+0000 I NETWORK [conn4084595] end connection 10.0.216.90:59807 (1505 connections now open) 2016-03-13T09:37:39.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59813 #4084607 (1506 connections now open) 2016-03-13T09:37:39.345+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59814 #4084608 (1507 connections now open) 2016-03-13T09:37:40.409+0000 I NETWORK [conn4084598] end connection 10.0.20.53:53234 (1506 connections now open) 2016-03-13T09:37:40.409+0000 I NETWORK [conn4084597] end connection 10.0.20.53:53235 (1506 connections now open) 2016-03-13T09:37:40.418+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53247 #4084609 (1506 connections now open) 2016-03-13T09:37:40.418+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53246 #4084610 (1507 connections now open) 2016-03-13T09:37:42.811+0000 I NETWORK [conn4084599] end connection 10.0.179.209:42909 (1506 connections now open) 2016-03-13T09:37:42.811+0000 I NETWORK [conn4084600] end connection 10.0.179.209:42910 (1506 connections now open) 2016-03-13T09:37:42.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42916 #4084611 (1506 connections now open) 2016-03-13T09:37:42.812+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42915 #4084612 (1507 connections now open) 2016-03-13T09:37:43.008+0000 I SHARDING [conn1051637] ChunkManager: time to load chunks for mydomain.Sessions: 696ms sequenceNumber: 17425 version: 30725|1||563ba74d869758a4542b9075 based on: 30724|1||563ba74d869758a4542b9075 2016-03-13T09:37:43.300+0000 I SHARDING [Balancer] about to log metadata event into actionlog: { _id: "ip-10-0-231-120-2016-03-13T09:37:43.300+0000-56e534e718fa188537623de8", server: "ip-10-0-231-120", clientAddr: "", time: new Date(1457861863300), what: "balancer.round", ns: "", details: { executionTimeMillis: 16591, errorOccured: false, candidateChunks: 1, chunksMoved: 1 } } 2016-03-13T09:37:43.463+0000 I NETWORK [conn4084601] end connection 10.0.26.69:44910 (1506 connections now open) 2016-03-13T09:37:43.463+0000 I NETWORK [conn4084602] end connection 10.0.26.69:44909 (1506 connections now open) 2016-03-13T09:37:43.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44927 #4084613 (1507 connections now open) 2016-03-13T09:37:43.464+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44928 #4084614 (1507 connections now open) 2016-03-13T09:37:43.481+0000 I SHARDING [Balancer] distributed lock 'balancer/ip-10-0-231-120:27017:1457539873:229444446' unlocked. 2016-03-13T09:37:46.117+0000 I NETWORK [conn4082064] end connection 10.0.216.90:57840 (1506 connections now open) 2016-03-13T09:37:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59824 #4084615 (1507 connections now open) 2016-03-13T09:37:46.150+0000 I NETWORK [conn4084603] end connection 10.0.27.237:38278 (1506 connections now open) 2016-03-13T09:37:46.150+0000 I NETWORK [conn4084604] end connection 10.0.27.237:38279 (1506 connections now open) 2016-03-13T09:37:46.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38299 #4084616 (1506 connections now open) 2016-03-13T09:37:46.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38300 #4084617 (1507 connections now open) 2016-03-13T09:37:48.396+0000 I NETWORK [conn4084605] end connection 10.0.7.113:33557 (1506 connections now open) 2016-03-13T09:37:48.396+0000 I NETWORK [conn4084606] end connection 10.0.7.113:33556 (1506 connections now open) 2016-03-13T09:37:48.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33570 #4084618 (1506 connections now open) 2016-03-13T09:37:48.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33569 #4084619 (1507 connections now open) 2016-03-13T09:37:49.358+0000 I NETWORK [conn4084608] end connection 10.0.216.90:59814 (1506 connections now open) 2016-03-13T09:37:49.358+0000 I NETWORK [conn4084607] end connection 10.0.216.90:59813 (1506 connections now open) 2016-03-13T09:37:49.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59827 #4084620 (1506 connections now open) 2016-03-13T09:37:49.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59826 #4084621 (1507 connections now open) 2016-03-13T09:37:50.430+0000 I NETWORK [conn4084610] end connection 10.0.20.53:53246 (1506 connections now open) 2016-03-13T09:37:50.430+0000 I NETWORK [conn4084609] end connection 10.0.20.53:53247 (1506 connections now open) 2016-03-13T09:37:50.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53267 #4084622 (1506 connections now open) 2016-03-13T09:37:50.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53268 #4084623 (1507 connections now open) 2016-03-13T09:37:52.824+0000 I NETWORK [conn4084611] end connection 10.0.179.209:42916 (1506 connections now open) 2016-03-13T09:37:52.824+0000 I NETWORK [conn4084612] end connection 10.0.179.209:42915 (1506 connections now open) 2016-03-13T09:37:52.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42929 #4084624 (1506 connections now open) 2016-03-13T09:37:52.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42930 #4084625 (1507 connections now open) 2016-03-13T09:37:53.477+0000 I NETWORK [conn4084613] end connection 10.0.26.69:44927 (1506 connections now open) 2016-03-13T09:37:53.477+0000 I NETWORK [conn4084614] end connection 10.0.26.69:44928 (1506 connections now open) 2016-03-13T09:37:53.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44937 #4084626 (1506 connections now open) 2016-03-13T09:37:53.478+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44938 #4084627 (1507 connections now open) 2016-03-13T09:37:56.171+0000 I NETWORK [conn4084616] end connection 10.0.27.237:38299 (1506 connections now open) 2016-03-13T09:37:56.171+0000 I NETWORK [conn4084617] end connection 10.0.27.237:38300 (1506 connections now open) 2016-03-13T09:37:56.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38307 #4084628 (1506 connections now open) 2016-03-13T09:37:56.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38308 #4084629 (1507 connections now open) 2016-03-13T09:37:58.114+0000 I NETWORK [conn4081461] end connection 10.0.27.237:35967 (1506 connections now open) 2016-03-13T09:37:58.114+0000 I NETWORK [conn4081551] end connection 10.0.26.69:42705 (1506 connections now open) 2016-03-13T09:37:58.114+0000 I NETWORK [conn4081409] end connection 10.0.179.209:40748 (1506 connections now open) 2016-03-13T09:37:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38317 #4084630 (1505 connections now open) 2016-03-13T09:37:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42945 #4084631 (1506 connections now open) 2016-03-13T09:37:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44943 #4084632 (1507 connections now open) 2016-03-13T09:37:58.409+0000 I NETWORK [conn4084619] end connection 10.0.7.113:33569 (1506 connections now open) 2016-03-13T09:37:58.409+0000 I NETWORK [conn4084618] end connection 10.0.7.113:33570 (1506 connections now open) 2016-03-13T09:37:58.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33590 #4084633 (1506 connections now open) 2016-03-13T09:37:58.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33589 #4084634 (1507 connections now open) 2016-03-13T09:37:59.371+0000 I NETWORK [conn4084620] end connection 10.0.216.90:59827 (1506 connections now open) 2016-03-13T09:37:59.371+0000 I NETWORK [conn4084621] end connection 10.0.216.90:59826 (1506 connections now open) 2016-03-13T09:37:59.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59846 #4084635 (1506 connections now open) 2016-03-13T09:37:59.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59847 #4084636 (1507 connections now open) 2016-03-13T09:38:00.444+0000 I NETWORK [conn4084622] end connection 10.0.20.53:53267 (1506 connections now open) 2016-03-13T09:38:00.444+0000 I NETWORK [conn4084623] end connection 10.0.20.53:53268 (1506 connections now open) 2016-03-13T09:38:00.444+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53276 #4084637 (1506 connections now open) 2016-03-13T09:38:00.444+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53277 #4084638 (1507 connections now open) 2016-03-13T09:38:02.846+0000 I NETWORK [conn4084625] end connection 10.0.179.209:42930 (1506 connections now open) 2016-03-13T09:38:02.846+0000 I NETWORK [conn4084624] end connection 10.0.179.209:42929 (1506 connections now open) 2016-03-13T09:38:02.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42951 #4084639 (1506 connections now open) 2016-03-13T09:38:02.847+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42952 #4084640 (1507 connections now open) 2016-03-13T09:38:03.490+0000 I NETWORK [conn4084626] end connection 10.0.26.69:44937 (1506 connections now open) 2016-03-13T09:38:03.490+0000 I NETWORK [conn4084627] end connection 10.0.26.69:44938 (1506 connections now open) 2016-03-13T09:38:03.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44952 #4084641 (1506 connections now open) 2016-03-13T09:38:03.491+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44953 #4084642 (1507 connections now open) 2016-03-13T09:38:03.781+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:38:03.493+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:38:03.781+0000 I SHARDING [LockPinger] trying to delete 1 old lock entries for process ip-10-0-231-120:27017:1457539873:229444446 2016-03-13T09:38:03.916+0000 I SHARDING [LockPinger] handled late remove of old distributed lock with ts 56e534e818fa188537623de9 2016-03-13T09:38:06.184+0000 I NETWORK [conn4084629] end connection 10.0.27.237:38308 (1506 connections now open) 2016-03-13T09:38:06.184+0000 I NETWORK [conn4084628] end connection 10.0.27.237:38307 (1506 connections now open) 2016-03-13T09:38:06.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38324 #4084643 (1506 connections now open) 2016-03-13T09:38:06.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38323 #4084644 (1507 connections now open) 2016-03-13T09:38:08.422+0000 I NETWORK [conn4084634] end connection 10.0.7.113:33589 (1506 connections now open) 2016-03-13T09:38:08.422+0000 I NETWORK [conn4084633] end connection 10.0.7.113:33590 (1506 connections now open) 2016-03-13T09:38:08.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33603 #4084645 (1506 connections now open) 2016-03-13T09:38:08.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33602 #4084646 (1507 connections now open) 2016-03-13T09:38:09.384+0000 I NETWORK [conn4084636] end connection 10.0.216.90:59847 (1506 connections now open) 2016-03-13T09:38:09.384+0000 I NETWORK [conn4084635] end connection 10.0.216.90:59846 (1506 connections now open) 2016-03-13T09:38:09.384+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59856 #4084647 (1506 connections now open) 2016-03-13T09:38:09.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59857 #4084648 (1507 connections now open) 2016-03-13T09:38:10.456+0000 I NETWORK [conn4084638] end connection 10.0.20.53:53277 (1506 connections now open) 2016-03-13T09:38:10.456+0000 I NETWORK [conn4084637] end connection 10.0.20.53:53276 (1506 connections now open) 2016-03-13T09:38:10.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53287 #4084649 (1506 connections now open) 2016-03-13T09:38:10.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53288 #4084650 (1507 connections now open) 2016-03-13T09:38:12.859+0000 I NETWORK [conn4084639] end connection 10.0.179.209:42951 (1506 connections now open) 2016-03-13T09:38:12.859+0000 I NETWORK [conn4084640] end connection 10.0.179.209:42952 (1506 connections now open) 2016-03-13T09:38:12.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42960 #4084651 (1506 connections now open) 2016-03-13T09:38:12.860+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42961 #4084652 (1507 connections now open) 2016-03-13T09:38:13.503+0000 I NETWORK [conn4084641] end connection 10.0.26.69:44952 (1506 connections now open) 2016-03-13T09:38:13.503+0000 I NETWORK [conn4084642] end connection 10.0.26.69:44953 (1505 connections now open) 2016-03-13T09:38:13.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44971 #4084653 (1507 connections now open) 2016-03-13T09:38:13.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44972 #4084654 (1507 connections now open) 2016-03-13T09:38:16.117+0000 I NETWORK [conn4082230] end connection 10.0.179.209:41005 (1506 connections now open) 2016-03-13T09:38:16.117+0000 I NETWORK [conn4077058] end connection 10.0.216.90:53645 (1505 connections now open) 2016-03-13T09:38:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42968 #4084655 (1506 connections now open) 2016-03-13T09:38:16.117+0000 I NETWORK [conn4081867] end connection 10.0.27.237:36171 (1505 connections now open) 2016-03-13T09:38:16.117+0000 I NETWORK [conn4083002] end connection 10.0.7.113:60215 (1504 connections now open) 2016-03-13T09:38:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59867 #4084656 (1505 connections now open) 2016-03-13T09:38:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38345 #4084657 (1506 connections now open) 2016-03-13T09:38:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33615 #4084658 (1507 connections now open) 2016-03-13T09:38:16.197+0000 I NETWORK [conn4084643] end connection 10.0.27.237:38324 (1506 connections now open) 2016-03-13T09:38:16.197+0000 I NETWORK [conn4084644] end connection 10.0.27.237:38323 (1506 connections now open) 2016-03-13T09:38:16.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38346 #4084659 (1506 connections now open) 2016-03-13T09:38:16.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38347 #4084660 (1507 connections now open) 2016-03-13T09:38:18.436+0000 I NETWORK [conn4084645] end connection 10.0.7.113:33603 (1506 connections now open) 2016-03-13T09:38:18.436+0000 I NETWORK [conn4084646] end connection 10.0.7.113:33602 (1506 connections now open) 2016-03-13T09:38:18.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33617 #4084661 (1506 connections now open) 2016-03-13T09:38:18.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33618 #4084662 (1507 connections now open) 2016-03-13T09:38:19.397+0000 I NETWORK [conn4084648] end connection 10.0.216.90:59857 (1506 connections now open) 2016-03-13T09:38:19.397+0000 I NETWORK [conn4084647] end connection 10.0.216.90:59856 (1506 connections now open) 2016-03-13T09:38:19.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59869 #4084663 (1506 connections now open) 2016-03-13T09:38:19.397+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59870 #4084664 (1507 connections now open) 2016-03-13T09:38:20.470+0000 I NETWORK [conn4084649] end connection 10.0.20.53:53287 (1506 connections now open) 2016-03-13T09:38:20.470+0000 I NETWORK [conn4084650] end connection 10.0.20.53:53288 (1506 connections now open) 2016-03-13T09:38:20.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53308 #4084665 (1506 connections now open) 2016-03-13T09:38:20.470+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53309 #4084666 (1507 connections now open) 2016-03-13T09:38:22.872+0000 I NETWORK [conn4084651] end connection 10.0.179.209:42960 (1506 connections now open) 2016-03-13T09:38:22.873+0000 I NETWORK [conn4084652] end connection 10.0.179.209:42961 (1505 connections now open) 2016-03-13T09:38:22.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42977 #4084667 (1506 connections now open) 2016-03-13T09:38:22.873+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42978 #4084668 (1507 connections now open) 2016-03-13T09:38:23.517+0000 I NETWORK [conn4084653] end connection 10.0.26.69:44971 (1506 connections now open) 2016-03-13T09:38:23.517+0000 I NETWORK [conn4084654] end connection 10.0.26.69:44972 (1506 connections now open) 2016-03-13T09:38:23.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44981 #4084669 (1506 connections now open) 2016-03-13T09:38:23.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44980 #4084670 (1507 connections now open) 2016-03-13T09:38:26.210+0000 I NETWORK [conn4084659] end connection 10.0.27.237:38346 (1506 connections now open) 2016-03-13T09:38:26.210+0000 I NETWORK [conn4084660] end connection 10.0.27.237:38347 (1506 connections now open) 2016-03-13T09:38:26.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38353 #4084671 (1506 connections now open) 2016-03-13T09:38:26.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38354 #4084672 (1507 connections now open) 2016-03-13T09:38:28.113+0000 I NETWORK [conn4081285] end connection 10.0.179.209:40723 (1506 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [conn4084093] end connection 10.0.20.53:52694 (1506 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [conn4082917] end connection 10.0.216.90:58232 (1504 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [conn4081649] end connection 10.0.26.69:42754 (1503 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [conn4081558] end connection 10.0.27.237:36015 (1502 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42992 #4084673 (1503 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59888 #4084674 (1504 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53316 #4084675 (1505 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44988 #4084676 (1506 connections now open) 2016-03-13T09:38:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38364 #4084677 (1507 connections now open) 2016-03-13T09:38:28.449+0000 I NETWORK [conn4084661] end connection 10.0.7.113:33617 (1506 connections now open) 2016-03-13T09:38:28.449+0000 I NETWORK [conn4084662] end connection 10.0.7.113:33618 (1506 connections now open) 2016-03-13T09:38:28.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33636 #4084678 (1506 connections now open) 2016-03-13T09:38:28.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33637 #4084679 (1507 connections now open) 2016-03-13T09:38:29.410+0000 I NETWORK [conn4084663] end connection 10.0.216.90:59869 (1506 connections now open) 2016-03-13T09:38:29.410+0000 I NETWORK [conn4084664] end connection 10.0.216.90:59870 (1506 connections now open) 2016-03-13T09:38:29.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59893 #4084680 (1506 connections now open) 2016-03-13T09:38:29.411+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59892 #4084681 (1507 connections now open) 2016-03-13T09:38:30.482+0000 I NETWORK [conn4084666] end connection 10.0.20.53:53309 (1506 connections now open) 2016-03-13T09:38:30.482+0000 I NETWORK [conn4084665] end connection 10.0.20.53:53308 (1505 connections now open) 2016-03-13T09:38:30.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53321 #4084682 (1506 connections now open) 2016-03-13T09:38:30.483+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53322 #4084683 (1507 connections now open) 2016-03-13T09:38:32.884+0000 I NETWORK [conn4084667] end connection 10.0.179.209:42977 (1506 connections now open) 2016-03-13T09:38:32.885+0000 I NETWORK [conn4084668] end connection 10.0.179.209:42978 (1505 connections now open) 2016-03-13T09:38:32.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42997 #4084684 (1506 connections now open) 2016-03-13T09:38:32.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:42998 #4084685 (1507 connections now open) 2016-03-13T09:38:33.530+0000 I NETWORK [conn4084669] end connection 10.0.26.69:44981 (1506 connections now open) 2016-03-13T09:38:33.530+0000 I NETWORK [conn4084670] end connection 10.0.26.69:44980 (1506 connections now open) 2016-03-13T09:38:33.540+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44996 #4084686 (1506 connections now open) 2016-03-13T09:38:33.540+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:44995 #4084687 (1507 connections now open) 2016-03-13T09:38:36.046+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:38:33.916+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:38:36.222+0000 I NETWORK [conn4084672] end connection 10.0.27.237:38354 (1506 connections now open) 2016-03-13T09:38:36.223+0000 I NETWORK [conn4084671] end connection 10.0.27.237:38353 (1505 connections now open) 2016-03-13T09:38:36.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38368 #4084688 (1506 connections now open) 2016-03-13T09:38:36.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38369 #4084689 (1507 connections now open) 2016-03-13T09:38:38.462+0000 I NETWORK [conn4084679] end connection 10.0.7.113:33637 (1506 connections now open) 2016-03-13T09:38:38.462+0000 I NETWORK [conn4084678] end connection 10.0.7.113:33636 (1506 connections now open) 2016-03-13T09:38:38.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33643 #4084690 (1506 connections now open) 2016-03-13T09:38:38.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33642 #4084691 (1507 connections now open) 2016-03-13T09:38:39.423+0000 I NETWORK [conn4084681] end connection 10.0.216.90:59892 (1506 connections now open) 2016-03-13T09:38:39.423+0000 I NETWORK [conn4084680] end connection 10.0.216.90:59893 (1506 connections now open) 2016-03-13T09:38:39.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59901 #4084692 (1506 connections now open) 2016-03-13T09:38:39.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59900 #4084693 (1507 connections now open) 2016-03-13T09:38:40.495+0000 I NETWORK [conn4084683] end connection 10.0.20.53:53322 (1506 connections now open) 2016-03-13T09:38:40.495+0000 I NETWORK [conn4084682] end connection 10.0.20.53:53321 (1506 connections now open) 2016-03-13T09:38:40.505+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53332 #4084694 (1506 connections now open) 2016-03-13T09:38:40.505+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53333 #4084695 (1507 connections now open) 2016-03-13T09:38:42.897+0000 I NETWORK [conn4084685] end connection 10.0.179.209:42998 (1506 connections now open) 2016-03-13T09:38:42.897+0000 I NETWORK [conn4084684] end connection 10.0.179.209:42997 (1506 connections now open) 2016-03-13T09:38:42.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43005 #4084696 (1506 connections now open) 2016-03-13T09:38:42.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43006 #4084697 (1507 connections now open) 2016-03-13T09:38:43.552+0000 I NETWORK [conn4084686] end connection 10.0.26.69:44996 (1506 connections now open) 2016-03-13T09:38:43.552+0000 I NETWORK [conn4084687] end connection 10.0.26.69:44995 (1506 connections now open) 2016-03-13T09:38:43.553+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45013 #4084698 (1507 connections now open) 2016-03-13T09:38:43.553+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45012 #4084699 (1507 connections now open) 2016-03-13T09:38:46.117+0000 I NETWORK [conn4076933] end connection 10.0.216.90:53509 (1506 connections now open) 2016-03-13T09:38:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59911 #4084700 (1507 connections now open) 2016-03-13T09:38:46.235+0000 I NETWORK [conn4084688] end connection 10.0.27.237:38368 (1506 connections now open) 2016-03-13T09:38:46.235+0000 I NETWORK [conn4084689] end connection 10.0.27.237:38369 (1505 connections now open) 2016-03-13T09:38:46.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38388 #4084701 (1506 connections now open) 2016-03-13T09:38:46.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38387 #4084702 (1507 connections now open) 2016-03-13T09:38:48.484+0000 I NETWORK [conn4084690] end connection 10.0.7.113:33643 (1506 connections now open) 2016-03-13T09:38:48.484+0000 I NETWORK [conn4084691] end connection 10.0.7.113:33642 (1505 connections now open) 2016-03-13T09:38:48.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33656 #4084703 (1506 connections now open) 2016-03-13T09:38:48.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33657 #4084704 (1507 connections now open) 2016-03-13T09:38:49.501+0000 I NETWORK [conn4084693] end connection 10.0.216.90:59900 (1506 connections now open) 2016-03-13T09:38:49.501+0000 I NETWORK [conn4084692] end connection 10.0.216.90:59901 (1505 connections now open) 2016-03-13T09:38:49.501+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59914 #4084705 (1506 connections now open) 2016-03-13T09:38:49.501+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59915 #4084706 (1507 connections now open) 2016-03-13T09:38:50.517+0000 I NETWORK [conn4084694] end connection 10.0.20.53:53332 (1506 connections now open) 2016-03-13T09:38:50.517+0000 I NETWORK [conn4084695] end connection 10.0.20.53:53333 (1506 connections now open) 2016-03-13T09:38:50.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53355 #4084707 (1506 connections now open) 2016-03-13T09:38:50.518+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53354 #4084708 (1507 connections now open) 2016-03-13T09:38:52.910+0000 I NETWORK [conn4084696] end connection 10.0.179.209:43005 (1506 connections now open) 2016-03-13T09:38:52.910+0000 I NETWORK [conn4084697] end connection 10.0.179.209:43006 (1506 connections now open) 2016-03-13T09:38:52.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43018 #4084709 (1506 connections now open) 2016-03-13T09:38:52.918+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43019 #4084710 (1507 connections now open) 2016-03-13T09:38:53.566+0000 I NETWORK [conn4084699] end connection 10.0.26.69:45012 (1506 connections now open) 2016-03-13T09:38:53.566+0000 I NETWORK [conn4084698] end connection 10.0.26.69:45013 (1506 connections now open) 2016-03-13T09:38:53.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45022 #4084711 (1506 connections now open) 2016-03-13T09:38:53.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45021 #4084712 (1507 connections now open) 2016-03-13T09:38:56.256+0000 I NETWORK [conn4084702] end connection 10.0.27.237:38387 (1506 connections now open) 2016-03-13T09:38:56.256+0000 I NETWORK [conn4084701] end connection 10.0.27.237:38388 (1506 connections now open) 2016-03-13T09:38:56.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38393 #4084713 (1506 connections now open) 2016-03-13T09:38:56.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38394 #4084714 (1507 connections now open) 2016-03-13T09:38:58.114+0000 I NETWORK [conn4083925] end connection 10.0.179.209:42215 (1506 connections now open) 2016-03-13T09:38:58.114+0000 I NETWORK [conn4083003] end connection 10.0.216.90:58263 (1506 connections now open) 2016-03-13T09:38:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59931 #4084715 (1506 connections now open) 2016-03-13T09:38:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43033 #4084716 (1507 connections now open) 2016-03-13T09:38:58.498+0000 I NETWORK [conn4084703] end connection 10.0.7.113:33656 (1506 connections now open) 2016-03-13T09:38:58.498+0000 I NETWORK [conn4084704] end connection 10.0.7.113:33657 (1506 connections now open) 2016-03-13T09:38:58.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33675 #4084717 (1506 connections now open) 2016-03-13T09:38:58.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33676 #4084718 (1507 connections now open) 2016-03-13T09:38:59.513+0000 I NETWORK [conn4084705] end connection 10.0.216.90:59914 (1506 connections now open) 2016-03-13T09:38:59.513+0000 I NETWORK [conn4084706] end connection 10.0.216.90:59915 (1505 connections now open) 2016-03-13T09:38:59.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59932 #4084719 (1506 connections now open) 2016-03-13T09:38:59.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59933 #4084720 (1507 connections now open) 2016-03-13T09:39:00.530+0000 I NETWORK [conn4084707] end connection 10.0.20.53:53355 (1506 connections now open) 2016-03-13T09:39:00.530+0000 I NETWORK [conn4084708] end connection 10.0.20.53:53354 (1506 connections now open) 2016-03-13T09:39:00.531+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53364 #4084721 (1506 connections now open) 2016-03-13T09:39:00.531+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53365 #4084722 (1507 connections now open) 2016-03-13T09:39:02.930+0000 I NETWORK [conn4084710] end connection 10.0.179.209:43019 (1506 connections now open) 2016-03-13T09:39:02.930+0000 I NETWORK [conn4084709] end connection 10.0.179.209:43018 (1506 connections now open) 2016-03-13T09:39:02.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43039 #4084723 (1506 connections now open) 2016-03-13T09:39:02.931+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43040 #4084724 (1507 connections now open) 2016-03-13T09:39:03.579+0000 I NETWORK [conn4084712] end connection 10.0.26.69:45021 (1506 connections now open) 2016-03-13T09:39:03.579+0000 I NETWORK [conn4084711] end connection 10.0.26.69:45022 (1506 connections now open) 2016-03-13T09:39:03.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45036 #4084725 (1506 connections now open) 2016-03-13T09:39:03.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45037 #4084726 (1507 connections now open) 2016-03-13T09:39:06.269+0000 I NETWORK [conn4084714] end connection 10.0.27.237:38394 (1506 connections now open) 2016-03-13T09:39:06.269+0000 I NETWORK [conn4084713] end connection 10.0.27.237:38393 (1506 connections now open) 2016-03-13T09:39:06.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38410 #4084727 (1506 connections now open) 2016-03-13T09:39:06.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38409 #4084728 (1507 connections now open) 2016-03-13T09:39:06.297+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:39:06.046+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:39:08.511+0000 I NETWORK [conn4084717] end connection 10.0.7.113:33675 (1506 connections now open) 2016-03-13T09:39:08.511+0000 I NETWORK [conn4084718] end connection 10.0.7.113:33676 (1505 connections now open) 2016-03-13T09:39:08.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33688 #4084729 (1506 connections now open) 2016-03-13T09:39:08.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33687 #4084730 (1507 connections now open) 2016-03-13T09:39:09.526+0000 I NETWORK [conn4084719] end connection 10.0.216.90:59932 (1506 connections now open) 2016-03-13T09:39:09.526+0000 I NETWORK [conn4084720] end connection 10.0.216.90:59933 (1505 connections now open) 2016-03-13T09:39:09.527+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59942 #4084731 (1506 connections now open) 2016-03-13T09:39:09.527+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59943 #4084732 (1507 connections now open) 2016-03-13T09:39:10.543+0000 I NETWORK [conn4084721] end connection 10.0.20.53:53364 (1506 connections now open) 2016-03-13T09:39:10.543+0000 I NETWORK [conn4084722] end connection 10.0.20.53:53365 (1505 connections now open) 2016-03-13T09:39:10.544+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53376 #4084733 (1506 connections now open) 2016-03-13T09:39:10.544+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53377 #4084734 (1507 connections now open) 2016-03-13T09:39:12.943+0000 I NETWORK [conn4084723] end connection 10.0.179.209:43039 (1506 connections now open) 2016-03-13T09:39:12.943+0000 I NETWORK [conn4084724] end connection 10.0.179.209:43040 (1506 connections now open) 2016-03-13T09:39:12.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43049 #4084735 (1506 connections now open) 2016-03-13T09:39:12.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43048 #4084736 (1507 connections now open) 2016-03-13T09:39:13.592+0000 I NETWORK [conn4084726] end connection 10.0.26.69:45037 (1506 connections now open) 2016-03-13T09:39:13.592+0000 I NETWORK [conn4084725] end connection 10.0.26.69:45036 (1506 connections now open) 2016-03-13T09:39:13.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45056 #4084737 (1507 connections now open) 2016-03-13T09:39:13.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45057 #4084738 (1507 connections now open) 2016-03-13T09:39:16.117+0000 I NETWORK [conn4081863] end connection 10.0.179.209:40844 (1506 connections now open) 2016-03-13T09:39:16.118+0000 I NETWORK [conn4083909] end connection 10.0.26.69:44214 (1505 connections now open) 2016-03-13T09:39:16.118+0000 I NETWORK [conn4081906] end connection 10.0.27.237:36184 (1504 connections now open) 2016-03-13T09:39:16.118+0000 I NETWORK [conn4082958] end connection 10.0.7.113:60180 (1503 connections now open) 2016-03-13T09:39:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43056 #4084739 (1504 connections now open) 2016-03-13T09:39:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45060 #4084740 (1505 connections now open) 2016-03-13T09:39:16.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38431 #4084741 (1506 connections now open) 2016-03-13T09:39:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33697 #4084742 (1507 connections now open) 2016-03-13T09:39:16.282+0000 I NETWORK [conn4084727] end connection 10.0.27.237:38410 (1506 connections now open) 2016-03-13T09:39:16.282+0000 I NETWORK [conn4084728] end connection 10.0.27.237:38409 (1506 connections now open) 2016-03-13T09:39:16.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38432 #4084743 (1506 connections now open) 2016-03-13T09:39:16.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38433 #4084744 (1507 connections now open) 2016-03-13T09:39:18.524+0000 I NETWORK [conn4084729] end connection 10.0.7.113:33688 (1506 connections now open) 2016-03-13T09:39:18.524+0000 I NETWORK [conn4084730] end connection 10.0.7.113:33687 (1506 connections now open) 2016-03-13T09:39:18.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33700 #4084745 (1506 connections now open) 2016-03-13T09:39:18.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33699 #4084746 (1507 connections now open) 2016-03-13T09:39:19.538+0000 I NETWORK [conn4084731] end connection 10.0.216.90:59942 (1506 connections now open) 2016-03-13T09:39:19.539+0000 I NETWORK [conn4084732] end connection 10.0.216.90:59943 (1505 connections now open) 2016-03-13T09:39:19.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59956 #4084747 (1506 connections now open) 2016-03-13T09:39:19.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59957 #4084748 (1507 connections now open) 2016-03-13T09:39:20.557+0000 I NETWORK [conn4084733] end connection 10.0.20.53:53376 (1506 connections now open) 2016-03-13T09:39:20.557+0000 I NETWORK [conn4084734] end connection 10.0.20.53:53377 (1506 connections now open) 2016-03-13T09:39:20.557+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53396 #4084749 (1506 connections now open) 2016-03-13T09:39:20.557+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53397 #4084750 (1507 connections now open) 2016-03-13T09:39:22.956+0000 I NETWORK [conn4084736] end connection 10.0.179.209:43048 (1506 connections now open) 2016-03-13T09:39:22.956+0000 I NETWORK [conn4084735] end connection 10.0.179.209:43049 (1506 connections now open) 2016-03-13T09:39:22.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43064 #4084751 (1506 connections now open) 2016-03-13T09:39:22.957+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43065 #4084752 (1507 connections now open) 2016-03-13T09:39:23.605+0000 I NETWORK [conn4084738] end connection 10.0.26.69:45057 (1506 connections now open) 2016-03-13T09:39:23.605+0000 I NETWORK [conn4084737] end connection 10.0.26.69:45056 (1506 connections now open) 2016-03-13T09:39:23.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45064 #4084753 (1506 connections now open) 2016-03-13T09:39:23.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45065 #4084754 (1507 connections now open) 2016-03-13T09:39:26.296+0000 I NETWORK [conn4084743] end connection 10.0.27.237:38432 (1506 connections now open) 2016-03-13T09:39:26.296+0000 I NETWORK [conn4084744] end connection 10.0.27.237:38433 (1506 connections now open) 2016-03-13T09:39:26.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38439 #4084755 (1506 connections now open) 2016-03-13T09:39:26.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38440 #4084756 (1507 connections now open) 2016-03-13T09:39:28.113+0000 I NETWORK [conn4083419] end connection 10.0.179.209:41698 (1506 connections now open) 2016-03-13T09:39:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43079 #4084757 (1507 connections now open) 2016-03-13T09:39:28.114+0000 I NETWORK [conn4081525] end connection 10.0.27.237:36000 (1506 connections now open) 2016-03-13T09:39:28.114+0000 I NETWORK [conn4081144] end connection 10.0.7.113:59590 (1506 connections now open) 2016-03-13T09:39:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33718 #4084758 (1506 connections now open) 2016-03-13T09:39:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38450 #4084759 (1507 connections now open) 2016-03-13T09:39:28.537+0000 I NETWORK [conn4084746] end connection 10.0.7.113:33699 (1506 connections now open) 2016-03-13T09:39:28.537+0000 I NETWORK [conn4084745] end connection 10.0.7.113:33700 (1505 connections now open) 2016-03-13T09:39:28.537+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33719 #4084760 (1506 connections now open) 2016-03-13T09:39:28.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33720 #4084761 (1507 connections now open) 2016-03-13T09:39:29.559+0000 I NETWORK [conn4084747] end connection 10.0.216.90:59956 (1506 connections now open) 2016-03-13T09:39:29.559+0000 I NETWORK [conn4084748] end connection 10.0.216.90:59957 (1505 connections now open) 2016-03-13T09:39:29.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59980 #4084762 (1506 connections now open) 2016-03-13T09:39:29.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59979 #4084763 (1507 connections now open) 2016-03-13T09:39:30.570+0000 I NETWORK [conn4084749] end connection 10.0.20.53:53396 (1506 connections now open) 2016-03-13T09:39:30.570+0000 I NETWORK [conn4084750] end connection 10.0.20.53:53397 (1505 connections now open) 2016-03-13T09:39:30.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53410 #4084764 (1506 connections now open) 2016-03-13T09:39:30.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53409 #4084765 (1507 connections now open) 2016-03-13T09:39:32.969+0000 I NETWORK [conn4084752] end connection 10.0.179.209:43065 (1506 connections now open) 2016-03-13T09:39:32.969+0000 I NETWORK [conn4084751] end connection 10.0.179.209:43064 (1506 connections now open) 2016-03-13T09:39:32.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43084 #4084766 (1506 connections now open) 2016-03-13T09:39:32.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43085 #4084767 (1507 connections now open) 2016-03-13T09:39:33.618+0000 I NETWORK [conn4084754] end connection 10.0.26.69:45065 (1506 connections now open) 2016-03-13T09:39:33.618+0000 I NETWORK [conn4084753] end connection 10.0.26.69:45064 (1506 connections now open) 2016-03-13T09:39:33.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45078 #4084768 (1506 connections now open) 2016-03-13T09:39:33.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45079 #4084769 (1507 connections now open) 2016-03-13T09:39:36.309+0000 I NETWORK [conn4084755] end connection 10.0.27.237:38439 (1506 connections now open) 2016-03-13T09:39:36.309+0000 I NETWORK [conn4084756] end connection 10.0.27.237:38440 (1506 connections now open) 2016-03-13T09:39:36.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38454 #4084770 (1506 connections now open) 2016-03-13T09:39:36.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38455 #4084771 (1507 connections now open) 2016-03-13T09:39:36.506+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:39:36.297+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:39:38.550+0000 I NETWORK [conn4084760] end connection 10.0.7.113:33719 (1506 connections now open) 2016-03-13T09:39:38.550+0000 I NETWORK [conn4084761] end connection 10.0.7.113:33720 (1506 connections now open) 2016-03-13T09:39:38.557+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33726 #4084772 (1506 connections now open) 2016-03-13T09:39:38.557+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33727 #4084773 (1507 connections now open) 2016-03-13T09:39:39.572+0000 I NETWORK [conn4084763] end connection 10.0.216.90:59979 (1506 connections now open) 2016-03-13T09:39:39.572+0000 I NETWORK [conn4084762] end connection 10.0.216.90:59980 (1506 connections now open) 2016-03-13T09:39:39.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59986 #4084774 (1506 connections now open) 2016-03-13T09:39:39.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59985 #4084775 (1507 connections now open) 2016-03-13T09:39:40.583+0000 I NETWORK [conn4084764] end connection 10.0.20.53:53410 (1506 connections now open) 2016-03-13T09:39:40.583+0000 I NETWORK [conn4084765] end connection 10.0.20.53:53409 (1506 connections now open) 2016-03-13T09:39:40.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53421 #4084776 (1506 connections now open) 2016-03-13T09:39:40.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53420 #4084777 (1507 connections now open) 2016-03-13T09:39:42.982+0000 I NETWORK [conn4084767] end connection 10.0.179.209:43085 (1506 connections now open) 2016-03-13T09:39:42.982+0000 I NETWORK [conn4084766] end connection 10.0.179.209:43084 (1506 connections now open) 2016-03-13T09:39:42.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43093 #4084778 (1506 connections now open) 2016-03-13T09:39:42.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43094 #4084779 (1507 connections now open) 2016-03-13T09:39:43.640+0000 I NETWORK [conn4084768] end connection 10.0.26.69:45078 (1506 connections now open) 2016-03-13T09:39:43.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45096 #4084780 (1508 connections now open) 2016-03-13T09:39:43.640+0000 I NETWORK [conn4084769] end connection 10.0.26.69:45079 (1506 connections now open) 2016-03-13T09:39:43.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45095 #4084781 (1507 connections now open) 2016-03-13T09:39:46.119+0000 I NETWORK [conn4082149] end connection 10.0.179.209:40967 (1506 connections now open) 2016-03-13T09:39:46.119+0000 I NETWORK [conn4082704] end connection 10.0.27.237:36528 (1506 connections now open) 2016-03-13T09:39:46.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43102 #4084782 (1506 connections now open) 2016-03-13T09:39:46.128+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38473 #4084783 (1507 connections now open) 2016-03-13T09:39:46.323+0000 I NETWORK [conn4084770] end connection 10.0.27.237:38454 (1506 connections now open) 2016-03-13T09:39:46.323+0000 I NETWORK [conn4084771] end connection 10.0.27.237:38455 (1506 connections now open) 2016-03-13T09:39:46.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38474 #4084784 (1506 connections now open) 2016-03-13T09:39:46.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38475 #4084785 (1507 connections now open) 2016-03-13T09:39:48.570+0000 I NETWORK [conn4084772] end connection 10.0.7.113:33726 (1506 connections now open) 2016-03-13T09:39:48.570+0000 I NETWORK [conn4084773] end connection 10.0.7.113:33727 (1506 connections now open) 2016-03-13T09:39:48.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33741 #4084786 (1506 connections now open) 2016-03-13T09:39:48.571+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33740 #4084787 (1507 connections now open) 2016-03-13T09:39:49.593+0000 I NETWORK [conn4084775] end connection 10.0.216.90:59985 (1506 connections now open) 2016-03-13T09:39:49.593+0000 I NETWORK [conn4084774] end connection 10.0.216.90:59986 (1506 connections now open) 2016-03-13T09:39:49.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60000 #4084788 (1506 connections now open) 2016-03-13T09:39:49.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:59999 #4084789 (1507 connections now open) 2016-03-13T09:39:50.598+0000 I NETWORK [conn4084777] end connection 10.0.20.53:53420 (1506 connections now open) 2016-03-13T09:39:50.598+0000 I NETWORK [conn4084776] end connection 10.0.20.53:53421 (1506 connections now open) 2016-03-13T09:39:50.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53440 #4084790 (1506 connections now open) 2016-03-13T09:39:50.598+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53441 #4084791 (1507 connections now open) 2016-03-13T09:39:52.996+0000 I NETWORK [conn4084778] end connection 10.0.179.209:43093 (1506 connections now open) 2016-03-13T09:39:52.996+0000 I NETWORK [conn4084779] end connection 10.0.179.209:43094 (1506 connections now open) 2016-03-13T09:39:52.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43109 #4084792 (1506 connections now open) 2016-03-13T09:39:52.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43108 #4084793 (1507 connections now open) 2016-03-13T09:39:53.654+0000 I NETWORK [conn4084781] end connection 10.0.26.69:45095 (1506 connections now open) 2016-03-13T09:39:53.654+0000 I NETWORK [conn4084780] end connection 10.0.26.69:45096 (1506 connections now open) 2016-03-13T09:39:53.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45104 #4084794 (1506 connections now open) 2016-03-13T09:39:53.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45105 #4084795 (1507 connections now open) 2016-03-13T09:39:56.335+0000 I NETWORK [conn4084784] end connection 10.0.27.237:38474 (1506 connections now open) 2016-03-13T09:39:56.335+0000 I NETWORK [conn4084785] end connection 10.0.27.237:38475 (1506 connections now open) 2016-03-13T09:39:56.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38480 #4084796 (1506 connections now open) 2016-03-13T09:39:56.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38481 #4084797 (1507 connections now open) 2016-03-13T09:39:58.113+0000 I NETWORK [conn4081254] end connection 10.0.179.209:40716 (1506 connections now open) 2016-03-13T09:39:58.113+0000 I NETWORK [conn4081128] end connection 10.0.20.53:50764 (1505 connections now open) 2016-03-13T09:39:58.113+0000 I NETWORK [conn4080880] end connection 10.0.26.69:42426 (1504 connections now open) 2016-03-13T09:39:58.114+0000 I NETWORK [conn4081589] end connection 10.0.7.113:59701 (1503 connections now open) 2016-03-13T09:39:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53447 #4084798 (1504 connections now open) 2016-03-13T09:39:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45112 #4084799 (1505 connections now open) 2016-03-13T09:39:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33759 #4084800 (1506 connections now open) 2016-03-13T09:39:58.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43124 #4084801 (1507 connections now open) 2016-03-13T09:39:58.584+0000 I NETWORK [conn4084786] end connection 10.0.7.113:33741 (1506 connections now open) 2016-03-13T09:39:58.584+0000 I NETWORK [conn4084787] end connection 10.0.7.113:33740 (1506 connections now open) 2016-03-13T09:39:58.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33761 #4084802 (1506 connections now open) 2016-03-13T09:39:58.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33760 #4084803 (1507 connections now open) 2016-03-13T09:39:59.606+0000 I NETWORK [conn4084788] end connection 10.0.216.90:60000 (1506 connections now open) 2016-03-13T09:39:59.606+0000 I NETWORK [conn4084789] end connection 10.0.216.90:59999 (1506 connections now open) 2016-03-13T09:39:59.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60019 #4084804 (1506 connections now open) 2016-03-13T09:39:59.607+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60018 #4084805 (1507 connections now open) 2016-03-13T09:40:00.611+0000 I NETWORK [conn4084791] end connection 10.0.20.53:53441 (1506 connections now open) 2016-03-13T09:40:00.611+0000 I NETWORK [conn4084790] end connection 10.0.20.53:53440 (1506 connections now open) 2016-03-13T09:40:00.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53450 #4084806 (1506 connections now open) 2016-03-13T09:40:00.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53449 #4084807 (1507 connections now open) 2016-03-13T09:40:03.009+0000 I NETWORK [conn4084793] end connection 10.0.179.209:43108 (1506 connections now open) 2016-03-13T09:40:03.009+0000 I NETWORK [conn4084792] end connection 10.0.179.209:43109 (1506 connections now open) 2016-03-13T09:40:03.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43133 #4084808 (1506 connections now open) 2016-03-13T09:40:03.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43132 #4084809 (1507 connections now open) 2016-03-13T09:40:03.666+0000 I NETWORK [conn4084795] end connection 10.0.26.69:45105 (1506 connections now open) 2016-03-13T09:40:03.666+0000 I NETWORK [conn4084794] end connection 10.0.26.69:45104 (1506 connections now open) 2016-03-13T09:40:03.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45122 #4084810 (1506 connections now open) 2016-03-13T09:40:03.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45123 #4084811 (1507 connections now open) 2016-03-13T09:40:06.349+0000 I NETWORK [conn4084796] end connection 10.0.27.237:38480 (1506 connections now open) 2016-03-13T09:40:06.349+0000 I NETWORK [conn4084797] end connection 10.0.27.237:38481 (1506 connections now open) 2016-03-13T09:40:06.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38500 #4084812 (1506 connections now open) 2016-03-13T09:40:06.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38499 #4084813 (1507 connections now open) 2016-03-13T09:40:06.665+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:40:06.506+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:40:08.598+0000 I NETWORK [conn4084803] end connection 10.0.7.113:33760 (1506 connections now open) 2016-03-13T09:40:08.598+0000 I NETWORK [conn4084802] end connection 10.0.7.113:33761 (1506 connections now open) 2016-03-13T09:40:08.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33774 #4084814 (1506 connections now open) 2016-03-13T09:40:08.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33775 #4084815 (1507 connections now open) 2016-03-13T09:40:09.619+0000 I NETWORK [conn4084804] end connection 10.0.216.90:60019 (1506 connections now open) 2016-03-13T09:40:09.619+0000 I NETWORK [conn4084805] end connection 10.0.216.90:60018 (1506 connections now open) 2016-03-13T09:40:09.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60028 #4084816 (1506 connections now open) 2016-03-13T09:40:09.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60029 #4084817 (1507 connections now open) 2016-03-13T09:40:10.624+0000 I NETWORK [conn4084806] end connection 10.0.20.53:53450 (1506 connections now open) 2016-03-13T09:40:10.624+0000 I NETWORK [conn4084807] end connection 10.0.20.53:53449 (1505 connections now open) 2016-03-13T09:40:10.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53464 #4084818 (1506 connections now open) 2016-03-13T09:40:10.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53465 #4084819 (1507 connections now open) 2016-03-13T09:40:13.022+0000 I NETWORK [conn4084809] end connection 10.0.179.209:43132 (1506 connections now open) 2016-03-13T09:40:13.022+0000 I NETWORK [conn4084808] end connection 10.0.179.209:43133 (1506 connections now open) 2016-03-13T09:40:13.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43140 #4084820 (1506 connections now open) 2016-03-13T09:40:13.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43141 #4084821 (1507 connections now open) 2016-03-13T09:40:13.679+0000 I NETWORK [conn4084811] end connection 10.0.26.69:45123 (1506 connections now open) 2016-03-13T09:40:13.679+0000 I NETWORK [conn4084810] end connection 10.0.26.69:45122 (1506 connections now open) 2016-03-13T09:40:13.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45142 #4084822 (1507 connections now open) 2016-03-13T09:40:13.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45143 #4084823 (1507 connections now open) 2016-03-13T09:40:16.116+0000 I NETWORK [conn4083072] end connection 10.0.20.53:51630 (1506 connections now open) 2016-03-13T09:40:16.117+0000 I NETWORK [conn4081800] end connection 10.0.216.90:57719 (1505 connections now open) 2016-03-13T09:40:16.117+0000 I NETWORK [conn4081960] end connection 10.0.27.237:36207 (1504 connections now open) 2016-03-13T09:40:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53471 #4084824 (1505 connections now open) 2016-03-13T09:40:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60041 #4084825 (1506 connections now open) 2016-03-13T09:40:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38523 #4084826 (1507 connections now open) 2016-03-13T09:40:16.362+0000 I NETWORK [conn4084812] end connection 10.0.27.237:38500 (1506 connections now open) 2016-03-13T09:40:16.362+0000 I NETWORK [conn4084813] end connection 10.0.27.237:38499 (1506 connections now open) 2016-03-13T09:40:16.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38525 #4084827 (1506 connections now open) 2016-03-13T09:40:16.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38524 #4084828 (1507 connections now open) 2016-03-13T09:40:18.611+0000 I NETWORK [conn4084815] end connection 10.0.7.113:33775 (1506 connections now open) 2016-03-13T09:40:18.612+0000 I NETWORK [conn4084814] end connection 10.0.7.113:33774 (1505 connections now open) 2016-03-13T09:40:18.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33787 #4084829 (1506 connections now open) 2016-03-13T09:40:18.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33788 #4084830 (1507 connections now open) 2016-03-13T09:40:19.633+0000 I NETWORK [conn4084817] end connection 10.0.216.90:60029 (1506 connections now open) 2016-03-13T09:40:19.633+0000 I NETWORK [conn4084816] end connection 10.0.216.90:60028 (1506 connections now open) 2016-03-13T09:40:19.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60044 #4084831 (1506 connections now open) 2016-03-13T09:40:19.633+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60043 #4084832 (1507 connections now open) 2016-03-13T09:40:20.637+0000 I NETWORK [conn4084818] end connection 10.0.20.53:53464 (1506 connections now open) 2016-03-13T09:40:20.637+0000 I NETWORK [conn4084819] end connection 10.0.20.53:53465 (1506 connections now open) 2016-03-13T09:40:20.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53484 #4084833 (1506 connections now open) 2016-03-13T09:40:20.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53485 #4084834 (1507 connections now open) 2016-03-13T09:40:23.035+0000 I NETWORK [conn4084821] end connection 10.0.179.209:43141 (1506 connections now open) 2016-03-13T09:40:23.035+0000 I NETWORK [conn4084820] end connection 10.0.179.209:43140 (1506 connections now open) 2016-03-13T09:40:23.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43156 #4084835 (1506 connections now open) 2016-03-13T09:40:23.036+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43157 #4084836 (1507 connections now open) 2016-03-13T09:40:23.693+0000 I NETWORK [conn4084822] end connection 10.0.26.69:45142 (1506 connections now open) 2016-03-13T09:40:23.693+0000 I NETWORK [conn4084823] end connection 10.0.26.69:45143 (1506 connections now open) 2016-03-13T09:40:23.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45150 #4084837 (1506 connections now open) 2016-03-13T09:40:23.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45151 #4084838 (1507 connections now open) 2016-03-13T09:40:26.376+0000 I NETWORK [conn4084827] end connection 10.0.27.237:38525 (1506 connections now open) 2016-03-13T09:40:26.376+0000 I NETWORK [conn4084828] end connection 10.0.27.237:38524 (1506 connections now open) 2016-03-13T09:40:26.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38533 #4084839 (1506 connections now open) 2016-03-13T09:40:26.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38532 #4084840 (1507 connections now open) 2016-03-13T09:40:28.116+0000 I NETWORK [conn4082142] end connection 10.0.179.209:40964 (1506 connections now open) 2016-03-13T09:40:28.116+0000 I NETWORK [conn4082729] end connection 10.0.27.237:36540 (1505 connections now open) 2016-03-13T09:40:28.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43171 #4084841 (1506 connections now open) 2016-03-13T09:40:28.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38544 #4084842 (1507 connections now open) 2016-03-13T09:40:28.625+0000 I NETWORK [conn4084830] end connection 10.0.7.113:33788 (1506 connections now open) 2016-03-13T09:40:28.625+0000 I NETWORK [conn4084829] end connection 10.0.7.113:33787 (1506 connections now open) 2016-03-13T09:40:28.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33805 #4084843 (1506 connections now open) 2016-03-13T09:40:28.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33806 #4084844 (1507 connections now open) 2016-03-13T09:40:29.645+0000 I NETWORK [conn4084832] end connection 10.0.216.90:60043 (1506 connections now open) 2016-03-13T09:40:29.646+0000 I NETWORK [conn4084831] end connection 10.0.216.90:60044 (1505 connections now open) 2016-03-13T09:40:29.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60065 #4084845 (1506 connections now open) 2016-03-13T09:40:29.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60066 #4084846 (1507 connections now open) 2016-03-13T09:40:30.650+0000 I NETWORK [conn4084834] end connection 10.0.20.53:53485 (1506 connections now open) 2016-03-13T09:40:30.650+0000 I NETWORK [conn4084833] end connection 10.0.20.53:53484 (1506 connections now open) 2016-03-13T09:40:30.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53496 #4084847 (1506 connections now open) 2016-03-13T09:40:30.651+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53497 #4084848 (1507 connections now open) 2016-03-13T09:40:33.048+0000 I NETWORK [conn4084836] end connection 10.0.179.209:43157 (1506 connections now open) 2016-03-13T09:40:33.048+0000 I NETWORK [conn4084835] end connection 10.0.179.209:43156 (1506 connections now open) 2016-03-13T09:40:33.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43175 #4084849 (1506 connections now open) 2016-03-13T09:40:33.049+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43176 #4084850 (1507 connections now open) 2016-03-13T09:40:33.706+0000 I NETWORK [conn4084837] end connection 10.0.26.69:45150 (1506 connections now open) 2016-03-13T09:40:33.706+0000 I NETWORK [conn4084838] end connection 10.0.26.69:45151 (1506 connections now open) 2016-03-13T09:40:33.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45164 #4084851 (1506 connections now open) 2016-03-13T09:40:33.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45163 #4084852 (1507 connections now open) 2016-03-13T09:40:36.389+0000 I NETWORK [conn4084839] end connection 10.0.27.237:38533 (1506 connections now open) 2016-03-13T09:40:36.389+0000 I NETWORK [conn4084840] end connection 10.0.27.237:38532 (1506 connections now open) 2016-03-13T09:40:36.389+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38548 #4084853 (1506 connections now open) 2016-03-13T09:40:36.390+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38547 #4084854 (1507 connections now open) 2016-03-13T09:40:36.976+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:40:36.665+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:40:38.639+0000 I NETWORK [conn4084844] end connection 10.0.7.113:33806 (1506 connections now open) 2016-03-13T09:40:38.639+0000 I NETWORK [conn4084843] end connection 10.0.7.113:33805 (1506 connections now open) 2016-03-13T09:40:38.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33813 #4084855 (1506 connections now open) 2016-03-13T09:40:38.648+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33812 #4084856 (1507 connections now open) 2016-03-13T09:40:39.658+0000 I NETWORK [conn4084846] end connection 10.0.216.90:60066 (1506 connections now open) 2016-03-13T09:40:39.658+0000 I NETWORK [conn4084845] end connection 10.0.216.90:60065 (1506 connections now open) 2016-03-13T09:40:39.668+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60073 #4084857 (1506 connections now open) 2016-03-13T09:40:39.668+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60072 #4084858 (1507 connections now open) 2016-03-13T09:40:40.664+0000 I NETWORK [conn4084848] end connection 10.0.20.53:53497 (1506 connections now open) 2016-03-13T09:40:40.664+0000 I NETWORK [conn4084847] end connection 10.0.20.53:53496 (1506 connections now open) 2016-03-13T09:40:40.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53507 #4084859 (1506 connections now open) 2016-03-13T09:40:40.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53508 #4084860 (1507 connections now open) 2016-03-13T09:40:43.061+0000 I NETWORK [conn4084849] end connection 10.0.179.209:43175 (1506 connections now open) 2016-03-13T09:40:43.061+0000 I NETWORK [conn4084850] end connection 10.0.179.209:43176 (1506 connections now open) 2016-03-13T09:40:43.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43183 #4084861 (1506 connections now open) 2016-03-13T09:40:43.062+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43184 #4084862 (1507 connections now open) 2016-03-13T09:40:43.720+0000 I NETWORK [conn4084852] end connection 10.0.26.69:45163 (1506 connections now open) 2016-03-13T09:40:43.721+0000 I NETWORK [conn4084851] end connection 10.0.26.69:45164 (1505 connections now open) 2016-03-13T09:40:43.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45181 #4084863 (1506 connections now open) 2016-03-13T09:40:43.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45182 #4084864 (1507 connections now open) 2016-03-13T09:40:46.116+0000 I NETWORK [conn4081775] end connection 10.0.179.209:40796 (1506 connections now open) 2016-03-13T09:40:46.117+0000 I NETWORK [conn4081999] end connection 10.0.26.69:42926 (1505 connections now open) 2016-03-13T09:40:46.117+0000 I NETWORK [conn4082916] end connection 10.0.27.237:36630 (1504 connections now open) 2016-03-13T09:40:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43188 #4084865 (1505 connections now open) 2016-03-13T09:40:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45185 #4084866 (1506 connections now open) 2016-03-13T09:40:46.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38568 #4084867 (1507 connections now open) 2016-03-13T09:40:46.402+0000 I NETWORK [conn4084853] end connection 10.0.27.237:38548 (1506 connections now open) 2016-03-13T09:40:46.402+0000 I NETWORK [conn4084854] end connection 10.0.27.237:38547 (1506 connections now open) 2016-03-13T09:40:46.403+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38569 #4084868 (1506 connections now open) 2016-03-13T09:40:46.403+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38570 #4084869 (1507 connections now open) 2016-03-13T09:40:48.661+0000 I NETWORK [conn4084855] end connection 10.0.7.113:33813 (1506 connections now open) 2016-03-13T09:40:48.661+0000 I NETWORK [conn4084856] end connection 10.0.7.113:33812 (1506 connections now open) 2016-03-13T09:40:48.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33827 #4084870 (1506 connections now open) 2016-03-13T09:40:48.661+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33826 #4084871 (1507 connections now open) 2016-03-13T09:40:49.680+0000 I NETWORK [conn4084857] end connection 10.0.216.90:60073 (1506 connections now open) 2016-03-13T09:40:49.680+0000 I NETWORK [conn4084858] end connection 10.0.216.90:60072 (1505 connections now open) 2016-03-13T09:40:49.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60086 #4084872 (1506 connections now open) 2016-03-13T09:40:49.681+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60087 #4084873 (1507 connections now open) 2016-03-13T09:40:50.685+0000 I NETWORK [conn4084859] end connection 10.0.20.53:53507 (1506 connections now open) 2016-03-13T09:40:50.685+0000 I NETWORK [conn4084860] end connection 10.0.20.53:53508 (1506 connections now open) 2016-03-13T09:40:50.686+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53528 #4084874 (1506 connections now open) 2016-03-13T09:40:50.686+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53529 #4084875 (1507 connections now open) 2016-03-13T09:40:53.074+0000 I NETWORK [conn4084861] end connection 10.0.179.209:43183 (1506 connections now open) 2016-03-13T09:40:53.074+0000 I NETWORK [conn4084862] end connection 10.0.179.209:43184 (1506 connections now open) 2016-03-13T09:40:53.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43198 #4084876 (1506 connections now open) 2016-03-13T09:40:53.075+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43199 #4084877 (1507 connections now open) 2016-03-13T09:40:53.734+0000 I NETWORK [conn4084864] end connection 10.0.26.69:45182 (1506 connections now open) 2016-03-13T09:40:53.734+0000 I NETWORK [conn4084863] end connection 10.0.26.69:45181 (1505 connections now open) 2016-03-13T09:40:53.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45191 #4084878 (1506 connections now open) 2016-03-13T09:40:53.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45192 #4084879 (1507 connections now open) 2016-03-13T09:40:56.415+0000 I NETWORK [conn4084869] end connection 10.0.27.237:38570 (1506 connections now open) 2016-03-13T09:40:56.415+0000 I NETWORK [conn4084868] end connection 10.0.27.237:38569 (1506 connections now open) 2016-03-13T09:40:56.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38578 #4084880 (1506 connections now open) 2016-03-13T09:40:56.416+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38577 #4084881 (1507 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [conn4081947] end connection 10.0.20.53:51156 (1506 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [conn4082774] end connection 10.0.216.90:58149 (1505 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [conn4079387] end connection 10.0.27.237:34349 (1505 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [conn4081395] end connection 10.0.26.69:42620 (1503 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [conn4083018] end connection 10.0.7.113:60221 (1502 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53537 #4084882 (1503 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38589 #4084883 (1504 connections now open) 2016-03-13T09:40:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60103 #4084884 (1505 connections now open) 2016-03-13T09:40:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45198 #4084885 (1506 connections now open) 2016-03-13T09:40:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33844 #4084886 (1507 connections now open) 2016-03-13T09:40:58.674+0000 I NETWORK [conn4084871] end connection 10.0.7.113:33826 (1506 connections now open) 2016-03-13T09:40:58.674+0000 I NETWORK [conn4084870] end connection 10.0.7.113:33827 (1506 connections now open) 2016-03-13T09:40:58.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33846 #4084887 (1506 connections now open) 2016-03-13T09:40:58.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33847 #4084888 (1507 connections now open) 2016-03-13T09:40:59.693+0000 I NETWORK [conn4084872] end connection 10.0.216.90:60086 (1506 connections now open) 2016-03-13T09:40:59.693+0000 I NETWORK [conn4084873] end connection 10.0.216.90:60087 (1506 connections now open) 2016-03-13T09:40:59.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60104 #4084889 (1506 connections now open) 2016-03-13T09:40:59.694+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60105 #4084890 (1507 connections now open) 2016-03-13T09:41:00.698+0000 I NETWORK [conn4084874] end connection 10.0.20.53:53528 (1506 connections now open) 2016-03-13T09:41:00.698+0000 I NETWORK [conn4084875] end connection 10.0.20.53:53529 (1506 connections now open) 2016-03-13T09:41:00.699+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53539 #4084891 (1506 connections now open) 2016-03-13T09:41:00.699+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53540 #4084892 (1507 connections now open) 2016-03-13T09:41:03.087+0000 I NETWORK [conn4084876] end connection 10.0.179.209:43198 (1506 connections now open) 2016-03-13T09:41:03.087+0000 I NETWORK [conn4084877] end connection 10.0.179.209:43199 (1506 connections now open) 2016-03-13T09:41:03.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43221 #4084893 (1506 connections now open) 2016-03-13T09:41:03.151+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43220 #4084894 (1507 connections now open) 2016-03-13T09:41:03.747+0000 I NETWORK [conn4084878] end connection 10.0.26.69:45191 (1506 connections now open) 2016-03-13T09:41:03.747+0000 I NETWORK [conn4084879] end connection 10.0.26.69:45192 (1506 connections now open) 2016-03-13T09:41:03.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45207 #4084895 (1506 connections now open) 2016-03-13T09:41:03.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45208 #4084896 (1507 connections now open) 2016-03-13T09:41:06.428+0000 I NETWORK [conn4084881] end connection 10.0.27.237:38577 (1506 connections now open) 2016-03-13T09:41:06.428+0000 I NETWORK [conn4084880] end connection 10.0.27.237:38578 (1506 connections now open) 2016-03-13T09:41:06.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38594 #4084897 (1506 connections now open) 2016-03-13T09:41:06.429+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38595 #4084898 (1507 connections now open) 2016-03-13T09:41:07.131+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:41:06.976+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:41:08.688+0000 I NETWORK [conn4084887] end connection 10.0.7.113:33846 (1506 connections now open) 2016-03-13T09:41:08.688+0000 I NETWORK [conn4084888] end connection 10.0.7.113:33847 (1506 connections now open) 2016-03-13T09:41:08.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33857 #4084899 (1506 connections now open) 2016-03-13T09:41:08.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33858 #4084900 (1507 connections now open) 2016-03-13T09:41:09.706+0000 I NETWORK [conn4084889] end connection 10.0.216.90:60104 (1506 connections now open) 2016-03-13T09:41:09.706+0000 I NETWORK [conn4084890] end connection 10.0.216.90:60105 (1506 connections now open) 2016-03-13T09:41:09.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60112 #4084901 (1506 connections now open) 2016-03-13T09:41:09.707+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60113 #4084902 (1507 connections now open) 2016-03-13T09:41:10.711+0000 I NETWORK [conn4084891] end connection 10.0.20.53:53539 (1506 connections now open) 2016-03-13T09:41:10.711+0000 I NETWORK [conn4084892] end connection 10.0.20.53:53540 (1505 connections now open) 2016-03-13T09:41:10.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53552 #4084903 (1506 connections now open) 2016-03-13T09:41:10.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53553 #4084904 (1507 connections now open) 2016-03-13T09:41:13.163+0000 I NETWORK [conn4084894] end connection 10.0.179.209:43220 (1506 connections now open) 2016-03-13T09:41:13.164+0000 I NETWORK [conn4084893] end connection 10.0.179.209:43221 (1505 connections now open) 2016-03-13T09:41:13.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43229 #4084905 (1506 connections now open) 2016-03-13T09:41:13.164+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43230 #4084906 (1507 connections now open) 2016-03-13T09:41:13.761+0000 I NETWORK [conn4084895] end connection 10.0.26.69:45207 (1506 connections now open) 2016-03-13T09:41:13.761+0000 I NETWORK [conn4084896] end connection 10.0.26.69:45208 (1506 connections now open) 2016-03-13T09:41:13.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45228 #4084907 (1506 connections now open) 2016-03-13T09:41:13.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45227 #4084908 (1507 connections now open) 2016-03-13T09:41:16.117+0000 I NETWORK [conn4081844] end connection 10.0.179.209:40821 (1506 connections now open) 2016-03-13T09:41:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43234 #4084909 (1507 connections now open) 2016-03-13T09:41:16.118+0000 I NETWORK [conn4081611] end connection 10.0.20.53:50982 (1506 connections now open) 2016-03-13T09:41:16.118+0000 I NETWORK [conn4081950] end connection 10.0.27.237:36199 (1505 connections now open) 2016-03-13T09:41:16.118+0000 I NETWORK [conn4081026] end connection 10.0.7.113:59558 (1505 connections now open) 2016-03-13T09:41:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38617 #4084910 (1505 connections now open) 2016-03-13T09:41:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53561 #4084911 (1506 connections now open) 2016-03-13T09:41:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33868 #4084912 (1507 connections now open) 2016-03-13T09:41:16.441+0000 I NETWORK [conn4084898] end connection 10.0.27.237:38595 (1506 connections now open) 2016-03-13T09:41:16.441+0000 I NETWORK [conn4084897] end connection 10.0.27.237:38594 (1506 connections now open) 2016-03-13T09:41:16.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38618 #4084913 (1506 connections now open) 2016-03-13T09:41:16.442+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38619 #4084914 (1507 connections now open) 2016-03-13T09:41:18.701+0000 I NETWORK [conn4084899] end connection 10.0.7.113:33857 (1506 connections now open) 2016-03-13T09:41:18.701+0000 I NETWORK [conn4084900] end connection 10.0.7.113:33858 (1506 connections now open) 2016-03-13T09:41:18.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33870 #4084915 (1506 connections now open) 2016-03-13T09:41:18.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33871 #4084916 (1507 connections now open) 2016-03-13T09:41:19.719+0000 I NETWORK [conn4084902] end connection 10.0.216.90:60113 (1506 connections now open) 2016-03-13T09:41:19.719+0000 I NETWORK [conn4084901] end connection 10.0.216.90:60112 (1505 connections now open) 2016-03-13T09:41:19.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60125 #4084917 (1506 connections now open) 2016-03-13T09:41:19.720+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60126 #4084918 (1507 connections now open) 2016-03-13T09:41:20.734+0000 I NETWORK [conn4084904] end connection 10.0.20.53:53553 (1506 connections now open) 2016-03-13T09:41:20.734+0000 I NETWORK [conn4084903] end connection 10.0.20.53:53552 (1506 connections now open) 2016-03-13T09:41:20.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53574 #4084919 (1506 connections now open) 2016-03-13T09:41:20.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53573 #4084920 (1507 connections now open) 2016-03-13T09:41:23.177+0000 I NETWORK [conn4084906] end connection 10.0.179.209:43230 (1506 connections now open) 2016-03-13T09:41:23.177+0000 I NETWORK [conn4084905] end connection 10.0.179.209:43229 (1506 connections now open) 2016-03-13T09:41:23.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43247 #4084921 (1506 connections now open) 2016-03-13T09:41:23.177+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43246 #4084922 (1507 connections now open) 2016-03-13T09:41:23.782+0000 I NETWORK [conn4084907] end connection 10.0.26.69:45228 (1506 connections now open) 2016-03-13T09:41:23.782+0000 I NETWORK [conn4084908] end connection 10.0.26.69:45227 (1506 connections now open) 2016-03-13T09:41:23.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45236 #4084923 (1506 connections now open) 2016-03-13T09:41:23.783+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45235 #4084924 (1507 connections now open) 2016-03-13T09:41:26.455+0000 I NETWORK [conn4084914] end connection 10.0.27.237:38619 (1506 connections now open) 2016-03-13T09:41:26.455+0000 I NETWORK [conn4084913] end connection 10.0.27.237:38618 (1506 connections now open) 2016-03-13T09:41:26.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38627 #4084925 (1506 connections now open) 2016-03-13T09:41:26.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38626 #4084926 (1507 connections now open) 2016-03-13T09:41:28.114+0000 I NETWORK [conn4082873] end connection 10.0.216.90:58207 (1506 connections now open) 2016-03-13T09:41:28.114+0000 I NETWORK [conn4081399] end connection 10.0.7.113:59668 (1505 connections now open) 2016-03-13T09:41:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60142 #4084927 (1506 connections now open) 2016-03-13T09:41:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33888 #4084928 (1507 connections now open) 2016-03-13T09:41:28.714+0000 I NETWORK [conn4084916] end connection 10.0.7.113:33871 (1506 connections now open) 2016-03-13T09:41:28.714+0000 I NETWORK [conn4084915] end connection 10.0.7.113:33870 (1506 connections now open) 2016-03-13T09:41:28.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33889 #4084929 (1506 connections now open) 2016-03-13T09:41:28.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33890 #4084930 (1507 connections now open) 2016-03-13T09:41:29.732+0000 I NETWORK [conn4084917] end connection 10.0.216.90:60125 (1506 connections now open) 2016-03-13T09:41:29.732+0000 I NETWORK [conn4084918] end connection 10.0.216.90:60126 (1505 connections now open) 2016-03-13T09:41:29.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60148 #4084931 (1506 connections now open) 2016-03-13T09:41:29.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60147 #4084932 (1507 connections now open) 2016-03-13T09:41:30.747+0000 I NETWORK [conn4084919] end connection 10.0.20.53:53574 (1506 connections now open) 2016-03-13T09:41:30.747+0000 I NETWORK [conn4084920] end connection 10.0.20.53:53573 (1506 connections now open) 2016-03-13T09:41:30.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53586 #4084933 (1506 connections now open) 2016-03-13T09:41:30.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53587 #4084934 (1507 connections now open) 2016-03-13T09:41:33.190+0000 I NETWORK [conn4084921] end connection 10.0.179.209:43247 (1506 connections now open) 2016-03-13T09:41:33.190+0000 I NETWORK [conn4084922] end connection 10.0.179.209:43246 (1506 connections now open) 2016-03-13T09:41:33.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43266 #4084935 (1506 connections now open) 2016-03-13T09:41:33.190+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43267 #4084936 (1507 connections now open) 2016-03-13T09:41:33.795+0000 I NETWORK [conn4084924] end connection 10.0.26.69:45235 (1506 connections now open) 2016-03-13T09:41:33.795+0000 I NETWORK [conn4084923] end connection 10.0.26.69:45236 (1506 connections now open) 2016-03-13T09:41:33.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45250 #4084937 (1506 connections now open) 2016-03-13T09:41:33.796+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45249 #4084938 (1507 connections now open) 2016-03-13T09:41:36.468+0000 I NETWORK [conn4084925] end connection 10.0.27.237:38627 (1506 connections now open) 2016-03-13T09:41:36.468+0000 I NETWORK [conn4084926] end connection 10.0.27.237:38626 (1506 connections now open) 2016-03-13T09:41:36.468+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38640 #4084939 (1506 connections now open) 2016-03-13T09:41:36.468+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38641 #4084940 (1507 connections now open) 2016-03-13T09:41:37.299+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:41:37.131+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:41:38.728+0000 I NETWORK [conn4084930] end connection 10.0.7.113:33890 (1506 connections now open) 2016-03-13T09:41:38.728+0000 I NETWORK [conn4084929] end connection 10.0.7.113:33889 (1506 connections now open) 2016-03-13T09:41:38.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33896 #4084941 (1506 connections now open) 2016-03-13T09:41:38.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33897 #4084942 (1507 connections now open) 2016-03-13T09:41:39.745+0000 I NETWORK [conn4084932] end connection 10.0.216.90:60147 (1506 connections now open) 2016-03-13T09:41:39.745+0000 I NETWORK [conn4084931] end connection 10.0.216.90:60148 (1506 connections now open) 2016-03-13T09:41:39.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60155 #4084943 (1506 connections now open) 2016-03-13T09:41:39.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60156 #4084944 (1507 connections now open) 2016-03-13T09:41:40.761+0000 I NETWORK [conn4084934] end connection 10.0.20.53:53587 (1506 connections now open) 2016-03-13T09:41:40.761+0000 I NETWORK [conn4084933] end connection 10.0.20.53:53586 (1506 connections now open) 2016-03-13T09:41:40.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53599 #4084945 (1506 connections now open) 2016-03-13T09:41:40.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53598 #4084946 (1507 connections now open) 2016-03-13T09:41:43.203+0000 I NETWORK [conn4084935] end connection 10.0.179.209:43266 (1506 connections now open) 2016-03-13T09:41:43.203+0000 I NETWORK [conn4084936] end connection 10.0.179.209:43267 (1506 connections now open) 2016-03-13T09:41:43.203+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43275 #4084947 (1506 connections now open) 2016-03-13T09:41:43.204+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43276 #4084948 (1507 connections now open) 2016-03-13T09:41:43.809+0000 I NETWORK [conn4084938] end connection 10.0.26.69:45249 (1506 connections now open) 2016-03-13T09:41:43.809+0000 I NETWORK [conn4084937] end connection 10.0.26.69:45250 (1506 connections now open) 2016-03-13T09:41:43.818+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45267 #4084949 (1506 connections now open) 2016-03-13T09:41:43.818+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45268 #4084950 (1507 connections now open) 2016-03-13T09:41:46.117+0000 I NETWORK [conn4081818] end connection 10.0.179.209:40808 (1506 connections now open) 2016-03-13T09:41:46.117+0000 I NETWORK [conn4080904] end connection 10.0.20.53:50689 (1506 connections now open) 2016-03-13T09:41:46.117+0000 I NETWORK [conn4082108] end connection 10.0.27.237:36278 (1504 connections now open) 2016-03-13T09:41:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43279 #4084951 (1505 connections now open) 2016-03-13T09:41:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53607 #4084952 (1506 connections now open) 2016-03-13T09:41:46.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38660 #4084953 (1507 connections now open) 2016-03-13T09:41:46.481+0000 I NETWORK [conn4084940] end connection 10.0.27.237:38641 (1506 connections now open) 2016-03-13T09:41:46.481+0000 I NETWORK [conn4084939] end connection 10.0.27.237:38640 (1506 connections now open) 2016-03-13T09:41:46.481+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38662 #4084954 (1506 connections now open) 2016-03-13T09:41:46.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38663 #4084955 (1507 connections now open) 2016-03-13T09:41:48.750+0000 I NETWORK [conn4084941] end connection 10.0.7.113:33896 (1506 connections now open) 2016-03-13T09:41:48.750+0000 I NETWORK [conn4084942] end connection 10.0.7.113:33897 (1506 connections now open) 2016-03-13T09:41:48.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33910 #4084956 (1506 connections now open) 2016-03-13T09:41:48.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33909 #4084957 (1507 connections now open) 2016-03-13T09:41:49.767+0000 I NETWORK [conn4084944] end connection 10.0.216.90:60156 (1506 connections now open) 2016-03-13T09:41:49.767+0000 I NETWORK [conn4084943] end connection 10.0.216.90:60155 (1506 connections now open) 2016-03-13T09:41:49.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60168 #4084958 (1506 connections now open) 2016-03-13T09:41:49.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60169 #4084959 (1507 connections now open) 2016-03-13T09:41:50.783+0000 I NETWORK [conn4084946] end connection 10.0.20.53:53598 (1506 connections now open) 2016-03-13T09:41:50.783+0000 I NETWORK [conn4084945] end connection 10.0.20.53:53599 (1506 connections now open) 2016-03-13T09:41:50.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53620 #4084960 (1506 connections now open) 2016-03-13T09:41:50.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53619 #4084961 (1507 connections now open) 2016-03-13T09:41:53.216+0000 I NETWORK [conn4084948] end connection 10.0.179.209:43276 (1506 connections now open) 2016-03-13T09:41:53.216+0000 I NETWORK [conn4084947] end connection 10.0.179.209:43275 (1506 connections now open) 2016-03-13T09:41:53.216+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43291 #4084962 (1506 connections now open) 2016-03-13T09:41:53.216+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43290 #4084963 (1507 connections now open) 2016-03-13T09:41:53.831+0000 I NETWORK [conn4084950] end connection 10.0.26.69:45268 (1506 connections now open) 2016-03-13T09:41:53.831+0000 I NETWORK [conn4084949] end connection 10.0.26.69:45267 (1506 connections now open) 2016-03-13T09:41:53.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45276 #4084964 (1507 connections now open) 2016-03-13T09:41:53.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45277 #4084965 (1507 connections now open) 2016-03-13T09:41:56.494+0000 I NETWORK [conn4084955] end connection 10.0.27.237:38663 (1506 connections now open) 2016-03-13T09:41:56.494+0000 I NETWORK [conn4084954] end connection 10.0.27.237:38662 (1506 connections now open) 2016-03-13T09:41:56.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38668 #4084966 (1506 connections now open) 2016-03-13T09:41:56.494+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38669 #4084967 (1507 connections now open) 2016-03-13T09:41:58.114+0000 I NETWORK [conn4083033] end connection 10.0.216.90:58277 (1506 connections now open) 2016-03-13T09:41:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60185 #4084968 (1507 connections now open) 2016-03-13T09:41:58.115+0000 I NETWORK [conn4081651] end connection 10.0.26.69:42756 (1506 connections now open) 2016-03-13T09:41:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45284 #4084969 (1507 connections now open) 2016-03-13T09:41:58.764+0000 I NETWORK [conn4084956] end connection 10.0.7.113:33910 (1506 connections now open) 2016-03-13T09:41:58.764+0000 I NETWORK [conn4084957] end connection 10.0.7.113:33909 (1506 connections now open) 2016-03-13T09:41:58.765+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33928 #4084970 (1506 connections now open) 2016-03-13T09:41:58.765+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33929 #4084971 (1507 connections now open) 2016-03-13T09:41:59.780+0000 I NETWORK [conn4084958] end connection 10.0.216.90:60168 (1506 connections now open) 2016-03-13T09:41:59.780+0000 I NETWORK [conn4084959] end connection 10.0.216.90:60169 (1506 connections now open) 2016-03-13T09:41:59.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60187 #4084972 (1506 connections now open) 2016-03-13T09:41:59.781+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60188 #4084973 (1507 connections now open) 2016-03-13T09:42:00.797+0000 I NETWORK [conn4084961] end connection 10.0.20.53:53619 (1506 connections now open) 2016-03-13T09:42:00.797+0000 I NETWORK [conn4084960] end connection 10.0.20.53:53620 (1506 connections now open) 2016-03-13T09:42:00.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53630 #4084974 (1506 connections now open) 2016-03-13T09:42:00.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53629 #4084975 (1507 connections now open) 2016-03-13T09:42:03.228+0000 I NETWORK [conn4084962] end connection 10.0.179.209:43291 (1506 connections now open) 2016-03-13T09:42:03.228+0000 I NETWORK [conn4084963] end connection 10.0.179.209:43290 (1506 connections now open) 2016-03-13T09:42:03.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43312 #4084976 (1506 connections now open) 2016-03-13T09:42:03.237+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43313 #4084977 (1507 connections now open) 2016-03-13T09:42:03.844+0000 I NETWORK [conn4084965] end connection 10.0.26.69:45277 (1506 connections now open) 2016-03-13T09:42:03.844+0000 I NETWORK [conn4084964] end connection 10.0.26.69:45276 (1505 connections now open) 2016-03-13T09:42:03.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45291 #4084978 (1506 connections now open) 2016-03-13T09:42:03.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45292 #4084979 (1507 connections now open) 2016-03-13T09:42:06.507+0000 I NETWORK [conn4084967] end connection 10.0.27.237:38669 (1506 connections now open) 2016-03-13T09:42:06.507+0000 I NETWORK [conn4084966] end connection 10.0.27.237:38668 (1506 connections now open) 2016-03-13T09:42:06.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38685 #4084980 (1506 connections now open) 2016-03-13T09:42:06.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38686 #4084981 (1507 connections now open) 2016-03-13T09:42:07.603+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:42:07.299+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:42:08.778+0000 I NETWORK [conn4084970] end connection 10.0.7.113:33928 (1506 connections now open) 2016-03-13T09:42:08.778+0000 I NETWORK [conn4084971] end connection 10.0.7.113:33929 (1506 connections now open) 2016-03-13T09:42:08.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33941 #4084982 (1506 connections now open) 2016-03-13T09:42:08.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33942 #4084983 (1507 connections now open) 2016-03-13T09:42:09.793+0000 I NETWORK [conn4084973] end connection 10.0.216.90:60188 (1506 connections now open) 2016-03-13T09:42:09.793+0000 I NETWORK [conn4084972] end connection 10.0.216.90:60187 (1506 connections now open) 2016-03-13T09:42:09.794+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60195 #4084984 (1506 connections now open) 2016-03-13T09:42:09.794+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60196 #4084985 (1507 connections now open) 2016-03-13T09:42:10.810+0000 I NETWORK [conn4084975] end connection 10.0.20.53:53629 (1506 connections now open) 2016-03-13T09:42:10.810+0000 I NETWORK [conn4084974] end connection 10.0.20.53:53630 (1506 connections now open) 2016-03-13T09:42:10.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53644 #4084986 (1506 connections now open) 2016-03-13T09:42:10.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53643 #4084987 (1507 connections now open) 2016-03-13T09:42:13.249+0000 I NETWORK [conn4084976] end connection 10.0.179.209:43312 (1506 connections now open) 2016-03-13T09:42:13.250+0000 I NETWORK [conn4084977] end connection 10.0.179.209:43313 (1505 connections now open) 2016-03-13T09:42:13.250+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43320 #4084988 (1507 connections now open) 2016-03-13T09:42:13.250+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43321 #4084989 (1507 connections now open) 2016-03-13T09:42:13.857+0000 I NETWORK [conn4084978] end connection 10.0.26.69:45291 (1506 connections now open) 2016-03-13T09:42:13.857+0000 I NETWORK [conn4084979] end connection 10.0.26.69:45292 (1506 connections now open) 2016-03-13T09:42:13.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45313 #4084990 (1506 connections now open) 2016-03-13T09:42:13.858+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45312 #4084991 (1507 connections now open) 2016-03-13T09:42:16.117+0000 I NETWORK [conn4082013] end connection 10.0.179.209:40902 (1506 connections now open) 2016-03-13T09:42:16.117+0000 I NETWORK [conn4082920] end connection 10.0.216.90:58235 (1506 connections now open) 2016-03-13T09:42:16.118+0000 I NETWORK [conn4081434] end connection 10.0.26.69:42641 (1505 connections now open) 2016-03-13T09:42:16.118+0000 I NETWORK [conn4082312] end connection 10.0.27.237:36376 (1503 connections now open) 2016-03-13T09:42:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60201 #4084992 (1504 connections now open) 2016-03-13T09:42:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45316 #4084993 (1505 connections now open) 2016-03-13T09:42:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43323 #4084994 (1506 connections now open) 2016-03-13T09:42:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38706 #4084995 (1507 connections now open) 2016-03-13T09:42:16.520+0000 I NETWORK [conn4084981] end connection 10.0.27.237:38686 (1506 connections now open) 2016-03-13T09:42:16.520+0000 I NETWORK [conn4084980] end connection 10.0.27.237:38685 (1505 connections now open) 2016-03-13T09:42:16.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38708 #4084996 (1506 connections now open) 2016-03-13T09:42:16.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38709 #4084997 (1507 connections now open) 2016-03-13T09:42:18.791+0000 I NETWORK [conn4084982] end connection 10.0.7.113:33941 (1506 connections now open) 2016-03-13T09:42:18.791+0000 I NETWORK [conn4084983] end connection 10.0.7.113:33942 (1506 connections now open) 2016-03-13T09:42:18.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33954 #4084998 (1506 connections now open) 2016-03-13T09:42:18.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33955 #4084999 (1507 connections now open) 2016-03-13T09:42:19.806+0000 I NETWORK [conn4084984] end connection 10.0.216.90:60195 (1506 connections now open) 2016-03-13T09:42:19.806+0000 I NETWORK [conn4084985] end connection 10.0.216.90:60196 (1506 connections now open) 2016-03-13T09:42:19.807+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60207 #4085000 (1506 connections now open) 2016-03-13T09:42:19.807+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60208 #4085001 (1507 connections now open) 2016-03-13T09:42:20.824+0000 I NETWORK [conn4084986] end connection 10.0.20.53:53644 (1506 connections now open) 2016-03-13T09:42:20.824+0000 I NETWORK [conn4084987] end connection 10.0.20.53:53643 (1506 connections now open) 2016-03-13T09:42:20.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53664 #4085002 (1506 connections now open) 2016-03-13T09:42:20.825+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53665 #4085003 (1507 connections now open) 2016-03-13T09:42:23.262+0000 I NETWORK [conn4084988] end connection 10.0.179.209:43320 (1506 connections now open) 2016-03-13T09:42:23.262+0000 I NETWORK [conn4084989] end connection 10.0.179.209:43321 (1505 connections now open) 2016-03-13T09:42:23.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43337 #4085004 (1506 connections now open) 2016-03-13T09:42:23.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43338 #4085005 (1507 connections now open) 2016-03-13T09:42:23.871+0000 I NETWORK [conn4084991] end connection 10.0.26.69:45312 (1506 connections now open) 2016-03-13T09:42:23.871+0000 I NETWORK [conn4084990] end connection 10.0.26.69:45313 (1506 connections now open) 2016-03-13T09:42:23.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45322 #4085006 (1506 connections now open) 2016-03-13T09:42:23.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45321 #4085007 (1507 connections now open) 2016-03-13T09:42:26.533+0000 I NETWORK [conn4084997] end connection 10.0.27.237:38709 (1506 connections now open) 2016-03-13T09:42:26.533+0000 I NETWORK [conn4084996] end connection 10.0.27.237:38708 (1506 connections now open) 2016-03-13T09:42:26.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38714 #4085008 (1506 connections now open) 2016-03-13T09:42:26.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38715 #4085009 (1507 connections now open) 2016-03-13T09:42:28.113+0000 I NETWORK [conn4082847] end connection 10.0.216.90:58194 (1506 connections now open) 2016-03-13T09:42:28.113+0000 I NETWORK [conn4081980] end connection 10.0.20.53:51169 (1505 connections now open) 2016-03-13T09:42:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60225 #4085010 (1506 connections now open) 2016-03-13T09:42:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53673 #4085011 (1507 connections now open) 2016-03-13T09:42:28.805+0000 I NETWORK [conn4084998] end connection 10.0.7.113:33954 (1506 connections now open) 2016-03-13T09:42:28.805+0000 I NETWORK [conn4084999] end connection 10.0.7.113:33955 (1506 connections now open) 2016-03-13T09:42:28.806+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33973 #4085012 (1506 connections now open) 2016-03-13T09:42:28.806+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33974 #4085013 (1507 connections now open) 2016-03-13T09:42:29.819+0000 I NETWORK [conn4085000] end connection 10.0.216.90:60207 (1506 connections now open) 2016-03-13T09:42:29.819+0000 I NETWORK [conn4085001] end connection 10.0.216.90:60208 (1505 connections now open) 2016-03-13T09:42:29.820+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60229 #4085014 (1506 connections now open) 2016-03-13T09:42:29.820+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60230 #4085015 (1507 connections now open) 2016-03-13T09:42:30.837+0000 I NETWORK [conn4085003] end connection 10.0.20.53:53665 (1506 connections now open) 2016-03-13T09:42:30.837+0000 I NETWORK [conn4085002] end connection 10.0.20.53:53664 (1506 connections now open) 2016-03-13T09:42:30.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53677 #4085016 (1506 connections now open) 2016-03-13T09:42:30.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53678 #4085017 (1507 connections now open) 2016-03-13T09:42:33.275+0000 I NETWORK [conn4085005] end connection 10.0.179.209:43338 (1506 connections now open) 2016-03-13T09:42:33.275+0000 I NETWORK [conn4085004] end connection 10.0.179.209:43337 (1506 connections now open) 2016-03-13T09:42:33.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43357 #4085018 (1506 connections now open) 2016-03-13T09:42:33.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43358 #4085019 (1507 connections now open) 2016-03-13T09:42:33.884+0000 I NETWORK [conn4085006] end connection 10.0.26.69:45322 (1506 connections now open) 2016-03-13T09:42:33.884+0000 I NETWORK [conn4085007] end connection 10.0.26.69:45321 (1506 connections now open) 2016-03-13T09:42:33.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45336 #4085020 (1506 connections now open) 2016-03-13T09:42:33.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45335 #4085021 (1507 connections now open) 2016-03-13T09:42:36.546+0000 I NETWORK [conn4085008] end connection 10.0.27.237:38714 (1506 connections now open) 2016-03-13T09:42:36.546+0000 I NETWORK [conn4085009] end connection 10.0.27.237:38715 (1505 connections now open) 2016-03-13T09:42:36.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38728 #4085022 (1506 connections now open) 2016-03-13T09:42:36.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38729 #4085023 (1507 connections now open) 2016-03-13T09:42:37.806+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:42:37.603+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:42:38.819+0000 I NETWORK [conn4085012] end connection 10.0.7.113:33973 (1506 connections now open) 2016-03-13T09:42:38.819+0000 I NETWORK [conn4085013] end connection 10.0.7.113:33974 (1506 connections now open) 2016-03-13T09:42:38.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33981 #4085024 (1506 connections now open) 2016-03-13T09:42:38.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33982 #4085025 (1507 connections now open) 2016-03-13T09:42:39.832+0000 I NETWORK [conn4085014] end connection 10.0.216.90:60229 (1506 connections now open) 2016-03-13T09:42:39.832+0000 I NETWORK [conn4085015] end connection 10.0.216.90:60230 (1506 connections now open) 2016-03-13T09:42:39.841+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60235 #4085026 (1506 connections now open) 2016-03-13T09:42:39.841+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60236 #4085027 (1507 connections now open) 2016-03-13T09:42:40.851+0000 I NETWORK [conn4085016] end connection 10.0.20.53:53677 (1506 connections now open) 2016-03-13T09:42:40.851+0000 I NETWORK [conn4085017] end connection 10.0.20.53:53678 (1506 connections now open) 2016-03-13T09:42:40.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53689 #4085028 (1506 connections now open) 2016-03-13T09:42:40.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53690 #4085029 (1507 connections now open) 2016-03-13T09:42:43.288+0000 I NETWORK [conn4085019] end connection 10.0.179.209:43358 (1506 connections now open) 2016-03-13T09:42:43.288+0000 I NETWORK [conn4085018] end connection 10.0.179.209:43357 (1506 connections now open) 2016-03-13T09:42:43.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43365 #4085030 (1506 connections now open) 2016-03-13T09:42:43.289+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43366 #4085031 (1507 connections now open) 2016-03-13T09:42:43.898+0000 I NETWORK [conn4085021] end connection 10.0.26.69:45335 (1506 connections now open) 2016-03-13T09:42:43.898+0000 I NETWORK [conn4085020] end connection 10.0.26.69:45336 (1506 connections now open) 2016-03-13T09:42:43.907+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45353 #4085032 (1506 connections now open) 2016-03-13T09:42:43.907+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45354 #4085033 (1507 connections now open) 2016-03-13T09:42:46.118+0000 I NETWORK [conn4081001] end connection 10.0.7.113:59547 (1506 connections now open) 2016-03-13T09:42:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33992 #4085034 (1507 connections now open) 2016-03-13T09:42:46.559+0000 I NETWORK [conn4085022] end connection 10.0.27.237:38728 (1506 connections now open) 2016-03-13T09:42:46.559+0000 I NETWORK [conn4085023] end connection 10.0.27.237:38729 (1506 connections now open) 2016-03-13T09:42:46.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38749 #4085035 (1506 connections now open) 2016-03-13T09:42:46.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38748 #4085036 (1507 connections now open) 2016-03-13T09:42:48.839+0000 I NETWORK [conn4085024] end connection 10.0.7.113:33981 (1506 connections now open) 2016-03-13T09:42:48.839+0000 I NETWORK [conn4085025] end connection 10.0.7.113:33982 (1506 connections now open) 2016-03-13T09:42:48.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33995 #4085037 (1506 connections now open) 2016-03-13T09:42:48.840+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:33994 #4085038 (1507 connections now open) 2016-03-13T09:42:49.853+0000 I NETWORK [conn4085026] end connection 10.0.216.90:60235 (1506 connections now open) 2016-03-13T09:42:49.853+0000 I NETWORK [conn4085027] end connection 10.0.216.90:60236 (1505 connections now open) 2016-03-13T09:42:49.854+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60248 #4085039 (1506 connections now open) 2016-03-13T09:42:49.854+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60249 #4085040 (1507 connections now open) 2016-03-13T09:42:50.871+0000 I NETWORK [conn4085029] end connection 10.0.20.53:53690 (1506 connections now open) 2016-03-13T09:42:50.871+0000 I NETWORK [conn4085028] end connection 10.0.20.53:53689 (1506 connections now open) 2016-03-13T09:42:50.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53710 #4085041 (1506 connections now open) 2016-03-13T09:42:50.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53711 #4085042 (1507 connections now open) 2016-03-13T09:42:53.301+0000 I NETWORK [conn4085030] end connection 10.0.179.209:43365 (1506 connections now open) 2016-03-13T09:42:53.301+0000 I NETWORK [conn4085031] end connection 10.0.179.209:43366 (1505 connections now open) 2016-03-13T09:42:53.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43379 #4085043 (1506 connections now open) 2016-03-13T09:42:53.301+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43380 #4085044 (1507 connections now open) 2016-03-13T09:42:53.920+0000 I NETWORK [conn4085032] end connection 10.0.26.69:45353 (1506 connections now open) 2016-03-13T09:42:53.920+0000 I NETWORK [conn4085033] end connection 10.0.26.69:45354 (1506 connections now open) 2016-03-13T09:42:53.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45363 #4085045 (1506 connections now open) 2016-03-13T09:42:53.920+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45364 #4085046 (1507 connections now open) 2016-03-13T09:42:56.581+0000 I NETWORK [conn4085035] end connection 10.0.27.237:38749 (1506 connections now open) 2016-03-13T09:42:56.581+0000 I NETWORK [conn4085036] end connection 10.0.27.237:38748 (1506 connections now open) 2016-03-13T09:42:56.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38756 #4085047 (1506 connections now open) 2016-03-13T09:42:56.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38755 #4085048 (1507 connections now open) 2016-03-13T09:42:58.114+0000 I NETWORK [conn4082914] end connection 10.0.26.69:43308 (1506 connections now open) 2016-03-13T09:42:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45370 #4085049 (1507 connections now open) 2016-03-13T09:42:58.853+0000 I NETWORK [conn4085038] end connection 10.0.7.113:33994 (1506 connections now open) 2016-03-13T09:42:58.853+0000 I NETWORK [conn4085037] end connection 10.0.7.113:33995 (1506 connections now open) 2016-03-13T09:42:58.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34014 #4085050 (1506 connections now open) 2016-03-13T09:42:58.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34013 #4085051 (1507 connections now open) 2016-03-13T09:42:59.865+0000 I NETWORK [conn4085039] end connection 10.0.216.90:60248 (1506 connections now open) 2016-03-13T09:42:59.865+0000 I NETWORK [conn4085040] end connection 10.0.216.90:60249 (1506 connections now open) 2016-03-13T09:42:59.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60267 #4085052 (1506 connections now open) 2016-03-13T09:42:59.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60266 #4085053 (1507 connections now open) 2016-03-13T09:43:00.884+0000 I NETWORK [conn4085041] end connection 10.0.20.53:53710 (1506 connections now open) 2016-03-13T09:43:00.884+0000 I NETWORK [conn4085042] end connection 10.0.20.53:53711 (1506 connections now open) 2016-03-13T09:43:00.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53721 #4085054 (1506 connections now open) 2016-03-13T09:43:00.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53722 #4085055 (1507 connections now open) 2016-03-13T09:43:03.314+0000 I NETWORK [conn4085044] end connection 10.0.179.209:43380 (1506 connections now open) 2016-03-13T09:43:03.314+0000 I NETWORK [conn4085043] end connection 10.0.179.209:43379 (1506 connections now open) 2016-03-13T09:43:03.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43401 #4085056 (1506 connections now open) 2016-03-13T09:43:03.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43402 #4085057 (1507 connections now open) 2016-03-13T09:43:03.932+0000 I NETWORK [conn4085046] end connection 10.0.26.69:45364 (1506 connections now open) 2016-03-13T09:43:03.932+0000 I NETWORK [conn4085045] end connection 10.0.26.69:45363 (1506 connections now open) 2016-03-13T09:43:03.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45379 #4085058 (1506 connections now open) 2016-03-13T09:43:03.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45380 #4085059 (1507 connections now open) 2016-03-13T09:43:06.593+0000 I NETWORK [conn4085048] end connection 10.0.27.237:38755 (1506 connections now open) 2016-03-13T09:43:06.594+0000 I NETWORK [conn4085047] end connection 10.0.27.237:38756 (1505 connections now open) 2016-03-13T09:43:06.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38771 #4085060 (1506 connections now open) 2016-03-13T09:43:06.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38772 #4085061 (1507 connections now open) 2016-03-13T09:43:08.059+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:43:07.806+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:43:08.866+0000 I NETWORK [conn4085051] end connection 10.0.7.113:34013 (1506 connections now open) 2016-03-13T09:43:08.866+0000 I NETWORK [conn4085050] end connection 10.0.7.113:34014 (1506 connections now open) 2016-03-13T09:43:08.867+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34026 #4085062 (1506 connections now open) 2016-03-13T09:43:08.867+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34025 #4085063 (1507 connections now open) 2016-03-13T09:43:09.878+0000 I NETWORK [conn4085053] end connection 10.0.216.90:60266 (1506 connections now open) 2016-03-13T09:43:09.878+0000 I NETWORK [conn4085052] end connection 10.0.216.90:60267 (1506 connections now open) 2016-03-13T09:43:09.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60277 #4085064 (1506 connections now open) 2016-03-13T09:43:09.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60278 #4085065 (1507 connections now open) 2016-03-13T09:43:10.897+0000 I NETWORK [conn4085054] end connection 10.0.20.53:53721 (1506 connections now open) 2016-03-13T09:43:10.897+0000 I NETWORK [conn4085055] end connection 10.0.20.53:53722 (1505 connections now open) 2016-03-13T09:43:10.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53734 #4085066 (1506 connections now open) 2016-03-13T09:43:10.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53735 #4085067 (1507 connections now open) 2016-03-13T09:43:13.335+0000 I NETWORK [conn4085056] end connection 10.0.179.209:43401 (1506 connections now open) 2016-03-13T09:43:13.335+0000 I NETWORK [conn4085057] end connection 10.0.179.209:43402 (1506 connections now open) 2016-03-13T09:43:13.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43411 #4085068 (1506 connections now open) 2016-03-13T09:43:13.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43410 #4085069 (1507 connections now open) 2016-03-13T09:43:13.945+0000 I NETWORK [conn4085058] end connection 10.0.26.69:45379 (1506 connections now open) 2016-03-13T09:43:13.945+0000 I NETWORK [conn4085059] end connection 10.0.26.69:45380 (1506 connections now open) 2016-03-13T09:43:13.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45399 #4085070 (1506 connections now open) 2016-03-13T09:43:13.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45400 #4085071 (1507 connections now open) 2016-03-13T09:43:16.117+0000 I NETWORK [conn4081817] end connection 10.0.179.209:40807 (1506 connections now open) 2016-03-13T09:43:16.117+0000 I NETWORK [conn4082685] end connection 10.0.216.90:58107 (1505 connections now open) 2016-03-13T09:43:16.117+0000 I NETWORK [conn4083993] end connection 10.0.26.69:44302 (1505 connections now open) 2016-03-13T09:43:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43414 #4085072 (1505 connections now open) 2016-03-13T09:43:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60284 #4085073 (1506 connections now open) 2016-03-13T09:43:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45404 #4085074 (1507 connections now open) 2016-03-13T09:43:16.606+0000 I NETWORK [conn4085061] end connection 10.0.27.237:38772 (1506 connections now open) 2016-03-13T09:43:16.606+0000 I NETWORK [conn4085060] end connection 10.0.27.237:38771 (1505 connections now open) 2016-03-13T09:43:16.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38794 #4085075 (1506 connections now open) 2016-03-13T09:43:16.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38795 #4085076 (1507 connections now open) 2016-03-13T09:43:18.879+0000 I NETWORK [conn4085062] end connection 10.0.7.113:34026 (1506 connections now open) 2016-03-13T09:43:18.879+0000 I NETWORK [conn4085063] end connection 10.0.7.113:34025 (1506 connections now open) 2016-03-13T09:43:18.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34038 #4085077 (1507 connections now open) 2016-03-13T09:43:18.880+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34039 #4085078 (1507 connections now open) 2016-03-13T09:43:19.892+0000 I NETWORK [conn4085065] end connection 10.0.216.90:60278 (1506 connections now open) 2016-03-13T09:43:19.892+0000 I NETWORK [conn4085064] end connection 10.0.216.90:60277 (1506 connections now open) 2016-03-13T09:43:19.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60291 #4085079 (1506 connections now open) 2016-03-13T09:43:19.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60292 #4085080 (1507 connections now open) 2016-03-13T09:43:20.909+0000 I NETWORK [conn4085067] end connection 10.0.20.53:53735 (1506 connections now open) 2016-03-13T09:43:20.909+0000 I NETWORK [conn4085066] end connection 10.0.20.53:53734 (1505 connections now open) 2016-03-13T09:43:20.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53754 #4085081 (1506 connections now open) 2016-03-13T09:43:20.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53755 #4085082 (1507 connections now open) 2016-03-13T09:43:23.348+0000 I NETWORK [conn4085068] end connection 10.0.179.209:43411 (1506 connections now open) 2016-03-13T09:43:23.348+0000 I NETWORK [conn4085069] end connection 10.0.179.209:43410 (1505 connections now open) 2016-03-13T09:43:23.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43426 #4085083 (1506 connections now open) 2016-03-13T09:43:23.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43427 #4085084 (1507 connections now open) 2016-03-13T09:43:23.957+0000 I NETWORK [conn4085071] end connection 10.0.26.69:45400 (1506 connections now open) 2016-03-13T09:43:23.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45409 #4085085 (1507 connections now open) 2016-03-13T09:43:23.958+0000 I NETWORK [conn4085070] end connection 10.0.26.69:45399 (1505 connections now open) 2016-03-13T09:43:23.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45410 #4085086 (1507 connections now open) 2016-03-13T09:43:26.619+0000 I NETWORK [conn4085075] end connection 10.0.27.237:38794 (1506 connections now open) 2016-03-13T09:43:26.619+0000 I NETWORK [conn4085076] end connection 10.0.27.237:38795 (1506 connections now open) 2016-03-13T09:43:26.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38802 #4085087 (1506 connections now open) 2016-03-13T09:43:26.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38803 #4085088 (1507 connections now open) 2016-03-13T09:43:28.114+0000 I NETWORK [conn4080976] end connection 10.0.27.237:35776 (1506 connections now open) 2016-03-13T09:43:28.114+0000 I NETWORK [conn4081464] end connection 10.0.7.113:59682 (1506 connections now open) 2016-03-13T09:43:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34057 #4085089 (1506 connections now open) 2016-03-13T09:43:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38808 #4085090 (1507 connections now open) 2016-03-13T09:43:28.893+0000 I NETWORK [conn4085077] end connection 10.0.7.113:34038 (1506 connections now open) 2016-03-13T09:43:28.893+0000 I NETWORK [conn4085078] end connection 10.0.7.113:34039 (1506 connections now open) 2016-03-13T09:43:28.894+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34058 #4085091 (1506 connections now open) 2016-03-13T09:43:28.894+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34059 #4085092 (1507 connections now open) 2016-03-13T09:43:29.905+0000 I NETWORK [conn4085080] end connection 10.0.216.90:60292 (1506 connections now open) 2016-03-13T09:43:29.905+0000 I NETWORK [conn4085079] end connection 10.0.216.90:60291 (1506 connections now open) 2016-03-13T09:43:29.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60315 #4085093 (1506 connections now open) 2016-03-13T09:43:29.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60314 #4085094 (1507 connections now open) 2016-03-13T09:43:30.922+0000 I NETWORK [conn4085081] end connection 10.0.20.53:53754 (1506 connections now open) 2016-03-13T09:43:30.922+0000 I NETWORK [conn4085082] end connection 10.0.20.53:53755 (1506 connections now open) 2016-03-13T09:43:30.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53770 #4085095 (1506 connections now open) 2016-03-13T09:43:30.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53769 #4085096 (1507 connections now open) 2016-03-13T09:43:33.361+0000 I NETWORK [conn4085083] end connection 10.0.179.209:43426 (1506 connections now open) 2016-03-13T09:43:33.361+0000 I NETWORK [conn4085084] end connection 10.0.179.209:43427 (1506 connections now open) 2016-03-13T09:43:33.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43448 #4085097 (1506 connections now open) 2016-03-13T09:43:33.361+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43447 #4085098 (1507 connections now open) 2016-03-13T09:43:33.971+0000 I NETWORK [conn4085085] end connection 10.0.26.69:45409 (1506 connections now open) 2016-03-13T09:43:33.971+0000 I NETWORK [conn4085086] end connection 10.0.26.69:45410 (1505 connections now open) 2016-03-13T09:43:33.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45421 #4085099 (1506 connections now open) 2016-03-13T09:43:33.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45422 #4085100 (1507 connections now open) 2016-03-13T09:43:36.631+0000 I NETWORK [conn4085087] end connection 10.0.27.237:38802 (1506 connections now open) 2016-03-13T09:43:36.631+0000 I NETWORK [conn4085088] end connection 10.0.27.237:38803 (1505 connections now open) 2016-03-13T09:43:36.631+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38816 #4085101 (1506 connections now open) 2016-03-13T09:43:36.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38817 #4085102 (1507 connections now open) 2016-03-13T09:43:38.343+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:43:38.060+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:43:38.907+0000 I NETWORK [conn4085091] end connection 10.0.7.113:34058 (1506 connections now open) 2016-03-13T09:43:38.907+0000 I NETWORK [conn4085092] end connection 10.0.7.113:34059 (1506 connections now open) 2016-03-13T09:43:38.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34065 #4085103 (1506 connections now open) 2016-03-13T09:43:38.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34066 #4085104 (1507 connections now open) 2016-03-13T09:43:39.918+0000 I NETWORK [conn4085093] end connection 10.0.216.90:60315 (1506 connections now open) 2016-03-13T09:43:39.918+0000 I NETWORK [conn4085094] end connection 10.0.216.90:60314 (1506 connections now open) 2016-03-13T09:43:39.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60323 #4085105 (1506 connections now open) 2016-03-13T09:43:39.927+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60322 #4085106 (1507 connections now open) 2016-03-13T09:43:40.935+0000 I NETWORK [conn4085096] end connection 10.0.20.53:53769 (1506 connections now open) 2016-03-13T09:43:40.935+0000 I NETWORK [conn4085095] end connection 10.0.20.53:53770 (1505 connections now open) 2016-03-13T09:43:40.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53777 #4085107 (1506 connections now open) 2016-03-13T09:43:40.944+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53778 #4085108 (1507 connections now open) 2016-03-13T09:43:43.373+0000 I NETWORK [conn4085098] end connection 10.0.179.209:43447 (1506 connections now open) 2016-03-13T09:43:43.374+0000 I NETWORK [conn4085097] end connection 10.0.179.209:43448 (1505 connections now open) 2016-03-13T09:43:43.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43455 #4085109 (1506 connections now open) 2016-03-13T09:43:43.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43456 #4085110 (1507 connections now open) 2016-03-13T09:43:43.983+0000 I NETWORK [conn4085099] end connection 10.0.26.69:45421 (1506 connections now open) 2016-03-13T09:43:43.983+0000 I NETWORK [conn4085100] end connection 10.0.26.69:45422 (1505 connections now open) 2016-03-13T09:43:43.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45439 #4085111 (1506 connections now open) 2016-03-13T09:43:43.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45438 #4085112 (1507 connections now open) 2016-03-13T09:43:46.116+0000 I NETWORK [conn4083568] end connection 10.0.179.209:41848 (1506 connections now open) 2016-03-13T09:43:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43458 #4085113 (1507 connections now open) 2016-03-13T09:43:46.644+0000 I NETWORK [conn4085101] end connection 10.0.27.237:38816 (1506 connections now open) 2016-03-13T09:43:46.644+0000 I NETWORK [conn4085102] end connection 10.0.27.237:38817 (1505 connections now open) 2016-03-13T09:43:46.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38837 #4085114 (1506 connections now open) 2016-03-13T09:43:46.645+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38836 #4085115 (1507 connections now open) 2016-03-13T09:43:48.922+0000 I NETWORK [conn4085103] end connection 10.0.7.113:34065 (1506 connections now open) 2016-03-13T09:43:48.922+0000 I NETWORK [conn4085104] end connection 10.0.7.113:34066 (1506 connections now open) 2016-03-13T09:43:48.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34080 #4085116 (1506 connections now open) 2016-03-13T09:43:48.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34079 #4085117 (1507 connections now open) 2016-03-13T09:43:49.940+0000 I NETWORK [conn4085106] end connection 10.0.216.90:60322 (1506 connections now open) 2016-03-13T09:43:49.940+0000 I NETWORK [conn4085105] end connection 10.0.216.90:60323 (1506 connections now open) 2016-03-13T09:43:49.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60337 #4085118 (1506 connections now open) 2016-03-13T09:43:49.940+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60336 #4085119 (1507 connections now open) 2016-03-13T09:43:50.957+0000 I NETWORK [conn4085107] end connection 10.0.20.53:53777 (1506 connections now open) 2016-03-13T09:43:50.957+0000 I NETWORK [conn4085108] end connection 10.0.20.53:53778 (1506 connections now open) 2016-03-13T09:43:50.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53792 #4085120 (1506 connections now open) 2016-03-13T09:43:50.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53793 #4085121 (1507 connections now open) 2016-03-13T09:43:53.386+0000 I NETWORK [conn4085109] end connection 10.0.179.209:43455 (1506 connections now open) 2016-03-13T09:43:53.386+0000 I NETWORK [conn4085110] end connection 10.0.179.209:43456 (1506 connections now open) 2016-03-13T09:43:53.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43467 #4085122 (1506 connections now open) 2016-03-13T09:43:53.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43468 #4085123 (1507 connections now open) 2016-03-13T09:43:53.997+0000 I NETWORK [conn4085111] end connection 10.0.26.69:45439 (1506 connections now open) 2016-03-13T09:43:53.997+0000 I NETWORK [conn4085112] end connection 10.0.26.69:45438 (1506 connections now open) 2016-03-13T09:43:54.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45450 #4085124 (1506 connections now open) 2016-03-13T09:43:54.006+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45449 #4085125 (1507 connections now open) 2016-03-13T09:43:56.657+0000 I NETWORK [conn4085115] end connection 10.0.27.237:38836 (1506 connections now open) 2016-03-13T09:43:56.657+0000 I NETWORK [conn4085114] end connection 10.0.27.237:38837 (1506 connections now open) 2016-03-13T09:43:56.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38845 #4085126 (1506 connections now open) 2016-03-13T09:43:56.658+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38844 #4085127 (1507 connections now open) 2016-03-13T09:43:58.114+0000 I NETWORK [conn4081858] end connection 10.0.179.209:40842 (1506 connections now open) 2016-03-13T09:43:58.114+0000 I NETWORK [conn4084758] end connection 10.0.7.113:33718 (1505 connections now open) 2016-03-13T09:43:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43482 #4085128 (1506 connections now open) 2016-03-13T09:43:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34097 #4085129 (1507 connections now open) 2016-03-13T09:43:58.935+0000 I NETWORK [conn4085117] end connection 10.0.7.113:34079 (1506 connections now open) 2016-03-13T09:43:58.935+0000 I NETWORK [conn4085116] end connection 10.0.7.113:34080 (1506 connections now open) 2016-03-13T09:43:58.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34099 #4085130 (1506 connections now open) 2016-03-13T09:43:58.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34098 #4085131 (1507 connections now open) 2016-03-13T09:43:59.953+0000 I NETWORK [conn4085118] end connection 10.0.216.90:60337 (1506 connections now open) 2016-03-13T09:43:59.953+0000 I NETWORK [conn4085119] end connection 10.0.216.90:60336 (1506 connections now open) 2016-03-13T09:43:59.953+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60355 #4085132 (1506 connections now open) 2016-03-13T09:43:59.954+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60356 #4085133 (1507 connections now open) 2016-03-13T09:44:00.970+0000 I NETWORK [conn4085120] end connection 10.0.20.53:53792 (1506 connections now open) 2016-03-13T09:44:00.970+0000 I NETWORK [conn4085121] end connection 10.0.20.53:53793 (1506 connections now open) 2016-03-13T09:44:00.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53812 #4085134 (1506 connections now open) 2016-03-13T09:44:00.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53813 #4085135 (1507 connections now open) 2016-03-13T09:44:03.399+0000 I NETWORK [conn4085122] end connection 10.0.179.209:43467 (1506 connections now open) 2016-03-13T09:44:03.399+0000 I NETWORK [conn4085123] end connection 10.0.179.209:43468 (1506 connections now open) 2016-03-13T09:44:03.408+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43490 #4085136 (1506 connections now open) 2016-03-13T09:44:03.408+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43489 #4085137 (1507 connections now open) 2016-03-13T09:44:04.019+0000 I NETWORK [conn4085124] end connection 10.0.26.69:45450 (1506 connections now open) 2016-03-13T09:44:04.019+0000 I NETWORK [conn4085125] end connection 10.0.26.69:45449 (1505 connections now open) 2016-03-13T09:44:04.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45465 #4085138 (1506 connections now open) 2016-03-13T09:44:04.020+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45466 #4085139 (1507 connections now open) 2016-03-13T09:44:06.670+0000 I NETWORK [conn4085127] end connection 10.0.27.237:38844 (1506 connections now open) 2016-03-13T09:44:06.670+0000 I NETWORK [conn4085126] end connection 10.0.27.237:38845 (1505 connections now open) 2016-03-13T09:44:06.670+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38859 #4085140 (1506 connections now open) 2016-03-13T09:44:06.671+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38860 #4085141 (1507 connections now open) 2016-03-13T09:44:08.477+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:44:08.343+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:44:08.949+0000 I NETWORK [conn4085131] end connection 10.0.7.113:34098 (1506 connections now open) 2016-03-13T09:44:08.949+0000 I NETWORK [conn4085130] end connection 10.0.7.113:34099 (1506 connections now open) 2016-03-13T09:44:08.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34110 #4085142 (1506 connections now open) 2016-03-13T09:44:08.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34111 #4085143 (1507 connections now open) 2016-03-13T09:44:09.966+0000 I NETWORK [conn4085132] end connection 10.0.216.90:60355 (1506 connections now open) 2016-03-13T09:44:09.966+0000 I NETWORK [conn4085133] end connection 10.0.216.90:60356 (1506 connections now open) 2016-03-13T09:44:09.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60364 #4085144 (1506 connections now open) 2016-03-13T09:44:09.967+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60365 #4085145 (1507 connections now open) 2016-03-13T09:44:10.983+0000 I NETWORK [conn4085135] end connection 10.0.20.53:53813 (1506 connections now open) 2016-03-13T09:44:10.983+0000 I NETWORK [conn4085134] end connection 10.0.20.53:53812 (1506 connections now open) 2016-03-13T09:44:10.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53822 #4085146 (1506 connections now open) 2016-03-13T09:44:10.984+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53821 #4085147 (1507 connections now open) 2016-03-13T09:44:13.421+0000 I NETWORK [conn4085137] end connection 10.0.179.209:43489 (1506 connections now open) 2016-03-13T09:44:13.421+0000 I NETWORK [conn4085136] end connection 10.0.179.209:43490 (1506 connections now open) 2016-03-13T09:44:13.421+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43498 #4085148 (1506 connections now open) 2016-03-13T09:44:13.421+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43499 #4085149 (1507 connections now open) 2016-03-13T09:44:14.032+0000 I NETWORK [conn4085139] end connection 10.0.26.69:45466 (1506 connections now open) 2016-03-13T09:44:14.032+0000 I NETWORK [conn4085138] end connection 10.0.26.69:45465 (1506 connections now open) 2016-03-13T09:44:14.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45486 #4085150 (1506 connections now open) 2016-03-13T09:44:14.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45487 #4085151 (1507 connections now open) 2016-03-13T09:44:16.117+0000 I NETWORK [conn4077016] end connection 10.0.216.90:53600 (1506 connections now open) 2016-03-13T09:44:16.117+0000 I NETWORK [conn4084036] end connection 10.0.26.69:44341 (1505 connections now open) 2016-03-13T09:44:16.117+0000 I NETWORK [conn4082425] end connection 10.0.27.237:36401 (1504 connections now open) 2016-03-13T09:44:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60372 #4085152 (1505 connections now open) 2016-03-13T09:44:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38882 #4085153 (1506 connections now open) 2016-03-13T09:44:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45490 #4085154 (1507 connections now open) 2016-03-13T09:44:16.683+0000 I NETWORK [conn4085141] end connection 10.0.27.237:38860 (1506 connections now open) 2016-03-13T09:44:16.683+0000 I NETWORK [conn4085140] end connection 10.0.27.237:38859 (1506 connections now open) 2016-03-13T09:44:16.683+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38884 #4085155 (1506 connections now open) 2016-03-13T09:44:16.684+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38883 #4085156 (1507 connections now open) 2016-03-13T09:44:18.971+0000 I NETWORK [conn4085143] end connection 10.0.7.113:34111 (1506 connections now open) 2016-03-13T09:44:18.971+0000 I NETWORK [conn4085142] end connection 10.0.7.113:34110 (1506 connections now open) 2016-03-13T09:44:18.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34123 #4085157 (1507 connections now open) 2016-03-13T09:44:18.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34124 #4085158 (1507 connections now open) 2016-03-13T09:44:19.979+0000 I NETWORK [conn4085145] end connection 10.0.216.90:60365 (1506 connections now open) 2016-03-13T09:44:19.979+0000 I NETWORK [conn4085144] end connection 10.0.216.90:60364 (1506 connections now open) 2016-03-13T09:44:19.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60379 #4085159 (1506 connections now open) 2016-03-13T09:44:19.980+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60378 #4085160 (1507 connections now open) 2016-03-13T09:44:20.996+0000 I NETWORK [conn4085147] end connection 10.0.20.53:53821 (1506 connections now open) 2016-03-13T09:44:20.996+0000 I NETWORK [conn4085146] end connection 10.0.20.53:53822 (1506 connections now open) 2016-03-13T09:44:20.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53836 #4085161 (1506 connections now open) 2016-03-13T09:44:20.997+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53837 #4085162 (1507 connections now open) 2016-03-13T09:44:23.434+0000 I NETWORK [conn4085149] end connection 10.0.179.209:43499 (1506 connections now open) 2016-03-13T09:44:23.434+0000 I NETWORK [conn4085148] end connection 10.0.179.209:43498 (1506 connections now open) 2016-03-13T09:44:23.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43516 #4085163 (1506 connections now open) 2016-03-13T09:44:23.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43515 #4085164 (1507 connections now open) 2016-03-13T09:44:24.044+0000 I NETWORK [conn4085151] end connection 10.0.26.69:45487 (1506 connections now open) 2016-03-13T09:44:24.044+0000 I NETWORK [conn4085150] end connection 10.0.26.69:45486 (1505 connections now open) 2016-03-13T09:44:24.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45496 #4085165 (1507 connections now open) 2016-03-13T09:44:24.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45497 #4085166 (1507 connections now open) 2016-03-13T09:44:26.695+0000 I NETWORK [conn4085156] end connection 10.0.27.237:38883 (1506 connections now open) 2016-03-13T09:44:26.696+0000 I NETWORK [conn4085155] end connection 10.0.27.237:38884 (1505 connections now open) 2016-03-13T09:44:26.696+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38892 #4085167 (1506 connections now open) 2016-03-13T09:44:26.696+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38893 #4085168 (1507 connections now open) 2016-03-13T09:44:28.114+0000 I NETWORK [conn4082427] end connection 10.0.27.237:36402 (1506 connections now open) 2016-03-13T09:44:28.114+0000 I NETWORK [conn4081654] end connection 10.0.26.69:42758 (1506 connections now open) 2016-03-13T09:44:28.114+0000 I NETWORK [conn4082816] end connection 10.0.216.90:58170 (1506 connections now open) 2016-03-13T09:44:28.114+0000 I NETWORK [conn4081274] end connection 10.0.7.113:59632 (1503 connections now open) 2016-03-13T09:44:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38897 #4085169 (1504 connections now open) 2016-03-13T09:44:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45502 #4085170 (1505 connections now open) 2016-03-13T09:44:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60396 #4085171 (1506 connections now open) 2016-03-13T09:44:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34141 #4085172 (1507 connections now open) 2016-03-13T09:44:28.985+0000 I NETWORK [conn4085158] end connection 10.0.7.113:34124 (1506 connections now open) 2016-03-13T09:44:28.985+0000 I NETWORK [conn4085157] end connection 10.0.7.113:34123 (1506 connections now open) 2016-03-13T09:44:28.986+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34144 #4085173 (1506 connections now open) 2016-03-13T09:44:28.986+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34143 #4085174 (1507 connections now open) 2016-03-13T09:44:29.992+0000 I NETWORK [conn4085159] end connection 10.0.216.90:60379 (1506 connections now open) 2016-03-13T09:44:29.992+0000 I NETWORK [conn4085160] end connection 10.0.216.90:60378 (1506 connections now open) 2016-03-13T09:44:29.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60400 #4085175 (1506 connections now open) 2016-03-13T09:44:29.993+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60401 #4085176 (1507 connections now open) 2016-03-13T09:44:31.009+0000 I NETWORK [conn4085161] end connection 10.0.20.53:53836 (1506 connections now open) 2016-03-13T09:44:31.009+0000 I NETWORK [conn4085162] end connection 10.0.20.53:53837 (1505 connections now open) 2016-03-13T09:44:31.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53860 #4085177 (1506 connections now open) 2016-03-13T09:44:31.010+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53861 #4085178 (1507 connections now open) 2016-03-13T09:44:33.447+0000 I NETWORK [conn4085163] end connection 10.0.179.209:43516 (1506 connections now open) 2016-03-13T09:44:33.447+0000 I NETWORK [conn4085164] end connection 10.0.179.209:43515 (1506 connections now open) 2016-03-13T09:44:33.448+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43536 #4085179 (1506 connections now open) 2016-03-13T09:44:33.448+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43537 #4085180 (1507 connections now open) 2016-03-13T09:44:34.057+0000 I NETWORK [conn4085165] end connection 10.0.26.69:45496 (1506 connections now open) 2016-03-13T09:44:34.057+0000 I NETWORK [conn4085166] end connection 10.0.26.69:45497 (1506 connections now open) 2016-03-13T09:44:34.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45509 #4085181 (1506 connections now open) 2016-03-13T09:44:34.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45510 #4085182 (1507 connections now open) 2016-03-13T09:44:36.708+0000 I NETWORK [conn4085167] end connection 10.0.27.237:38892 (1506 connections now open) 2016-03-13T09:44:36.709+0000 I NETWORK [conn4085168] end connection 10.0.27.237:38893 (1505 connections now open) 2016-03-13T09:44:36.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38905 #4085183 (1506 connections now open) 2016-03-13T09:44:36.709+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38906 #4085184 (1507 connections now open) 2016-03-13T09:44:38.673+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:44:38.477+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:44:38.998+0000 I NETWORK [conn4085174] end connection 10.0.7.113:34143 (1506 connections now open) 2016-03-13T09:44:38.998+0000 I NETWORK [conn4085173] end connection 10.0.7.113:34144 (1506 connections now open) 2016-03-13T09:44:38.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34151 #4085185 (1506 connections now open) 2016-03-13T09:44:38.999+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34150 #4085186 (1507 connections now open) 2016-03-13T09:44:40.005+0000 I NETWORK [conn4085176] end connection 10.0.216.90:60401 (1506 connections now open) 2016-03-13T09:44:40.005+0000 I NETWORK [conn4085175] end connection 10.0.216.90:60400 (1505 connections now open) 2016-03-13T09:44:40.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60407 #4085187 (1506 connections now open) 2016-03-13T09:44:40.014+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60406 #4085188 (1507 connections now open) 2016-03-13T09:44:41.022+0000 I NETWORK [conn4085177] end connection 10.0.20.53:53860 (1506 connections now open) 2016-03-13T09:44:41.022+0000 I NETWORK [conn4085178] end connection 10.0.20.53:53861 (1506 connections now open) 2016-03-13T09:44:41.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53867 #4085189 (1506 connections now open) 2016-03-13T09:44:41.032+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53866 #4085190 (1507 connections now open) 2016-03-13T09:44:43.460+0000 I NETWORK [conn4085179] end connection 10.0.179.209:43536 (1506 connections now open) 2016-03-13T09:44:43.460+0000 I NETWORK [conn4085180] end connection 10.0.179.209:43537 (1505 connections now open) 2016-03-13T09:44:43.461+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43545 #4085191 (1506 connections now open) 2016-03-13T09:44:43.461+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43544 #4085192 (1507 connections now open) 2016-03-13T09:44:44.070+0000 I NETWORK [conn4085182] end connection 10.0.26.69:45510 (1506 connections now open) 2016-03-13T09:44:44.070+0000 I NETWORK [conn4085181] end connection 10.0.26.69:45509 (1505 connections now open) 2016-03-13T09:44:44.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45526 #4085193 (1506 connections now open) 2016-03-13T09:44:44.071+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45527 #4085194 (1507 connections now open) 2016-03-13T09:44:46.117+0000 I NETWORK [conn4083236] end connection 10.0.216.90:58432 (1506 connections now open) 2016-03-13T09:44:46.117+0000 I NETWORK [conn4081732] end connection 10.0.26.69:42812 (1506 connections now open) 2016-03-13T09:44:46.117+0000 I NETWORK [conn4082053] end connection 10.0.27.237:36254 (1504 connections now open) 2016-03-13T09:44:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60414 #4085195 (1505 connections now open) 2016-03-13T09:44:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45529 #4085196 (1506 connections now open) 2016-03-13T09:44:46.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38923 #4085197 (1507 connections now open) 2016-03-13T09:44:46.721+0000 I NETWORK [conn4085183] end connection 10.0.27.237:38905 (1506 connections now open) 2016-03-13T09:44:46.721+0000 I NETWORK [conn4085184] end connection 10.0.27.237:38906 (1505 connections now open) 2016-03-13T09:44:46.721+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38924 #4085198 (1506 connections now open) 2016-03-13T09:44:46.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38925 #4085199 (1507 connections now open) 2016-03-13T09:44:49.011+0000 I NETWORK [conn4085186] end connection 10.0.7.113:34150 (1506 connections now open) 2016-03-13T09:44:49.011+0000 I NETWORK [conn4085185] end connection 10.0.7.113:34151 (1506 connections now open) 2016-03-13T09:44:49.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34164 #4085200 (1506 connections now open) 2016-03-13T09:44:49.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34165 #4085201 (1507 connections now open) 2016-03-13T09:44:50.026+0000 I NETWORK [conn4085187] end connection 10.0.216.90:60407 (1506 connections now open) 2016-03-13T09:44:50.026+0000 I NETWORK [conn4085188] end connection 10.0.216.90:60406 (1506 connections now open) 2016-03-13T09:44:50.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60419 #4085202 (1506 connections now open) 2016-03-13T09:44:50.027+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60420 #4085203 (1507 connections now open) 2016-03-13T09:44:51.044+0000 I NETWORK [conn4085190] end connection 10.0.20.53:53866 (1506 connections now open) 2016-03-13T09:44:51.044+0000 I NETWORK [conn4085189] end connection 10.0.20.53:53867 (1506 connections now open) 2016-03-13T09:44:51.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53883 #4085204 (1506 connections now open) 2016-03-13T09:44:51.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53882 #4085205 (1507 connections now open) 2016-03-13T09:44:53.473+0000 I NETWORK [conn4085191] end connection 10.0.179.209:43545 (1506 connections now open) 2016-03-13T09:44:53.473+0000 I NETWORK [conn4085192] end connection 10.0.179.209:43544 (1505 connections now open) 2016-03-13T09:44:53.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43558 #4085206 (1506 connections now open) 2016-03-13T09:44:53.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43559 #4085207 (1507 connections now open) 2016-03-13T09:44:54.083+0000 I NETWORK [conn4085194] end connection 10.0.26.69:45527 (1506 connections now open) 2016-03-13T09:44:54.083+0000 I NETWORK [conn4085193] end connection 10.0.26.69:45526 (1506 connections now open) 2016-03-13T09:44:54.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45535 #4085208 (1506 connections now open) 2016-03-13T09:44:54.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45536 #4085209 (1507 connections now open) 2016-03-13T09:44:56.734+0000 I NETWORK [conn4085199] end connection 10.0.27.237:38925 (1506 connections now open) 2016-03-13T09:44:56.734+0000 I NETWORK [conn4085198] end connection 10.0.27.237:38924 (1506 connections now open) 2016-03-13T09:44:56.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38932 #4085210 (1506 connections now open) 2016-03-13T09:44:56.734+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38933 #4085211 (1507 connections now open) 2016-03-13T09:44:58.114+0000 I NETWORK [conn4081261] end connection 10.0.179.209:40719 (1506 connections now open) 2016-03-13T09:44:58.114+0000 I NETWORK [conn4080322] end connection 10.0.27.237:35334 (1505 connections now open) 2016-03-13T09:44:58.114+0000 I NETWORK [conn4081493] end connection 10.0.7.113:59688 (1504 connections now open) 2016-03-13T09:44:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43563 #4085212 (1505 connections now open) 2016-03-13T09:44:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38938 #4085213 (1506 connections now open) 2016-03-13T09:44:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34181 #4085214 (1507 connections now open) 2016-03-13T09:44:59.025+0000 I NETWORK [conn4085200] end connection 10.0.7.113:34164 (1506 connections now open) 2016-03-13T09:44:59.025+0000 I NETWORK [conn4085201] end connection 10.0.7.113:34165 (1506 connections now open) 2016-03-13T09:44:59.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34183 #4085215 (1506 connections now open) 2016-03-13T09:44:59.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34184 #4085216 (1507 connections now open) 2016-03-13T09:45:00.040+0000 I NETWORK [conn4085202] end connection 10.0.216.90:60419 (1506 connections now open) 2016-03-13T09:45:00.040+0000 I NETWORK [conn4085203] end connection 10.0.216.90:60420 (1506 connections now open) 2016-03-13T09:45:00.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60439 #4085217 (1506 connections now open) 2016-03-13T09:45:00.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60438 #4085218 (1507 connections now open) 2016-03-13T09:45:01.058+0000 I NETWORK [conn4085205] end connection 10.0.20.53:53882 (1506 connections now open) 2016-03-13T09:45:01.058+0000 I NETWORK [conn4085204] end connection 10.0.20.53:53883 (1505 connections now open) 2016-03-13T09:45:01.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53902 #4085219 (1506 connections now open) 2016-03-13T09:45:01.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53903 #4085220 (1507 connections now open) 2016-03-13T09:45:03.486+0000 I NETWORK [conn4085206] end connection 10.0.179.209:43558 (1506 connections now open) 2016-03-13T09:45:03.486+0000 I NETWORK [conn4085207] end connection 10.0.179.209:43559 (1506 connections now open) 2016-03-13T09:45:03.495+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43582 #4085221 (1506 connections now open) 2016-03-13T09:45:03.495+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43583 #4085222 (1507 connections now open) 2016-03-13T09:45:04.104+0000 I NETWORK [conn4085209] end connection 10.0.26.69:45536 (1506 connections now open) 2016-03-13T09:45:04.104+0000 I NETWORK [conn4085208] end connection 10.0.26.69:45535 (1506 connections now open) 2016-03-13T09:45:04.104+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45552 #4085223 (1506 connections now open) 2016-03-13T09:45:04.105+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45553 #4085224 (1507 connections now open) 2016-03-13T09:45:06.746+0000 I NETWORK [conn4085210] end connection 10.0.27.237:38932 (1506 connections now open) 2016-03-13T09:45:06.746+0000 I NETWORK [conn4085211] end connection 10.0.27.237:38933 (1506 connections now open) 2016-03-13T09:45:06.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38950 #4085225 (1506 connections now open) 2016-03-13T09:45:06.747+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38949 #4085226 (1507 connections now open) 2016-03-13T09:45:08.889+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:45:08.673+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:45:09.038+0000 I NETWORK [conn4085216] end connection 10.0.7.113:34184 (1506 connections now open) 2016-03-13T09:45:09.038+0000 I NETWORK [conn4085215] end connection 10.0.7.113:34183 (1505 connections now open) 2016-03-13T09:45:09.046+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34197 #4085227 (1506 connections now open) 2016-03-13T09:45:09.046+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34198 #4085228 (1507 connections now open) 2016-03-13T09:45:10.053+0000 I NETWORK [conn4085218] end connection 10.0.216.90:60438 (1506 connections now open) 2016-03-13T09:45:10.053+0000 I NETWORK [conn4085217] end connection 10.0.216.90:60439 (1506 connections now open) 2016-03-13T09:45:10.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60449 #4085229 (1506 connections now open) 2016-03-13T09:45:10.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60450 #4085230 (1507 connections now open) 2016-03-13T09:45:11.071+0000 I NETWORK [conn4085220] end connection 10.0.20.53:53903 (1506 connections now open) 2016-03-13T09:45:11.071+0000 I NETWORK [conn4085219] end connection 10.0.20.53:53902 (1506 connections now open) 2016-03-13T09:45:11.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53912 #4085231 (1506 connections now open) 2016-03-13T09:45:11.072+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53911 #4085232 (1507 connections now open) 2016-03-13T09:45:13.507+0000 I NETWORK [conn4085221] end connection 10.0.179.209:43582 (1506 connections now open) 2016-03-13T09:45:13.508+0000 I NETWORK [conn4085222] end connection 10.0.179.209:43583 (1505 connections now open) 2016-03-13T09:45:13.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43593 #4085233 (1506 connections now open) 2016-03-13T09:45:13.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43592 #4085234 (1507 connections now open) 2016-03-13T09:45:14.117+0000 I NETWORK [conn4085224] end connection 10.0.26.69:45553 (1506 connections now open) 2016-03-13T09:45:14.117+0000 I NETWORK [conn4085223] end connection 10.0.26.69:45552 (1506 connections now open) 2016-03-13T09:45:14.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45574 #4085235 (1507 connections now open) 2016-03-13T09:45:14.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45575 #4085236 (1507 connections now open) 2016-03-13T09:45:16.116+0000 I NETWORK [conn4081974] end connection 10.0.20.53:51166 (1506 connections now open) 2016-03-13T09:45:16.117+0000 I NETWORK [conn4081136] end connection 10.0.7.113:59586 (1505 connections now open) 2016-03-13T09:45:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53924 #4085237 (1506 connections now open) 2016-03-13T09:45:16.117+0000 I NETWORK [conn4082262] end connection 10.0.179.209:41013 (1505 connections now open) 2016-03-13T09:45:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34204 #4085238 (1506 connections now open) 2016-03-13T09:45:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43596 #4085239 (1507 connections now open) 2016-03-13T09:45:16.759+0000 I NETWORK [conn4085226] end connection 10.0.27.237:38949 (1506 connections now open) 2016-03-13T09:45:16.759+0000 I NETWORK [conn4085225] end connection 10.0.27.237:38950 (1506 connections now open) 2016-03-13T09:45:16.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38973 #4085240 (1506 connections now open) 2016-03-13T09:45:16.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38974 #4085241 (1507 connections now open) 2016-03-13T09:45:19.059+0000 I NETWORK [conn4085228] end connection 10.0.7.113:34198 (1506 connections now open) 2016-03-13T09:45:19.059+0000 I NETWORK [conn4085227] end connection 10.0.7.113:34197 (1506 connections now open) 2016-03-13T09:45:19.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34210 #4085242 (1506 connections now open) 2016-03-13T09:45:19.060+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34211 #4085243 (1507 connections now open) 2016-03-13T09:45:20.066+0000 I NETWORK [conn4085230] end connection 10.0.216.90:60450 (1506 connections now open) 2016-03-13T09:45:20.066+0000 I NETWORK [conn4085229] end connection 10.0.216.90:60449 (1506 connections now open) 2016-03-13T09:45:20.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60463 #4085244 (1506 connections now open) 2016-03-13T09:45:20.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60464 #4085245 (1507 connections now open) 2016-03-13T09:45:21.084+0000 I NETWORK [conn4085231] end connection 10.0.20.53:53912 (1506 connections now open) 2016-03-13T09:45:21.084+0000 I NETWORK [conn4085232] end connection 10.0.20.53:53911 (1506 connections now open) 2016-03-13T09:45:21.085+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53927 #4085246 (1506 connections now open) 2016-03-13T09:45:21.085+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53928 #4085247 (1507 connections now open) 2016-03-13T09:45:23.520+0000 I NETWORK [conn4085234] end connection 10.0.179.209:43592 (1506 connections now open) 2016-03-13T09:45:23.520+0000 I NETWORK [conn4085233] end connection 10.0.179.209:43593 (1506 connections now open) 2016-03-13T09:45:23.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43609 #4085248 (1506 connections now open) 2016-03-13T09:45:23.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43610 #4085249 (1507 connections now open) 2016-03-13T09:45:24.130+0000 I NETWORK [conn4085236] end connection 10.0.26.69:45575 (1506 connections now open) 2016-03-13T09:45:24.131+0000 I NETWORK [conn4085235] end connection 10.0.26.69:45574 (1506 connections now open) 2016-03-13T09:45:24.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45582 #4085250 (1506 connections now open) 2016-03-13T09:45:24.131+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45583 #4085251 (1507 connections now open) 2016-03-13T09:45:26.773+0000 I NETWORK [conn4085241] end connection 10.0.27.237:38974 (1506 connections now open) 2016-03-13T09:45:26.773+0000 I NETWORK [conn4085240] end connection 10.0.27.237:38973 (1506 connections now open) 2016-03-13T09:45:26.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38983 #4085252 (1506 connections now open) 2016-03-13T09:45:26.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38982 #4085253 (1507 connections now open) 2016-03-13T09:45:28.113+0000 I NETWORK [conn4082063] end connection 10.0.179.209:40926 (1506 connections now open) 2016-03-13T09:45:28.113+0000 I NETWORK [conn4081695] end connection 10.0.20.53:51040 (1506 connections now open) 2016-03-13T09:45:28.113+0000 I NETWORK [conn4082851] end connection 10.0.216.90:58196 (1504 connections now open) 2016-03-13T09:45:28.113+0000 I NETWORK [conn4082342] end connection 10.0.7.113:59950 (1503 connections now open) 2016-03-13T09:45:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43614 #4085254 (1504 connections now open) 2016-03-13T09:45:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53944 #4085255 (1505 connections now open) 2016-03-13T09:45:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60473 #4085256 (1506 connections now open) 2016-03-13T09:45:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34228 #4085257 (1507 connections now open) 2016-03-13T09:45:29.072+0000 I NETWORK [conn4085243] end connection 10.0.7.113:34211 (1506 connections now open) 2016-03-13T09:45:29.072+0000 I NETWORK [conn4085242] end connection 10.0.7.113:34210 (1506 connections now open) 2016-03-13T09:45:29.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34229 #4085258 (1506 connections now open) 2016-03-13T09:45:29.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34230 #4085259 (1507 connections now open) 2016-03-13T09:45:30.079+0000 I NETWORK [conn4085245] end connection 10.0.216.90:60464 (1506 connections now open) 2016-03-13T09:45:30.079+0000 I NETWORK [conn4085244] end connection 10.0.216.90:60463 (1506 connections now open) 2016-03-13T09:45:30.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60488 #4085260 (1506 connections now open) 2016-03-13T09:45:30.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60487 #4085261 (1507 connections now open) 2016-03-13T09:45:31.097+0000 I NETWORK [conn4085246] end connection 10.0.20.53:53927 (1506 connections now open) 2016-03-13T09:45:31.097+0000 I NETWORK [conn4085247] end connection 10.0.20.53:53928 (1506 connections now open) 2016-03-13T09:45:31.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53949 #4085262 (1506 connections now open) 2016-03-13T09:45:31.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53950 #4085263 (1507 connections now open) 2016-03-13T09:45:33.533+0000 I NETWORK [conn4085248] end connection 10.0.179.209:43609 (1506 connections now open) 2016-03-13T09:45:33.533+0000 I NETWORK [conn4085249] end connection 10.0.179.209:43610 (1506 connections now open) 2016-03-13T09:45:33.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43629 #4085264 (1506 connections now open) 2016-03-13T09:45:33.534+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43630 #4085265 (1507 connections now open) 2016-03-13T09:45:34.144+0000 I NETWORK [conn4085250] end connection 10.0.26.69:45582 (1506 connections now open) 2016-03-13T09:45:34.144+0000 I NETWORK [conn4085251] end connection 10.0.26.69:45583 (1506 connections now open) 2016-03-13T09:45:34.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45593 #4085266 (1506 connections now open) 2016-03-13T09:45:34.144+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45594 #4085267 (1507 connections now open) 2016-03-13T09:45:36.786+0000 I NETWORK [conn4085253] end connection 10.0.27.237:38982 (1506 connections now open) 2016-03-13T09:45:36.786+0000 I NETWORK [conn4085252] end connection 10.0.27.237:38983 (1506 connections now open) 2016-03-13T09:45:36.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38997 #4085268 (1506 connections now open) 2016-03-13T09:45:36.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:38998 #4085269 (1507 connections now open) 2016-03-13T09:45:39.033+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:45:38.889+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:45:39.086+0000 I NETWORK [conn4085259] end connection 10.0.7.113:34230 (1506 connections now open) 2016-03-13T09:45:39.086+0000 I NETWORK [conn4085258] end connection 10.0.7.113:34229 (1506 connections now open) 2016-03-13T09:45:39.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34236 #4085270 (1506 connections now open) 2016-03-13T09:45:39.087+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34237 #4085271 (1507 connections now open) 2016-03-13T09:45:40.092+0000 I NETWORK [conn4085260] end connection 10.0.216.90:60488 (1506 connections now open) 2016-03-13T09:45:40.092+0000 I NETWORK [conn4085261] end connection 10.0.216.90:60487 (1506 connections now open) 2016-03-13T09:45:40.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60496 #4085272 (1506 connections now open) 2016-03-13T09:45:40.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60495 #4085273 (1507 connections now open) 2016-03-13T09:45:41.111+0000 I NETWORK [conn4085262] end connection 10.0.20.53:53949 (1506 connections now open) 2016-03-13T09:45:41.111+0000 I NETWORK [conn4085263] end connection 10.0.20.53:53950 (1506 connections now open) 2016-03-13T09:45:41.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53958 #4085274 (1506 connections now open) 2016-03-13T09:45:41.120+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53957 #4085275 (1507 connections now open) 2016-03-13T09:45:43.546+0000 I NETWORK [conn4085265] end connection 10.0.179.209:43630 (1506 connections now open) 2016-03-13T09:45:43.546+0000 I NETWORK [conn4085264] end connection 10.0.179.209:43629 (1506 connections now open) 2016-03-13T09:45:43.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43637 #4085276 (1506 connections now open) 2016-03-13T09:45:43.547+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43638 #4085277 (1507 connections now open) 2016-03-13T09:45:44.156+0000 I NETWORK [conn4085266] end connection 10.0.26.69:45593 (1506 connections now open) 2016-03-13T09:45:44.157+0000 I NETWORK [conn4085267] end connection 10.0.26.69:45594 (1505 connections now open) 2016-03-13T09:45:44.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45610 #4085278 (1506 connections now open) 2016-03-13T09:45:44.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45611 #4085279 (1507 connections now open) 2016-03-13T09:45:46.116+0000 I NETWORK [conn4085113] end connection 10.0.179.209:43458 (1506 connections now open) 2016-03-13T09:45:46.116+0000 I NETWORK [conn4082289] end connection 10.0.20.53:51295 (1505 connections now open) 2016-03-13T09:45:46.117+0000 I NETWORK [conn4081884] end connection 10.0.27.237:36179 (1504 connections now open) 2016-03-13T09:45:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43641 #4085280 (1505 connections now open) 2016-03-13T09:45:46.117+0000 I NETWORK [conn4082999] end connection 10.0.7.113:60213 (1504 connections now open) 2016-03-13T09:45:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53970 #4085281 (1506 connections now open) 2016-03-13T09:45:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34244 #4085282 (1506 connections now open) 2016-03-13T09:45:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39016 #4085283 (1507 connections now open) 2016-03-13T09:45:46.799+0000 I NETWORK [conn4085269] end connection 10.0.27.237:38998 (1506 connections now open) 2016-03-13T09:45:46.799+0000 I NETWORK [conn4085268] end connection 10.0.27.237:38997 (1506 connections now open) 2016-03-13T09:45:46.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39017 #4085284 (1506 connections now open) 2016-03-13T09:45:46.799+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39018 #4085285 (1507 connections now open) 2016-03-13T09:45:49.100+0000 I NETWORK [conn4085270] end connection 10.0.7.113:34236 (1506 connections now open) 2016-03-13T09:45:49.100+0000 I NETWORK [conn4085271] end connection 10.0.7.113:34237 (1506 connections now open) 2016-03-13T09:45:49.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34250 #4085286 (1507 connections now open) 2016-03-13T09:45:49.101+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34251 #4085287 (1507 connections now open) 2016-03-13T09:45:50.114+0000 I NETWORK [conn4085273] end connection 10.0.216.90:60495 (1506 connections now open) 2016-03-13T09:45:50.114+0000 I NETWORK [conn4085272] end connection 10.0.216.90:60496 (1506 connections now open) 2016-03-13T09:45:50.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60510 #4085288 (1506 connections now open) 2016-03-13T09:45:50.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60509 #4085289 (1507 connections now open) 2016-03-13T09:45:51.133+0000 I NETWORK [conn4085275] end connection 10.0.20.53:53957 (1506 connections now open) 2016-03-13T09:45:51.133+0000 I NETWORK [conn4085274] end connection 10.0.20.53:53958 (1506 connections now open) 2016-03-13T09:45:51.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53974 #4085290 (1506 connections now open) 2016-03-13T09:45:51.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53973 #4085291 (1507 connections now open) 2016-03-13T09:45:53.559+0000 I NETWORK [conn4085277] end connection 10.0.179.209:43638 (1506 connections now open) 2016-03-13T09:45:53.559+0000 I NETWORK [conn4085276] end connection 10.0.179.209:43637 (1506 connections now open) 2016-03-13T09:45:53.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43650 #4085292 (1506 connections now open) 2016-03-13T09:45:53.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43651 #4085293 (1507 connections now open) 2016-03-13T09:45:54.169+0000 I NETWORK [conn4085278] end connection 10.0.26.69:45610 (1506 connections now open) 2016-03-13T09:45:54.169+0000 I NETWORK [conn4085279] end connection 10.0.26.69:45611 (1506 connections now open) 2016-03-13T09:45:54.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45620 #4085294 (1506 connections now open) 2016-03-13T09:45:54.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45621 #4085295 (1507 connections now open) 2016-03-13T09:45:56.812+0000 I NETWORK [conn4085285] end connection 10.0.27.237:39018 (1506 connections now open) 2016-03-13T09:45:56.812+0000 I NETWORK [conn4085284] end connection 10.0.27.237:39017 (1506 connections now open) 2016-03-13T09:45:56.876+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39025 #4085296 (1506 connections now open) 2016-03-13T09:45:56.876+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39026 #4085297 (1507 connections now open) 2016-03-13T09:45:58.114+0000 I NETWORK [conn4081794] end connection 10.0.20.53:51091 (1506 connections now open) 2016-03-13T09:45:58.114+0000 I NETWORK [conn4081914] end connection 10.0.26.69:42892 (1506 connections now open) 2016-03-13T09:45:58.114+0000 I NETWORK [conn4083798] end connection 10.0.216.90:58997 (1506 connections now open) 2016-03-13T09:45:58.114+0000 I NETWORK [conn4081491] end connection 10.0.7.113:59687 (1503 connections now open) 2016-03-13T09:45:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45626 #4085298 (1504 connections now open) 2016-03-13T09:45:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60518 #4085299 (1505 connections now open) 2016-03-13T09:45:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53992 #4085300 (1506 connections now open) 2016-03-13T09:45:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34269 #4085301 (1507 connections now open) 2016-03-13T09:45:59.114+0000 I NETWORK [conn4085286] end connection 10.0.7.113:34250 (1506 connections now open) 2016-03-13T09:45:59.114+0000 I NETWORK [conn4085287] end connection 10.0.7.113:34251 (1506 connections now open) 2016-03-13T09:45:59.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34270 #4085302 (1506 connections now open) 2016-03-13T09:45:59.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34271 #4085303 (1507 connections now open) 2016-03-13T09:46:00.127+0000 I NETWORK [conn4085289] end connection 10.0.216.90:60509 (1506 connections now open) 2016-03-13T09:46:00.127+0000 I NETWORK [conn4085288] end connection 10.0.216.90:60510 (1505 connections now open) 2016-03-13T09:46:00.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60529 #4085304 (1506 connections now open) 2016-03-13T09:46:00.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60530 #4085305 (1507 connections now open) 2016-03-13T09:46:01.146+0000 I NETWORK [conn4085291] end connection 10.0.20.53:53973 (1506 connections now open) 2016-03-13T09:46:01.146+0000 I NETWORK [conn4085290] end connection 10.0.20.53:53974 (1506 connections now open) 2016-03-13T09:46:01.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53994 #4085306 (1506 connections now open) 2016-03-13T09:46:01.147+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:53995 #4085307 (1507 connections now open) 2016-03-13T09:46:03.572+0000 I NETWORK [conn4085293] end connection 10.0.179.209:43651 (1506 connections now open) 2016-03-13T09:46:03.572+0000 I NETWORK [conn4085292] end connection 10.0.179.209:43650 (1505 connections now open) 2016-03-13T09:46:03.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43673 #4085308 (1506 connections now open) 2016-03-13T09:46:03.582+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43674 #4085309 (1507 connections now open) 2016-03-13T09:46:04.183+0000 I NETWORK [conn4085295] end connection 10.0.26.69:45621 (1506 connections now open) 2016-03-13T09:46:04.183+0000 I NETWORK [conn4085294] end connection 10.0.26.69:45620 (1506 connections now open) 2016-03-13T09:46:04.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45636 #4085310 (1506 connections now open) 2016-03-13T09:46:04.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45635 #4085311 (1507 connections now open) 2016-03-13T09:46:06.889+0000 I NETWORK [conn4085297] end connection 10.0.27.237:39026 (1506 connections now open) 2016-03-13T09:46:06.889+0000 I NETWORK [conn4085296] end connection 10.0.27.237:39025 (1506 connections now open) 2016-03-13T09:46:06.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39043 #4085312 (1506 connections now open) 2016-03-13T09:46:06.890+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39042 #4085313 (1507 connections now open) 2016-03-13T09:46:09.127+0000 I NETWORK [conn4085303] end connection 10.0.7.113:34271 (1506 connections now open) 2016-03-13T09:46:09.127+0000 I NETWORK [conn4085302] end connection 10.0.7.113:34270 (1506 connections now open) 2016-03-13T09:46:09.135+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34283 #4085314 (1506 connections now open) 2016-03-13T09:46:09.135+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34282 #4085315 (1507 connections now open) 2016-03-13T09:46:09.318+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:46:09.033+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:46:10.140+0000 I NETWORK [conn4085304] end connection 10.0.216.90:60529 (1506 connections now open) 2016-03-13T09:46:10.140+0000 I NETWORK [conn4085305] end connection 10.0.216.90:60530 (1506 connections now open) 2016-03-13T09:46:10.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60539 #4085316 (1506 connections now open) 2016-03-13T09:46:10.141+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60538 #4085317 (1507 connections now open) 2016-03-13T09:46:11.159+0000 I NETWORK [conn4085307] end connection 10.0.20.53:53995 (1506 connections now open) 2016-03-13T09:46:11.159+0000 I NETWORK [conn4085306] end connection 10.0.20.53:53994 (1506 connections now open) 2016-03-13T09:46:11.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54002 #4085318 (1506 connections now open) 2016-03-13T09:46:11.160+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54001 #4085319 (1507 connections now open) 2016-03-13T09:46:13.594+0000 I NETWORK [conn4085308] end connection 10.0.179.209:43673 (1506 connections now open) 2016-03-13T09:46:13.594+0000 I NETWORK [conn4085309] end connection 10.0.179.209:43674 (1506 connections now open) 2016-03-13T09:46:13.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43681 #4085320 (1506 connections now open) 2016-03-13T09:46:13.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43682 #4085321 (1507 connections now open) 2016-03-13T09:46:14.196+0000 I NETWORK [conn4085310] end connection 10.0.26.69:45636 (1506 connections now open) 2016-03-13T09:46:14.196+0000 I NETWORK [conn4085311] end connection 10.0.26.69:45635 (1506 connections now open) 2016-03-13T09:46:14.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45657 #4085322 (1506 connections now open) 2016-03-13T09:46:14.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45658 #4085323 (1507 connections now open) 2016-03-13T09:46:16.117+0000 I NETWORK [conn4081363] end connection 10.0.20.53:50850 (1506 connections now open) 2016-03-13T09:46:16.117+0000 I NETWORK [conn4081921] end connection 10.0.179.209:40867 (1506 connections now open) 2016-03-13T09:46:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43686 #4085324 (1506 connections now open) 2016-03-13T09:46:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54014 #4085325 (1507 connections now open) 2016-03-13T09:46:16.119+0000 I NETWORK [conn4081733] end connection 10.0.26.69:42813 (1506 connections now open) 2016-03-13T09:46:16.119+0000 I NETWORK [conn4083034] end connection 10.0.27.237:36674 (1506 connections now open) 2016-03-13T09:46:16.119+0000 I NETWORK [conn4080801] end connection 10.0.7.113:59420 (1506 connections now open) 2016-03-13T09:46:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45660 #4085326 (1505 connections now open) 2016-03-13T09:46:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39064 #4085327 (1506 connections now open) 2016-03-13T09:46:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34290 #4085328 (1507 connections now open) 2016-03-13T09:46:16.903+0000 I NETWORK [conn4085312] end connection 10.0.27.237:39043 (1506 connections now open) 2016-03-13T09:46:16.903+0000 I NETWORK [conn4085313] end connection 10.0.27.237:39042 (1506 connections now open) 2016-03-13T09:46:16.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39066 #4085329 (1506 connections now open) 2016-03-13T09:46:16.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39065 #4085330 (1507 connections now open) 2016-03-13T09:46:19.148+0000 I NETWORK [conn4085314] end connection 10.0.7.113:34283 (1506 connections now open) 2016-03-13T09:46:19.148+0000 I NETWORK [conn4085315] end connection 10.0.7.113:34282 (1506 connections now open) 2016-03-13T09:46:19.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34296 #4085331 (1507 connections now open) 2016-03-13T09:46:19.149+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34295 #4085332 (1507 connections now open) 2016-03-13T09:46:20.153+0000 I NETWORK [conn4085317] end connection 10.0.216.90:60538 (1506 connections now open) 2016-03-13T09:46:20.153+0000 I NETWORK [conn4085316] end connection 10.0.216.90:60539 (1506 connections now open) 2016-03-13T09:46:20.153+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60549 #4085333 (1506 connections now open) 2016-03-13T09:46:20.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60550 #4085334 (1507 connections now open) 2016-03-13T09:46:21.173+0000 I NETWORK [conn4085318] end connection 10.0.20.53:54002 (1506 connections now open) 2016-03-13T09:46:21.173+0000 I NETWORK [conn4085319] end connection 10.0.20.53:54001 (1506 connections now open) 2016-03-13T09:46:21.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54016 #4085335 (1506 connections now open) 2016-03-13T09:46:21.173+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54017 #4085336 (1507 connections now open) 2016-03-13T09:46:23.615+0000 I NETWORK [conn4085320] end connection 10.0.179.209:43681 (1506 connections now open) 2016-03-13T09:46:23.615+0000 I NETWORK [conn4085321] end connection 10.0.179.209:43682 (1506 connections now open) 2016-03-13T09:46:23.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43699 #4085337 (1506 connections now open) 2016-03-13T09:46:23.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43698 #4085338 (1507 connections now open) 2016-03-13T09:46:24.210+0000 I NETWORK [conn4085322] end connection 10.0.26.69:45657 (1506 connections now open) 2016-03-13T09:46:24.210+0000 I NETWORK [conn4085323] end connection 10.0.26.69:45658 (1506 connections now open) 2016-03-13T09:46:24.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45666 #4085339 (1506 connections now open) 2016-03-13T09:46:24.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45667 #4085340 (1507 connections now open) 2016-03-13T09:46:26.915+0000 I NETWORK [conn4085330] end connection 10.0.27.237:39065 (1506 connections now open) 2016-03-13T09:46:26.915+0000 I NETWORK [conn4085329] end connection 10.0.27.237:39066 (1506 connections now open) 2016-03-13T09:46:26.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39074 #4085341 (1506 connections now open) 2016-03-13T09:46:26.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39075 #4085342 (1507 connections now open) 2016-03-13T09:46:28.114+0000 I NETWORK [conn4082211] end connection 10.0.7.113:59902 (1506 connections now open) 2016-03-13T09:46:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34314 #4085343 (1507 connections now open) 2016-03-13T09:46:29.161+0000 I NETWORK [conn4085331] end connection 10.0.7.113:34296 (1506 connections now open) 2016-03-13T09:46:29.161+0000 I NETWORK [conn4085332] end connection 10.0.7.113:34295 (1506 connections now open) 2016-03-13T09:46:29.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34315 #4085344 (1506 connections now open) 2016-03-13T09:46:29.162+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34316 #4085345 (1507 connections now open) 2016-03-13T09:46:30.166+0000 I NETWORK [conn4085334] end connection 10.0.216.90:60550 (1506 connections now open) 2016-03-13T09:46:30.166+0000 I NETWORK [conn4085333] end connection 10.0.216.90:60549 (1506 connections now open) 2016-03-13T09:46:30.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60573 #4085346 (1506 connections now open) 2016-03-13T09:46:30.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60572 #4085347 (1507 connections now open) 2016-03-13T09:46:31.186+0000 I NETWORK [conn4085336] end connection 10.0.20.53:54017 (1506 connections now open) 2016-03-13T09:46:31.186+0000 I NETWORK [conn4085335] end connection 10.0.20.53:54016 (1506 connections now open) 2016-03-13T09:46:31.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54041 #4085348 (1506 connections now open) 2016-03-13T09:46:31.187+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54042 #4085349 (1507 connections now open) 2016-03-13T09:46:33.628+0000 I NETWORK [conn4085337] end connection 10.0.179.209:43699 (1506 connections now open) 2016-03-13T09:46:33.629+0000 I NETWORK [conn4085338] end connection 10.0.179.209:43698 (1505 connections now open) 2016-03-13T09:46:33.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43718 #4085350 (1506 connections now open) 2016-03-13T09:46:33.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43719 #4085351 (1507 connections now open) 2016-03-13T09:46:34.223+0000 I NETWORK [conn4085339] end connection 10.0.26.69:45666 (1506 connections now open) 2016-03-13T09:46:34.223+0000 I NETWORK [conn4085340] end connection 10.0.26.69:45667 (1506 connections now open) 2016-03-13T09:46:34.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45680 #4085352 (1506 connections now open) 2016-03-13T09:46:34.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45679 #4085353 (1507 connections now open) 2016-03-13T09:46:36.929+0000 I NETWORK [conn4085342] end connection 10.0.27.237:39075 (1506 connections now open) 2016-03-13T09:46:36.929+0000 I NETWORK [conn4085341] end connection 10.0.27.237:39074 (1506 connections now open) 2016-03-13T09:46:36.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39090 #4085354 (1506 connections now open) 2016-03-13T09:46:36.930+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39089 #4085355 (1507 connections now open) 2016-03-13T09:46:39.175+0000 I NETWORK [conn4085344] end connection 10.0.7.113:34315 (1506 connections now open) 2016-03-13T09:46:39.175+0000 I NETWORK [conn4085345] end connection 10.0.7.113:34316 (1506 connections now open) 2016-03-13T09:46:39.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34324 #4085356 (1506 connections now open) 2016-03-13T09:46:39.175+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34323 #4085357 (1507 connections now open) 2016-03-13T09:46:39.498+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:46:39.318+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:46:40.179+0000 I NETWORK [conn4085346] end connection 10.0.216.90:60573 (1506 connections now open) 2016-03-13T09:46:40.179+0000 I NETWORK [conn4085347] end connection 10.0.216.90:60572 (1506 connections now open) 2016-03-13T09:46:40.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60581 #4085358 (1506 connections now open) 2016-03-13T09:46:40.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60582 #4085359 (1507 connections now open) 2016-03-13T09:46:41.199+0000 I NETWORK [conn4085348] end connection 10.0.20.53:54041 (1506 connections now open) 2016-03-13T09:46:41.199+0000 I NETWORK [conn4085349] end connection 10.0.20.53:54042 (1506 connections now open) 2016-03-13T09:46:41.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54048 #4085360 (1506 connections now open) 2016-03-13T09:46:41.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54049 #4085361 (1507 connections now open) 2016-03-13T09:46:43.641+0000 I NETWORK [conn4085350] end connection 10.0.179.209:43718 (1506 connections now open) 2016-03-13T09:46:43.641+0000 I NETWORK [conn4085351] end connection 10.0.179.209:43719 (1505 connections now open) 2016-03-13T09:46:43.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43725 #4085362 (1506 connections now open) 2016-03-13T09:46:43.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43726 #4085363 (1507 connections now open) 2016-03-13T09:46:44.237+0000 I NETWORK [conn4085352] end connection 10.0.26.69:45680 (1506 connections now open) 2016-03-13T09:46:44.237+0000 I NETWORK [conn4085353] end connection 10.0.26.69:45679 (1506 connections now open) 2016-03-13T09:46:44.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45698 #4085364 (1506 connections now open) 2016-03-13T09:46:44.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45697 #4085365 (1507 connections now open) 2016-03-13T09:46:46.117+0000 I NETWORK [conn4082935] end connection 10.0.20.53:51569 (1506 connections now open) 2016-03-13T09:46:46.117+0000 I NETWORK [conn4081157] end connection 10.0.7.113:59594 (1505 connections now open) 2016-03-13T09:46:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54060 #4085366 (1506 connections now open) 2016-03-13T09:46:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34330 #4085367 (1507 connections now open) 2016-03-13T09:46:46.942+0000 I NETWORK [conn4085354] end connection 10.0.27.237:39090 (1506 connections now open) 2016-03-13T09:46:46.942+0000 I NETWORK [conn4085355] end connection 10.0.27.237:39089 (1506 connections now open) 2016-03-13T09:46:46.943+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39110 #4085368 (1506 connections now open) 2016-03-13T09:46:46.943+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39111 #4085369 (1507 connections now open) 2016-03-13T09:46:49.188+0000 I NETWORK [conn4085357] end connection 10.0.7.113:34323 (1506 connections now open) 2016-03-13T09:46:49.188+0000 I NETWORK [conn4085356] end connection 10.0.7.113:34324 (1506 connections now open) 2016-03-13T09:46:49.189+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34336 #4085370 (1507 connections now open) 2016-03-13T09:46:49.189+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34337 #4085371 (1507 connections now open) 2016-03-13T09:46:50.256+0000 I NETWORK [conn4085358] end connection 10.0.216.90:60581 (1506 connections now open) 2016-03-13T09:46:50.256+0000 I NETWORK [conn4085359] end connection 10.0.216.90:60582 (1506 connections now open) 2016-03-13T09:46:50.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60594 #4085372 (1506 connections now open) 2016-03-13T09:46:50.257+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60595 #4085373 (1507 connections now open) 2016-03-13T09:46:51.222+0000 I NETWORK [conn4085361] end connection 10.0.20.53:54049 (1506 connections now open) 2016-03-13T09:46:51.222+0000 I NETWORK [conn4085360] end connection 10.0.20.53:54048 (1506 connections now open) 2016-03-13T09:46:51.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54063 #4085374 (1506 connections now open) 2016-03-13T09:46:51.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54062 #4085375 (1507 connections now open) 2016-03-13T09:46:53.654+0000 I NETWORK [conn4085363] end connection 10.0.179.209:43726 (1506 connections now open) 2016-03-13T09:46:53.654+0000 I NETWORK [conn4085362] end connection 10.0.179.209:43725 (1506 connections now open) 2016-03-13T09:46:53.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43740 #4085376 (1506 connections now open) 2016-03-13T09:46:53.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43739 #4085377 (1507 connections now open) 2016-03-13T09:46:54.258+0000 I NETWORK [conn4085364] end connection 10.0.26.69:45698 (1506 connections now open) 2016-03-13T09:46:54.258+0000 I NETWORK [conn4085365] end connection 10.0.26.69:45697 (1506 connections now open) 2016-03-13T09:46:54.259+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45707 #4085378 (1507 connections now open) 2016-03-13T09:46:54.259+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45708 #4085379 (1507 connections now open) 2016-03-13T09:46:56.955+0000 I NETWORK [conn4085368] end connection 10.0.27.237:39110 (1506 connections now open) 2016-03-13T09:46:56.955+0000 I NETWORK [conn4085369] end connection 10.0.27.237:39111 (1506 connections now open) 2016-03-13T09:46:56.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39117 #4085380 (1506 connections now open) 2016-03-13T09:46:56.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39118 #4085381 (1507 connections now open) 2016-03-13T09:46:58.113+0000 I NETWORK [conn4084631] end connection 10.0.179.209:42945 (1506 connections now open) 2016-03-13T09:46:58.113+0000 I NETWORK [conn4083671] end connection 10.0.216.90:58868 (1505 connections now open) 2016-03-13T09:46:58.113+0000 I NETWORK [conn4082309] end connection 10.0.26.69:43050 (1504 connections now open) 2016-03-13T09:46:58.113+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43743 #4085382 (1506 connections now open) 2016-03-13T09:46:58.113+0000 I NETWORK [conn4081313] end connection 10.0.7.113:59644 (1504 connections now open) 2016-03-13T09:46:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60604 #4085383 (1505 connections now open) 2016-03-13T09:46:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45711 #4085384 (1506 connections now open) 2016-03-13T09:46:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34353 #4085385 (1507 connections now open) 2016-03-13T09:46:59.201+0000 I NETWORK [conn4085370] end connection 10.0.7.113:34336 (1506 connections now open) 2016-03-13T09:46:59.201+0000 I NETWORK [conn4085371] end connection 10.0.7.113:34337 (1506 connections now open) 2016-03-13T09:46:59.202+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34355 #4085386 (1506 connections now open) 2016-03-13T09:46:59.202+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34356 #4085387 (1507 connections now open) 2016-03-13T09:47:00.269+0000 I NETWORK [conn4085372] end connection 10.0.216.90:60594 (1506 connections now open) 2016-03-13T09:47:00.269+0000 I NETWORK [conn4085373] end connection 10.0.216.90:60595 (1506 connections now open) 2016-03-13T09:47:00.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60615 #4085388 (1506 connections now open) 2016-03-13T09:47:00.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60614 #4085389 (1507 connections now open) 2016-03-13T09:47:01.235+0000 I NETWORK [conn4085375] end connection 10.0.20.53:54062 (1506 connections now open) 2016-03-13T09:47:01.235+0000 I NETWORK [conn4085374] end connection 10.0.20.53:54063 (1506 connections now open) 2016-03-13T09:47:01.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54084 #4085390 (1506 connections now open) 2016-03-13T09:47:01.236+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54083 #4085391 (1507 connections now open) 2016-03-13T09:47:03.666+0000 I NETWORK [conn4085376] end connection 10.0.179.209:43740 (1506 connections now open) 2016-03-13T09:47:03.666+0000 I NETWORK [conn4085377] end connection 10.0.179.209:43739 (1506 connections now open) 2016-03-13T09:47:03.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43761 #4085392 (1506 connections now open) 2016-03-13T09:47:03.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43762 #4085393 (1507 connections now open) 2016-03-13T09:47:04.272+0000 I NETWORK [conn4085379] end connection 10.0.26.69:45708 (1506 connections now open) 2016-03-13T09:47:04.272+0000 I NETWORK [conn4085378] end connection 10.0.26.69:45707 (1506 connections now open) 2016-03-13T09:47:04.273+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45722 #4085394 (1506 connections now open) 2016-03-13T09:47:04.273+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45723 #4085395 (1507 connections now open) 2016-03-13T09:47:06.976+0000 I NETWORK [conn4085381] end connection 10.0.27.237:39118 (1506 connections now open) 2016-03-13T09:47:06.976+0000 I NETWORK [conn4085380] end connection 10.0.27.237:39117 (1506 connections now open) 2016-03-13T09:47:06.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39135 #4085396 (1506 connections now open) 2016-03-13T09:47:06.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39134 #4085397 (1507 connections now open) 2016-03-13T09:47:09.215+0000 I NETWORK [conn4085386] end connection 10.0.7.113:34355 (1506 connections now open) 2016-03-13T09:47:09.215+0000 I NETWORK [conn4085387] end connection 10.0.7.113:34356 (1506 connections now open) 2016-03-13T09:47:09.225+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34367 #4085398 (1506 connections now open) 2016-03-13T09:47:09.225+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34368 #4085399 (1507 connections now open) 2016-03-13T09:47:09.747+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:47:09.498+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:47:10.283+0000 I NETWORK [conn4085389] end connection 10.0.216.90:60614 (1506 connections now open) 2016-03-13T09:47:10.283+0000 I NETWORK [conn4085388] end connection 10.0.216.90:60615 (1506 connections now open) 2016-03-13T09:47:10.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60625 #4085400 (1506 connections now open) 2016-03-13T09:47:10.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60626 #4085401 (1507 connections now open) 2016-03-13T09:47:11.249+0000 I NETWORK [conn4085391] end connection 10.0.20.53:54083 (1506 connections now open) 2016-03-13T09:47:11.249+0000 I NETWORK [conn4085390] end connection 10.0.20.53:54084 (1506 connections now open) 2016-03-13T09:47:11.249+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54092 #4085402 (1506 connections now open) 2016-03-13T09:47:11.249+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54091 #4085403 (1507 connections now open) 2016-03-13T09:47:13.688+0000 I NETWORK [conn4085392] end connection 10.0.179.209:43761 (1506 connections now open) 2016-03-13T09:47:13.688+0000 I NETWORK [conn4085393] end connection 10.0.179.209:43762 (1506 connections now open) 2016-03-13T09:47:13.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43770 #4085404 (1506 connections now open) 2016-03-13T09:47:13.698+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43771 #4085405 (1507 connections now open) 2016-03-13T09:47:14.286+0000 I NETWORK [conn4085394] end connection 10.0.26.69:45722 (1506 connections now open) 2016-03-13T09:47:14.286+0000 I NETWORK [conn4085395] end connection 10.0.26.69:45723 (1506 connections now open) 2016-03-13T09:47:14.286+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45743 #4085406 (1506 connections now open) 2016-03-13T09:47:14.286+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45742 #4085407 (1507 connections now open) 2016-03-13T09:47:16.117+0000 I NETWORK [conn4082181] end connection 10.0.179.209:40972 (1506 connections now open) 2016-03-13T09:47:16.117+0000 I NETWORK [conn4081369] end connection 10.0.20.53:50854 (1505 connections now open) 2016-03-13T09:47:16.117+0000 I NETWORK [conn4081772] end connection 10.0.216.90:57711 (1504 connections now open) 2016-03-13T09:47:16.117+0000 I NETWORK [conn4081910] end connection 10.0.26.69:42887 (1503 connections now open) 2016-03-13T09:47:16.117+0000 I NETWORK [conn4082494] end connection 10.0.27.237:36438 (1502 connections now open) 2016-03-13T09:47:16.117+0000 I NETWORK [conn4080769] end connection 10.0.7.113:59403 (1501 connections now open) 2016-03-13T09:47:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43775 #4085408 (1502 connections now open) 2016-03-13T09:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54104 #4085409 (1503 connections now open) 2016-03-13T09:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60631 #4085410 (1504 connections now open) 2016-03-13T09:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45747 #4085411 (1505 connections now open) 2016-03-13T09:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39157 #4085412 (1506 connections now open) 2016-03-13T09:47:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34374 #4085413 (1507 connections now open) 2016-03-13T09:47:16.989+0000 I NETWORK [conn4085396] end connection 10.0.27.237:39135 (1506 connections now open) 2016-03-13T09:47:16.989+0000 I NETWORK [conn4085397] end connection 10.0.27.237:39134 (1506 connections now open) 2016-03-13T09:47:16.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39158 #4085414 (1506 connections now open) 2016-03-13T09:47:16.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39159 #4085415 (1507 connections now open) 2016-03-13T09:47:19.238+0000 I NETWORK [conn4085398] end connection 10.0.7.113:34367 (1506 connections now open) 2016-03-13T09:47:19.238+0000 I NETWORK [conn4085399] end connection 10.0.7.113:34368 (1506 connections now open) 2016-03-13T09:47:19.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34380 #4085416 (1506 connections now open) 2016-03-13T09:47:19.239+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34381 #4085417 (1507 connections now open) 2016-03-13T09:47:20.296+0000 I NETWORK [conn4085400] end connection 10.0.216.90:60625 (1506 connections now open) 2016-03-13T09:47:20.296+0000 I NETWORK [conn4085401] end connection 10.0.216.90:60626 (1506 connections now open) 2016-03-13T09:47:20.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60637 #4085418 (1506 connections now open) 2016-03-13T09:47:20.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60638 #4085419 (1507 connections now open) 2016-03-13T09:47:21.262+0000 I NETWORK [conn4085403] end connection 10.0.20.53:54091 (1506 connections now open) 2016-03-13T09:47:21.262+0000 I NETWORK [conn4085402] end connection 10.0.20.53:54092 (1506 connections now open) 2016-03-13T09:47:21.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54106 #4085420 (1506 connections now open) 2016-03-13T09:47:21.263+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54107 #4085421 (1507 connections now open) 2016-03-13T09:47:23.711+0000 I NETWORK [conn4085405] end connection 10.0.179.209:43771 (1506 connections now open) 2016-03-13T09:47:23.711+0000 I NETWORK [conn4085404] end connection 10.0.179.209:43770 (1506 connections now open) 2016-03-13T09:47:23.712+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43789 #4085422 (1506 connections now open) 2016-03-13T09:47:23.712+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43788 #4085423 (1507 connections now open) 2016-03-13T09:47:24.299+0000 I NETWORK [conn4085407] end connection 10.0.26.69:45742 (1506 connections now open) 2016-03-13T09:47:24.299+0000 I NETWORK [conn4085406] end connection 10.0.26.69:45743 (1506 connections now open) 2016-03-13T09:47:24.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45754 #4085424 (1506 connections now open) 2016-03-13T09:47:24.300+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45753 #4085425 (1507 connections now open) 2016-03-13T09:47:27.002+0000 I NETWORK [conn4085415] end connection 10.0.27.237:39159 (1506 connections now open) 2016-03-13T09:47:27.003+0000 I NETWORK [conn4085414] end connection 10.0.27.237:39158 (1505 connections now open) 2016-03-13T09:47:27.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39166 #4085426 (1506 connections now open) 2016-03-13T09:47:27.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39167 #4085427 (1507 connections now open) 2016-03-13T09:47:28.114+0000 I NETWORK [conn4082426] end connection 10.0.27.237:36400 (1506 connections now open) 2016-03-13T09:47:28.114+0000 I NETWORK [conn4081334] end connection 10.0.7.113:59649 (1505 connections now open) 2016-03-13T09:47:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39171 #4085428 (1506 connections now open) 2016-03-13T09:47:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34399 #4085429 (1507 connections now open) 2016-03-13T09:47:29.251+0000 I NETWORK [conn4085417] end connection 10.0.7.113:34381 (1506 connections now open) 2016-03-13T09:47:29.251+0000 I NETWORK [conn4085416] end connection 10.0.7.113:34380 (1506 connections now open) 2016-03-13T09:47:29.252+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34401 #4085430 (1506 connections now open) 2016-03-13T09:47:29.252+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34400 #4085431 (1507 connections now open) 2016-03-13T09:47:30.308+0000 I NETWORK [conn4085419] end connection 10.0.216.90:60638 (1506 connections now open) 2016-03-13T09:47:30.308+0000 I NETWORK [conn4085418] end connection 10.0.216.90:60637 (1506 connections now open) 2016-03-13T09:47:30.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60662 #4085432 (1506 connections now open) 2016-03-13T09:47:30.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60661 #4085433 (1507 connections now open) 2016-03-13T09:47:31.275+0000 I NETWORK [conn4085421] end connection 10.0.20.53:54107 (1506 connections now open) 2016-03-13T09:47:31.275+0000 I NETWORK [conn4085420] end connection 10.0.20.53:54106 (1506 connections now open) 2016-03-13T09:47:31.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54130 #4085434 (1506 connections now open) 2016-03-13T09:47:31.276+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54131 #4085435 (1507 connections now open) 2016-03-13T09:47:33.724+0000 I NETWORK [conn4085423] end connection 10.0.179.209:43788 (1506 connections now open) 2016-03-13T09:47:33.724+0000 I NETWORK [conn4085422] end connection 10.0.179.209:43789 (1506 connections now open) 2016-03-13T09:47:33.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43809 #4085436 (1506 connections now open) 2016-03-13T09:47:33.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43810 #4085437 (1507 connections now open) 2016-03-13T09:47:34.313+0000 I NETWORK [conn4085425] end connection 10.0.26.69:45753 (1506 connections now open) 2016-03-13T09:47:34.313+0000 I NETWORK [conn4085424] end connection 10.0.26.69:45754 (1506 connections now open) 2016-03-13T09:47:34.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45767 #4085438 (1506 connections now open) 2016-03-13T09:47:34.314+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45766 #4085439 (1507 connections now open) 2016-03-13T09:47:37.015+0000 I NETWORK [conn4085426] end connection 10.0.27.237:39166 (1506 connections now open) 2016-03-13T09:47:37.015+0000 I NETWORK [conn4085427] end connection 10.0.27.237:39167 (1505 connections now open) 2016-03-13T09:47:37.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39177 #4085440 (1506 connections now open) 2016-03-13T09:47:37.016+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39178 #4085441 (1507 connections now open) 2016-03-13T09:47:39.265+0000 I NETWORK [conn4085431] end connection 10.0.7.113:34400 (1506 connections now open) 2016-03-13T09:47:39.265+0000 I NETWORK [conn4085430] end connection 10.0.7.113:34401 (1506 connections now open) 2016-03-13T09:47:39.265+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34408 #4085442 (1506 connections now open) 2016-03-13T09:47:39.266+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34409 #4085443 (1507 connections now open) 2016-03-13T09:47:40.010+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:47:39.748+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:47:40.321+0000 I NETWORK [conn4085432] end connection 10.0.216.90:60662 (1506 connections now open) 2016-03-13T09:47:40.321+0000 I NETWORK [conn4085433] end connection 10.0.216.90:60661 (1506 connections now open) 2016-03-13T09:47:40.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60670 #4085444 (1506 connections now open) 2016-03-13T09:47:40.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60669 #4085445 (1507 connections now open) 2016-03-13T09:47:41.288+0000 I NETWORK [conn4085434] end connection 10.0.20.53:54130 (1506 connections now open) 2016-03-13T09:47:41.288+0000 I NETWORK [conn4085435] end connection 10.0.20.53:54131 (1505 connections now open) 2016-03-13T09:47:41.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54138 #4085446 (1506 connections now open) 2016-03-13T09:47:41.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54137 #4085447 (1507 connections now open) 2016-03-13T09:47:43.737+0000 I NETWORK [conn4085436] end connection 10.0.179.209:43809 (1506 connections now open) 2016-03-13T09:47:43.737+0000 I NETWORK [conn4085437] end connection 10.0.179.209:43810 (1506 connections now open) 2016-03-13T09:47:43.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43817 #4085448 (1506 connections now open) 2016-03-13T09:47:43.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43818 #4085449 (1507 connections now open) 2016-03-13T09:47:44.326+0000 I NETWORK [conn4085438] end connection 10.0.26.69:45767 (1506 connections now open) 2016-03-13T09:47:44.326+0000 I NETWORK [conn4085439] end connection 10.0.26.69:45766 (1506 connections now open) 2016-03-13T09:47:44.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45784 #4085450 (1506 connections now open) 2016-03-13T09:47:44.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45783 #4085451 (1507 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [conn4082440] end connection 10.0.179.209:41077 (1506 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [conn4082883] end connection 10.0.20.53:51535 (1506 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [conn4083400] end connection 10.0.216.90:58596 (1504 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [conn4081815] end connection 10.0.26.69:42846 (1503 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [conn4082143] end connection 10.0.27.237:36296 (1502 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43822 #4085452 (1503 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54149 #4085453 (1504 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60676 #4085454 (1505 connections now open) 2016-03-13T09:47:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45788 #4085455 (1506 connections now open) 2016-03-13T09:47:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39197 #4085456 (1507 connections now open) 2016-03-13T09:47:47.029+0000 I NETWORK [conn4085441] end connection 10.0.27.237:39178 (1506 connections now open) 2016-03-13T09:47:47.029+0000 I NETWORK [conn4085440] end connection 10.0.27.237:39177 (1506 connections now open) 2016-03-13T09:47:47.029+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39198 #4085457 (1506 connections now open) 2016-03-13T09:47:47.030+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39199 #4085458 (1507 connections now open) 2016-03-13T09:47:49.278+0000 I NETWORK [conn4085442] end connection 10.0.7.113:34408 (1506 connections now open) 2016-03-13T09:47:49.278+0000 I NETWORK [conn4085443] end connection 10.0.7.113:34409 (1506 connections now open) 2016-03-13T09:47:49.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34420 #4085459 (1506 connections now open) 2016-03-13T09:47:49.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34421 #4085460 (1507 connections now open) 2016-03-13T09:47:50.342+0000 I NETWORK [conn4085444] end connection 10.0.216.90:60670 (1506 connections now open) 2016-03-13T09:47:50.342+0000 I NETWORK [conn4085445] end connection 10.0.216.90:60669 (1506 connections now open) 2016-03-13T09:47:50.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60684 #4085461 (1506 connections now open) 2016-03-13T09:47:50.343+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60683 #4085462 (1507 connections now open) 2016-03-13T09:47:51.309+0000 I NETWORK [conn4085447] end connection 10.0.20.53:54137 (1506 connections now open) 2016-03-13T09:47:51.309+0000 I NETWORK [conn4085446] end connection 10.0.20.53:54138 (1506 connections now open) 2016-03-13T09:47:51.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54151 #4085463 (1506 connections now open) 2016-03-13T09:47:51.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54152 #4085464 (1507 connections now open) 2016-03-13T09:47:53.750+0000 I NETWORK [conn4085448] end connection 10.0.179.209:43817 (1506 connections now open) 2016-03-13T09:47:53.750+0000 I NETWORK [conn4085449] end connection 10.0.179.209:43818 (1506 connections now open) 2016-03-13T09:47:53.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43832 #4085465 (1506 connections now open) 2016-03-13T09:47:53.751+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43833 #4085466 (1507 connections now open) 2016-03-13T09:47:54.349+0000 I NETWORK [conn4085451] end connection 10.0.26.69:45783 (1506 connections now open) 2016-03-13T09:47:54.349+0000 I NETWORK [conn4085450] end connection 10.0.26.69:45784 (1506 connections now open) 2016-03-13T09:47:54.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45794 #4085467 (1506 connections now open) 2016-03-13T09:47:54.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45795 #4085468 (1507 connections now open) 2016-03-13T09:47:57.043+0000 I NETWORK [conn4085458] end connection 10.0.27.237:39199 (1506 connections now open) 2016-03-13T09:47:57.043+0000 I NETWORK [conn4085457] end connection 10.0.27.237:39198 (1506 connections now open) 2016-03-13T09:47:57.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39206 #4085469 (1506 connections now open) 2016-03-13T09:47:57.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39207 #4085470 (1507 connections now open) 2016-03-13T09:47:58.114+0000 I NETWORK [conn4081802] end connection 10.0.27.237:36136 (1506 connections now open) 2016-03-13T09:47:58.114+0000 I NETWORK [conn4081348] end connection 10.0.7.113:59653 (1505 connections now open) 2016-03-13T09:47:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39212 #4085471 (1506 connections now open) 2016-03-13T09:47:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34437 #4085472 (1507 connections now open) 2016-03-13T09:47:59.293+0000 I NETWORK [conn4085460] end connection 10.0.7.113:34421 (1506 connections now open) 2016-03-13T09:47:59.293+0000 I NETWORK [conn4085459] end connection 10.0.7.113:34420 (1506 connections now open) 2016-03-13T09:47:59.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34438 #4085473 (1506 connections now open) 2016-03-13T09:47:59.293+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34439 #4085474 (1507 connections now open) 2016-03-13T09:48:00.356+0000 I NETWORK [conn4085461] end connection 10.0.216.90:60684 (1506 connections now open) 2016-03-13T09:48:00.356+0000 I NETWORK [conn4085462] end connection 10.0.216.90:60683 (1506 connections now open) 2016-03-13T09:48:00.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60703 #4085475 (1506 connections now open) 2016-03-13T09:48:00.356+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60702 #4085476 (1507 connections now open) 2016-03-13T09:48:01.322+0000 I NETWORK [conn4085463] end connection 10.0.20.53:54151 (1506 connections now open) 2016-03-13T09:48:01.322+0000 I NETWORK [conn4085464] end connection 10.0.20.53:54152 (1506 connections now open) 2016-03-13T09:48:01.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54171 #4085477 (1506 connections now open) 2016-03-13T09:48:01.323+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54170 #4085478 (1507 connections now open) 2016-03-13T09:48:03.763+0000 I NETWORK [conn4085465] end connection 10.0.179.209:43832 (1506 connections now open) 2016-03-13T09:48:03.763+0000 I NETWORK [conn4085466] end connection 10.0.179.209:43833 (1506 connections now open) 2016-03-13T09:48:03.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43854 #4085479 (1506 connections now open) 2016-03-13T09:48:03.764+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43855 #4085480 (1507 connections now open) 2016-03-13T09:48:04.362+0000 I NETWORK [conn4085467] end connection 10.0.26.69:45794 (1506 connections now open) 2016-03-13T09:48:04.362+0000 I NETWORK [conn4085468] end connection 10.0.26.69:45795 (1506 connections now open) 2016-03-13T09:48:04.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45808 #4085481 (1506 connections now open) 2016-03-13T09:48:04.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45809 #4085482 (1507 connections now open) 2016-03-13T09:48:07.065+0000 I NETWORK [conn4085470] end connection 10.0.27.237:39207 (1506 connections now open) 2016-03-13T09:48:07.065+0000 I NETWORK [conn4085469] end connection 10.0.27.237:39206 (1506 connections now open) 2016-03-13T09:48:07.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39222 #4085483 (1506 connections now open) 2016-03-13T09:48:07.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39221 #4085484 (1507 connections now open) 2016-03-13T09:48:09.307+0000 I NETWORK [conn4085474] end connection 10.0.7.113:34439 (1506 connections now open) 2016-03-13T09:48:09.307+0000 I NETWORK [conn4085473] end connection 10.0.7.113:34438 (1506 connections now open) 2016-03-13T09:48:09.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34452 #4085485 (1506 connections now open) 2016-03-13T09:48:09.308+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34451 #4085486 (1507 connections now open) 2016-03-13T09:48:10.337+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:48:10.010+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:48:10.369+0000 I NETWORK [conn4085476] end connection 10.0.216.90:60702 (1506 connections now open) 2016-03-13T09:48:10.369+0000 I NETWORK [conn4085475] end connection 10.0.216.90:60703 (1506 connections now open) 2016-03-13T09:48:10.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60712 #4085487 (1506 connections now open) 2016-03-13T09:48:10.369+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60713 #4085488 (1507 connections now open) 2016-03-13T09:48:11.335+0000 I NETWORK [conn4085477] end connection 10.0.20.53:54171 (1506 connections now open) 2016-03-13T09:48:11.335+0000 I NETWORK [conn4085478] end connection 10.0.20.53:54170 (1506 connections now open) 2016-03-13T09:48:11.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54180 #4085489 (1506 connections now open) 2016-03-13T09:48:11.336+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54179 #4085490 (1507 connections now open) 2016-03-13T09:48:13.777+0000 I NETWORK [conn4085479] end connection 10.0.179.209:43854 (1506 connections now open) 2016-03-13T09:48:13.777+0000 I NETWORK [conn4085480] end connection 10.0.179.209:43855 (1506 connections now open) 2016-03-13T09:48:13.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43862 #4085491 (1506 connections now open) 2016-03-13T09:48:13.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43863 #4085492 (1507 connections now open) 2016-03-13T09:48:14.376+0000 I NETWORK [conn4085482] end connection 10.0.26.69:45809 (1506 connections now open) 2016-03-13T09:48:14.376+0000 I NETWORK [conn4085481] end connection 10.0.26.69:45808 (1505 connections now open) 2016-03-13T09:48:14.377+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45828 #4085493 (1506 connections now open) 2016-03-13T09:48:14.377+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45829 #4085494 (1507 connections now open) 2016-03-13T09:48:16.116+0000 I NETWORK [conn4082503] end connection 10.0.20.53:51373 (1506 connections now open) 2016-03-13T09:48:16.116+0000 I NETWORK [conn4082349] end connection 10.0.179.209:41050 (1506 connections now open) 2016-03-13T09:48:16.116+0000 I NETWORK [conn4080362] end connection 10.0.216.90:57037 (1506 connections now open) 2016-03-13T09:48:16.117+0000 I NETWORK [conn4082743] end connection 10.0.27.237:36552 (1503 connections now open) 2016-03-13T09:48:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43866 #4085495 (1504 connections now open) 2016-03-13T09:48:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60720 #4085496 (1505 connections now open) 2016-03-13T09:48:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54192 #4085497 (1506 connections now open) 2016-03-13T09:48:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39243 #4085498 (1507 connections now open) 2016-03-13T09:48:17.078+0000 I NETWORK [conn4085484] end connection 10.0.27.237:39221 (1506 connections now open) 2016-03-13T09:48:17.078+0000 I NETWORK [conn4085483] end connection 10.0.27.237:39222 (1506 connections now open) 2016-03-13T09:48:17.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39245 #4085499 (1506 connections now open) 2016-03-13T09:48:17.079+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39246 #4085500 (1507 connections now open) 2016-03-13T09:48:19.321+0000 I NETWORK [conn4085485] end connection 10.0.7.113:34452 (1506 connections now open) 2016-03-13T09:48:19.321+0000 I NETWORK [conn4085486] end connection 10.0.7.113:34451 (1506 connections now open) 2016-03-13T09:48:19.330+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34464 #4085501 (1506 connections now open) 2016-03-13T09:48:19.330+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34465 #4085502 (1507 connections now open) 2016-03-13T09:48:20.382+0000 I NETWORK [conn4085488] end connection 10.0.216.90:60713 (1506 connections now open) 2016-03-13T09:48:20.382+0000 I NETWORK [conn4085487] end connection 10.0.216.90:60712 (1506 connections now open) 2016-03-13T09:48:20.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60725 #4085503 (1506 connections now open) 2016-03-13T09:48:20.382+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60726 #4085504 (1507 connections now open) 2016-03-13T09:48:21.349+0000 I NETWORK [conn4085490] end connection 10.0.20.53:54179 (1506 connections now open) 2016-03-13T09:48:21.349+0000 I NETWORK [conn4085489] end connection 10.0.20.53:54180 (1506 connections now open) 2016-03-13T09:48:21.349+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54194 #4085505 (1506 connections now open) 2016-03-13T09:48:21.350+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54195 #4085506 (1507 connections now open) 2016-03-13T09:48:23.796+0000 I NETWORK [conn4085491] end connection 10.0.179.209:43862 (1506 connections now open) 2016-03-13T09:48:23.796+0000 I NETWORK [conn4085492] end connection 10.0.179.209:43863 (1506 connections now open) 2016-03-13T09:48:23.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43880 #4085507 (1506 connections now open) 2016-03-13T09:48:23.797+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43879 #4085508 (1507 connections now open) 2016-03-13T09:48:24.390+0000 I NETWORK [conn4085494] end connection 10.0.26.69:45829 (1506 connections now open) 2016-03-13T09:48:24.390+0000 I NETWORK [conn4085493] end connection 10.0.26.69:45828 (1506 connections now open) 2016-03-13T09:48:24.390+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45838 #4085509 (1507 connections now open) 2016-03-13T09:48:24.390+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45839 #4085510 (1507 connections now open) 2016-03-13T09:48:27.092+0000 I NETWORK [conn4085500] end connection 10.0.27.237:39246 (1506 connections now open) 2016-03-13T09:48:27.092+0000 I NETWORK [conn4085499] end connection 10.0.27.237:39245 (1506 connections now open) 2016-03-13T09:48:27.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39254 #4085511 (1506 connections now open) 2016-03-13T09:48:27.093+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39255 #4085512 (1507 connections now open) 2016-03-13T09:48:28.113+0000 I NETWORK [conn4081504] end connection 10.0.20.53:50929 (1506 connections now open) 2016-03-13T09:48:28.113+0000 I NETWORK [conn4080825] end connection 10.0.179.209:40516 (1505 connections now open) 2016-03-13T09:48:28.113+0000 I NETWORK [conn4084715] end connection 10.0.216.90:59931 (1505 connections now open) 2016-03-13T09:48:28.113+0000 I NETWORK [conn4080992] end connection 10.0.26.69:42457 (1503 connections now open) 2016-03-13T09:48:28.114+0000 I NETWORK [conn4081536] end connection 10.0.27.237:36005 (1502 connections now open) 2016-03-13T09:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60734 #4085513 (1503 connections now open) 2016-03-13T09:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54211 #4085514 (1504 connections now open) 2016-03-13T09:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43884 #4085515 (1505 connections now open) 2016-03-13T09:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45841 #4085516 (1506 connections now open) 2016-03-13T09:48:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39258 #4085517 (1507 connections now open) 2016-03-13T09:48:29.343+0000 I NETWORK [conn4085502] end connection 10.0.7.113:34465 (1506 connections now open) 2016-03-13T09:48:29.343+0000 I NETWORK [conn4085501] end connection 10.0.7.113:34464 (1506 connections now open) 2016-03-13T09:48:29.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34484 #4085518 (1506 connections now open) 2016-03-13T09:48:29.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34485 #4085519 (1507 connections now open) 2016-03-13T09:48:30.395+0000 I NETWORK [conn4085504] end connection 10.0.216.90:60726 (1506 connections now open) 2016-03-13T09:48:30.395+0000 I NETWORK [conn4085503] end connection 10.0.216.90:60725 (1506 connections now open) 2016-03-13T09:48:30.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60748 #4085520 (1506 connections now open) 2016-03-13T09:48:30.395+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60749 #4085521 (1507 connections now open) 2016-03-13T09:48:31.362+0000 I NETWORK [conn4085506] end connection 10.0.20.53:54195 (1506 connections now open) 2016-03-13T09:48:31.362+0000 I NETWORK [conn4085505] end connection 10.0.20.53:54194 (1505 connections now open) 2016-03-13T09:48:31.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54216 #4085522 (1506 connections now open) 2016-03-13T09:48:31.363+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54217 #4085523 (1507 connections now open) 2016-03-13T09:48:33.809+0000 I NETWORK [conn4085507] end connection 10.0.179.209:43880 (1506 connections now open) 2016-03-13T09:48:33.809+0000 I NETWORK [conn4085508] end connection 10.0.179.209:43879 (1506 connections now open) 2016-03-13T09:48:33.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43901 #4085524 (1506 connections now open) 2016-03-13T09:48:33.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43900 #4085525 (1507 connections now open) 2016-03-13T09:48:34.403+0000 I NETWORK [conn4085509] end connection 10.0.26.69:45838 (1506 connections now open) 2016-03-13T09:48:34.403+0000 I NETWORK [conn4085510] end connection 10.0.26.69:45839 (1506 connections now open) 2016-03-13T09:48:34.403+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45850 #4085526 (1506 connections now open) 2016-03-13T09:48:34.403+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45851 #4085527 (1507 connections now open) 2016-03-13T09:48:37.105+0000 I NETWORK [conn4085512] end connection 10.0.27.237:39255 (1506 connections now open) 2016-03-13T09:48:37.105+0000 I NETWORK [conn4085511] end connection 10.0.27.237:39254 (1506 connections now open) 2016-03-13T09:48:37.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39267 #4085528 (1506 connections now open) 2016-03-13T09:48:37.106+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39268 #4085529 (1507 connections now open) 2016-03-13T09:48:39.357+0000 I NETWORK [conn4085519] end connection 10.0.7.113:34485 (1506 connections now open) 2016-03-13T09:48:39.357+0000 I NETWORK [conn4085518] end connection 10.0.7.113:34484 (1506 connections now open) 2016-03-13T09:48:39.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34493 #4085530 (1506 connections now open) 2016-03-13T09:48:39.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34494 #4085531 (1507 connections now open) 2016-03-13T09:48:40.408+0000 I NETWORK [conn4085520] end connection 10.0.216.90:60748 (1506 connections now open) 2016-03-13T09:48:40.408+0000 I NETWORK [conn4085521] end connection 10.0.216.90:60749 (1506 connections now open) 2016-03-13T09:48:40.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60756 #4085532 (1506 connections now open) 2016-03-13T09:48:40.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60755 #4085533 (1507 connections now open) 2016-03-13T09:48:40.500+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:48:40.337+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:48:41.375+0000 I NETWORK [conn4085523] end connection 10.0.20.53:54217 (1506 connections now open) 2016-03-13T09:48:41.375+0000 I NETWORK [conn4085522] end connection 10.0.20.53:54216 (1506 connections now open) 2016-03-13T09:48:41.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54222 #4085534 (1506 connections now open) 2016-03-13T09:48:41.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54223 #4085535 (1507 connections now open) 2016-03-13T09:48:43.822+0000 I NETWORK [conn4085524] end connection 10.0.179.209:43901 (1506 connections now open) 2016-03-13T09:48:43.822+0000 I NETWORK [conn4085525] end connection 10.0.179.209:43900 (1506 connections now open) 2016-03-13T09:48:43.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43908 #4085536 (1506 connections now open) 2016-03-13T09:48:43.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43909 #4085537 (1507 connections now open) 2016-03-13T09:48:44.416+0000 I NETWORK [conn4085527] end connection 10.0.26.69:45851 (1506 connections now open) 2016-03-13T09:48:44.416+0000 I NETWORK [conn4085526] end connection 10.0.26.69:45850 (1506 connections now open) 2016-03-13T09:48:44.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45867 #4085538 (1506 connections now open) 2016-03-13T09:48:44.417+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45868 #4085539 (1507 connections now open) 2016-03-13T09:48:46.117+0000 I NETWORK [conn4082892] end connection 10.0.20.53:51550 (1506 connections now open) 2016-03-13T09:48:46.117+0000 I NETWORK [conn4081308] end connection 10.0.216.90:57592 (1506 connections now open) 2016-03-13T09:48:46.117+0000 I NETWORK [conn4082099] end connection 10.0.27.237:36272 (1504 connections now open) 2016-03-13T09:48:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54235 #4085540 (1505 connections now open) 2016-03-13T09:48:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60762 #4085541 (1506 connections now open) 2016-03-13T09:48:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39287 #4085542 (1507 connections now open) 2016-03-13T09:48:47.117+0000 I NETWORK [conn4085529] end connection 10.0.27.237:39268 (1506 connections now open) 2016-03-13T09:48:47.118+0000 I NETWORK [conn4085528] end connection 10.0.27.237:39267 (1505 connections now open) 2016-03-13T09:48:47.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39289 #4085543 (1506 connections now open) 2016-03-13T09:48:47.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39290 #4085544 (1507 connections now open) 2016-03-13T09:48:49.371+0000 I NETWORK [conn4085530] end connection 10.0.7.113:34493 (1506 connections now open) 2016-03-13T09:48:49.371+0000 I NETWORK [conn4085531] end connection 10.0.7.113:34494 (1506 connections now open) 2016-03-13T09:48:49.372+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34506 #4085545 (1506 connections now open) 2016-03-13T09:48:49.372+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34505 #4085546 (1507 connections now open) 2016-03-13T09:48:50.430+0000 I NETWORK [conn4085533] end connection 10.0.216.90:60755 (1506 connections now open) 2016-03-13T09:48:50.430+0000 I NETWORK [conn4085532] end connection 10.0.216.90:60756 (1506 connections now open) 2016-03-13T09:48:50.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60768 #4085547 (1506 connections now open) 2016-03-13T09:48:50.430+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60769 #4085548 (1507 connections now open) 2016-03-13T09:48:51.398+0000 I NETWORK [conn4085534] end connection 10.0.20.53:54222 (1506 connections now open) 2016-03-13T09:48:51.398+0000 I NETWORK [conn4085535] end connection 10.0.20.53:54223 (1506 connections now open) 2016-03-13T09:48:51.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54237 #4085549 (1506 connections now open) 2016-03-13T09:48:51.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54238 #4085550 (1507 connections now open) 2016-03-13T09:48:53.835+0000 I NETWORK [conn4085536] end connection 10.0.179.209:43908 (1506 connections now open) 2016-03-13T09:48:53.835+0000 I NETWORK [conn4085537] end connection 10.0.179.209:43909 (1506 connections now open) 2016-03-13T09:48:53.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43923 #4085551 (1506 connections now open) 2016-03-13T09:48:53.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43924 #4085552 (1507 connections now open) 2016-03-13T09:48:54.430+0000 I NETWORK [conn4085538] end connection 10.0.26.69:45867 (1506 connections now open) 2016-03-13T09:48:54.430+0000 I NETWORK [conn4085539] end connection 10.0.26.69:45868 (1506 connections now open) 2016-03-13T09:48:54.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45877 #4085553 (1507 connections now open) 2016-03-13T09:48:54.431+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45878 #4085554 (1507 connections now open) 2016-03-13T09:48:57.130+0000 I NETWORK [conn4085543] end connection 10.0.27.237:39289 (1506 connections now open) 2016-03-13T09:48:57.130+0000 I NETWORK [conn4085544] end connection 10.0.27.237:39290 (1506 connections now open) 2016-03-13T09:48:57.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39296 #4085555 (1506 connections now open) 2016-03-13T09:48:57.138+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39297 #4085556 (1507 connections now open) 2016-03-13T09:48:58.114+0000 I NETWORK [conn4081663] end connection 10.0.26.69:42763 (1506 connections now open) 2016-03-13T09:48:58.114+0000 I NETWORK [conn4083884] end connection 10.0.27.237:37512 (1506 connections now open) 2016-03-13T09:48:58.114+0000 I NETWORK [conn4081577] end connection 10.0.20.53:50966 (1504 connections now open) 2016-03-13T09:48:58.114+0000 I NETWORK [conn4081569] end connection 10.0.7.113:59698 (1504 connections now open) 2016-03-13T09:48:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39301 #4085557 (1504 connections now open) 2016-03-13T09:48:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34515 #4085558 (1505 connections now open) 2016-03-13T09:48:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54254 #4085559 (1506 connections now open) 2016-03-13T09:48:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45881 #4085560 (1507 connections now open) 2016-03-13T09:48:59.385+0000 I NETWORK [conn4085546] end connection 10.0.7.113:34505 (1506 connections now open) 2016-03-13T09:48:59.385+0000 I NETWORK [conn4085545] end connection 10.0.7.113:34506 (1506 connections now open) 2016-03-13T09:48:59.386+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34526 #4085561 (1506 connections now open) 2016-03-13T09:48:59.386+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34527 #4085562 (1507 connections now open) 2016-03-13T09:49:00.443+0000 I NETWORK [conn4085547] end connection 10.0.216.90:60768 (1506 connections now open) 2016-03-13T09:49:00.443+0000 I NETWORK [conn4085548] end connection 10.0.216.90:60769 (1505 connections now open) 2016-03-13T09:49:00.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60787 #4085563 (1506 connections now open) 2016-03-13T09:49:00.443+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60788 #4085564 (1507 connections now open) 2016-03-13T09:49:01.411+0000 I NETWORK [conn4085549] end connection 10.0.20.53:54237 (1506 connections now open) 2016-03-13T09:49:01.411+0000 I NETWORK [conn4085550] end connection 10.0.20.53:54238 (1506 connections now open) 2016-03-13T09:49:01.412+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54256 #4085565 (1506 connections now open) 2016-03-13T09:49:01.412+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54257 #4085566 (1507 connections now open) 2016-03-13T09:49:03.849+0000 I NETWORK [conn4085552] end connection 10.0.179.209:43924 (1506 connections now open) 2016-03-13T09:49:03.849+0000 I NETWORK [conn4085551] end connection 10.0.179.209:43923 (1506 connections now open) 2016-03-13T09:49:03.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43945 #4085567 (1506 connections now open) 2016-03-13T09:49:03.849+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43946 #4085568 (1507 connections now open) 2016-03-13T09:49:04.444+0000 I NETWORK [conn4085553] end connection 10.0.26.69:45877 (1506 connections now open) 2016-03-13T09:49:04.444+0000 I NETWORK [conn4085554] end connection 10.0.26.69:45878 (1506 connections now open) 2016-03-13T09:49:04.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45892 #4085569 (1506 connections now open) 2016-03-13T09:49:04.445+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45893 #4085570 (1507 connections now open) 2016-03-13T09:49:07.151+0000 I NETWORK [conn4085556] end connection 10.0.27.237:39297 (1506 connections now open) 2016-03-13T09:49:07.151+0000 I NETWORK [conn4085555] end connection 10.0.27.237:39296 (1506 connections now open) 2016-03-13T09:49:07.152+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39313 #4085571 (1506 connections now open) 2016-03-13T09:49:07.152+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39312 #4085572 (1507 connections now open) 2016-03-13T09:49:09.399+0000 I NETWORK [conn4085561] end connection 10.0.7.113:34526 (1506 connections now open) 2016-03-13T09:49:09.399+0000 I NETWORK [conn4085562] end connection 10.0.7.113:34527 (1506 connections now open) 2016-03-13T09:49:09.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34538 #4085573 (1506 connections now open) 2016-03-13T09:49:09.399+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34539 #4085574 (1507 connections now open) 2016-03-13T09:49:10.455+0000 I NETWORK [conn4085564] end connection 10.0.216.90:60788 (1506 connections now open) 2016-03-13T09:49:10.455+0000 I NETWORK [conn4085563] end connection 10.0.216.90:60787 (1506 connections now open) 2016-03-13T09:49:10.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60797 #4085575 (1506 connections now open) 2016-03-13T09:49:10.456+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60796 #4085576 (1507 connections now open) 2016-03-13T09:49:10.770+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:49:10.500+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:49:11.425+0000 I NETWORK [conn4085565] end connection 10.0.20.53:54256 (1506 connections now open) 2016-03-13T09:49:11.425+0000 I NETWORK [conn4085566] end connection 10.0.20.53:54257 (1506 connections now open) 2016-03-13T09:49:11.425+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54264 #4085577 (1506 connections now open) 2016-03-13T09:49:11.425+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54265 #4085578 (1507 connections now open) 2016-03-13T09:49:13.862+0000 I NETWORK [conn4085567] end connection 10.0.179.209:43945 (1506 connections now open) 2016-03-13T09:49:13.862+0000 I NETWORK [conn4085568] end connection 10.0.179.209:43946 (1506 connections now open) 2016-03-13T09:49:13.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43954 #4085579 (1506 connections now open) 2016-03-13T09:49:13.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43955 #4085580 (1507 connections now open) 2016-03-13T09:49:14.458+0000 I NETWORK [conn4085570] end connection 10.0.26.69:45893 (1506 connections now open) 2016-03-13T09:49:14.458+0000 I NETWORK [conn4085569] end connection 10.0.26.69:45892 (1506 connections now open) 2016-03-13T09:49:14.458+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45912 #4085581 (1506 connections now open) 2016-03-13T09:49:14.458+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45913 #4085582 (1507 connections now open) 2016-03-13T09:49:16.117+0000 I NETWORK [conn4083654] end connection 10.0.216.90:58848 (1506 connections now open) 2016-03-13T09:49:16.117+0000 I NETWORK [conn4081137] end connection 10.0.26.69:42512 (1505 connections now open) 2016-03-13T09:49:16.117+0000 I NETWORK [conn4081870] end connection 10.0.27.237:36175 (1504 connections now open) 2016-03-13T09:49:16.117+0000 I NETWORK [conn4082078] end connection 10.0.7.113:59843 (1504 connections now open) 2016-03-13T09:49:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60803 #4085583 (1504 connections now open) 2016-03-13T09:49:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45916 #4085584 (1505 connections now open) 2016-03-13T09:49:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39334 #4085585 (1506 connections now open) 2016-03-13T09:49:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34546 #4085586 (1507 connections now open) 2016-03-13T09:49:17.164+0000 I NETWORK [conn4085572] end connection 10.0.27.237:39312 (1506 connections now open) 2016-03-13T09:49:17.164+0000 I NETWORK [conn4085571] end connection 10.0.27.237:39313 (1506 connections now open) 2016-03-13T09:49:17.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39337 #4085587 (1506 connections now open) 2016-03-13T09:49:17.165+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39336 #4085588 (1507 connections now open) 2016-03-13T09:49:19.412+0000 I NETWORK [conn4085574] end connection 10.0.7.113:34539 (1506 connections now open) 2016-03-13T09:49:19.412+0000 I NETWORK [conn4085573] end connection 10.0.7.113:34538 (1506 connections now open) 2016-03-13T09:49:19.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34553 #4085589 (1506 connections now open) 2016-03-13T09:49:19.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34552 #4085590 (1507 connections now open) 2016-03-13T09:49:20.468+0000 I NETWORK [conn4085576] end connection 10.0.216.90:60796 (1506 connections now open) 2016-03-13T09:49:20.469+0000 I NETWORK [conn4085575] end connection 10.0.216.90:60797 (1506 connections now open) 2016-03-13T09:49:20.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60810 #4085591 (1506 connections now open) 2016-03-13T09:49:20.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60809 #4085592 (1507 connections now open) 2016-03-13T09:49:21.438+0000 I NETWORK [conn4085577] end connection 10.0.20.53:54264 (1506 connections now open) 2016-03-13T09:49:21.438+0000 I NETWORK [conn4085578] end connection 10.0.20.53:54265 (1506 connections now open) 2016-03-13T09:49:21.439+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54281 #4085593 (1506 connections now open) 2016-03-13T09:49:21.439+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54280 #4085594 (1507 connections now open) 2016-03-13T09:49:23.883+0000 I NETWORK [conn4085579] end connection 10.0.179.209:43954 (1506 connections now open) 2016-03-13T09:49:23.883+0000 I NETWORK [conn4085580] end connection 10.0.179.209:43955 (1506 connections now open) 2016-03-13T09:49:23.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43970 #4085595 (1506 connections now open) 2016-03-13T09:49:23.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43971 #4085596 (1507 connections now open) 2016-03-13T09:49:24.471+0000 I NETWORK [conn4085581] end connection 10.0.26.69:45912 (1506 connections now open) 2016-03-13T09:49:24.471+0000 I NETWORK [conn4085582] end connection 10.0.26.69:45913 (1506 connections now open) 2016-03-13T09:49:24.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45921 #4085597 (1507 connections now open) 2016-03-13T09:49:24.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45922 #4085598 (1507 connections now open) 2016-03-13T09:49:27.177+0000 I NETWORK [conn4085588] end connection 10.0.27.237:39336 (1506 connections now open) 2016-03-13T09:49:27.177+0000 I NETWORK [conn4085587] end connection 10.0.27.237:39337 (1506 connections now open) 2016-03-13T09:49:27.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39346 #4085599 (1506 connections now open) 2016-03-13T09:49:27.178+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39345 #4085600 (1507 connections now open) 2016-03-13T09:49:28.114+0000 I NETWORK [conn4081112] end connection 10.0.7.113:59578 (1506 connections now open) 2016-03-13T09:49:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34561 #4085601 (1507 connections now open) 2016-03-13T09:49:29.434+0000 I NETWORK [conn4085589] end connection 10.0.7.113:34553 (1506 connections now open) 2016-03-13T09:49:29.434+0000 I NETWORK [conn4085590] end connection 10.0.7.113:34552 (1506 connections now open) 2016-03-13T09:49:29.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34572 #4085602 (1506 connections now open) 2016-03-13T09:49:29.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34573 #4085603 (1507 connections now open) 2016-03-13T09:49:30.482+0000 I NETWORK [conn4085591] end connection 10.0.216.90:60810 (1506 connections now open) 2016-03-13T09:49:30.482+0000 I NETWORK [conn4085592] end connection 10.0.216.90:60809 (1506 connections now open) 2016-03-13T09:49:30.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60832 #4085604 (1506 connections now open) 2016-03-13T09:49:30.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60833 #4085605 (1507 connections now open) 2016-03-13T09:49:31.451+0000 I NETWORK [conn4085593] end connection 10.0.20.53:54281 (1506 connections now open) 2016-03-13T09:49:31.451+0000 I NETWORK [conn4085594] end connection 10.0.20.53:54280 (1505 connections now open) 2016-03-13T09:49:31.451+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54301 #4085606 (1506 connections now open) 2016-03-13T09:49:31.452+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54302 #4085607 (1507 connections now open) 2016-03-13T09:49:33.896+0000 I NETWORK [conn4085595] end connection 10.0.179.209:43970 (1506 connections now open) 2016-03-13T09:49:33.896+0000 I NETWORK [conn4085596] end connection 10.0.179.209:43971 (1506 connections now open) 2016-03-13T09:49:33.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43991 #4085608 (1506 connections now open) 2016-03-13T09:49:33.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43990 #4085609 (1507 connections now open) 2016-03-13T09:49:34.484+0000 I NETWORK [conn4085597] end connection 10.0.26.69:45921 (1506 connections now open) 2016-03-13T09:49:34.484+0000 I NETWORK [conn4085598] end connection 10.0.26.69:45922 (1506 connections now open) 2016-03-13T09:49:34.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45934 #4085610 (1506 connections now open) 2016-03-13T09:49:34.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45933 #4085611 (1507 connections now open) 2016-03-13T09:49:37.190+0000 I NETWORK [conn4085599] end connection 10.0.27.237:39346 (1506 connections now open) 2016-03-13T09:49:37.190+0000 I NETWORK [conn4085600] end connection 10.0.27.237:39345 (1506 connections now open) 2016-03-13T09:49:37.191+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39358 #4085612 (1506 connections now open) 2016-03-13T09:49:37.191+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39359 #4085613 (1507 connections now open) 2016-03-13T09:49:39.447+0000 I NETWORK [conn4085602] end connection 10.0.7.113:34572 (1506 connections now open) 2016-03-13T09:49:39.447+0000 I NETWORK [conn4085603] end connection 10.0.7.113:34573 (1506 connections now open) 2016-03-13T09:49:39.448+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34579 #4085614 (1506 connections now open) 2016-03-13T09:49:39.448+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34580 #4085615 (1507 connections now open) 2016-03-13T09:49:40.495+0000 I NETWORK [conn4085605] end connection 10.0.216.90:60833 (1506 connections now open) 2016-03-13T09:49:40.495+0000 I NETWORK [conn4085604] end connection 10.0.216.90:60832 (1506 connections now open) 2016-03-13T09:49:40.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60841 #4085616 (1506 connections now open) 2016-03-13T09:49:40.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60842 #4085617 (1507 connections now open) 2016-03-13T09:49:41.176+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:49:40.770+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:49:41.463+0000 I NETWORK [conn4085606] end connection 10.0.20.53:54301 (1506 connections now open) 2016-03-13T09:49:41.464+0000 I NETWORK [conn4085607] end connection 10.0.20.53:54302 (1505 connections now open) 2016-03-13T09:49:41.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54308 #4085618 (1506 connections now open) 2016-03-13T09:49:41.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54307 #4085619 (1507 connections now open) 2016-03-13T09:49:43.909+0000 I NETWORK [conn4085609] end connection 10.0.179.209:43990 (1506 connections now open) 2016-03-13T09:49:43.909+0000 I NETWORK [conn4085608] end connection 10.0.179.209:43991 (1506 connections now open) 2016-03-13T09:49:43.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44000 #4085620 (1506 connections now open) 2016-03-13T09:49:43.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:43999 #4085621 (1507 connections now open) 2016-03-13T09:49:44.497+0000 I NETWORK [conn4085611] end connection 10.0.26.69:45933 (1506 connections now open) 2016-03-13T09:49:44.498+0000 I NETWORK [conn4085610] end connection 10.0.26.69:45934 (1505 connections now open) 2016-03-13T09:49:44.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45951 #4085622 (1506 connections now open) 2016-03-13T09:49:44.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45952 #4085623 (1507 connections now open) 2016-03-13T09:49:46.117+0000 I NETWORK [conn4082128] end connection 10.0.27.237:36287 (1506 connections now open) 2016-03-13T09:49:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39376 #4085624 (1507 connections now open) 2016-03-13T09:49:47.204+0000 I NETWORK [conn4085613] end connection 10.0.27.237:39359 (1506 connections now open) 2016-03-13T09:49:47.204+0000 I NETWORK [conn4085612] end connection 10.0.27.237:39358 (1506 connections now open) 2016-03-13T09:49:47.204+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39378 #4085625 (1506 connections now open) 2016-03-13T09:49:47.204+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39379 #4085626 (1507 connections now open) 2016-03-13T09:49:49.460+0000 I NETWORK [conn4085614] end connection 10.0.7.113:34579 (1506 connections now open) 2016-03-13T09:49:49.460+0000 I NETWORK [conn4085615] end connection 10.0.7.113:34580 (1506 connections now open) 2016-03-13T09:49:49.461+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34590 #4085627 (1506 connections now open) 2016-03-13T09:49:49.461+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34591 #4085628 (1507 connections now open) 2016-03-13T09:49:50.516+0000 I NETWORK [conn4085616] end connection 10.0.216.90:60841 (1506 connections now open) 2016-03-13T09:49:50.516+0000 I NETWORK [conn4085617] end connection 10.0.216.90:60842 (1506 connections now open) 2016-03-13T09:49:50.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60853 #4085629 (1506 connections now open) 2016-03-13T09:49:50.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60854 #4085630 (1507 connections now open) 2016-03-13T09:49:51.485+0000 I NETWORK [conn4085618] end connection 10.0.20.53:54308 (1506 connections now open) 2016-03-13T09:49:51.485+0000 I NETWORK [conn4085619] end connection 10.0.20.53:54307 (1506 connections now open) 2016-03-13T09:49:51.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54321 #4085631 (1506 connections now open) 2016-03-13T09:49:51.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54320 #4085632 (1507 connections now open) 2016-03-13T09:49:53.922+0000 I NETWORK [conn4085620] end connection 10.0.179.209:44000 (1506 connections now open) 2016-03-13T09:49:53.922+0000 I NETWORK [conn4085621] end connection 10.0.179.209:43999 (1506 connections now open) 2016-03-13T09:49:53.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44015 #4085633 (1506 connections now open) 2016-03-13T09:49:53.923+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44014 #4085634 (1507 connections now open) 2016-03-13T09:49:54.510+0000 I NETWORK [conn4085622] end connection 10.0.26.69:45951 (1506 connections now open) 2016-03-13T09:49:54.510+0000 I NETWORK [conn4085623] end connection 10.0.26.69:45952 (1505 connections now open) 2016-03-13T09:49:54.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45962 #4085635 (1507 connections now open) 2016-03-13T09:49:54.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45963 #4085636 (1507 connections now open) 2016-03-13T09:49:57.217+0000 I NETWORK [conn4085625] end connection 10.0.27.237:39378 (1506 connections now open) 2016-03-13T09:49:57.217+0000 I NETWORK [conn4085626] end connection 10.0.27.237:39379 (1506 connections now open) 2016-03-13T09:49:57.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39386 #4085637 (1506 connections now open) 2016-03-13T09:49:57.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39385 #4085638 (1507 connections now open) 2016-03-13T09:49:58.116+0000 I NETWORK [conn4081021] end connection 10.0.179.209:40644 (1506 connections now open) 2016-03-13T09:49:58.116+0000 I NETWORK [conn4083420] end connection 10.0.216.90:58617 (1505 connections now open) 2016-03-13T09:49:58.116+0000 I NETWORK [conn4081666] end connection 10.0.26.69:42765 (1504 connections now open) 2016-03-13T09:49:58.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44019 #4085639 (1505 connections now open) 2016-03-13T09:49:58.116+0000 I NETWORK [conn4081919] end connection 10.0.27.237:36193 (1504 connections now open) 2016-03-13T09:49:58.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60862 #4085640 (1505 connections now open) 2016-03-13T09:49:58.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39389 #4085641 (1506 connections now open) 2016-03-13T09:49:58.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45965 #4085642 (1507 connections now open) 2016-03-13T09:49:59.474+0000 I NETWORK [conn4085627] end connection 10.0.7.113:34590 (1506 connections now open) 2016-03-13T09:49:59.474+0000 I NETWORK [conn4085628] end connection 10.0.7.113:34591 (1506 connections now open) 2016-03-13T09:49:59.475+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34610 #4085643 (1506 connections now open) 2016-03-13T09:49:59.475+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34611 #4085644 (1507 connections now open) 2016-03-13T09:50:00.529+0000 I NETWORK [conn4085629] end connection 10.0.216.90:60853 (1506 connections now open) 2016-03-13T09:50:00.529+0000 I NETWORK [conn4085630] end connection 10.0.216.90:60854 (1505 connections now open) 2016-03-13T09:50:00.530+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60873 #4085645 (1506 connections now open) 2016-03-13T09:50:00.530+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60874 #4085646 (1507 connections now open) 2016-03-13T09:50:01.498+0000 I NETWORK [conn4085632] end connection 10.0.20.53:54320 (1506 connections now open) 2016-03-13T09:50:01.498+0000 I NETWORK [conn4085631] end connection 10.0.20.53:54321 (1506 connections now open) 2016-03-13T09:50:01.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54340 #4085647 (1506 connections now open) 2016-03-13T09:50:01.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54341 #4085648 (1507 connections now open) 2016-03-13T09:50:03.935+0000 I NETWORK [conn4085633] end connection 10.0.179.209:44015 (1506 connections now open) 2016-03-13T09:50:03.935+0000 I NETWORK [conn4085634] end connection 10.0.179.209:44014 (1506 connections now open) 2016-03-13T09:50:03.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44038 #4085649 (1506 connections now open) 2016-03-13T09:50:03.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44039 #4085650 (1507 connections now open) 2016-03-13T09:50:04.524+0000 I NETWORK [conn4085636] end connection 10.0.26.69:45963 (1506 connections now open) 2016-03-13T09:50:04.524+0000 I NETWORK [conn4085635] end connection 10.0.26.69:45962 (1506 connections now open) 2016-03-13T09:50:04.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45978 #4085651 (1506 connections now open) 2016-03-13T09:50:04.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45979 #4085652 (1507 connections now open) 2016-03-13T09:50:07.295+0000 I NETWORK [conn4085638] end connection 10.0.27.237:39385 (1506 connections now open) 2016-03-13T09:50:07.295+0000 I NETWORK [conn4085637] end connection 10.0.27.237:39386 (1506 connections now open) 2016-03-13T09:50:07.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39402 #4085653 (1506 connections now open) 2016-03-13T09:50:07.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39401 #4085654 (1507 connections now open) 2016-03-13T09:50:09.489+0000 I NETWORK [conn4085643] end connection 10.0.7.113:34610 (1506 connections now open) 2016-03-13T09:50:09.489+0000 I NETWORK [conn4085644] end connection 10.0.7.113:34611 (1506 connections now open) 2016-03-13T09:50:09.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34624 #4085655 (1506 connections now open) 2016-03-13T09:50:09.489+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34625 #4085656 (1507 connections now open) 2016-03-13T09:50:10.542+0000 I NETWORK [conn4085646] end connection 10.0.216.90:60874 (1506 connections now open) 2016-03-13T09:50:10.542+0000 I NETWORK [conn4085645] end connection 10.0.216.90:60873 (1505 connections now open) 2016-03-13T09:50:10.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60884 #4085657 (1506 connections now open) 2016-03-13T09:50:10.543+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60885 #4085658 (1507 connections now open) 2016-03-13T09:50:11.386+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:50:11.177+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:50:11.511+0000 I NETWORK [conn4085648] end connection 10.0.20.53:54341 (1506 connections now open) 2016-03-13T09:50:11.511+0000 I NETWORK [conn4085647] end connection 10.0.20.53:54340 (1506 connections now open) 2016-03-13T09:50:11.512+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54349 #4085659 (1506 connections now open) 2016-03-13T09:50:11.512+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54350 #4085660 (1507 connections now open) 2016-03-13T09:50:13.948+0000 I NETWORK [conn4085650] end connection 10.0.179.209:44039 (1506 connections now open) 2016-03-13T09:50:13.948+0000 I NETWORK [conn4085649] end connection 10.0.179.209:44038 (1506 connections now open) 2016-03-13T09:50:13.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44047 #4085661 (1506 connections now open) 2016-03-13T09:50:13.958+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44046 #4085662 (1507 connections now open) 2016-03-13T09:50:14.537+0000 I NETWORK [conn4085652] end connection 10.0.26.69:45979 (1506 connections now open) 2016-03-13T09:50:14.537+0000 I NETWORK [conn4085651] end connection 10.0.26.69:45978 (1505 connections now open) 2016-03-13T09:50:14.537+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45989 #4085663 (1506 connections now open) 2016-03-13T09:50:14.537+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:45990 #4085664 (1507 connections now open) 2016-03-13T09:50:16.117+0000 I NETWORK [conn4081646] end connection 10.0.20.53:51009 (1506 connections now open) 2016-03-13T09:50:16.117+0000 I NETWORK [conn4081866] end connection 10.0.179.209:40847 (1506 connections now open) 2016-03-13T09:50:16.117+0000 I NETWORK [conn4082362] end connection 10.0.7.113:59957 (1504 connections now open) 2016-03-13T09:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54361 #4085665 (1505 connections now open) 2016-03-13T09:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44051 #4085666 (1506 connections now open) 2016-03-13T09:50:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34631 #4085667 (1507 connections now open) 2016-03-13T09:50:16.118+0000 I NETWORK [conn4081901] end connection 10.0.26.69:42883 (1506 connections now open) 2016-03-13T09:50:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46002 #4085668 (1507 connections now open) 2016-03-13T09:50:17.308+0000 I NETWORK [conn4085654] end connection 10.0.27.237:39401 (1506 connections now open) 2016-03-13T09:50:17.308+0000 I NETWORK [conn4085653] end connection 10.0.27.237:39402 (1506 connections now open) 2016-03-13T09:50:17.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39425 #4085669 (1506 connections now open) 2016-03-13T09:50:17.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39426 #4085670 (1507 connections now open) 2016-03-13T09:50:19.503+0000 I NETWORK [conn4085655] end connection 10.0.7.113:34624 (1506 connections now open) 2016-03-13T09:50:19.503+0000 I NETWORK [conn4085656] end connection 10.0.7.113:34625 (1506 connections now open) 2016-03-13T09:50:19.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34637 #4085671 (1506 connections now open) 2016-03-13T09:50:19.504+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34638 #4085672 (1507 connections now open) 2016-03-13T09:50:20.555+0000 I NETWORK [conn4085657] end connection 10.0.216.90:60884 (1506 connections now open) 2016-03-13T09:50:20.555+0000 I NETWORK [conn4085658] end connection 10.0.216.90:60885 (1505 connections now open) 2016-03-13T09:50:20.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60897 #4085673 (1506 connections now open) 2016-03-13T09:50:20.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60898 #4085674 (1507 connections now open) 2016-03-13T09:50:21.525+0000 I NETWORK [conn4085660] end connection 10.0.20.53:54350 (1506 connections now open) 2016-03-13T09:50:21.525+0000 I NETWORK [conn4085659] end connection 10.0.20.53:54349 (1506 connections now open) 2016-03-13T09:50:21.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54364 #4085675 (1506 connections now open) 2016-03-13T09:50:21.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54365 #4085676 (1507 connections now open) 2016-03-13T09:50:23.970+0000 I NETWORK [conn4085661] end connection 10.0.179.209:44047 (1506 connections now open) 2016-03-13T09:50:23.970+0000 I NETWORK [conn4085662] end connection 10.0.179.209:44046 (1505 connections now open) 2016-03-13T09:50:23.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44064 #4085677 (1506 connections now open) 2016-03-13T09:50:23.971+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44065 #4085678 (1507 connections now open) 2016-03-13T09:50:24.550+0000 I NETWORK [conn4085663] end connection 10.0.26.69:45989 (1506 connections now open) 2016-03-13T09:50:24.550+0000 I NETWORK [conn4085664] end connection 10.0.26.69:45990 (1506 connections now open) 2016-03-13T09:50:24.550+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46007 #4085679 (1507 connections now open) 2016-03-13T09:50:24.550+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46008 #4085680 (1507 connections now open) 2016-03-13T09:50:27.321+0000 I NETWORK [conn4085669] end connection 10.0.27.237:39425 (1506 connections now open) 2016-03-13T09:50:27.321+0000 I NETWORK [conn4085670] end connection 10.0.27.237:39426 (1506 connections now open) 2016-03-13T09:50:27.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39435 #4085681 (1506 connections now open) 2016-03-13T09:50:27.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39434 #4085682 (1507 connections now open) 2016-03-13T09:50:28.114+0000 I NETWORK [conn4081740] end connection 10.0.7.113:59709 (1506 connections now open) 2016-03-13T09:50:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34647 #4085683 (1507 connections now open) 2016-03-13T09:50:29.517+0000 I NETWORK [conn4085672] end connection 10.0.7.113:34638 (1506 connections now open) 2016-03-13T09:50:29.517+0000 I NETWORK [conn4085671] end connection 10.0.7.113:34637 (1506 connections now open) 2016-03-13T09:50:29.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34657 #4085684 (1506 connections now open) 2016-03-13T09:50:29.517+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34658 #4085685 (1507 connections now open) 2016-03-13T09:50:30.567+0000 I NETWORK [conn4085673] end connection 10.0.216.90:60897 (1506 connections now open) 2016-03-13T09:50:30.568+0000 I NETWORK [conn4085674] end connection 10.0.216.90:60898 (1505 connections now open) 2016-03-13T09:50:30.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60909 #4085686 (1506 connections now open) 2016-03-13T09:50:30.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60910 #4085687 (1507 connections now open) 2016-03-13T09:50:31.538+0000 I NETWORK [conn4085675] end connection 10.0.20.53:54364 (1506 connections now open) 2016-03-13T09:50:31.538+0000 I NETWORK [conn4085676] end connection 10.0.20.53:54365 (1506 connections now open) 2016-03-13T09:50:31.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54385 #4085688 (1506 connections now open) 2016-03-13T09:50:31.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54386 #4085689 (1507 connections now open) 2016-03-13T09:50:33.983+0000 I NETWORK [conn4085678] end connection 10.0.179.209:44065 (1506 connections now open) 2016-03-13T09:50:33.983+0000 I NETWORK [conn4085677] end connection 10.0.179.209:44064 (1506 connections now open) 2016-03-13T09:50:33.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44083 #4085690 (1506 connections now open) 2016-03-13T09:50:33.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44084 #4085691 (1507 connections now open) 2016-03-13T09:50:34.563+0000 I NETWORK [conn4085679] end connection 10.0.26.69:46007 (1506 connections now open) 2016-03-13T09:50:34.563+0000 I NETWORK [conn4085680] end connection 10.0.26.69:46008 (1506 connections now open) 2016-03-13T09:50:34.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46016 #4085692 (1506 connections now open) 2016-03-13T09:50:34.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46015 #4085693 (1507 connections now open) 2016-03-13T09:50:37.334+0000 I NETWORK [conn4085682] end connection 10.0.27.237:39434 (1506 connections now open) 2016-03-13T09:50:37.334+0000 I NETWORK [conn4085681] end connection 10.0.27.237:39435 (1506 connections now open) 2016-03-13T09:50:37.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39448 #4085694 (1506 connections now open) 2016-03-13T09:50:37.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39449 #4085695 (1507 connections now open) 2016-03-13T09:50:39.529+0000 I NETWORK [conn4085684] end connection 10.0.7.113:34657 (1506 connections now open) 2016-03-13T09:50:39.530+0000 I NETWORK [conn4085685] end connection 10.0.7.113:34658 (1505 connections now open) 2016-03-13T09:50:39.530+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34665 #4085696 (1506 connections now open) 2016-03-13T09:50:39.530+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34666 #4085697 (1507 connections now open) 2016-03-13T09:50:40.580+0000 I NETWORK [conn4085686] end connection 10.0.216.90:60909 (1506 connections now open) 2016-03-13T09:50:40.580+0000 I NETWORK [conn4085687] end connection 10.0.216.90:60910 (1505 connections now open) 2016-03-13T09:50:40.589+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60925 #4085698 (1506 connections now open) 2016-03-13T09:50:40.589+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60926 #4085699 (1507 connections now open) 2016-03-13T09:50:41.551+0000 I NETWORK [conn4085689] end connection 10.0.20.53:54386 (1506 connections now open) 2016-03-13T09:50:41.551+0000 I NETWORK [conn4085688] end connection 10.0.20.53:54385 (1506 connections now open) 2016-03-13T09:50:41.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54392 #4085700 (1506 connections now open) 2016-03-13T09:50:41.560+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54391 #4085701 (1507 connections now open) 2016-03-13T09:50:41.649+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:50:41.386+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:50:43.996+0000 I NETWORK [conn4085690] end connection 10.0.179.209:44083 (1506 connections now open) 2016-03-13T09:50:43.996+0000 I NETWORK [conn4085691] end connection 10.0.179.209:44084 (1505 connections now open) 2016-03-13T09:50:43.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44092 #4085702 (1506 connections now open) 2016-03-13T09:50:43.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44093 #4085703 (1507 connections now open) 2016-03-13T09:50:44.577+0000 I NETWORK [conn4085692] end connection 10.0.26.69:46016 (1506 connections now open) 2016-03-13T09:50:44.577+0000 I NETWORK [conn4085693] end connection 10.0.26.69:46015 (1506 connections now open) 2016-03-13T09:50:44.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46027 #4085704 (1506 connections now open) 2016-03-13T09:50:44.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46028 #4085705 (1507 connections now open) 2016-03-13T09:50:46.117+0000 I NETWORK [conn4078252] end connection 10.0.216.90:54905 (1506 connections now open) 2016-03-13T09:50:46.117+0000 I NETWORK [conn4082452] end connection 10.0.26.69:43121 (1505 connections now open) 2016-03-13T09:50:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60932 #4085706 (1506 connections now open) 2016-03-13T09:50:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46039 #4085707 (1507 connections now open) 2016-03-13T09:50:47.347+0000 I NETWORK [conn4085694] end connection 10.0.27.237:39448 (1506 connections now open) 2016-03-13T09:50:47.347+0000 I NETWORK [conn4085695] end connection 10.0.27.237:39449 (1506 connections now open) 2016-03-13T09:50:47.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39469 #4085708 (1506 connections now open) 2016-03-13T09:50:47.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39468 #4085709 (1507 connections now open) 2016-03-13T09:50:49.543+0000 I NETWORK [conn4085696] end connection 10.0.7.113:34665 (1506 connections now open) 2016-03-13T09:50:49.543+0000 I NETWORK [conn4085697] end connection 10.0.7.113:34666 (1506 connections now open) 2016-03-13T09:50:49.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34677 #4085710 (1506 connections now open) 2016-03-13T09:50:49.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34676 #4085711 (1507 connections now open) 2016-03-13T09:50:50.601+0000 I NETWORK [conn4085699] end connection 10.0.216.90:60926 (1506 connections now open) 2016-03-13T09:50:50.601+0000 I NETWORK [conn4085698] end connection 10.0.216.90:60925 (1506 connections now open) 2016-03-13T09:50:50.602+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60934 #4085712 (1506 connections now open) 2016-03-13T09:50:50.602+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60935 #4085713 (1507 connections now open) 2016-03-13T09:50:51.573+0000 I NETWORK [conn4085700] end connection 10.0.20.53:54392 (1506 connections now open) 2016-03-13T09:50:51.573+0000 I NETWORK [conn4085701] end connection 10.0.20.53:54391 (1506 connections now open) 2016-03-13T09:50:51.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54407 #4085714 (1506 connections now open) 2016-03-13T09:50:51.573+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54406 #4085715 (1507 connections now open) 2016-03-13T09:50:54.008+0000 I NETWORK [conn4085703] end connection 10.0.179.209:44093 (1506 connections now open) 2016-03-13T09:50:54.008+0000 I NETWORK [conn4085702] end connection 10.0.179.209:44092 (1505 connections now open) 2016-03-13T09:50:54.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44105 #4085716 (1506 connections now open) 2016-03-13T09:50:54.009+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44106 #4085717 (1507 connections now open) 2016-03-13T09:50:54.590+0000 I NETWORK [conn4085705] end connection 10.0.26.69:46028 (1506 connections now open) 2016-03-13T09:50:54.590+0000 I NETWORK [conn4085704] end connection 10.0.26.69:46027 (1506 connections now open) 2016-03-13T09:50:54.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46046 #4085718 (1507 connections now open) 2016-03-13T09:50:54.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46045 #4085719 (1507 connections now open) 2016-03-13T09:50:57.361+0000 I NETWORK [conn4085709] end connection 10.0.27.237:39468 (1506 connections now open) 2016-03-13T09:50:57.361+0000 I NETWORK [conn4085708] end connection 10.0.27.237:39469 (1506 connections now open) 2016-03-13T09:50:57.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39477 #4085720 (1506 connections now open) 2016-03-13T09:50:57.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39476 #4085721 (1507 connections now open) 2016-03-13T09:50:58.114+0000 I NETWORK [conn4081302] end connection 10.0.179.209:40726 (1506 connections now open) 2016-03-13T09:50:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44109 #4085722 (1507 connections now open) 2016-03-13T09:50:59.565+0000 I NETWORK [conn4085710] end connection 10.0.7.113:34677 (1506 connections now open) 2016-03-13T09:50:59.565+0000 I NETWORK [conn4085711] end connection 10.0.7.113:34676 (1506 connections now open) 2016-03-13T09:50:59.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34696 #4085723 (1506 connections now open) 2016-03-13T09:50:59.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34697 #4085724 (1507 connections now open) 2016-03-13T09:51:00.614+0000 I NETWORK [conn4085713] end connection 10.0.216.90:60935 (1506 connections now open) 2016-03-13T09:51:00.614+0000 I NETWORK [conn4085712] end connection 10.0.216.90:60934 (1506 connections now open) 2016-03-13T09:51:00.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60950 #4085725 (1506 connections now open) 2016-03-13T09:51:00.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60949 #4085726 (1507 connections now open) 2016-03-13T09:51:01.586+0000 I NETWORK [conn4085714] end connection 10.0.20.53:54407 (1506 connections now open) 2016-03-13T09:51:01.586+0000 I NETWORK [conn4085715] end connection 10.0.20.53:54406 (1506 connections now open) 2016-03-13T09:51:01.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54426 #4085727 (1506 connections now open) 2016-03-13T09:51:01.587+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54427 #4085728 (1507 connections now open) 2016-03-13T09:51:04.022+0000 I NETWORK [conn4085717] end connection 10.0.179.209:44106 (1506 connections now open) 2016-03-13T09:51:04.022+0000 I NETWORK [conn4085716] end connection 10.0.179.209:44105 (1506 connections now open) 2016-03-13T09:51:04.022+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44126 #4085729 (1506 connections now open) 2016-03-13T09:51:04.023+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44127 #4085730 (1507 connections now open) 2016-03-13T09:51:04.603+0000 I NETWORK [conn4085718] end connection 10.0.26.69:46046 (1506 connections now open) 2016-03-13T09:51:04.604+0000 I NETWORK [conn4085719] end connection 10.0.26.69:46045 (1505 connections now open) 2016-03-13T09:51:04.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46057 #4085731 (1506 connections now open) 2016-03-13T09:51:04.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46058 #4085732 (1507 connections now open) 2016-03-13T09:51:05.043+0000 I NETWORK [conn4084030] end connection 10.0.91.115:55845 (1506 connections now open) 2016-03-13T09:51:05.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:56880 #4085733 (1507 connections now open) 2016-03-13T09:51:07.382+0000 I NETWORK [conn4085720] end connection 10.0.27.237:39477 (1506 connections now open) 2016-03-13T09:51:07.382+0000 I NETWORK [conn4085721] end connection 10.0.27.237:39476 (1506 connections now open) 2016-03-13T09:51:07.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39493 #4085734 (1506 connections now open) 2016-03-13T09:51:07.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39492 #4085735 (1507 connections now open) 2016-03-13T09:51:09.578+0000 I NETWORK [conn4085723] end connection 10.0.7.113:34696 (1506 connections now open) 2016-03-13T09:51:09.578+0000 I NETWORK [conn4085724] end connection 10.0.7.113:34697 (1506 connections now open) 2016-03-13T09:51:09.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34710 #4085736 (1506 connections now open) 2016-03-13T09:51:09.579+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34709 #4085737 (1507 connections now open) 2016-03-13T09:51:10.627+0000 I NETWORK [conn4085726] end connection 10.0.216.90:60949 (1506 connections now open) 2016-03-13T09:51:10.627+0000 I NETWORK [conn4085725] end connection 10.0.216.90:60950 (1506 connections now open) 2016-03-13T09:51:10.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60971 #4085738 (1506 connections now open) 2016-03-13T09:51:10.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60970 #4085739 (1507 connections now open) 2016-03-13T09:51:11.599+0000 I NETWORK [conn4085728] end connection 10.0.20.53:54427 (1506 connections now open) 2016-03-13T09:51:11.599+0000 I NETWORK [conn4085727] end connection 10.0.20.53:54426 (1506 connections now open) 2016-03-13T09:51:11.600+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54433 #4085740 (1506 connections now open) 2016-03-13T09:51:11.600+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54434 #4085741 (1507 connections now open) 2016-03-13T09:51:11.817+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:51:11.649+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:51:14.035+0000 I NETWORK [conn4085729] end connection 10.0.179.209:44126 (1506 connections now open) 2016-03-13T09:51:14.035+0000 I NETWORK [conn4085730] end connection 10.0.179.209:44127 (1506 connections now open) 2016-03-13T09:51:14.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44135 #4085742 (1506 connections now open) 2016-03-13T09:51:14.045+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44134 #4085743 (1507 connections now open) 2016-03-13T09:51:14.626+0000 I NETWORK [conn4085732] end connection 10.0.26.69:46058 (1506 connections now open) 2016-03-13T09:51:14.626+0000 I NETWORK [conn4085731] end connection 10.0.26.69:46057 (1506 connections now open) 2016-03-13T09:51:14.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46073 #4085744 (1506 connections now open) 2016-03-13T09:51:14.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46074 #4085745 (1507 connections now open) 2016-03-13T09:51:16.117+0000 I NETWORK [conn4081925] end connection 10.0.179.209:40869 (1506 connections now open) 2016-03-13T09:51:16.117+0000 I NETWORK [conn4081840] end connection 10.0.26.69:42859 (1505 connections now open) 2016-03-13T09:51:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44138 #4085746 (1506 connections now open) 2016-03-13T09:51:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46086 #4085747 (1507 connections now open) 2016-03-13T09:51:17.395+0000 I NETWORK [conn4085735] end connection 10.0.27.237:39492 (1506 connections now open) 2016-03-13T09:51:17.395+0000 I NETWORK [conn4085734] end connection 10.0.27.237:39493 (1506 connections now open) 2016-03-13T09:51:17.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39516 #4085748 (1506 connections now open) 2016-03-13T09:51:17.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39517 #4085749 (1507 connections now open) 2016-03-13T09:51:19.592+0000 I NETWORK [conn4085737] end connection 10.0.7.113:34709 (1506 connections now open) 2016-03-13T09:51:19.592+0000 I NETWORK [conn4085736] end connection 10.0.7.113:34710 (1506 connections now open) 2016-03-13T09:51:19.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34722 #4085750 (1506 connections now open) 2016-03-13T09:51:19.592+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34721 #4085751 (1507 connections now open) 2016-03-13T09:51:20.640+0000 I NETWORK [conn4085738] end connection 10.0.216.90:60971 (1506 connections now open) 2016-03-13T09:51:20.640+0000 I NETWORK [conn4085739] end connection 10.0.216.90:60970 (1505 connections now open) 2016-03-13T09:51:20.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60979 #4085752 (1506 connections now open) 2016-03-13T09:51:20.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60980 #4085753 (1507 connections now open) 2016-03-13T09:51:21.612+0000 I NETWORK [conn4085741] end connection 10.0.20.53:54434 (1506 connections now open) 2016-03-13T09:51:21.613+0000 I NETWORK [conn4085740] end connection 10.0.20.53:54433 (1506 connections now open) 2016-03-13T09:51:21.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54447 #4085754 (1506 connections now open) 2016-03-13T09:51:21.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54448 #4085755 (1507 connections now open) 2016-03-13T09:51:24.057+0000 I NETWORK [conn4085742] end connection 10.0.179.209:44135 (1506 connections now open) 2016-03-13T09:51:24.057+0000 I NETWORK [conn4085743] end connection 10.0.179.209:44134 (1505 connections now open) 2016-03-13T09:51:24.057+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44149 #4085756 (1506 connections now open) 2016-03-13T09:51:24.058+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44150 #4085757 (1507 connections now open) 2016-03-13T09:51:24.639+0000 I NETWORK [conn4085744] end connection 10.0.26.69:46073 (1506 connections now open) 2016-03-13T09:51:24.639+0000 I NETWORK [conn4085745] end connection 10.0.26.69:46074 (1506 connections now open) 2016-03-13T09:51:24.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46092 #4085758 (1507 connections now open) 2016-03-13T09:51:24.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46091 #4085759 (1507 connections now open) 2016-03-13T09:51:27.408+0000 I NETWORK [conn4085748] end connection 10.0.27.237:39516 (1506 connections now open) 2016-03-13T09:51:27.408+0000 I NETWORK [conn4085749] end connection 10.0.27.237:39517 (1506 connections now open) 2016-03-13T09:51:27.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39524 #4085760 (1506 connections now open) 2016-03-13T09:51:27.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39525 #4085761 (1507 connections now open) 2016-03-13T09:51:28.114+0000 I NETWORK [conn4081938] end connection 10.0.26.69:42900 (1506 connections now open) 2016-03-13T09:51:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46096 #4085762 (1507 connections now open) 2016-03-13T09:51:29.605+0000 I NETWORK [conn4085751] end connection 10.0.7.113:34721 (1506 connections now open) 2016-03-13T09:51:29.605+0000 I NETWORK [conn4085750] end connection 10.0.7.113:34722 (1506 connections now open) 2016-03-13T09:51:29.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34740 #4085763 (1506 connections now open) 2016-03-13T09:51:29.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34741 #4085764 (1507 connections now open) 2016-03-13T09:51:30.652+0000 I NETWORK [conn4085753] end connection 10.0.216.90:60980 (1506 connections now open) 2016-03-13T09:51:30.653+0000 I NETWORK [conn4085752] end connection 10.0.216.90:60979 (1505 connections now open) 2016-03-13T09:51:30.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60996 #4085765 (1506 connections now open) 2016-03-13T09:51:30.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:60997 #4085766 (1507 connections now open) 2016-03-13T09:51:31.625+0000 I NETWORK [conn4085754] end connection 10.0.20.53:54447 (1506 connections now open) 2016-03-13T09:51:31.626+0000 I NETWORK [conn4085755] end connection 10.0.20.53:54448 (1505 connections now open) 2016-03-13T09:51:31.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54469 #4085767 (1506 connections now open) 2016-03-13T09:51:31.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54470 #4085768 (1507 connections now open) 2016-03-13T09:51:34.070+0000 I NETWORK [conn4085756] end connection 10.0.179.209:44149 (1506 connections now open) 2016-03-13T09:51:34.070+0000 I NETWORK [conn4085757] end connection 10.0.179.209:44150 (1506 connections now open) 2016-03-13T09:51:34.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44170 #4085769 (1506 connections now open) 2016-03-13T09:51:34.070+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44171 #4085770 (1507 connections now open) 2016-03-13T09:51:34.652+0000 I NETWORK [conn4085759] end connection 10.0.26.69:46091 (1506 connections now open) 2016-03-13T09:51:34.652+0000 I NETWORK [conn4085758] end connection 10.0.26.69:46092 (1506 connections now open) 2016-03-13T09:51:34.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46101 #4085771 (1506 connections now open) 2016-03-13T09:51:34.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46100 #4085772 (1507 connections now open) 2016-03-13T09:51:37.421+0000 I NETWORK [conn4085761] end connection 10.0.27.237:39525 (1506 connections now open) 2016-03-13T09:51:37.422+0000 I NETWORK [conn4085760] end connection 10.0.27.237:39524 (1505 connections now open) 2016-03-13T09:51:37.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39538 #4085773 (1506 connections now open) 2016-03-13T09:51:37.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39539 #4085774 (1507 connections now open) 2016-03-13T09:51:39.619+0000 I NETWORK [conn4085764] end connection 10.0.7.113:34741 (1506 connections now open) 2016-03-13T09:51:39.619+0000 I NETWORK [conn4085763] end connection 10.0.7.113:34740 (1506 connections now open) 2016-03-13T09:51:39.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34748 #4085775 (1506 connections now open) 2016-03-13T09:51:39.620+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34749 #4085776 (1507 connections now open) 2016-03-13T09:51:40.665+0000 I NETWORK [conn4085765] end connection 10.0.216.90:60996 (1506 connections now open) 2016-03-13T09:51:40.665+0000 I NETWORK [conn4085766] end connection 10.0.216.90:60997 (1505 connections now open) 2016-03-13T09:51:40.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32781 #4085777 (1506 connections now open) 2016-03-13T09:51:40.674+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32780 #4085778 (1507 connections now open) 2016-03-13T09:51:41.639+0000 I NETWORK [conn4085768] end connection 10.0.20.53:54470 (1506 connections now open) 2016-03-13T09:51:41.639+0000 I NETWORK [conn4085767] end connection 10.0.20.53:54469 (1506 connections now open) 2016-03-13T09:51:41.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54476 #4085779 (1506 connections now open) 2016-03-13T09:51:41.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54475 #4085780 (1507 connections now open) 2016-03-13T09:51:42.071+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:51:41.818+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:51:44.083+0000 I NETWORK [conn4085770] end connection 10.0.179.209:44171 (1506 connections now open) 2016-03-13T09:51:44.083+0000 I NETWORK [conn4085769] end connection 10.0.179.209:44170 (1506 connections now open) 2016-03-13T09:51:44.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44179 #4085781 (1506 connections now open) 2016-03-13T09:51:44.084+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44178 #4085782 (1507 connections now open) 2016-03-13T09:51:44.666+0000 I NETWORK [conn4085772] end connection 10.0.26.69:46100 (1506 connections now open) 2016-03-13T09:51:44.666+0000 I NETWORK [conn4085771] end connection 10.0.26.69:46101 (1506 connections now open) 2016-03-13T09:51:44.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46113 #4085783 (1506 connections now open) 2016-03-13T09:51:44.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46114 #4085784 (1507 connections now open) 2016-03-13T09:51:46.117+0000 I NETWORK [conn4084739] end connection 10.0.179.209:43056 (1506 connections now open) 2016-03-13T09:51:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44182 #4085785 (1507 connections now open) 2016-03-13T09:51:47.434+0000 I NETWORK [conn4085774] end connection 10.0.27.237:39539 (1506 connections now open) 2016-03-13T09:51:47.434+0000 I NETWORK [conn4085773] end connection 10.0.27.237:39538 (1506 connections now open) 2016-03-13T09:51:47.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39559 #4085786 (1506 connections now open) 2016-03-13T09:51:47.435+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39560 #4085787 (1507 connections now open) 2016-03-13T09:51:49.632+0000 I NETWORK [conn4085776] end connection 10.0.7.113:34749 (1506 connections now open) 2016-03-13T09:51:49.632+0000 I NETWORK [conn4085775] end connection 10.0.7.113:34748 (1506 connections now open) 2016-03-13T09:51:49.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34762 #4085788 (1506 connections now open) 2016-03-13T09:51:49.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34763 #4085789 (1507 connections now open) 2016-03-13T09:51:50.687+0000 I NETWORK [conn4085778] end connection 10.0.216.90:32780 (1506 connections now open) 2016-03-13T09:51:50.687+0000 I NETWORK [conn4085777] end connection 10.0.216.90:32781 (1506 connections now open) 2016-03-13T09:51:50.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32789 #4085790 (1506 connections now open) 2016-03-13T09:51:50.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32790 #4085791 (1507 connections now open) 2016-03-13T09:51:51.652+0000 I NETWORK [conn4085780] end connection 10.0.20.53:54475 (1506 connections now open) 2016-03-13T09:51:51.652+0000 I NETWORK [conn4085779] end connection 10.0.20.53:54476 (1506 connections now open) 2016-03-13T09:51:51.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54489 #4085792 (1506 connections now open) 2016-03-13T09:51:51.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54488 #4085793 (1507 connections now open) 2016-03-13T09:51:54.096+0000 I NETWORK [conn4085781] end connection 10.0.179.209:44179 (1506 connections now open) 2016-03-13T09:51:54.096+0000 I NETWORK [conn4085782] end connection 10.0.179.209:44178 (1506 connections now open) 2016-03-13T09:51:54.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44193 #4085794 (1506 connections now open) 2016-03-13T09:51:54.097+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44192 #4085795 (1507 connections now open) 2016-03-13T09:51:54.679+0000 I NETWORK [conn4085784] end connection 10.0.26.69:46114 (1506 connections now open) 2016-03-13T09:51:54.679+0000 I NETWORK [conn4085783] end connection 10.0.26.69:46113 (1506 connections now open) 2016-03-13T09:51:54.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46132 #4085796 (1507 connections now open) 2016-03-13T09:51:54.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46133 #4085797 (1507 connections now open) 2016-03-13T09:51:57.447+0000 I NETWORK [conn4085786] end connection 10.0.27.237:39559 (1506 connections now open) 2016-03-13T09:51:57.447+0000 I NETWORK [conn4085787] end connection 10.0.27.237:39560 (1505 connections now open) 2016-03-13T09:51:57.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39567 #4085798 (1506 connections now open) 2016-03-13T09:51:57.457+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39568 #4085799 (1507 connections now open) 2016-03-13T09:51:58.114+0000 I NETWORK [conn4084507] end connection 10.0.179.209:42820 (1506 connections now open) 2016-03-13T09:51:58.114+0000 I NETWORK [conn4082848] end connection 10.0.216.90:58195 (1506 connections now open) 2016-03-13T09:51:58.114+0000 I NETWORK [conn4081667] end connection 10.0.26.69:42767 (1506 connections now open) 2016-03-13T09:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32801 #4085800 (1505 connections now open) 2016-03-13T09:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44198 #4085801 (1506 connections now open) 2016-03-13T09:51:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46136 #4085802 (1507 connections now open) 2016-03-13T09:51:59.654+0000 I NETWORK [conn4085789] end connection 10.0.7.113:34763 (1506 connections now open) 2016-03-13T09:51:59.654+0000 I NETWORK [conn4085788] end connection 10.0.7.113:34762 (1506 connections now open) 2016-03-13T09:51:59.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34782 #4085803 (1506 connections now open) 2016-03-13T09:51:59.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34783 #4085804 (1507 connections now open) 2016-03-13T09:52:00.700+0000 I NETWORK [conn4085790] end connection 10.0.216.90:32789 (1506 connections now open) 2016-03-13T09:52:00.700+0000 I NETWORK [conn4085791] end connection 10.0.216.90:32790 (1506 connections now open) 2016-03-13T09:52:00.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32803 #4085805 (1506 connections now open) 2016-03-13T09:52:00.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32804 #4085806 (1507 connections now open) 2016-03-13T09:52:01.665+0000 I NETWORK [conn4085793] end connection 10.0.20.53:54488 (1506 connections now open) 2016-03-13T09:52:01.665+0000 I NETWORK [conn4085792] end connection 10.0.20.53:54489 (1506 connections now open) 2016-03-13T09:52:01.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54508 #4085807 (1506 connections now open) 2016-03-13T09:52:01.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54509 #4085808 (1507 connections now open) 2016-03-13T09:52:04.110+0000 I NETWORK [conn4085794] end connection 10.0.179.209:44193 (1506 connections now open) 2016-03-13T09:52:04.110+0000 I NETWORK [conn4085795] end connection 10.0.179.209:44192 (1506 connections now open) 2016-03-13T09:52:04.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44215 #4085809 (1506 connections now open) 2016-03-13T09:52:04.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44216 #4085810 (1507 connections now open) 2016-03-13T09:52:04.692+0000 I NETWORK [conn4085796] end connection 10.0.26.69:46132 (1506 connections now open) 2016-03-13T09:52:04.692+0000 I NETWORK [conn4085797] end connection 10.0.26.69:46133 (1506 connections now open) 2016-03-13T09:52:04.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46142 #4085811 (1506 connections now open) 2016-03-13T09:52:04.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46143 #4085812 (1507 connections now open) 2016-03-13T09:52:07.469+0000 I NETWORK [conn4085799] end connection 10.0.27.237:39568 (1506 connections now open) 2016-03-13T09:52:07.469+0000 I NETWORK [conn4085798] end connection 10.0.27.237:39567 (1506 connections now open) 2016-03-13T09:52:07.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39583 #4085813 (1506 connections now open) 2016-03-13T09:52:07.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39584 #4085814 (1507 connections now open) 2016-03-13T09:52:09.667+0000 I NETWORK [conn4085803] end connection 10.0.7.113:34782 (1506 connections now open) 2016-03-13T09:52:09.667+0000 I NETWORK [conn4085804] end connection 10.0.7.113:34783 (1506 connections now open) 2016-03-13T09:52:09.668+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34795 #4085815 (1506 connections now open) 2016-03-13T09:52:09.668+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34794 #4085816 (1507 connections now open) 2016-03-13T09:52:10.712+0000 I NETWORK [conn4085805] end connection 10.0.216.90:32803 (1506 connections now open) 2016-03-13T09:52:10.713+0000 I NETWORK [conn4085806] end connection 10.0.216.90:32804 (1505 connections now open) 2016-03-13T09:52:10.713+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32822 #4085817 (1506 connections now open) 2016-03-13T09:52:10.713+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32823 #4085818 (1507 connections now open) 2016-03-13T09:52:11.679+0000 I NETWORK [conn4085807] end connection 10.0.20.53:54508 (1506 connections now open) 2016-03-13T09:52:11.679+0000 I NETWORK [conn4085808] end connection 10.0.20.53:54509 (1506 connections now open) 2016-03-13T09:52:11.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54516 #4085819 (1506 connections now open) 2016-03-13T09:52:11.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54515 #4085820 (1507 connections now open) 2016-03-13T09:52:12.347+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:52:12.071+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:52:14.124+0000 I NETWORK [conn4085809] end connection 10.0.179.209:44215 (1506 connections now open) 2016-03-13T09:52:14.124+0000 I NETWORK [conn4085810] end connection 10.0.179.209:44216 (1506 connections now open) 2016-03-13T09:52:14.132+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44224 #4085821 (1506 connections now open) 2016-03-13T09:52:14.133+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44223 #4085822 (1507 connections now open) 2016-03-13T09:52:14.715+0000 I NETWORK [conn4085812] end connection 10.0.26.69:46143 (1506 connections now open) 2016-03-13T09:52:14.715+0000 I NETWORK [conn4085811] end connection 10.0.26.69:46142 (1506 connections now open) 2016-03-13T09:52:14.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46156 #4085823 (1506 connections now open) 2016-03-13T09:52:14.716+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46157 #4085824 (1507 connections now open) 2016-03-13T09:52:16.117+0000 I NETWORK [conn4081561] end connection 10.0.20.53:50959 (1506 connections now open) 2016-03-13T09:52:16.117+0000 I NETWORK [conn4082414] end connection 10.0.26.69:43104 (1505 connections now open) 2016-03-13T09:52:16.117+0000 I NETWORK [conn4085641] end connection 10.0.27.237:39389 (1505 connections now open) 2016-03-13T09:52:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39605 #4085825 (1505 connections now open) 2016-03-13T09:52:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46169 #4085826 (1506 connections now open) 2016-03-13T09:52:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54526 #4085827 (1507 connections now open) 2016-03-13T09:52:17.482+0000 I NETWORK [conn4085813] end connection 10.0.27.237:39583 (1506 connections now open) 2016-03-13T09:52:17.482+0000 I NETWORK [conn4085814] end connection 10.0.27.237:39584 (1506 connections now open) 2016-03-13T09:52:17.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39606 #4085828 (1506 connections now open) 2016-03-13T09:52:17.482+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39607 #4085829 (1507 connections now open) 2016-03-13T09:52:19.681+0000 I NETWORK [conn4085816] end connection 10.0.7.113:34794 (1506 connections now open) 2016-03-13T09:52:19.681+0000 I NETWORK [conn4085815] end connection 10.0.7.113:34795 (1506 connections now open) 2016-03-13T09:52:19.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34809 #4085830 (1506 connections now open) 2016-03-13T09:52:19.682+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34808 #4085831 (1507 connections now open) 2016-03-13T09:52:20.725+0000 I NETWORK [conn4085817] end connection 10.0.216.90:32822 (1506 connections now open) 2016-03-13T09:52:20.725+0000 I NETWORK [conn4085818] end connection 10.0.216.90:32823 (1505 connections now open) 2016-03-13T09:52:20.725+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32831 #4085832 (1506 connections now open) 2016-03-13T09:52:20.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32832 #4085833 (1507 connections now open) 2016-03-13T09:52:21.692+0000 I NETWORK [conn4085819] end connection 10.0.20.53:54516 (1506 connections now open) 2016-03-13T09:52:21.692+0000 I NETWORK [conn4085820] end connection 10.0.20.53:54515 (1506 connections now open) 2016-03-13T09:52:21.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54529 #4085834 (1506 connections now open) 2016-03-13T09:52:21.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54530 #4085835 (1507 connections now open) 2016-03-13T09:52:24.145+0000 I NETWORK [conn4085822] end connection 10.0.179.209:44223 (1506 connections now open) 2016-03-13T09:52:24.145+0000 I NETWORK [conn4085821] end connection 10.0.179.209:44224 (1506 connections now open) 2016-03-13T09:52:24.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44238 #4085836 (1506 connections now open) 2016-03-13T09:52:24.146+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44239 #4085837 (1507 connections now open) 2016-03-13T09:52:24.729+0000 I NETWORK [conn4085823] end connection 10.0.26.69:46156 (1506 connections now open) 2016-03-13T09:52:24.729+0000 I NETWORK [conn4085824] end connection 10.0.26.69:46157 (1506 connections now open) 2016-03-13T09:52:24.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46175 #4085838 (1507 connections now open) 2016-03-13T09:52:24.729+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46174 #4085839 (1507 connections now open) 2016-03-13T09:52:27.495+0000 I NETWORK [conn4085829] end connection 10.0.27.237:39607 (1506 connections now open) 2016-03-13T09:52:27.495+0000 I NETWORK [conn4085828] end connection 10.0.27.237:39606 (1506 connections now open) 2016-03-13T09:52:27.495+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39615 #4085840 (1506 connections now open) 2016-03-13T09:52:27.495+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39616 #4085841 (1507 connections now open) 2016-03-13T09:52:28.113+0000 I NETWORK [conn4085300] end connection 10.0.20.53:53992 (1506 connections now open) 2016-03-13T09:52:28.113+0000 I NETWORK [conn4082573] end connection 10.0.179.209:41121 (1506 connections now open) 2016-03-13T09:52:28.113+0000 I NETWORK [conn4083168] end connection 10.0.216.90:58370 (1506 connections now open) 2016-03-13T09:52:28.113+0000 I NETWORK [conn4081812] end connection 10.0.26.69:42842 (1503 connections now open) 2016-03-13T09:52:28.113+0000 I NETWORK [conn4082155] end connection 10.0.27.237:36300 (1502 connections now open) 2016-03-13T09:52:28.114+0000 I NETWORK [conn4081030] end connection 10.0.7.113:59559 (1501 connections now open) 2016-03-13T09:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32843 #4085842 (1502 connections now open) 2016-03-13T09:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54547 #4085843 (1503 connections now open) 2016-03-13T09:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44243 #4085844 (1504 connections now open) 2016-03-13T09:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39618 #4085845 (1505 connections now open) 2016-03-13T09:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46179 #4085846 (1506 connections now open) 2016-03-13T09:52:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34818 #4085847 (1507 connections now open) 2016-03-13T09:52:29.694+0000 I NETWORK [conn4085830] end connection 10.0.7.113:34809 (1506 connections now open) 2016-03-13T09:52:29.694+0000 I NETWORK [conn4085831] end connection 10.0.7.113:34808 (1505 connections now open) 2016-03-13T09:52:29.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34819 #4085848 (1506 connections now open) 2016-03-13T09:52:29.695+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34820 #4085849 (1507 connections now open) 2016-03-13T09:52:30.738+0000 I NETWORK [conn4085832] end connection 10.0.216.90:32831 (1506 connections now open) 2016-03-13T09:52:30.738+0000 I NETWORK [conn4085833] end connection 10.0.216.90:32832 (1505 connections now open) 2016-03-13T09:52:30.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32848 #4085850 (1506 connections now open) 2016-03-13T09:52:30.738+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32849 #4085851 (1507 connections now open) 2016-03-13T09:52:31.705+0000 I NETWORK [conn4085835] end connection 10.0.20.53:54530 (1506 connections now open) 2016-03-13T09:52:31.705+0000 I NETWORK [conn4085834] end connection 10.0.20.53:54529 (1506 connections now open) 2016-03-13T09:52:31.705+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54553 #4085852 (1506 connections now open) 2016-03-13T09:52:31.705+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54552 #4085853 (1507 connections now open) 2016-03-13T09:52:34.158+0000 I NETWORK [conn4085837] end connection 10.0.179.209:44239 (1506 connections now open) 2016-03-13T09:52:34.158+0000 I NETWORK [conn4085836] end connection 10.0.179.209:44238 (1506 connections now open) 2016-03-13T09:52:34.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44259 #4085854 (1506 connections now open) 2016-03-13T09:52:34.159+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44260 #4085855 (1507 connections now open) 2016-03-13T09:52:34.742+0000 I NETWORK [conn4085838] end connection 10.0.26.69:46175 (1506 connections now open) 2016-03-13T09:52:34.742+0000 I NETWORK [conn4085839] end connection 10.0.26.69:46174 (1506 connections now open) 2016-03-13T09:52:34.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46185 #4085856 (1506 connections now open) 2016-03-13T09:52:34.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46184 #4085857 (1507 connections now open) 2016-03-13T09:52:37.507+0000 I NETWORK [conn4085841] end connection 10.0.27.237:39616 (1506 connections now open) 2016-03-13T09:52:37.508+0000 I NETWORK [conn4085840] end connection 10.0.27.237:39615 (1505 connections now open) 2016-03-13T09:52:37.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39628 #4085858 (1506 connections now open) 2016-03-13T09:52:37.508+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39629 #4085859 (1507 connections now open) 2016-03-13T09:52:39.707+0000 I NETWORK [conn4085849] end connection 10.0.7.113:34820 (1506 connections now open) 2016-03-13T09:52:39.708+0000 I NETWORK [conn4085848] end connection 10.0.7.113:34819 (1505 connections now open) 2016-03-13T09:52:39.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34836 #4085860 (1506 connections now open) 2016-03-13T09:52:39.708+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34837 #4085861 (1507 connections now open) 2016-03-13T09:52:40.750+0000 I NETWORK [conn4085850] end connection 10.0.216.90:32848 (1506 connections now open) 2016-03-13T09:52:40.751+0000 I NETWORK [conn4085851] end connection 10.0.216.90:32849 (1505 connections now open) 2016-03-13T09:52:40.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32864 #4085862 (1506 connections now open) 2016-03-13T09:52:40.759+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32863 #4085863 (1507 connections now open) 2016-03-13T09:52:41.718+0000 I NETWORK [conn4085853] end connection 10.0.20.53:54552 (1506 connections now open) 2016-03-13T09:52:41.718+0000 I NETWORK [conn4085852] end connection 10.0.20.53:54553 (1506 connections now open) 2016-03-13T09:52:41.719+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54561 #4085864 (1506 connections now open) 2016-03-13T09:52:41.719+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54562 #4085865 (1507 connections now open) 2016-03-13T09:52:42.476+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:52:42.348+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:52:44.171+0000 I NETWORK [conn4085855] end connection 10.0.179.209:44260 (1506 connections now open) 2016-03-13T09:52:44.171+0000 I NETWORK [conn4085854] end connection 10.0.179.209:44259 (1505 connections now open) 2016-03-13T09:52:44.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44267 #4085866 (1506 connections now open) 2016-03-13T09:52:44.172+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44268 #4085867 (1507 connections now open) 2016-03-13T09:52:44.756+0000 I NETWORK [conn4085857] end connection 10.0.26.69:46184 (1506 connections now open) 2016-03-13T09:52:44.756+0000 I NETWORK [conn4085856] end connection 10.0.26.69:46185 (1506 connections now open) 2016-03-13T09:52:44.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46197 #4085868 (1506 connections now open) 2016-03-13T09:52:44.757+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46198 #4085869 (1507 connections now open) 2016-03-13T09:52:46.117+0000 I NETWORK [conn4079854] end connection 10.0.216.90:56528 (1506 connections now open) 2016-03-13T09:52:46.118+0000 I NETWORK [conn4082699] end connection 10.0.27.237:36523 (1505 connections now open) 2016-03-13T09:52:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32871 #4085870 (1507 connections now open) 2016-03-13T09:52:46.118+0000 I NETWORK [conn4081723] end connection 10.0.7.113:59706 (1505 connections now open) 2016-03-13T09:52:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39647 #4085871 (1506 connections now open) 2016-03-13T09:52:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34842 #4085872 (1507 connections now open) 2016-03-13T09:52:47.520+0000 I NETWORK [conn4085859] end connection 10.0.27.237:39629 (1506 connections now open) 2016-03-13T09:52:47.520+0000 I NETWORK [conn4085858] end connection 10.0.27.237:39628 (1506 connections now open) 2016-03-13T09:52:47.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39649 #4085873 (1506 connections now open) 2016-03-13T09:52:47.521+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39648 #4085874 (1507 connections now open) 2016-03-13T09:52:49.720+0000 I NETWORK [conn4085860] end connection 10.0.7.113:34836 (1506 connections now open) 2016-03-13T09:52:49.720+0000 I NETWORK [conn4085861] end connection 10.0.7.113:34837 (1506 connections now open) 2016-03-13T09:52:49.730+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34845 #4085875 (1506 connections now open) 2016-03-13T09:52:49.730+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34844 #4085876 (1507 connections now open) 2016-03-13T09:52:50.772+0000 I NETWORK [conn4085863] end connection 10.0.216.90:32863 (1506 connections now open) 2016-03-13T09:52:50.772+0000 I NETWORK [conn4085862] end connection 10.0.216.90:32864 (1506 connections now open) 2016-03-13T09:52:50.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32874 #4085877 (1506 connections now open) 2016-03-13T09:52:50.772+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32873 #4085878 (1507 connections now open) 2016-03-13T09:52:51.732+0000 I NETWORK [conn4085865] end connection 10.0.20.53:54562 (1506 connections now open) 2016-03-13T09:52:51.732+0000 I NETWORK [conn4085864] end connection 10.0.20.53:54561 (1506 connections now open) 2016-03-13T09:52:51.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54576 #4085879 (1506 connections now open) 2016-03-13T09:52:51.733+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54575 #4085880 (1507 connections now open) 2016-03-13T09:52:54.185+0000 I NETWORK [conn4085866] end connection 10.0.179.209:44267 (1506 connections now open) 2016-03-13T09:52:54.185+0000 I NETWORK [conn4085867] end connection 10.0.179.209:44268 (1506 connections now open) 2016-03-13T09:52:54.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44281 #4085881 (1506 connections now open) 2016-03-13T09:52:54.185+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44282 #4085882 (1507 connections now open) 2016-03-13T09:52:54.769+0000 I NETWORK [conn4085868] end connection 10.0.26.69:46197 (1506 connections now open) 2016-03-13T09:52:54.769+0000 I NETWORK [conn4085869] end connection 10.0.26.69:46198 (1506 connections now open) 2016-03-13T09:52:54.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46215 #4085883 (1506 connections now open) 2016-03-13T09:52:54.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46216 #4085884 (1507 connections now open) 2016-03-13T09:52:57.533+0000 I NETWORK [conn4085873] end connection 10.0.27.237:39649 (1506 connections now open) 2016-03-13T09:52:57.533+0000 I NETWORK [conn4085874] end connection 10.0.27.237:39648 (1506 connections now open) 2016-03-13T09:52:57.542+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39656 #4085885 (1506 connections now open) 2016-03-13T09:52:57.542+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39657 #4085886 (1507 connections now open) 2016-03-13T09:52:58.113+0000 I NETWORK [conn4082518] end connection 10.0.20.53:51379 (1506 connections now open) 2016-03-13T09:52:58.113+0000 I NETWORK [conn4082706] end connection 10.0.27.237:36530 (1505 connections now open) 2016-03-13T09:52:58.113+0000 I NETWORK [conn4082283] end connection 10.0.26.69:43046 (1505 connections now open) 2016-03-13T09:52:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54594 #4085887 (1505 connections now open) 2016-03-13T09:52:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39660 #4085888 (1506 connections now open) 2016-03-13T09:52:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46220 #4085889 (1507 connections now open) 2016-03-13T09:52:59.742+0000 I NETWORK [conn4085876] end connection 10.0.7.113:34844 (1506 connections now open) 2016-03-13T09:52:59.742+0000 I NETWORK [conn4085875] end connection 10.0.7.113:34845 (1506 connections now open) 2016-03-13T09:52:59.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34859 #4085890 (1506 connections now open) 2016-03-13T09:52:59.743+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34858 #4085891 (1507 connections now open) 2016-03-13T09:53:00.784+0000 I NETWORK [conn4085878] end connection 10.0.216.90:32873 (1506 connections now open) 2016-03-13T09:53:00.784+0000 I NETWORK [conn4085877] end connection 10.0.216.90:32874 (1506 connections now open) 2016-03-13T09:53:00.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32887 #4085892 (1506 connections now open) 2016-03-13T09:53:00.785+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32888 #4085893 (1507 connections now open) 2016-03-13T09:53:01.746+0000 I NETWORK [conn4085879] end connection 10.0.20.53:54576 (1506 connections now open) 2016-03-13T09:53:01.746+0000 I NETWORK [conn4085880] end connection 10.0.20.53:54575 (1506 connections now open) 2016-03-13T09:53:01.746+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54597 #4085894 (1506 connections now open) 2016-03-13T09:53:01.746+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54598 #4085895 (1507 connections now open) 2016-03-13T09:53:04.198+0000 I NETWORK [conn4085882] end connection 10.0.179.209:44282 (1506 connections now open) 2016-03-13T09:53:04.198+0000 I NETWORK [conn4085881] end connection 10.0.179.209:44281 (1506 connections now open) 2016-03-13T09:53:04.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44305 #4085896 (1506 connections now open) 2016-03-13T09:53:04.199+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44304 #4085897 (1507 connections now open) 2016-03-13T09:53:04.782+0000 I NETWORK [conn4085884] end connection 10.0.26.69:46216 (1506 connections now open) 2016-03-13T09:53:04.782+0000 I NETWORK [conn4085883] end connection 10.0.26.69:46215 (1506 connections now open) 2016-03-13T09:53:04.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46228 #4085898 (1506 connections now open) 2016-03-13T09:53:04.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46227 #4085899 (1507 connections now open) 2016-03-13T09:53:07.554+0000 I NETWORK [conn4085885] end connection 10.0.27.237:39656 (1506 connections now open) 2016-03-13T09:53:07.554+0000 I NETWORK [conn4085886] end connection 10.0.27.237:39657 (1506 connections now open) 2016-03-13T09:53:07.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39672 #4085900 (1506 connections now open) 2016-03-13T09:53:07.555+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39673 #4085901 (1507 connections now open) 2016-03-13T09:53:09.756+0000 I NETWORK [conn4085891] end connection 10.0.7.113:34858 (1506 connections now open) 2016-03-13T09:53:09.756+0000 I NETWORK [conn4085890] end connection 10.0.7.113:34859 (1506 connections now open) 2016-03-13T09:53:09.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34881 #4085902 (1506 connections now open) 2016-03-13T09:53:09.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34882 #4085903 (1507 connections now open) 2016-03-13T09:53:10.797+0000 I NETWORK [conn4085892] end connection 10.0.216.90:32887 (1506 connections now open) 2016-03-13T09:53:10.797+0000 I NETWORK [conn4085893] end connection 10.0.216.90:32888 (1506 connections now open) 2016-03-13T09:53:10.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32905 #4085904 (1506 connections now open) 2016-03-13T09:53:10.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32906 #4085905 (1507 connections now open) 2016-03-13T09:53:11.759+0000 I NETWORK [conn4085895] end connection 10.0.20.53:54598 (1506 connections now open) 2016-03-13T09:53:11.759+0000 I NETWORK [conn4085894] end connection 10.0.20.53:54597 (1506 connections now open) 2016-03-13T09:53:11.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54606 #4085906 (1506 connections now open) 2016-03-13T09:53:11.760+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54607 #4085907 (1507 connections now open) 2016-03-13T09:53:12.628+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:53:12.477+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:53:14.212+0000 I NETWORK [conn4085897] end connection 10.0.179.209:44304 (1506 connections now open) 2016-03-13T09:53:14.212+0000 I NETWORK [conn4085896] end connection 10.0.179.209:44305 (1506 connections now open) 2016-03-13T09:53:14.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44312 #4085908 (1506 connections now open) 2016-03-13T09:53:14.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44313 #4085909 (1507 connections now open) 2016-03-13T09:53:14.805+0000 I NETWORK [conn4085899] end connection 10.0.26.69:46227 (1506 connections now open) 2016-03-13T09:53:14.805+0000 I NETWORK [conn4085898] end connection 10.0.26.69:46228 (1506 connections now open) 2016-03-13T09:53:14.805+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46244 #4085910 (1506 connections now open) 2016-03-13T09:53:14.805+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46243 #4085911 (1507 connections now open) 2016-03-13T09:53:16.116+0000 I NETWORK [conn4081304] end connection 10.0.20.53:50822 (1506 connections now open) 2016-03-13T09:53:16.116+0000 I NETWORK [conn4082787] end connection 10.0.179.209:41221 (1506 connections now open) 2016-03-13T09:53:16.117+0000 I NETWORK [conn4081826] end connection 10.0.27.237:36144 (1504 connections now open) 2016-03-13T09:53:16.117+0000 I NETWORK [conn4081291] end connection 10.0.7.113:59637 (1503 connections now open) 2016-03-13T09:53:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39695 #4085912 (1504 connections now open) 2016-03-13T09:53:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34887 #4085913 (1505 connections now open) 2016-03-13T09:53:16.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54616 #4085914 (1506 connections now open) 2016-03-13T09:53:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44316 #4085915 (1507 connections now open) 2016-03-13T09:53:17.567+0000 I NETWORK [conn4085900] end connection 10.0.27.237:39672 (1506 connections now open) 2016-03-13T09:53:17.567+0000 I NETWORK [conn4085901] end connection 10.0.27.237:39673 (1506 connections now open) 2016-03-13T09:53:17.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39697 #4085916 (1506 connections now open) 2016-03-13T09:53:17.568+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39696 #4085917 (1507 connections now open) 2016-03-13T09:53:19.770+0000 I NETWORK [conn4085902] end connection 10.0.7.113:34881 (1506 connections now open) 2016-03-13T09:53:19.770+0000 I NETWORK [conn4085903] end connection 10.0.7.113:34882 (1506 connections now open) 2016-03-13T09:53:19.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34889 #4085918 (1506 connections now open) 2016-03-13T09:53:19.770+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34890 #4085919 (1507 connections now open) 2016-03-13T09:53:20.810+0000 I NETWORK [conn4085905] end connection 10.0.216.90:32906 (1506 connections now open) 2016-03-13T09:53:20.810+0000 I NETWORK [conn4085904] end connection 10.0.216.90:32905 (1506 connections now open) 2016-03-13T09:53:20.810+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32914 #4085920 (1506 connections now open) 2016-03-13T09:53:20.811+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32915 #4085921 (1507 connections now open) 2016-03-13T09:53:21.773+0000 I NETWORK [conn4085906] end connection 10.0.20.53:54606 (1506 connections now open) 2016-03-13T09:53:21.773+0000 I NETWORK [conn4085907] end connection 10.0.20.53:54607 (1506 connections now open) 2016-03-13T09:53:21.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54619 #4085922 (1506 connections now open) 2016-03-13T09:53:21.773+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54620 #4085923 (1507 connections now open) 2016-03-13T09:53:24.227+0000 I NETWORK [conn4085909] end connection 10.0.179.209:44313 (1506 connections now open) 2016-03-13T09:53:24.227+0000 I NETWORK [conn4085908] end connection 10.0.179.209:44312 (1506 connections now open) 2016-03-13T09:53:24.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44328 #4085924 (1506 connections now open) 2016-03-13T09:53:24.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44329 #4085925 (1507 connections now open) 2016-03-13T09:53:24.818+0000 I NETWORK [conn4085911] end connection 10.0.26.69:46243 (1506 connections now open) 2016-03-13T09:53:24.818+0000 I NETWORK [conn4085910] end connection 10.0.26.69:46244 (1506 connections now open) 2016-03-13T09:53:24.819+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46262 #4085926 (1507 connections now open) 2016-03-13T09:53:24.819+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46261 #4085927 (1507 connections now open) 2016-03-13T09:53:27.580+0000 I NETWORK [conn4085916] end connection 10.0.27.237:39697 (1506 connections now open) 2016-03-13T09:53:27.580+0000 I NETWORK [conn4085917] end connection 10.0.27.237:39696 (1506 connections now open) 2016-03-13T09:53:27.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39706 #4085928 (1506 connections now open) 2016-03-13T09:53:27.581+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39705 #4085929 (1507 connections now open) 2016-03-13T09:53:28.114+0000 I NETWORK [conn4083398] end connection 10.0.20.53:51964 (1506 connections now open) 2016-03-13T09:53:28.114+0000 I NETWORK [conn4084550] end connection 10.0.216.90:59760 (1506 connections now open) 2016-03-13T09:53:28.114+0000 I NETWORK [conn4082489] end connection 10.0.27.237:36429 (1504 connections now open) 2016-03-13T09:53:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54637 #4085930 (1505 connections now open) 2016-03-13T09:53:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32927 #4085931 (1506 connections now open) 2016-03-13T09:53:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39709 #4085932 (1507 connections now open) 2016-03-13T09:53:29.783+0000 I NETWORK [conn4085918] end connection 10.0.7.113:34889 (1506 connections now open) 2016-03-13T09:53:29.783+0000 I NETWORK [conn4085919] end connection 10.0.7.113:34890 (1506 connections now open) 2016-03-13T09:53:29.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34902 #4085933 (1506 connections now open) 2016-03-13T09:53:29.784+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34903 #4085934 (1507 connections now open) 2016-03-13T09:53:30.823+0000 I NETWORK [conn4085920] end connection 10.0.216.90:32914 (1506 connections now open) 2016-03-13T09:53:30.823+0000 I NETWORK [conn4085921] end connection 10.0.216.90:32915 (1506 connections now open) 2016-03-13T09:53:30.823+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32931 #4085935 (1506 connections now open) 2016-03-13T09:53:30.824+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32932 #4085936 (1507 connections now open) 2016-03-13T09:53:31.786+0000 I NETWORK [conn4085923] end connection 10.0.20.53:54620 (1506 connections now open) 2016-03-13T09:53:31.786+0000 I NETWORK [conn4085922] end connection 10.0.20.53:54619 (1506 connections now open) 2016-03-13T09:53:31.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54643 #4085937 (1506 connections now open) 2016-03-13T09:53:31.786+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54642 #4085938 (1507 connections now open) 2016-03-13T09:53:34.241+0000 I NETWORK [conn4085925] end connection 10.0.179.209:44329 (1506 connections now open) 2016-03-13T09:53:34.241+0000 I NETWORK [conn4085924] end connection 10.0.179.209:44328 (1506 connections now open) 2016-03-13T09:53:34.242+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44348 #4085939 (1506 connections now open) 2016-03-13T09:53:34.242+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44349 #4085940 (1507 connections now open) 2016-03-13T09:53:34.831+0000 I NETWORK [conn4085926] end connection 10.0.26.69:46262 (1506 connections now open) 2016-03-13T09:53:34.831+0000 I NETWORK [conn4085927] end connection 10.0.26.69:46261 (1506 connections now open) 2016-03-13T09:53:34.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46271 #4085941 (1506 connections now open) 2016-03-13T09:53:34.832+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46272 #4085942 (1507 connections now open) 2016-03-13T09:53:37.593+0000 I NETWORK [conn4085928] end connection 10.0.27.237:39706 (1506 connections now open) 2016-03-13T09:53:37.593+0000 I NETWORK [conn4085929] end connection 10.0.27.237:39705 (1505 connections now open) 2016-03-13T09:53:37.593+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39719 #4085943 (1506 connections now open) 2016-03-13T09:53:37.594+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39720 #4085944 (1507 connections now open) 2016-03-13T09:53:39.797+0000 I NETWORK [conn4085933] end connection 10.0.7.113:34902 (1506 connections now open) 2016-03-13T09:53:39.797+0000 I NETWORK [conn4085934] end connection 10.0.7.113:34903 (1506 connections now open) 2016-03-13T09:53:39.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34919 #4085945 (1506 connections now open) 2016-03-13T09:53:39.798+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34920 #4085946 (1507 connections now open) 2016-03-13T09:53:40.836+0000 I NETWORK [conn4085935] end connection 10.0.216.90:32931 (1506 connections now open) 2016-03-13T09:53:40.836+0000 I NETWORK [conn4085936] end connection 10.0.216.90:32932 (1506 connections now open) 2016-03-13T09:53:40.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32949 #4085947 (1506 connections now open) 2016-03-13T09:53:40.844+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32948 #4085948 (1507 connections now open) 2016-03-13T09:53:41.799+0000 I NETWORK [conn4085938] end connection 10.0.20.53:54642 (1506 connections now open) 2016-03-13T09:53:41.799+0000 I NETWORK [conn4085937] end connection 10.0.20.53:54643 (1506 connections now open) 2016-03-13T09:53:41.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54650 #4085949 (1506 connections now open) 2016-03-13T09:53:41.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54651 #4085950 (1507 connections now open) 2016-03-13T09:53:42.893+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:53:42.629+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:53:44.255+0000 I NETWORK [conn4085939] end connection 10.0.179.209:44348 (1506 connections now open) 2016-03-13T09:53:44.255+0000 I NETWORK [conn4085940] end connection 10.0.179.209:44349 (1506 connections now open) 2016-03-13T09:53:44.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44358 #4085951 (1506 connections now open) 2016-03-13T09:53:44.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44357 #4085952 (1507 connections now open) 2016-03-13T09:53:44.845+0000 I NETWORK [conn4085941] end connection 10.0.26.69:46271 (1506 connections now open) 2016-03-13T09:53:44.845+0000 I NETWORK [conn4085942] end connection 10.0.26.69:46272 (1506 connections now open) 2016-03-13T09:53:44.846+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46282 #4085953 (1506 connections now open) 2016-03-13T09:53:44.846+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46283 #4085954 (1507 connections now open) 2016-03-13T09:53:46.117+0000 I NETWORK [conn4082844] end connection 10.0.26.69:43283 (1506 connections now open) 2016-03-13T09:53:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46287 #4085955 (1507 connections now open) 2016-03-13T09:53:47.606+0000 I NETWORK [conn4085943] end connection 10.0.27.237:39719 (1506 connections now open) 2016-03-13T09:53:47.606+0000 I NETWORK [conn4085944] end connection 10.0.27.237:39720 (1505 connections now open) 2016-03-13T09:53:47.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39739 #4085956 (1506 connections now open) 2016-03-13T09:53:47.606+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39740 #4085957 (1507 connections now open) 2016-03-13T09:53:49.812+0000 I NETWORK [conn4085946] end connection 10.0.7.113:34920 (1506 connections now open) 2016-03-13T09:53:49.812+0000 I NETWORK [conn4085945] end connection 10.0.7.113:34919 (1506 connections now open) 2016-03-13T09:53:49.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34928 #4085958 (1506 connections now open) 2016-03-13T09:53:49.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34929 #4085959 (1507 connections now open) 2016-03-13T09:53:50.856+0000 I NETWORK [conn4085948] end connection 10.0.216.90:32948 (1506 connections now open) 2016-03-13T09:53:50.856+0000 I NETWORK [conn4085947] end connection 10.0.216.90:32949 (1505 connections now open) 2016-03-13T09:53:50.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32957 #4085960 (1506 connections now open) 2016-03-13T09:53:50.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32958 #4085961 (1507 connections now open) 2016-03-13T09:53:51.812+0000 I NETWORK [conn4085950] end connection 10.0.20.53:54651 (1506 connections now open) 2016-03-13T09:53:51.812+0000 I NETWORK [conn4085949] end connection 10.0.20.53:54650 (1506 connections now open) 2016-03-13T09:53:51.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54665 #4085962 (1506 connections now open) 2016-03-13T09:53:51.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54664 #4085963 (1507 connections now open) 2016-03-13T09:53:54.269+0000 I NETWORK [conn4085952] end connection 10.0.179.209:44357 (1506 connections now open) 2016-03-13T09:53:54.269+0000 I NETWORK [conn4085951] end connection 10.0.179.209:44358 (1506 connections now open) 2016-03-13T09:53:54.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44371 #4085964 (1506 connections now open) 2016-03-13T09:53:54.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44370 #4085965 (1507 connections now open) 2016-03-13T09:53:54.858+0000 I NETWORK [conn4085953] end connection 10.0.26.69:46282 (1506 connections now open) 2016-03-13T09:53:54.858+0000 I NETWORK [conn4085954] end connection 10.0.26.69:46283 (1506 connections now open) 2016-03-13T09:53:54.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46303 #4085966 (1506 connections now open) 2016-03-13T09:53:54.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46302 #4085967 (1507 connections now open) 2016-03-13T09:53:57.619+0000 I NETWORK [conn4085957] end connection 10.0.27.237:39740 (1506 connections now open) 2016-03-13T09:53:57.619+0000 I NETWORK [conn4085956] end connection 10.0.27.237:39739 (1506 connections now open) 2016-03-13T09:53:57.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39747 #4085968 (1506 connections now open) 2016-03-13T09:53:57.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39746 #4085969 (1507 connections now open) 2016-03-13T09:53:58.114+0000 I NETWORK [conn4082279] end connection 10.0.27.237:36360 (1506 connections now open) 2016-03-13T09:53:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39749 #4085970 (1507 connections now open) 2016-03-13T09:53:59.826+0000 I NETWORK [conn4085959] end connection 10.0.7.113:34929 (1506 connections now open) 2016-03-13T09:53:59.826+0000 I NETWORK [conn4085958] end connection 10.0.7.113:34928 (1506 connections now open) 2016-03-13T09:53:59.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34942 #4085971 (1506 connections now open) 2016-03-13T09:53:59.836+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34943 #4085972 (1507 connections now open) 2016-03-13T09:54:00.869+0000 I NETWORK [conn4085960] end connection 10.0.216.90:32957 (1506 connections now open) 2016-03-13T09:54:00.869+0000 I NETWORK [conn4085961] end connection 10.0.216.90:32958 (1505 connections now open) 2016-03-13T09:54:00.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32970 #4085973 (1506 connections now open) 2016-03-13T09:54:00.869+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32971 #4085974 (1507 connections now open) 2016-03-13T09:54:01.825+0000 I NETWORK [conn4085963] end connection 10.0.20.53:54664 (1506 connections now open) 2016-03-13T09:54:01.825+0000 I NETWORK [conn4085962] end connection 10.0.20.53:54665 (1506 connections now open) 2016-03-13T09:54:01.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54685 #4085975 (1506 connections now open) 2016-03-13T09:54:01.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54686 #4085976 (1507 connections now open) 2016-03-13T09:54:04.282+0000 I NETWORK [conn4085964] end connection 10.0.179.209:44371 (1506 connections now open) 2016-03-13T09:54:04.282+0000 I NETWORK [conn4085965] end connection 10.0.179.209:44370 (1506 connections now open) 2016-03-13T09:54:04.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44394 #4085977 (1506 connections now open) 2016-03-13T09:54:04.283+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44393 #4085978 (1507 connections now open) 2016-03-13T09:54:04.872+0000 I NETWORK [conn4085966] end connection 10.0.26.69:46303 (1506 connections now open) 2016-03-13T09:54:04.872+0000 I NETWORK [conn4085967] end connection 10.0.26.69:46302 (1506 connections now open) 2016-03-13T09:54:04.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46313 #4085979 (1506 connections now open) 2016-03-13T09:54:04.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46314 #4085980 (1507 connections now open) 2016-03-13T09:54:07.639+0000 I NETWORK [conn4085969] end connection 10.0.27.237:39746 (1506 connections now open) 2016-03-13T09:54:07.639+0000 I NETWORK [conn4085968] end connection 10.0.27.237:39747 (1506 connections now open) 2016-03-13T09:54:07.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39761 #4085981 (1506 connections now open) 2016-03-13T09:54:07.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39760 #4085982 (1507 connections now open) 2016-03-13T09:54:09.849+0000 I NETWORK [conn4085972] end connection 10.0.7.113:34943 (1506 connections now open) 2016-03-13T09:54:09.849+0000 I NETWORK [conn4085971] end connection 10.0.7.113:34942 (1506 connections now open) 2016-03-13T09:54:09.850+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34965 #4085983 (1506 connections now open) 2016-03-13T09:54:09.850+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34964 #4085984 (1507 connections now open) 2016-03-13T09:54:10.881+0000 I NETWORK [conn4085973] end connection 10.0.216.90:32970 (1506 connections now open) 2016-03-13T09:54:10.882+0000 I NETWORK [conn4085974] end connection 10.0.216.90:32971 (1505 connections now open) 2016-03-13T09:54:10.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32988 #4085985 (1506 connections now open) 2016-03-13T09:54:10.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32989 #4085986 (1507 connections now open) 2016-03-13T09:54:11.839+0000 I NETWORK [conn4085975] end connection 10.0.20.53:54685 (1506 connections now open) 2016-03-13T09:54:11.839+0000 I NETWORK [conn4085976] end connection 10.0.20.53:54686 (1506 connections now open) 2016-03-13T09:54:11.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54694 #4085987 (1506 connections now open) 2016-03-13T09:54:11.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54693 #4085988 (1507 connections now open) 2016-03-13T09:54:13.065+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:54:12.893+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:54:14.295+0000 I NETWORK [conn4085978] end connection 10.0.179.209:44393 (1506 connections now open) 2016-03-13T09:54:14.295+0000 I NETWORK [conn4085977] end connection 10.0.179.209:44394 (1506 connections now open) 2016-03-13T09:54:14.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44402 #4085989 (1506 connections now open) 2016-03-13T09:54:14.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44403 #4085990 (1507 connections now open) 2016-03-13T09:54:14.894+0000 I NETWORK [conn4085979] end connection 10.0.26.69:46313 (1506 connections now open) 2016-03-13T09:54:14.894+0000 I NETWORK [conn4085980] end connection 10.0.26.69:46314 (1506 connections now open) 2016-03-13T09:54:14.894+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46328 #4085991 (1506 connections now open) 2016-03-13T09:54:14.894+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46329 #4085992 (1507 connections now open) 2016-03-13T09:54:16.117+0000 I NETWORK [conn4081673] end connection 10.0.20.53:51028 (1506 connections now open) 2016-03-13T09:54:16.117+0000 I NETWORK [conn4081101] end connection 10.0.27.237:35832 (1505 connections now open) 2016-03-13T09:54:16.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54700 #4085993 (1506 connections now open) 2016-03-13T09:54:16.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39782 #4085994 (1507 connections now open) 2016-03-13T09:54:17.652+0000 I NETWORK [conn4085982] end connection 10.0.27.237:39760 (1506 connections now open) 2016-03-13T09:54:17.652+0000 I NETWORK [conn4085981] end connection 10.0.27.237:39761 (1506 connections now open) 2016-03-13T09:54:17.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39784 #4085995 (1506 connections now open) 2016-03-13T09:54:17.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39783 #4085996 (1507 connections now open) 2016-03-13T09:54:19.863+0000 I NETWORK [conn4085983] end connection 10.0.7.113:34965 (1506 connections now open) 2016-03-13T09:54:19.863+0000 I NETWORK [conn4085984] end connection 10.0.7.113:34964 (1506 connections now open) 2016-03-13T09:54:19.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34974 #4085997 (1506 connections now open) 2016-03-13T09:54:19.871+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34975 #4085998 (1507 connections now open) 2016-03-13T09:54:20.894+0000 I NETWORK [conn4085986] end connection 10.0.216.90:32989 (1506 connections now open) 2016-03-13T09:54:20.894+0000 I NETWORK [conn4085985] end connection 10.0.216.90:32988 (1505 connections now open) 2016-03-13T09:54:20.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32997 #4085999 (1506 connections now open) 2016-03-13T09:54:20.903+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:32996 #4086000 (1507 connections now open) 2016-03-13T09:54:21.852+0000 I NETWORK [conn4085988] end connection 10.0.20.53:54693 (1506 connections now open) 2016-03-13T09:54:21.852+0000 I NETWORK [conn4085987] end connection 10.0.20.53:54694 (1506 connections now open) 2016-03-13T09:54:21.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54707 #4086001 (1506 connections now open) 2016-03-13T09:54:21.853+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54708 #4086002 (1507 connections now open) 2016-03-13T09:54:24.308+0000 I NETWORK [conn4085989] end connection 10.0.179.209:44402 (1506 connections now open) 2016-03-13T09:54:24.309+0000 I NETWORK [conn4085990] end connection 10.0.179.209:44403 (1505 connections now open) 2016-03-13T09:54:24.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44418 #4086003 (1506 connections now open) 2016-03-13T09:54:24.318+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44419 #4086004 (1507 connections now open) 2016-03-13T09:54:24.907+0000 I NETWORK [conn4085991] end connection 10.0.26.69:46328 (1506 connections now open) 2016-03-13T09:54:24.907+0000 I NETWORK [conn4085992] end connection 10.0.26.69:46329 (1506 connections now open) 2016-03-13T09:54:24.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46348 #4086005 (1507 connections now open) 2016-03-13T09:54:24.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46349 #4086006 (1507 connections now open) 2016-03-13T09:54:27.665+0000 I NETWORK [conn4085995] end connection 10.0.27.237:39784 (1506 connections now open) 2016-03-13T09:54:27.665+0000 I NETWORK [conn4085996] end connection 10.0.27.237:39783 (1506 connections now open) 2016-03-13T09:54:27.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39792 #4086007 (1506 connections now open) 2016-03-13T09:54:27.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39793 #4086008 (1507 connections now open) 2016-03-13T09:54:28.113+0000 I NETWORK [conn4082552] end connection 10.0.20.53:51394 (1506 connections now open) 2016-03-13T09:54:28.113+0000 I NETWORK [conn4085842] end connection 10.0.216.90:32843 (1506 connections now open) 2016-03-13T09:54:28.114+0000 I NETWORK [conn4081576] end connection 10.0.7.113:59699 (1504 connections now open) 2016-03-13T09:54:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33008 #4086009 (1505 connections now open) 2016-03-13T09:54:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54725 #4086010 (1506 connections now open) 2016-03-13T09:54:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34988 #4086011 (1507 connections now open) 2016-03-13T09:54:29.884+0000 I NETWORK [conn4085998] end connection 10.0.7.113:34975 (1506 connections now open) 2016-03-13T09:54:29.884+0000 I NETWORK [conn4085997] end connection 10.0.7.113:34974 (1506 connections now open) 2016-03-13T09:54:29.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34990 #4086012 (1506 connections now open) 2016-03-13T09:54:29.885+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:34989 #4086013 (1507 connections now open) 2016-03-13T09:54:30.915+0000 I NETWORK [conn4086000] end connection 10.0.216.90:32996 (1506 connections now open) 2016-03-13T09:54:30.915+0000 I NETWORK [conn4085999] end connection 10.0.216.90:32997 (1506 connections now open) 2016-03-13T09:54:30.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33013 #4086014 (1506 connections now open) 2016-03-13T09:54:30.916+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33014 #4086015 (1507 connections now open) 2016-03-13T09:54:31.865+0000 I NETWORK [conn4086002] end connection 10.0.20.53:54708 (1506 connections now open) 2016-03-13T09:54:31.865+0000 I NETWORK [conn4086001] end connection 10.0.20.53:54707 (1506 connections now open) 2016-03-13T09:54:31.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54730 #4086016 (1506 connections now open) 2016-03-13T09:54:31.866+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54731 #4086017 (1507 connections now open) 2016-03-13T09:54:34.330+0000 I NETWORK [conn4086004] end connection 10.0.179.209:44419 (1506 connections now open) 2016-03-13T09:54:34.330+0000 I NETWORK [conn4086003] end connection 10.0.179.209:44418 (1506 connections now open) 2016-03-13T09:54:34.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44440 #4086018 (1506 connections now open) 2016-03-13T09:54:34.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44439 #4086019 (1507 connections now open) 2016-03-13T09:54:34.920+0000 I NETWORK [conn4086006] end connection 10.0.26.69:46349 (1506 connections now open) 2016-03-13T09:54:34.920+0000 I NETWORK [conn4086005] end connection 10.0.26.69:46348 (1506 connections now open) 2016-03-13T09:54:34.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46358 #4086020 (1506 connections now open) 2016-03-13T09:54:34.921+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46357 #4086021 (1507 connections now open) 2016-03-13T09:54:37.678+0000 I NETWORK [conn4086008] end connection 10.0.27.237:39793 (1506 connections now open) 2016-03-13T09:54:37.678+0000 I NETWORK [conn4086007] end connection 10.0.27.237:39792 (1506 connections now open) 2016-03-13T09:54:37.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39805 #4086022 (1506 connections now open) 2016-03-13T09:54:37.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39804 #4086023 (1507 connections now open) 2016-03-13T09:54:39.898+0000 I NETWORK [conn4086012] end connection 10.0.7.113:34990 (1506 connections now open) 2016-03-13T09:54:39.898+0000 I NETWORK [conn4086013] end connection 10.0.7.113:34989 (1506 connections now open) 2016-03-13T09:54:39.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35008 #4086024 (1506 connections now open) 2016-03-13T09:54:39.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35007 #4086025 (1507 connections now open) 2016-03-13T09:54:40.928+0000 I NETWORK [conn4086015] end connection 10.0.216.90:33014 (1506 connections now open) 2016-03-13T09:54:40.928+0000 I NETWORK [conn4086014] end connection 10.0.216.90:33013 (1506 connections now open) 2016-03-13T09:54:40.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33031 #4086026 (1506 connections now open) 2016-03-13T09:54:40.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33030 #4086027 (1507 connections now open) 2016-03-13T09:54:41.878+0000 I NETWORK [conn4086016] end connection 10.0.20.53:54730 (1506 connections now open) 2016-03-13T09:54:41.878+0000 I NETWORK [conn4086017] end connection 10.0.20.53:54731 (1506 connections now open) 2016-03-13T09:54:41.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54739 #4086028 (1506 connections now open) 2016-03-13T09:54:41.879+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54740 #4086029 (1507 connections now open) 2016-03-13T09:54:43.333+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:54:43.065+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:54:44.344+0000 I NETWORK [conn4086019] end connection 10.0.179.209:44439 (1506 connections now open) 2016-03-13T09:54:44.344+0000 I NETWORK [conn4086018] end connection 10.0.179.209:44440 (1506 connections now open) 2016-03-13T09:54:44.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44449 #4086030 (1506 connections now open) 2016-03-13T09:54:44.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44448 #4086031 (1507 connections now open) 2016-03-13T09:54:44.933+0000 I NETWORK [conn4086021] end connection 10.0.26.69:46357 (1506 connections now open) 2016-03-13T09:54:44.933+0000 I NETWORK [conn4086020] end connection 10.0.26.69:46358 (1506 connections now open) 2016-03-13T09:54:44.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46371 #4086032 (1506 connections now open) 2016-03-13T09:54:44.934+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46370 #4086033 (1507 connections now open) 2016-03-13T09:54:46.117+0000 I NETWORK [conn4082314] end connection 10.0.7.113:59935 (1506 connections now open) 2016-03-13T09:54:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35013 #4086034 (1507 connections now open) 2016-03-13T09:54:47.691+0000 I NETWORK [conn4086022] end connection 10.0.27.237:39805 (1506 connections now open) 2016-03-13T09:54:47.691+0000 I NETWORK [conn4086023] end connection 10.0.27.237:39804 (1506 connections now open) 2016-03-13T09:54:47.692+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39825 #4086035 (1506 connections now open) 2016-03-13T09:54:47.692+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39826 #4086036 (1507 connections now open) 2016-03-13T09:54:49.912+0000 I NETWORK [conn4086024] end connection 10.0.7.113:35008 (1506 connections now open) 2016-03-13T09:54:49.912+0000 I NETWORK [conn4086025] end connection 10.0.7.113:35007 (1506 connections now open) 2016-03-13T09:54:49.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35016 #4086037 (1506 connections now open) 2016-03-13T09:54:49.913+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35015 #4086038 (1507 connections now open) 2016-03-13T09:54:50.951+0000 I NETWORK [conn4086027] end connection 10.0.216.90:33030 (1506 connections now open) 2016-03-13T09:54:50.951+0000 I NETWORK [conn4086026] end connection 10.0.216.90:33031 (1506 connections now open) 2016-03-13T09:54:50.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33039 #4086039 (1506 connections now open) 2016-03-13T09:54:50.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33040 #4086040 (1507 connections now open) 2016-03-13T09:54:51.892+0000 I NETWORK [conn4086028] end connection 10.0.20.53:54739 (1506 connections now open) 2016-03-13T09:54:51.892+0000 I NETWORK [conn4086029] end connection 10.0.20.53:54740 (1506 connections now open) 2016-03-13T09:54:51.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54753 #4086041 (1506 connections now open) 2016-03-13T09:54:51.892+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54754 #4086042 (1507 connections now open) 2016-03-13T09:54:54.357+0000 I NETWORK [conn4086030] end connection 10.0.179.209:44449 (1506 connections now open) 2016-03-13T09:54:54.357+0000 I NETWORK [conn4086031] end connection 10.0.179.209:44448 (1506 connections now open) 2016-03-13T09:54:54.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44462 #4086043 (1506 connections now open) 2016-03-13T09:54:54.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44461 #4086044 (1507 connections now open) 2016-03-13T09:54:54.947+0000 I NETWORK [conn4086032] end connection 10.0.26.69:46371 (1506 connections now open) 2016-03-13T09:54:54.947+0000 I NETWORK [conn4086033] end connection 10.0.26.69:46370 (1506 connections now open) 2016-03-13T09:54:54.947+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46389 #4086045 (1507 connections now open) 2016-03-13T09:54:54.947+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46390 #4086046 (1507 connections now open) 2016-03-13T09:54:57.704+0000 I NETWORK [conn4086036] end connection 10.0.27.237:39826 (1506 connections now open) 2016-03-13T09:54:57.704+0000 I NETWORK [conn4086035] end connection 10.0.27.237:39825 (1506 connections now open) 2016-03-13T09:54:57.713+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39834 #4086047 (1506 connections now open) 2016-03-13T09:54:57.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39835 #4086048 (1507 connections now open) 2016-03-13T09:54:58.114+0000 I NETWORK [conn4081318] end connection 10.0.179.209:40730 (1506 connections now open) 2016-03-13T09:54:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44466 #4086049 (1507 connections now open) 2016-03-13T09:54:59.926+0000 I NETWORK [conn4086038] end connection 10.0.7.113:35015 (1506 connections now open) 2016-03-13T09:54:59.926+0000 I NETWORK [conn4086037] end connection 10.0.7.113:35016 (1506 connections now open) 2016-03-13T09:54:59.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35029 #4086050 (1506 connections now open) 2016-03-13T09:54:59.935+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35028 #4086051 (1507 connections now open) 2016-03-13T09:55:00.964+0000 I NETWORK [conn4086039] end connection 10.0.216.90:33039 (1506 connections now open) 2016-03-13T09:55:00.964+0000 I NETWORK [conn4086040] end connection 10.0.216.90:33040 (1506 connections now open) 2016-03-13T09:55:00.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33054 #4086052 (1506 connections now open) 2016-03-13T09:55:00.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33055 #4086053 (1507 connections now open) 2016-03-13T09:55:01.905+0000 I NETWORK [conn4086041] end connection 10.0.20.53:54753 (1506 connections now open) 2016-03-13T09:55:01.905+0000 I NETWORK [conn4086042] end connection 10.0.20.53:54754 (1506 connections now open) 2016-03-13T09:55:01.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54776 #4086054 (1506 connections now open) 2016-03-13T09:55:01.906+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54777 #4086055 (1507 connections now open) 2016-03-13T09:55:04.371+0000 I NETWORK [conn4086043] end connection 10.0.179.209:44462 (1506 connections now open) 2016-03-13T09:55:04.371+0000 I NETWORK [conn4086044] end connection 10.0.179.209:44461 (1506 connections now open) 2016-03-13T09:55:04.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44485 #4086056 (1506 connections now open) 2016-03-13T09:55:04.371+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44484 #4086057 (1507 connections now open) 2016-03-13T09:55:04.960+0000 I NETWORK [conn4086045] end connection 10.0.26.69:46389 (1506 connections now open) 2016-03-13T09:55:04.960+0000 I NETWORK [conn4086046] end connection 10.0.26.69:46390 (1506 connections now open) 2016-03-13T09:55:04.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46402 #4086058 (1506 connections now open) 2016-03-13T09:55:04.970+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46403 #4086059 (1507 connections now open) 2016-03-13T09:55:07.726+0000 I NETWORK [conn4086048] end connection 10.0.27.237:39835 (1506 connections now open) 2016-03-13T09:55:07.726+0000 I NETWORK [conn4086047] end connection 10.0.27.237:39834 (1506 connections now open) 2016-03-13T09:55:07.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39852 #4086060 (1506 connections now open) 2016-03-13T09:55:07.727+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39853 #4086061 (1507 connections now open) 2016-03-13T09:55:09.948+0000 I NETWORK [conn4086051] end connection 10.0.7.113:35028 (1506 connections now open) 2016-03-13T09:55:09.948+0000 I NETWORK [conn4086050] end connection 10.0.7.113:35029 (1506 connections now open) 2016-03-13T09:55:09.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35054 #4086062 (1506 connections now open) 2016-03-13T09:55:09.949+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35053 #4086063 (1507 connections now open) 2016-03-13T09:55:10.977+0000 I NETWORK [conn4086052] end connection 10.0.216.90:33054 (1506 connections now open) 2016-03-13T09:55:10.977+0000 I NETWORK [conn4086053] end connection 10.0.216.90:33055 (1506 connections now open) 2016-03-13T09:55:10.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33073 #4086064 (1506 connections now open) 2016-03-13T09:55:10.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33074 #4086065 (1507 connections now open) 2016-03-13T09:55:11.918+0000 I NETWORK [conn4086054] end connection 10.0.20.53:54776 (1506 connections now open) 2016-03-13T09:55:11.918+0000 I NETWORK [conn4086055] end connection 10.0.20.53:54777 (1505 connections now open) 2016-03-13T09:55:11.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54785 #4086066 (1506 connections now open) 2016-03-13T09:55:11.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54786 #4086067 (1507 connections now open) 2016-03-13T09:55:13.472+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:55:13.334+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:55:14.384+0000 I NETWORK [conn4086057] end connection 10.0.179.209:44484 (1506 connections now open) 2016-03-13T09:55:14.384+0000 I NETWORK [conn4086056] end connection 10.0.179.209:44485 (1506 connections now open) 2016-03-13T09:55:14.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44493 #4086068 (1506 connections now open) 2016-03-13T09:55:14.385+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44494 #4086069 (1507 connections now open) 2016-03-13T09:55:14.982+0000 I NETWORK [conn4086058] end connection 10.0.26.69:46402 (1506 connections now open) 2016-03-13T09:55:14.982+0000 I NETWORK [conn4086059] end connection 10.0.26.69:46403 (1506 connections now open) 2016-03-13T09:55:14.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46418 #4086070 (1506 connections now open) 2016-03-13T09:55:14.983+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46419 #4086071 (1507 connections now open) 2016-03-13T09:55:16.117+0000 I NETWORK [conn4082040] end connection 10.0.179.209:40919 (1506 connections now open) 2016-03-13T09:55:16.117+0000 I NETWORK [conn4078505] end connection 10.0.216.90:55176 (1506 connections now open) 2016-03-13T09:55:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33081 #4086072 (1506 connections now open) 2016-03-13T09:55:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44498 #4086073 (1507 connections now open) 2016-03-13T09:55:17.739+0000 I NETWORK [conn4086060] end connection 10.0.27.237:39852 (1506 connections now open) 2016-03-13T09:55:17.739+0000 I NETWORK [conn4086061] end connection 10.0.27.237:39853 (1506 connections now open) 2016-03-13T09:55:17.739+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39877 #4086074 (1506 connections now open) 2016-03-13T09:55:17.740+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39876 #4086075 (1507 connections now open) 2016-03-13T09:55:19.962+0000 I NETWORK [conn4086062] end connection 10.0.7.113:35054 (1506 connections now open) 2016-03-13T09:55:19.962+0000 I NETWORK [conn4086063] end connection 10.0.7.113:35053 (1506 connections now open) 2016-03-13T09:55:19.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35064 #4086076 (1506 connections now open) 2016-03-13T09:55:19.963+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35063 #4086077 (1507 connections now open) 2016-03-13T09:55:20.990+0000 I NETWORK [conn4086065] end connection 10.0.216.90:33074 (1506 connections now open) 2016-03-13T09:55:20.990+0000 I NETWORK [conn4086064] end connection 10.0.216.90:33073 (1506 connections now open) 2016-03-13T09:55:20.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33083 #4086078 (1506 connections now open) 2016-03-13T09:55:20.991+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33084 #4086079 (1507 connections now open) 2016-03-13T09:55:21.931+0000 I NETWORK [conn4086066] end connection 10.0.20.53:54785 (1506 connections now open) 2016-03-13T09:55:21.931+0000 I NETWORK [conn4086067] end connection 10.0.20.53:54786 (1506 connections now open) 2016-03-13T09:55:21.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54798 #4086080 (1506 connections now open) 2016-03-13T09:55:21.941+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54799 #4086081 (1507 connections now open) 2016-03-13T09:55:24.398+0000 I NETWORK [conn4086069] end connection 10.0.179.209:44494 (1506 connections now open) 2016-03-13T09:55:24.398+0000 I NETWORK [conn4086068] end connection 10.0.179.209:44493 (1506 connections now open) 2016-03-13T09:55:24.406+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44513 #4086082 (1506 connections now open) 2016-03-13T09:55:24.406+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44512 #4086083 (1507 connections now open) 2016-03-13T09:55:24.995+0000 I NETWORK [conn4086070] end connection 10.0.26.69:46418 (1506 connections now open) 2016-03-13T09:55:24.995+0000 I NETWORK [conn4086071] end connection 10.0.26.69:46419 (1506 connections now open) 2016-03-13T09:55:24.996+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46439 #4086084 (1507 connections now open) 2016-03-13T09:55:24.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46440 #4086085 (1507 connections now open) 2016-03-13T09:55:27.752+0000 I NETWORK [conn4086075] end connection 10.0.27.237:39876 (1506 connections now open) 2016-03-13T09:55:27.752+0000 I NETWORK [conn4086074] end connection 10.0.27.237:39877 (1506 connections now open) 2016-03-13T09:55:27.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39885 #4086086 (1506 connections now open) 2016-03-13T09:55:27.753+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39886 #4086087 (1507 connections now open) 2016-03-13T09:55:28.114+0000 I NETWORK [conn4080920] end connection 10.0.179.209:40588 (1506 connections now open) 2016-03-13T09:55:28.114+0000 I NETWORK [conn4081555] end connection 10.0.26.69:42707 (1505 connections now open) 2016-03-13T09:55:28.114+0000 I NETWORK [conn4081583] end connection 10.0.7.113:59700 (1504 connections now open) 2016-03-13T09:55:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44517 #4086088 (1505 connections now open) 2016-03-13T09:55:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46444 #4086089 (1506 connections now open) 2016-03-13T09:55:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35075 #4086090 (1507 connections now open) 2016-03-13T09:55:29.976+0000 I NETWORK [conn4086076] end connection 10.0.7.113:35064 (1506 connections now open) 2016-03-13T09:55:29.976+0000 I NETWORK [conn4086077] end connection 10.0.7.113:35063 (1506 connections now open) 2016-03-13T09:55:29.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35078 #4086091 (1506 connections now open) 2016-03-13T09:55:29.977+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35077 #4086092 (1507 connections now open) 2016-03-13T09:55:31.002+0000 I NETWORK [conn4086078] end connection 10.0.216.90:33083 (1506 connections now open) 2016-03-13T09:55:31.002+0000 I NETWORK [conn4086079] end connection 10.0.216.90:33084 (1506 connections now open) 2016-03-13T09:55:31.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33100 #4086093 (1506 connections now open) 2016-03-13T09:55:31.003+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33101 #4086094 (1507 connections now open) 2016-03-13T09:55:31.954+0000 I NETWORK [conn4086080] end connection 10.0.20.53:54798 (1506 connections now open) 2016-03-13T09:55:31.954+0000 I NETWORK [conn4086081] end connection 10.0.20.53:54799 (1506 connections now open) 2016-03-13T09:55:31.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54823 #4086095 (1506 connections now open) 2016-03-13T09:55:31.955+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54824 #4086096 (1507 connections now open) 2016-03-13T09:55:34.419+0000 I NETWORK [conn4086082] end connection 10.0.179.209:44513 (1506 connections now open) 2016-03-13T09:55:34.419+0000 I NETWORK [conn4086083] end connection 10.0.179.209:44512 (1506 connections now open) 2016-03-13T09:55:34.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44532 #4086097 (1506 connections now open) 2016-03-13T09:55:34.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44533 #4086098 (1507 connections now open) 2016-03-13T09:55:35.011+0000 I NETWORK [conn4086084] end connection 10.0.26.69:46439 (1506 connections now open) 2016-03-13T09:55:35.011+0000 I NETWORK [conn4086085] end connection 10.0.26.69:46440 (1506 connections now open) 2016-03-13T09:55:35.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46451 #4086099 (1506 connections now open) 2016-03-13T09:55:35.011+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46450 #4086100 (1507 connections now open) 2016-03-13T09:55:37.765+0000 I NETWORK [conn4086086] end connection 10.0.27.237:39885 (1506 connections now open) 2016-03-13T09:55:37.765+0000 I NETWORK [conn4086087] end connection 10.0.27.237:39886 (1505 connections now open) 2016-03-13T09:55:37.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39899 #4086101 (1506 connections now open) 2016-03-13T09:55:37.766+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39900 #4086102 (1507 connections now open) 2016-03-13T09:55:39.989+0000 I NETWORK [conn4086092] end connection 10.0.7.113:35077 (1506 connections now open) 2016-03-13T09:55:39.990+0000 I NETWORK [conn4086091] end connection 10.0.7.113:35078 (1505 connections now open) 2016-03-13T09:55:39.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35095 #4086103 (1506 connections now open) 2016-03-13T09:55:39.990+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35094 #4086104 (1507 connections now open) 2016-03-13T09:55:41.015+0000 I NETWORK [conn4086093] end connection 10.0.216.90:33100 (1506 connections now open) 2016-03-13T09:55:41.015+0000 I NETWORK [conn4086094] end connection 10.0.216.90:33101 (1506 connections now open) 2016-03-13T09:55:41.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33117 #4086105 (1506 connections now open) 2016-03-13T09:55:41.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33118 #4086106 (1507 connections now open) 2016-03-13T09:55:41.967+0000 I NETWORK [conn4086095] end connection 10.0.20.53:54823 (1506 connections now open) 2016-03-13T09:55:41.967+0000 I NETWORK [conn4086096] end connection 10.0.20.53:54824 (1506 connections now open) 2016-03-13T09:55:41.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54831 #4086107 (1506 connections now open) 2016-03-13T09:55:41.968+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54832 #4086108 (1507 connections now open) 2016-03-13T09:55:43.733+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:55:43.472+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:55:44.433+0000 I NETWORK [conn4086097] end connection 10.0.179.209:44532 (1506 connections now open) 2016-03-13T09:55:44.433+0000 I NETWORK [conn4086098] end connection 10.0.179.209:44533 (1506 connections now open) 2016-03-13T09:55:44.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44541 #4086109 (1506 connections now open) 2016-03-13T09:55:44.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44542 #4086110 (1507 connections now open) 2016-03-13T09:55:45.024+0000 I NETWORK [conn4086100] end connection 10.0.26.69:46450 (1506 connections now open) 2016-03-13T09:55:45.024+0000 I NETWORK [conn4086099] end connection 10.0.26.69:46451 (1506 connections now open) 2016-03-13T09:55:45.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46461 #4086111 (1506 connections now open) 2016-03-13T09:55:45.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46462 #4086112 (1507 connections now open) 2016-03-13T09:55:46.117+0000 I NETWORK [conn4081679] end connection 10.0.20.53:51031 (1506 connections now open) 2016-03-13T09:55:46.117+0000 I NETWORK [conn4085785] end connection 10.0.179.209:44182 (1506 connections now open) 2016-03-13T09:55:46.117+0000 I NETWORK [conn4081920] end connection 10.0.26.69:42893 (1506 connections now open) 2016-03-13T09:55:46.117+0000 I NETWORK [conn4084536] end connection 10.0.7.113:33483 (1503 connections now open) 2016-03-13T09:55:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54838 #4086113 (1504 connections now open) 2016-03-13T09:55:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46464 #4086114 (1505 connections now open) 2016-03-13T09:55:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44545 #4086115 (1506 connections now open) 2016-03-13T09:55:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35102 #4086116 (1507 connections now open) 2016-03-13T09:55:47.778+0000 I NETWORK [conn4086101] end connection 10.0.27.237:39899 (1506 connections now open) 2016-03-13T09:55:47.778+0000 I NETWORK [conn4086102] end connection 10.0.27.237:39900 (1506 connections now open) 2016-03-13T09:55:47.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39918 #4086117 (1506 connections now open) 2016-03-13T09:55:47.779+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39919 #4086118 (1507 connections now open) 2016-03-13T09:55:50.003+0000 I NETWORK [conn4086104] end connection 10.0.7.113:35094 (1506 connections now open) 2016-03-13T09:55:50.003+0000 I NETWORK [conn4086103] end connection 10.0.7.113:35095 (1506 connections now open) 2016-03-13T09:55:50.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35104 #4086119 (1506 connections now open) 2016-03-13T09:55:50.004+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35105 #4086120 (1507 connections now open) 2016-03-13T09:55:51.037+0000 I NETWORK [conn4086106] end connection 10.0.216.90:33118 (1506 connections now open) 2016-03-13T09:55:51.037+0000 I NETWORK [conn4086105] end connection 10.0.216.90:33117 (1506 connections now open) 2016-03-13T09:55:51.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33126 #4086121 (1506 connections now open) 2016-03-13T09:55:51.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33125 #4086122 (1507 connections now open) 2016-03-13T09:55:51.980+0000 I NETWORK [conn4086107] end connection 10.0.20.53:54831 (1506 connections now open) 2016-03-13T09:55:51.980+0000 I NETWORK [conn4086108] end connection 10.0.20.53:54832 (1506 connections now open) 2016-03-13T09:55:51.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54845 #4086123 (1506 connections now open) 2016-03-13T09:55:51.981+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54846 #4086124 (1507 connections now open) 2016-03-13T09:55:54.446+0000 I NETWORK [conn4086109] end connection 10.0.179.209:44541 (1506 connections now open) 2016-03-13T09:55:54.446+0000 I NETWORK [conn4086110] end connection 10.0.179.209:44542 (1506 connections now open) 2016-03-13T09:55:54.447+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44556 #4086125 (1506 connections now open) 2016-03-13T09:55:54.447+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44555 #4086126 (1507 connections now open) 2016-03-13T09:55:55.038+0000 I NETWORK [conn4086112] end connection 10.0.26.69:46462 (1506 connections now open) 2016-03-13T09:55:55.038+0000 I NETWORK [conn4086111] end connection 10.0.26.69:46461 (1506 connections now open) 2016-03-13T09:55:55.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46481 #4086127 (1507 connections now open) 2016-03-13T09:55:55.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46480 #4086128 (1507 connections now open) 2016-03-13T09:55:57.791+0000 I NETWORK [conn4086117] end connection 10.0.27.237:39918 (1506 connections now open) 2016-03-13T09:55:57.791+0000 I NETWORK [conn4086118] end connection 10.0.27.237:39919 (1505 connections now open) 2016-03-13T09:55:57.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39928 #4086129 (1506 connections now open) 2016-03-13T09:55:57.792+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39927 #4086130 (1507 connections now open) 2016-03-13T09:55:58.114+0000 I NETWORK [conn4082876] end connection 10.0.216.90:58208 (1506 connections now open) 2016-03-13T09:55:58.114+0000 I NETWORK [conn4081341] end connection 10.0.179.209:40735 (1506 connections now open) 2016-03-13T09:55:58.114+0000 I NETWORK [conn4081680] end connection 10.0.26.69:42775 (1506 connections now open) 2016-03-13T09:55:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33138 #4086131 (1505 connections now open) 2016-03-13T09:55:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44560 #4086132 (1506 connections now open) 2016-03-13T09:55:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46485 #4086133 (1507 connections now open) 2016-03-13T09:56:00.016+0000 I NETWORK [conn4086119] end connection 10.0.7.113:35104 (1506 connections now open) 2016-03-13T09:56:00.016+0000 I NETWORK [conn4086120] end connection 10.0.7.113:35105 (1505 connections now open) 2016-03-13T09:56:00.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35120 #4086134 (1506 connections now open) 2016-03-13T09:56:00.026+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35119 #4086135 (1507 connections now open) 2016-03-13T09:56:01.050+0000 I NETWORK [conn4086122] end connection 10.0.216.90:33125 (1506 connections now open) 2016-03-13T09:56:01.050+0000 I NETWORK [conn4086121] end connection 10.0.216.90:33126 (1506 connections now open) 2016-03-13T09:56:01.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33141 #4086136 (1506 connections now open) 2016-03-13T09:56:01.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33140 #4086137 (1507 connections now open) 2016-03-13T09:56:01.993+0000 I NETWORK [conn4086123] end connection 10.0.20.53:54845 (1506 connections now open) 2016-03-13T09:56:01.993+0000 I NETWORK [conn4086124] end connection 10.0.20.53:54846 (1506 connections now open) 2016-03-13T09:56:01.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54867 #4086138 (1506 connections now open) 2016-03-13T09:56:01.994+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54868 #4086139 (1507 connections now open) 2016-03-13T09:56:04.459+0000 I NETWORK [conn4086125] end connection 10.0.179.209:44556 (1506 connections now open) 2016-03-13T09:56:04.459+0000 I NETWORK [conn4086126] end connection 10.0.179.209:44555 (1506 connections now open) 2016-03-13T09:56:04.460+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44577 #4086140 (1506 connections now open) 2016-03-13T09:56:04.460+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44578 #4086141 (1507 connections now open) 2016-03-13T09:56:05.051+0000 I NETWORK [conn4086127] end connection 10.0.26.69:46481 (1506 connections now open) 2016-03-13T09:56:05.051+0000 I NETWORK [conn4086128] end connection 10.0.26.69:46480 (1506 connections now open) 2016-03-13T09:56:05.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46493 #4086142 (1506 connections now open) 2016-03-13T09:56:05.061+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46492 #4086143 (1507 connections now open) 2016-03-13T09:56:07.805+0000 I NETWORK [conn4086129] end connection 10.0.27.237:39928 (1506 connections now open) 2016-03-13T09:56:07.805+0000 I NETWORK [conn4086130] end connection 10.0.27.237:39927 (1506 connections now open) 2016-03-13T09:56:07.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39942 #4086144 (1506 connections now open) 2016-03-13T09:56:07.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39943 #4086145 (1507 connections now open) 2016-03-13T09:56:10.039+0000 I NETWORK [conn4086134] end connection 10.0.7.113:35120 (1506 connections now open) 2016-03-13T09:56:10.039+0000 I NETWORK [conn4086135] end connection 10.0.7.113:35119 (1506 connections now open) 2016-03-13T09:56:10.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35142 #4086146 (1506 connections now open) 2016-03-13T09:56:10.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35141 #4086147 (1507 connections now open) 2016-03-13T09:56:11.063+0000 I NETWORK [conn4086136] end connection 10.0.216.90:33141 (1506 connections now open) 2016-03-13T09:56:11.063+0000 I NETWORK [conn4086137] end connection 10.0.216.90:33140 (1506 connections now open) 2016-03-13T09:56:11.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33159 #4086148 (1506 connections now open) 2016-03-13T09:56:11.064+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33158 #4086149 (1507 connections now open) 2016-03-13T09:56:12.006+0000 I NETWORK [conn4086138] end connection 10.0.20.53:54867 (1506 connections now open) 2016-03-13T09:56:12.006+0000 I NETWORK [conn4086139] end connection 10.0.20.53:54868 (1505 connections now open) 2016-03-13T09:56:12.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54876 #4086150 (1506 connections now open) 2016-03-13T09:56:12.007+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54877 #4086151 (1507 connections now open) 2016-03-13T09:56:14.025+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:56:13.734+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:56:14.473+0000 I NETWORK [conn4086140] end connection 10.0.179.209:44577 (1506 connections now open) 2016-03-13T09:56:14.473+0000 I NETWORK [conn4086141] end connection 10.0.179.209:44578 (1506 connections now open) 2016-03-13T09:56:14.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44585 #4086152 (1506 connections now open) 2016-03-13T09:56:14.473+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44586 #4086153 (1507 connections now open) 2016-03-13T09:56:15.073+0000 I NETWORK [conn4086143] end connection 10.0.26.69:46492 (1506 connections now open) 2016-03-13T09:56:15.073+0000 I NETWORK [conn4086142] end connection 10.0.26.69:46493 (1506 connections now open) 2016-03-13T09:56:15.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46507 #4086154 (1506 connections now open) 2016-03-13T09:56:15.083+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46508 #4086155 (1507 connections now open) 2016-03-13T09:56:16.117+0000 I NETWORK [conn4081380] end connection 10.0.216.90:57616 (1506 connections now open) 2016-03-13T09:56:16.117+0000 I NETWORK [conn4081546] end connection 10.0.20.53:50950 (1506 connections now open) 2016-03-13T09:56:16.117+0000 I NETWORK [conn4082386] end connection 10.0.26.69:43089 (1504 connections now open) 2016-03-13T09:56:16.117+0000 I NETWORK [conn4081033] end connection 10.0.27.237:35804 (1503 connections now open) 2016-03-13T09:56:16.117+0000 I NETWORK [conn4080963] end connection 10.0.7.113:59531 (1503 connections now open) 2016-03-13T09:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33166 #4086156 (1503 connections now open) 2016-03-13T09:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54883 #4086157 (1504 connections now open) 2016-03-13T09:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39964 #4086158 (1505 connections now open) 2016-03-13T09:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46510 #4086159 (1506 connections now open) 2016-03-13T09:56:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35149 #4086160 (1507 connections now open) 2016-03-13T09:56:17.825+0000 I NETWORK [conn4086144] end connection 10.0.27.237:39942 (1506 connections now open) 2016-03-13T09:56:17.825+0000 I NETWORK [conn4086145] end connection 10.0.27.237:39943 (1505 connections now open) 2016-03-13T09:56:17.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39966 #4086161 (1506 connections now open) 2016-03-13T09:56:17.826+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39967 #4086162 (1507 connections now open) 2016-03-13T09:56:20.052+0000 I NETWORK [conn4086147] end connection 10.0.7.113:35141 (1506 connections now open) 2016-03-13T09:56:20.052+0000 I NETWORK [conn4086146] end connection 10.0.7.113:35142 (1506 connections now open) 2016-03-13T09:56:20.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35152 #4086163 (1506 connections now open) 2016-03-13T09:56:20.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35151 #4086164 (1507 connections now open) 2016-03-13T09:56:21.076+0000 I NETWORK [conn4086149] end connection 10.0.216.90:33158 (1506 connections now open) 2016-03-13T09:56:21.076+0000 I NETWORK [conn4086148] end connection 10.0.216.90:33159 (1506 connections now open) 2016-03-13T09:56:21.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33169 #4086165 (1506 connections now open) 2016-03-13T09:56:21.077+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33168 #4086166 (1507 connections now open) 2016-03-13T09:56:22.019+0000 I NETWORK [conn4086151] end connection 10.0.20.53:54877 (1506 connections now open) 2016-03-13T09:56:22.019+0000 I NETWORK [conn4086150] end connection 10.0.20.53:54876 (1505 connections now open) 2016-03-13T09:56:22.028+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54890 #4086167 (1506 connections now open) 2016-03-13T09:56:22.028+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54891 #4086168 (1507 connections now open) 2016-03-13T09:56:24.486+0000 I NETWORK [conn4086153] end connection 10.0.179.209:44586 (1506 connections now open) 2016-03-13T09:56:24.486+0000 I NETWORK [conn4086152] end connection 10.0.179.209:44585 (1506 connections now open) 2016-03-13T09:56:24.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44603 #4086169 (1506 connections now open) 2016-03-13T09:56:24.496+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44604 #4086170 (1507 connections now open) 2016-03-13T09:56:25.095+0000 I NETWORK [conn4086154] end connection 10.0.26.69:46507 (1506 connections now open) 2016-03-13T09:56:25.095+0000 I NETWORK [conn4086155] end connection 10.0.26.69:46508 (1506 connections now open) 2016-03-13T09:56:25.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46526 #4086171 (1507 connections now open) 2016-03-13T09:56:25.096+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46527 #4086172 (1507 connections now open) 2016-03-13T09:56:27.838+0000 I NETWORK [conn4086161] end connection 10.0.27.237:39966 (1506 connections now open) 2016-03-13T09:56:27.838+0000 I NETWORK [conn4086162] end connection 10.0.27.237:39967 (1505 connections now open) 2016-03-13T09:56:27.838+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39975 #4086173 (1506 connections now open) 2016-03-13T09:56:27.839+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39976 #4086174 (1507 connections now open) 2016-03-13T09:56:28.113+0000 I NETWORK [conn4082076] end connection 10.0.20.53:51211 (1506 connections now open) 2016-03-13T09:56:28.113+0000 I NETWORK [conn4084008] end connection 10.0.216.90:59208 (1506 connections now open) 2016-03-13T09:56:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33180 #4086175 (1506 connections now open) 2016-03-13T09:56:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54907 #4086176 (1507 connections now open) 2016-03-13T09:56:30.065+0000 I NETWORK [conn4086164] end connection 10.0.7.113:35151 (1506 connections now open) 2016-03-13T09:56:30.065+0000 I NETWORK [conn4086163] end connection 10.0.7.113:35152 (1506 connections now open) 2016-03-13T09:56:30.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35165 #4086177 (1506 connections now open) 2016-03-13T09:56:30.066+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35166 #4086178 (1507 connections now open) 2016-03-13T09:56:31.089+0000 I NETWORK [conn4086166] end connection 10.0.216.90:33168 (1506 connections now open) 2016-03-13T09:56:31.089+0000 I NETWORK [conn4086165] end connection 10.0.216.90:33169 (1506 connections now open) 2016-03-13T09:56:31.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33186 #4086179 (1506 connections now open) 2016-03-13T09:56:31.090+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33185 #4086180 (1507 connections now open) 2016-03-13T09:56:32.040+0000 I NETWORK [conn4086167] end connection 10.0.20.53:54890 (1506 connections now open) 2016-03-13T09:56:32.040+0000 I NETWORK [conn4086168] end connection 10.0.20.53:54891 (1506 connections now open) 2016-03-13T09:56:32.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54914 #4086181 (1506 connections now open) 2016-03-13T09:56:32.041+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54913 #4086182 (1507 connections now open) 2016-03-13T09:56:34.509+0000 I NETWORK [conn4086170] end connection 10.0.179.209:44604 (1506 connections now open) 2016-03-13T09:56:34.509+0000 I NETWORK [conn4086169] end connection 10.0.179.209:44603 (1506 connections now open) 2016-03-13T09:56:34.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44624 #4086183 (1506 connections now open) 2016-03-13T09:56:34.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44625 #4086184 (1507 connections now open) 2016-03-13T09:56:35.108+0000 I NETWORK [conn4086172] end connection 10.0.26.69:46527 (1506 connections now open) 2016-03-13T09:56:35.108+0000 I NETWORK [conn4086171] end connection 10.0.26.69:46526 (1506 connections now open) 2016-03-13T09:56:35.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46534 #4086185 (1506 connections now open) 2016-03-13T09:56:35.109+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46535 #4086186 (1507 connections now open) 2016-03-13T09:56:37.850+0000 I NETWORK [conn4086173] end connection 10.0.27.237:39975 (1506 connections now open) 2016-03-13T09:56:37.851+0000 I NETWORK [conn4086174] end connection 10.0.27.237:39976 (1505 connections now open) 2016-03-13T09:56:37.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39988 #4086187 (1506 connections now open) 2016-03-13T09:56:37.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:39989 #4086188 (1507 connections now open) 2016-03-13T09:56:40.079+0000 I NETWORK [conn4086178] end connection 10.0.7.113:35166 (1506 connections now open) 2016-03-13T09:56:40.079+0000 I NETWORK [conn4086177] end connection 10.0.7.113:35165 (1506 connections now open) 2016-03-13T09:56:40.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35183 #4086189 (1506 connections now open) 2016-03-13T09:56:40.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35184 #4086190 (1507 connections now open) 2016-03-13T09:56:41.102+0000 I NETWORK [conn4086179] end connection 10.0.216.90:33186 (1506 connections now open) 2016-03-13T09:56:41.102+0000 I NETWORK [conn4086180] end connection 10.0.216.90:33185 (1506 connections now open) 2016-03-13T09:56:41.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33204 #4086191 (1506 connections now open) 2016-03-13T09:56:41.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33203 #4086192 (1507 connections now open) 2016-03-13T09:56:42.053+0000 I NETWORK [conn4086182] end connection 10.0.20.53:54913 (1506 connections now open) 2016-03-13T09:56:42.053+0000 I NETWORK [conn4086181] end connection 10.0.20.53:54914 (1506 connections now open) 2016-03-13T09:56:42.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54921 #4086193 (1506 connections now open) 2016-03-13T09:56:42.054+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54922 #4086194 (1507 connections now open) 2016-03-13T09:56:44.301+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:56:44.025+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:56:44.523+0000 I NETWORK [conn4086184] end connection 10.0.179.209:44625 (1506 connections now open) 2016-03-13T09:56:44.523+0000 I NETWORK [conn4086183] end connection 10.0.179.209:44624 (1506 connections now open) 2016-03-13T09:56:44.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44633 #4086195 (1506 connections now open) 2016-03-13T09:56:44.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44634 #4086196 (1507 connections now open) 2016-03-13T09:56:45.121+0000 I NETWORK [conn4086186] end connection 10.0.26.69:46535 (1506 connections now open) 2016-03-13T09:56:45.121+0000 I NETWORK [conn4086185] end connection 10.0.26.69:46534 (1506 connections now open) 2016-03-13T09:56:45.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46546 #4086197 (1506 connections now open) 2016-03-13T09:56:45.122+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46547 #4086198 (1507 connections now open) 2016-03-13T09:56:46.117+0000 I NETWORK [conn4081691] end connection 10.0.20.53:51037 (1506 connections now open) 2016-03-13T09:56:46.117+0000 I NETWORK [conn4084656] end connection 10.0.216.90:59867 (1505 connections now open) 2016-03-13T09:56:46.117+0000 I NETWORK [conn4082102] end connection 10.0.26.69:42974 (1504 connections now open) 2016-03-13T09:56:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54927 #4086199 (1505 connections now open) 2016-03-13T09:56:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33211 #4086200 (1506 connections now open) 2016-03-13T09:56:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46550 #4086201 (1507 connections now open) 2016-03-13T09:56:47.863+0000 I NETWORK [conn4086187] end connection 10.0.27.237:39988 (1506 connections now open) 2016-03-13T09:56:47.863+0000 I NETWORK [conn4086188] end connection 10.0.27.237:39989 (1505 connections now open) 2016-03-13T09:56:47.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40009 #4086202 (1506 connections now open) 2016-03-13T09:56:47.864+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40010 #4086203 (1507 connections now open) 2016-03-13T09:56:50.093+0000 I NETWORK [conn4086189] end connection 10.0.7.113:35183 (1506 connections now open) 2016-03-13T09:56:50.093+0000 I NETWORK [conn4086190] end connection 10.0.7.113:35184 (1506 connections now open) 2016-03-13T09:56:50.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35191 #4086204 (1506 connections now open) 2016-03-13T09:56:50.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35192 #4086205 (1507 connections now open) 2016-03-13T09:56:51.123+0000 I NETWORK [conn4086192] end connection 10.0.216.90:33203 (1506 connections now open) 2016-03-13T09:56:51.123+0000 I NETWORK [conn4086191] end connection 10.0.216.90:33204 (1506 connections now open) 2016-03-13T09:56:51.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33212 #4086206 (1506 connections now open) 2016-03-13T09:56:51.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33213 #4086207 (1507 connections now open) 2016-03-13T09:56:52.066+0000 I NETWORK [conn4086194] end connection 10.0.20.53:54922 (1506 connections now open) 2016-03-13T09:56:52.066+0000 I NETWORK [conn4086193] end connection 10.0.20.53:54921 (1506 connections now open) 2016-03-13T09:56:52.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54935 #4086208 (1506 connections now open) 2016-03-13T09:56:52.067+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54934 #4086209 (1507 connections now open) 2016-03-13T09:56:54.537+0000 I NETWORK [conn4086196] end connection 10.0.179.209:44634 (1506 connections now open) 2016-03-13T09:56:54.537+0000 I NETWORK [conn4086195] end connection 10.0.179.209:44633 (1506 connections now open) 2016-03-13T09:56:54.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44646 #4086210 (1506 connections now open) 2016-03-13T09:56:54.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44647 #4086211 (1507 connections now open) 2016-03-13T09:56:55.134+0000 I NETWORK [conn4086197] end connection 10.0.26.69:46546 (1506 connections now open) 2016-03-13T09:56:55.134+0000 I NETWORK [conn4086198] end connection 10.0.26.69:46547 (1506 connections now open) 2016-03-13T09:56:55.135+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46566 #4086212 (1507 connections now open) 2016-03-13T09:56:55.135+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46567 #4086213 (1507 connections now open) 2016-03-13T09:56:57.876+0000 I NETWORK [conn4086203] end connection 10.0.27.237:40010 (1506 connections now open) 2016-03-13T09:56:57.876+0000 I NETWORK [conn4086202] end connection 10.0.27.237:40009 (1506 connections now open) 2016-03-13T09:56:57.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40017 #4086214 (1506 connections now open) 2016-03-13T09:56:57.877+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40016 #4086215 (1507 connections now open) 2016-03-13T09:56:58.113+0000 I NETWORK [conn4081145] end connection 10.0.179.209:40690 (1506 connections now open) 2016-03-13T09:56:58.113+0000 I NETWORK [conn4082259] end connection 10.0.26.69:43037 (1506 connections now open) 2016-03-13T09:56:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44650 #4086216 (1506 connections now open) 2016-03-13T09:56:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46570 #4086217 (1507 connections now open) 2016-03-13T09:57:00.106+0000 I NETWORK [conn4086205] end connection 10.0.7.113:35192 (1506 connections now open) 2016-03-13T09:57:00.106+0000 I NETWORK [conn4086204] end connection 10.0.7.113:35191 (1505 connections now open) 2016-03-13T09:57:00.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35204 #4086218 (1506 connections now open) 2016-03-13T09:57:00.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35205 #4086219 (1507 connections now open) 2016-03-13T09:57:01.136+0000 I NETWORK [conn4086206] end connection 10.0.216.90:33212 (1506 connections now open) 2016-03-13T09:57:01.136+0000 I NETWORK [conn4086207] end connection 10.0.216.90:33213 (1506 connections now open) 2016-03-13T09:57:01.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33228 #4086220 (1506 connections now open) 2016-03-13T09:57:01.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33227 #4086221 (1507 connections now open) 2016-03-13T09:57:02.080+0000 I NETWORK [conn4086208] end connection 10.0.20.53:54935 (1506 connections now open) 2016-03-13T09:57:02.080+0000 I NETWORK [conn4086209] end connection 10.0.20.53:54934 (1506 connections now open) 2016-03-13T09:57:02.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54956 #4086222 (1506 connections now open) 2016-03-13T09:57:02.080+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54957 #4086223 (1507 connections now open) 2016-03-13T09:57:04.551+0000 I NETWORK [conn4086211] end connection 10.0.179.209:44647 (1506 connections now open) 2016-03-13T09:57:04.551+0000 I NETWORK [conn4086210] end connection 10.0.179.209:44646 (1506 connections now open) 2016-03-13T09:57:04.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44668 #4086224 (1506 connections now open) 2016-03-13T09:57:04.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44669 #4086225 (1507 connections now open) 2016-03-13T09:57:05.148+0000 I NETWORK [conn4086212] end connection 10.0.26.69:46566 (1506 connections now open) 2016-03-13T09:57:05.148+0000 I NETWORK [conn4086213] end connection 10.0.26.69:46567 (1506 connections now open) 2016-03-13T09:57:05.156+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46577 #4086226 (1506 connections now open) 2016-03-13T09:57:05.156+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46578 #4086227 (1507 connections now open) 2016-03-13T09:57:07.889+0000 I NETWORK [conn4086214] end connection 10.0.27.237:40017 (1506 connections now open) 2016-03-13T09:57:07.889+0000 I NETWORK [conn4086215] end connection 10.0.27.237:40016 (1506 connections now open) 2016-03-13T09:57:07.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40032 #4086228 (1506 connections now open) 2016-03-13T09:57:07.898+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40031 #4086229 (1507 connections now open) 2016-03-13T09:57:10.128+0000 I NETWORK [conn4086219] end connection 10.0.7.113:35205 (1506 connections now open) 2016-03-13T09:57:10.128+0000 I NETWORK [conn4086218] end connection 10.0.7.113:35204 (1506 connections now open) 2016-03-13T09:57:10.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35225 #4086230 (1506 connections now open) 2016-03-13T09:57:10.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35226 #4086231 (1507 connections now open) 2016-03-13T09:57:11.149+0000 I NETWORK [conn4086220] end connection 10.0.216.90:33228 (1506 connections now open) 2016-03-13T09:57:11.149+0000 I NETWORK [conn4086221] end connection 10.0.216.90:33227 (1506 connections now open) 2016-03-13T09:57:11.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33247 #4086232 (1506 connections now open) 2016-03-13T09:57:11.166+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33246 #4086233 (1507 connections now open) 2016-03-13T09:57:12.093+0000 I NETWORK [conn4086222] end connection 10.0.20.53:54956 (1506 connections now open) 2016-03-13T09:57:12.093+0000 I NETWORK [conn4086223] end connection 10.0.20.53:54957 (1506 connections now open) 2016-03-13T09:57:12.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54964 #4086234 (1507 connections now open) 2016-03-13T09:57:12.094+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54965 #4086235 (1507 connections now open) 2016-03-13T09:57:14.565+0000 I NETWORK [conn4086224] end connection 10.0.179.209:44668 (1506 connections now open) 2016-03-13T09:57:14.565+0000 I NETWORK [conn4086225] end connection 10.0.179.209:44669 (1506 connections now open) 2016-03-13T09:57:14.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44677 #4086236 (1506 connections now open) 2016-03-13T09:57:14.566+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44678 #4086237 (1507 connections now open) 2016-03-13T09:57:14.583+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:57:14.301+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:57:15.169+0000 I NETWORK [conn4086226] end connection 10.0.26.69:46577 (1506 connections now open) 2016-03-13T09:57:15.169+0000 I NETWORK [conn4086227] end connection 10.0.26.69:46578 (1506 connections now open) 2016-03-13T09:57:15.169+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46593 #4086238 (1506 connections now open) 2016-03-13T09:57:15.169+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46594 #4086239 (1507 connections now open) 2016-03-13T09:57:16.117+0000 I NETWORK [conn4082624] end connection 10.0.20.53:51420 (1506 connections now open) 2016-03-13T09:57:16.117+0000 I NETWORK [conn4081968] end connection 10.0.179.209:40887 (1505 connections now open) 2016-03-13T09:57:16.117+0000 I NETWORK [conn4082273] end connection 10.0.216.90:57940 (1505 connections now open) 2016-03-13T09:57:16.117+0000 I NETWORK [conn4081078] end connection 10.0.27.237:35824 (1503 connections now open) 2016-03-13T09:57:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54970 #4086240 (1504 connections now open) 2016-03-13T09:57:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33252 #4086241 (1505 connections now open) 2016-03-13T09:57:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44682 #4086242 (1506 connections now open) 2016-03-13T09:57:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40054 #4086243 (1507 connections now open) 2016-03-13T09:57:16.118+0000 I NETWORK [conn4081335] end connection 10.0.7.113:59650 (1505 connections now open) 2016-03-13T09:57:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35232 #4086244 (1507 connections now open) 2016-03-13T09:57:17.911+0000 I NETWORK [conn4086229] end connection 10.0.27.237:40031 (1506 connections now open) 2016-03-13T09:57:17.911+0000 I NETWORK [conn4086228] end connection 10.0.27.237:40032 (1506 connections now open) 2016-03-13T09:57:17.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40057 #4086245 (1506 connections now open) 2016-03-13T09:57:17.911+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40056 #4086246 (1507 connections now open) 2016-03-13T09:57:20.141+0000 I NETWORK [conn4086231] end connection 10.0.7.113:35226 (1506 connections now open) 2016-03-13T09:57:20.141+0000 I NETWORK [conn4086230] end connection 10.0.7.113:35225 (1506 connections now open) 2016-03-13T09:57:20.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35234 #4086247 (1506 connections now open) 2016-03-13T09:57:20.142+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35235 #4086248 (1507 connections now open) 2016-03-13T09:57:21.179+0000 I NETWORK [conn4086232] end connection 10.0.216.90:33247 (1506 connections now open) 2016-03-13T09:57:21.179+0000 I NETWORK [conn4086233] end connection 10.0.216.90:33246 (1506 connections now open) 2016-03-13T09:57:21.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33253 #4086249 (1506 connections now open) 2016-03-13T09:57:21.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33254 #4086250 (1507 connections now open) 2016-03-13T09:57:22.107+0000 I NETWORK [conn4086234] end connection 10.0.20.53:54964 (1506 connections now open) 2016-03-13T09:57:22.107+0000 I NETWORK [conn4086235] end connection 10.0.20.53:54965 (1506 connections now open) 2016-03-13T09:57:22.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54977 #4086251 (1506 connections now open) 2016-03-13T09:57:22.108+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:54978 #4086252 (1507 connections now open) 2016-03-13T09:57:24.579+0000 I NETWORK [conn4086237] end connection 10.0.179.209:44678 (1506 connections now open) 2016-03-13T09:57:24.579+0000 I NETWORK [conn4086236] end connection 10.0.179.209:44677 (1506 connections now open) 2016-03-13T09:57:24.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44695 #4086253 (1506 connections now open) 2016-03-13T09:57:24.588+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44696 #4086254 (1507 connections now open) 2016-03-13T09:57:25.182+0000 I NETWORK [conn4086239] end connection 10.0.26.69:46594 (1506 connections now open) 2016-03-13T09:57:25.182+0000 I NETWORK [conn4086238] end connection 10.0.26.69:46593 (1506 connections now open) 2016-03-13T09:57:25.182+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46613 #4086255 (1506 connections now open) 2016-03-13T09:57:25.182+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46614 #4086256 (1507 connections now open) 2016-03-13T09:57:27.924+0000 I NETWORK [conn4086246] end connection 10.0.27.237:40056 (1506 connections now open) 2016-03-13T09:57:27.924+0000 I NETWORK [conn4086245] end connection 10.0.27.237:40057 (1506 connections now open) 2016-03-13T09:57:27.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40065 #4086257 (1506 connections now open) 2016-03-13T09:57:27.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40066 #4086258 (1507 connections now open) 2016-03-13T09:57:28.114+0000 I NETWORK [conn4082880] end connection 10.0.216.90:58210 (1506 connections now open) 2016-03-13T09:57:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33266 #4086259 (1507 connections now open) 2016-03-13T09:57:30.155+0000 I NETWORK [conn4086248] end connection 10.0.7.113:35235 (1506 connections now open) 2016-03-13T09:57:30.155+0000 I NETWORK [conn4086247] end connection 10.0.7.113:35234 (1506 connections now open) 2016-03-13T09:57:30.156+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35250 #4086260 (1506 connections now open) 2016-03-13T09:57:30.156+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35251 #4086261 (1507 connections now open) 2016-03-13T09:57:31.192+0000 I NETWORK [conn4086249] end connection 10.0.216.90:33253 (1506 connections now open) 2016-03-13T09:57:31.192+0000 I NETWORK [conn4086250] end connection 10.0.216.90:33254 (1506 connections now open) 2016-03-13T09:57:31.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33271 #4086262 (1506 connections now open) 2016-03-13T09:57:31.192+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33272 #4086263 (1507 connections now open) 2016-03-13T09:57:32.121+0000 I NETWORK [conn4086251] end connection 10.0.20.53:54977 (1506 connections now open) 2016-03-13T09:57:32.121+0000 I NETWORK [conn4086252] end connection 10.0.20.53:54978 (1506 connections now open) 2016-03-13T09:57:32.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55002 #4086264 (1506 connections now open) 2016-03-13T09:57:32.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55001 #4086265 (1507 connections now open) 2016-03-13T09:57:34.601+0000 I NETWORK [conn4086254] end connection 10.0.179.209:44696 (1506 connections now open) 2016-03-13T09:57:34.601+0000 I NETWORK [conn4086253] end connection 10.0.179.209:44695 (1506 connections now open) 2016-03-13T09:57:34.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44716 #4086266 (1506 connections now open) 2016-03-13T09:57:34.601+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44715 #4086267 (1507 connections now open) 2016-03-13T09:57:35.195+0000 I NETWORK [conn4086256] end connection 10.0.26.69:46614 (1506 connections now open) 2016-03-13T09:57:35.195+0000 I NETWORK [conn4086255] end connection 10.0.26.69:46613 (1506 connections now open) 2016-03-13T09:57:35.195+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46622 #4086268 (1507 connections now open) 2016-03-13T09:57:35.195+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46621 #4086269 (1507 connections now open) 2016-03-13T09:57:37.937+0000 I NETWORK [conn4086258] end connection 10.0.27.237:40066 (1506 connections now open) 2016-03-13T09:57:37.937+0000 I NETWORK [conn4086257] end connection 10.0.27.237:40065 (1506 connections now open) 2016-03-13T09:57:37.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40077 #4086270 (1506 connections now open) 2016-03-13T09:57:37.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40078 #4086271 (1507 connections now open) 2016-03-13T09:57:40.168+0000 I NETWORK [conn4086260] end connection 10.0.7.113:35250 (1506 connections now open) 2016-03-13T09:57:40.168+0000 I NETWORK [conn4086261] end connection 10.0.7.113:35251 (1506 connections now open) 2016-03-13T09:57:40.169+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35268 #4086272 (1506 connections now open) 2016-03-13T09:57:40.169+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35269 #4086273 (1507 connections now open) 2016-03-13T09:57:41.205+0000 I NETWORK [conn4086263] end connection 10.0.216.90:33272 (1506 connections now open) 2016-03-13T09:57:41.205+0000 I NETWORK [conn4086262] end connection 10.0.216.90:33271 (1506 connections now open) 2016-03-13T09:57:41.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33289 #4086274 (1506 connections now open) 2016-03-13T09:57:41.214+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33290 #4086275 (1507 connections now open) 2016-03-13T09:57:42.133+0000 I NETWORK [conn4086264] end connection 10.0.20.53:55002 (1506 connections now open) 2016-03-13T09:57:42.134+0000 I NETWORK [conn4086265] end connection 10.0.20.53:55001 (1506 connections now open) 2016-03-13T09:57:42.143+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55010 #4086276 (1506 connections now open) 2016-03-13T09:57:42.143+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55009 #4086277 (1507 connections now open) 2016-03-13T09:57:44.614+0000 I NETWORK [conn4086267] end connection 10.0.179.209:44715 (1506 connections now open) 2016-03-13T09:57:44.614+0000 I NETWORK [conn4086266] end connection 10.0.179.209:44716 (1506 connections now open) 2016-03-13T09:57:44.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44723 #4086278 (1506 connections now open) 2016-03-13T09:57:44.615+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44722 #4086279 (1507 connections now open) 2016-03-13T09:57:44.857+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:57:44.583+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:57:45.208+0000 I NETWORK [conn4086269] end connection 10.0.26.69:46621 (1506 connections now open) 2016-03-13T09:57:45.208+0000 I NETWORK [conn4086268] end connection 10.0.26.69:46622 (1506 connections now open) 2016-03-13T09:57:45.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46634 #4086280 (1506 connections now open) 2016-03-13T09:57:45.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46635 #4086281 (1507 connections now open) 2016-03-13T09:57:46.117+0000 I NETWORK [conn4085237] end connection 10.0.20.53:53924 (1506 connections now open) 2016-03-13T09:57:46.117+0000 I NETWORK [conn4081983] end connection 10.0.26.69:42920 (1505 connections now open) 2016-03-13T09:57:46.117+0000 I NETWORK [conn4081488] end connection 10.0.27.237:35982 (1504 connections now open) 2016-03-13T09:57:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55016 #4086282 (1505 connections now open) 2016-03-13T09:57:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46639 #4086283 (1506 connections now open) 2016-03-13T09:57:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40097 #4086284 (1507 connections now open) 2016-03-13T09:57:47.950+0000 I NETWORK [conn4086271] end connection 10.0.27.237:40078 (1506 connections now open) 2016-03-13T09:57:47.950+0000 I NETWORK [conn4086270] end connection 10.0.27.237:40077 (1506 connections now open) 2016-03-13T09:57:47.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40099 #4086285 (1506 connections now open) 2016-03-13T09:57:47.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40100 #4086286 (1507 connections now open) 2016-03-13T09:57:50.181+0000 I NETWORK [conn4086273] end connection 10.0.7.113:35269 (1506 connections now open) 2016-03-13T09:57:50.181+0000 I NETWORK [conn4086272] end connection 10.0.7.113:35268 (1506 connections now open) 2016-03-13T09:57:50.182+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35279 #4086287 (1506 connections now open) 2016-03-13T09:57:50.182+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35280 #4086288 (1507 connections now open) 2016-03-13T09:57:51.226+0000 I NETWORK [conn4086274] end connection 10.0.216.90:33289 (1506 connections now open) 2016-03-13T09:57:51.226+0000 I NETWORK [conn4086275] end connection 10.0.216.90:33290 (1506 connections now open) 2016-03-13T09:57:51.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33298 #4086289 (1506 connections now open) 2016-03-13T09:57:51.227+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33297 #4086290 (1507 connections now open) 2016-03-13T09:57:52.156+0000 I NETWORK [conn4086276] end connection 10.0.20.53:55010 (1506 connections now open) 2016-03-13T09:57:52.156+0000 I NETWORK [conn4086277] end connection 10.0.20.53:55009 (1506 connections now open) 2016-03-13T09:57:52.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55024 #4086291 (1506 connections now open) 2016-03-13T09:57:52.157+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55023 #4086292 (1507 connections now open) 2016-03-13T09:57:54.628+0000 I NETWORK [conn4086279] end connection 10.0.179.209:44722 (1506 connections now open) 2016-03-13T09:57:54.628+0000 I NETWORK [conn4086278] end connection 10.0.179.209:44723 (1506 connections now open) 2016-03-13T09:57:54.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44737 #4086293 (1506 connections now open) 2016-03-13T09:57:54.628+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44738 #4086294 (1507 connections now open) 2016-03-13T09:57:55.221+0000 I NETWORK [conn4086281] end connection 10.0.26.69:46635 (1506 connections now open) 2016-03-13T09:57:55.221+0000 I NETWORK [conn4086280] end connection 10.0.26.69:46634 (1506 connections now open) 2016-03-13T09:57:55.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46655 #4086295 (1506 connections now open) 2016-03-13T09:57:55.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46654 #4086296 (1507 connections now open) 2016-03-13T09:57:57.964+0000 I NETWORK [conn4086286] end connection 10.0.27.237:40100 (1506 connections now open) 2016-03-13T09:57:57.964+0000 I NETWORK [conn4086285] end connection 10.0.27.237:40099 (1506 connections now open) 2016-03-13T09:57:57.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40107 #4086297 (1506 connections now open) 2016-03-13T09:57:57.964+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40108 #4086298 (1507 connections now open) 2016-03-13T09:57:58.114+0000 I NETWORK [conn4081880] end connection 10.0.179.209:40856 (1506 connections now open) 2016-03-13T09:57:58.114+0000 I NETWORK [conn4081690] end connection 10.0.26.69:42780 (1505 connections now open) 2016-03-13T09:57:58.114+0000 I NETWORK [conn4081591] end connection 10.0.7.113:59702 (1504 connections now open) 2016-03-13T09:57:58.114+0000 I NETWORK [conn4082601] end connection 10.0.20.53:51411 (1503 connections now open) 2016-03-13T09:57:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44741 #4086299 (1504 connections now open) 2016-03-13T09:57:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46659 #4086300 (1505 connections now open) 2016-03-13T09:57:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35292 #4086301 (1506 connections now open) 2016-03-13T09:57:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55034 #4086302 (1507 connections now open) 2016-03-13T09:58:00.194+0000 I NETWORK [conn4086287] end connection 10.0.7.113:35279 (1506 connections now open) 2016-03-13T09:58:00.194+0000 I NETWORK [conn4086288] end connection 10.0.7.113:35280 (1505 connections now open) 2016-03-13T09:58:00.195+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35295 #4086303 (1506 connections now open) 2016-03-13T09:58:00.195+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35294 #4086304 (1507 connections now open) 2016-03-13T09:58:01.240+0000 I NETWORK [conn4086289] end connection 10.0.216.90:33298 (1506 connections now open) 2016-03-13T09:58:01.240+0000 I NETWORK [conn4086290] end connection 10.0.216.90:33297 (1506 connections now open) 2016-03-13T09:58:01.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33313 #4086305 (1506 connections now open) 2016-03-13T09:58:01.240+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33312 #4086306 (1507 connections now open) 2016-03-13T09:58:02.169+0000 I NETWORK [conn4086291] end connection 10.0.20.53:55024 (1506 connections now open) 2016-03-13T09:58:02.169+0000 I NETWORK [conn4086292] end connection 10.0.20.53:55023 (1506 connections now open) 2016-03-13T09:58:02.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55044 #4086307 (1506 connections now open) 2016-03-13T09:58:02.170+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55045 #4086308 (1507 connections now open) 2016-03-13T09:58:04.641+0000 I NETWORK [conn4086293] end connection 10.0.179.209:44737 (1506 connections now open) 2016-03-13T09:58:04.641+0000 I NETWORK [conn4086294] end connection 10.0.179.209:44738 (1506 connections now open) 2016-03-13T09:58:04.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44758 #4086309 (1506 connections now open) 2016-03-13T09:58:04.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44759 #4086310 (1507 connections now open) 2016-03-13T09:58:05.233+0000 I NETWORK [conn4086296] end connection 10.0.26.69:46654 (1506 connections now open) 2016-03-13T09:58:05.233+0000 I NETWORK [conn4086295] end connection 10.0.26.69:46655 (1506 connections now open) 2016-03-13T09:58:05.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46665 #4086311 (1506 connections now open) 2016-03-13T09:58:05.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46666 #4086312 (1507 connections now open) 2016-03-13T09:58:07.977+0000 I NETWORK [conn4086297] end connection 10.0.27.237:40107 (1506 connections now open) 2016-03-13T09:58:07.977+0000 I NETWORK [conn4086298] end connection 10.0.27.237:40108 (1506 connections now open) 2016-03-13T09:58:07.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40121 #4086313 (1506 connections now open) 2016-03-13T09:58:07.987+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40122 #4086314 (1507 connections now open) 2016-03-13T09:58:10.208+0000 I NETWORK [conn4086303] end connection 10.0.7.113:35295 (1506 connections now open) 2016-03-13T09:58:10.208+0000 I NETWORK [conn4086304] end connection 10.0.7.113:35294 (1506 connections now open) 2016-03-13T09:58:10.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35316 #4086315 (1506 connections now open) 2016-03-13T09:58:10.209+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35315 #4086316 (1507 connections now open) 2016-03-13T09:58:11.253+0000 I NETWORK [conn4086306] end connection 10.0.216.90:33312 (1506 connections now open) 2016-03-13T09:58:11.253+0000 I NETWORK [conn4086305] end connection 10.0.216.90:33313 (1506 connections now open) 2016-03-13T09:58:11.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33332 #4086317 (1506 connections now open) 2016-03-13T09:58:11.254+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33333 #4086318 (1507 connections now open) 2016-03-13T09:58:12.183+0000 I NETWORK [conn4086308] end connection 10.0.20.53:55045 (1506 connections now open) 2016-03-13T09:58:12.183+0000 I NETWORK [conn4086307] end connection 10.0.20.53:55044 (1506 connections now open) 2016-03-13T09:58:12.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55052 #4086319 (1506 connections now open) 2016-03-13T09:58:12.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55051 #4086320 (1507 connections now open) 2016-03-13T09:58:14.654+0000 I NETWORK [conn4086310] end connection 10.0.179.209:44759 (1506 connections now open) 2016-03-13T09:58:14.654+0000 I NETWORK [conn4086309] end connection 10.0.179.209:44758 (1506 connections now open) 2016-03-13T09:58:14.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44768 #4086321 (1506 connections now open) 2016-03-13T09:58:14.655+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44767 #4086322 (1507 connections now open) 2016-03-13T09:58:15.017+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:58:14.857+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:58:15.247+0000 I NETWORK [conn4086312] end connection 10.0.26.69:46666 (1506 connections now open) 2016-03-13T09:58:15.247+0000 I NETWORK [conn4086311] end connection 10.0.26.69:46665 (1506 connections now open) 2016-03-13T09:58:15.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46681 #4086323 (1506 connections now open) 2016-03-13T09:58:15.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46680 #4086324 (1507 connections now open) 2016-03-13T09:58:16.117+0000 I NETWORK [conn4081693] end connection 10.0.20.53:51039 (1506 connections now open) 2016-03-13T09:58:16.117+0000 I NETWORK [conn4081891] end connection 10.0.179.209:40858 (1506 connections now open) 2016-03-13T09:58:16.117+0000 I NETWORK [conn4078206] end connection 10.0.216.90:54859 (1504 connections now open) 2016-03-13T09:58:16.117+0000 I NETWORK [conn4081227] end connection 10.0.27.237:35879 (1503 connections now open) 2016-03-13T09:58:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55058 #4086325 (1504 connections now open) 2016-03-13T09:58:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44771 #4086326 (1505 connections now open) 2016-03-13T09:58:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33339 #4086327 (1506 connections now open) 2016-03-13T09:58:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40144 #4086328 (1507 connections now open) 2016-03-13T09:58:17.999+0000 I NETWORK [conn4086314] end connection 10.0.27.237:40122 (1506 connections now open) 2016-03-13T09:58:17.999+0000 I NETWORK [conn4086313] end connection 10.0.27.237:40121 (1506 connections now open) 2016-03-13T09:58:18.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40147 #4086329 (1506 connections now open) 2016-03-13T09:58:18.000+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40146 #4086330 (1507 connections now open) 2016-03-13T09:58:20.221+0000 I NETWORK [conn4086315] end connection 10.0.7.113:35316 (1506 connections now open) 2016-03-13T09:58:20.221+0000 I NETWORK [conn4086316] end connection 10.0.7.113:35315 (1506 connections now open) 2016-03-13T09:58:20.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35326 #4086331 (1506 connections now open) 2016-03-13T09:58:20.222+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35325 #4086332 (1507 connections now open) 2016-03-13T09:58:21.266+0000 I NETWORK [conn4086317] end connection 10.0.216.90:33332 (1506 connections now open) 2016-03-13T09:58:21.266+0000 I NETWORK [conn4086318] end connection 10.0.216.90:33333 (1506 connections now open) 2016-03-13T09:58:21.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33341 #4086333 (1506 connections now open) 2016-03-13T09:58:21.267+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33340 #4086334 (1507 connections now open) 2016-03-13T09:58:22.196+0000 I NETWORK [conn4086320] end connection 10.0.20.53:55051 (1506 connections now open) 2016-03-13T09:58:22.196+0000 I NETWORK [conn4086319] end connection 10.0.20.53:55052 (1506 connections now open) 2016-03-13T09:58:22.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55065 #4086335 (1506 connections now open) 2016-03-13T09:58:22.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55066 #4086336 (1507 connections now open) 2016-03-13T09:58:24.667+0000 I NETWORK [conn4086321] end connection 10.0.179.209:44768 (1506 connections now open) 2016-03-13T09:58:24.667+0000 I NETWORK [conn4086322] end connection 10.0.179.209:44767 (1506 connections now open) 2016-03-13T09:58:24.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44781 #4086337 (1506 connections now open) 2016-03-13T09:58:24.676+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44782 #4086338 (1507 connections now open) 2016-03-13T09:58:25.261+0000 I NETWORK [conn4086323] end connection 10.0.26.69:46681 (1506 connections now open) 2016-03-13T09:58:25.261+0000 I NETWORK [conn4086324] end connection 10.0.26.69:46680 (1506 connections now open) 2016-03-13T09:58:25.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46699 #4086339 (1506 connections now open) 2016-03-13T09:58:25.261+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46700 #4086340 (1507 connections now open) 2016-03-13T09:58:28.011+0000 I NETWORK [conn4086330] end connection 10.0.27.237:40146 (1506 connections now open) 2016-03-13T09:58:28.012+0000 I NETWORK [conn4086329] end connection 10.0.27.237:40147 (1505 connections now open) 2016-03-13T09:58:28.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40155 #4086341 (1506 connections now open) 2016-03-13T09:58:28.012+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40156 #4086342 (1507 connections now open) 2016-03-13T09:58:28.114+0000 I NETWORK [conn4085514] end connection 10.0.20.53:54211 (1506 connections now open) 2016-03-13T09:58:28.115+0000 I NETWORK [conn4082240] end connection 10.0.26.69:43032 (1505 connections now open) 2016-03-13T09:58:28.115+0000 I NETWORK [conn4082351] end connection 10.0.7.113:59952 (1504 connections now open) 2016-03-13T09:58:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55072 #4086343 (1505 connections now open) 2016-03-13T09:58:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46703 #4086344 (1506 connections now open) 2016-03-13T09:58:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35337 #4086345 (1507 connections now open) 2016-03-13T09:58:30.234+0000 I NETWORK [conn4086331] end connection 10.0.7.113:35326 (1506 connections now open) 2016-03-13T09:58:30.234+0000 I NETWORK [conn4086332] end connection 10.0.7.113:35325 (1506 connections now open) 2016-03-13T09:58:30.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35339 #4086346 (1507 connections now open) 2016-03-13T09:58:30.235+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35338 #4086347 (1507 connections now open) 2016-03-13T09:58:31.279+0000 I NETWORK [conn4086334] end connection 10.0.216.90:33340 (1506 connections now open) 2016-03-13T09:58:31.279+0000 I NETWORK [conn4086333] end connection 10.0.216.90:33341 (1506 connections now open) 2016-03-13T09:58:31.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33358 #4086348 (1506 connections now open) 2016-03-13T09:58:31.279+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33359 #4086349 (1507 connections now open) 2016-03-13T09:58:32.210+0000 I NETWORK [conn4086335] end connection 10.0.20.53:55065 (1506 connections now open) 2016-03-13T09:58:32.210+0000 I NETWORK [conn4086336] end connection 10.0.20.53:55066 (1506 connections now open) 2016-03-13T09:58:32.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55088 #4086350 (1506 connections now open) 2016-03-13T09:58:32.210+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55087 #4086351 (1507 connections now open) 2016-03-13T09:58:34.689+0000 I NETWORK [conn4086338] end connection 10.0.179.209:44782 (1506 connections now open) 2016-03-13T09:58:34.689+0000 I NETWORK [conn4086337] end connection 10.0.179.209:44781 (1506 connections now open) 2016-03-13T09:58:34.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44795 #4086352 (1506 connections now open) 2016-03-13T09:58:34.690+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44796 #4086353 (1507 connections now open) 2016-03-13T09:58:35.274+0000 I NETWORK [conn4086340] end connection 10.0.26.69:46700 (1506 connections now open) 2016-03-13T09:58:35.274+0000 I NETWORK [conn4086339] end connection 10.0.26.69:46699 (1506 connections now open) 2016-03-13T09:58:35.274+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46708 #4086354 (1507 connections now open) 2016-03-13T09:58:35.275+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46709 #4086355 (1507 connections now open) 2016-03-13T09:58:38.024+0000 I NETWORK [conn4086341] end connection 10.0.27.237:40155 (1506 connections now open) 2016-03-13T09:58:38.024+0000 I NETWORK [conn4086342] end connection 10.0.27.237:40156 (1505 connections now open) 2016-03-13T09:58:38.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40168 #4086356 (1506 connections now open) 2016-03-13T09:58:38.025+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40167 #4086357 (1507 connections now open) 2016-03-13T09:58:40.247+0000 I NETWORK [conn4086347] end connection 10.0.7.113:35338 (1506 connections now open) 2016-03-13T09:58:40.247+0000 I NETWORK [conn4086346] end connection 10.0.7.113:35339 (1506 connections now open) 2016-03-13T09:58:40.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35357 #4086358 (1506 connections now open) 2016-03-13T09:58:40.248+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35356 #4086359 (1507 connections now open) 2016-03-13T09:58:41.292+0000 I NETWORK [conn4086348] end connection 10.0.216.90:33358 (1506 connections now open) 2016-03-13T09:58:41.292+0000 I NETWORK [conn4086349] end connection 10.0.216.90:33359 (1506 connections now open) 2016-03-13T09:58:41.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33375 #4086360 (1506 connections now open) 2016-03-13T09:58:41.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33376 #4086361 (1507 connections now open) 2016-03-13T09:58:42.223+0000 I NETWORK [conn4086350] end connection 10.0.20.53:55088 (1506 connections now open) 2016-03-13T09:58:42.223+0000 I NETWORK [conn4086351] end connection 10.0.20.53:55087 (1506 connections now open) 2016-03-13T09:58:42.232+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55096 #4086362 (1506 connections now open) 2016-03-13T09:58:42.232+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55097 #4086363 (1507 connections now open) 2016-03-13T09:58:44.703+0000 I NETWORK [conn4086353] end connection 10.0.179.209:44796 (1506 connections now open) 2016-03-13T09:58:44.703+0000 I NETWORK [conn4086352] end connection 10.0.179.209:44795 (1506 connections now open) 2016-03-13T09:58:44.704+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44813 #4086364 (1506 connections now open) 2016-03-13T09:58:44.704+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44814 #4086365 (1507 connections now open) 2016-03-13T09:58:45.287+0000 I NETWORK [conn4086354] end connection 10.0.26.69:46708 (1506 connections now open) 2016-03-13T09:58:45.287+0000 I NETWORK [conn4086355] end connection 10.0.26.69:46709 (1506 connections now open) 2016-03-13T09:58:45.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46720 #4086366 (1506 connections now open) 2016-03-13T09:58:45.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46721 #4086367 (1507 connections now open) 2016-03-13T09:58:45.302+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:58:45.017+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:58:46.117+0000 I NETWORK [conn4077702] end connection 10.0.216.90:54327 (1506 connections now open) 2016-03-13T09:58:46.117+0000 I NETWORK [conn4085455] end connection 10.0.26.69:45788 (1506 connections now open) 2016-03-13T09:58:46.117+0000 I NETWORK [conn4082011] end connection 10.0.27.237:36240 (1504 connections now open) 2016-03-13T09:58:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33383 #4086368 (1506 connections now open) 2016-03-13T09:58:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46724 #4086369 (1506 connections now open) 2016-03-13T09:58:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40185 #4086370 (1507 connections now open) 2016-03-13T09:58:48.037+0000 I NETWORK [conn4086356] end connection 10.0.27.237:40168 (1506 connections now open) 2016-03-13T09:58:48.038+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40187 #4086371 (1507 connections now open) 2016-03-13T09:58:48.039+0000 I NETWORK [conn4086357] end connection 10.0.27.237:40167 (1506 connections now open) 2016-03-13T09:58:48.039+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40188 #4086372 (1507 connections now open) 2016-03-13T09:58:50.260+0000 I NETWORK [conn4086359] end connection 10.0.7.113:35356 (1506 connections now open) 2016-03-13T09:58:50.260+0000 I NETWORK [conn4086358] end connection 10.0.7.113:35357 (1506 connections now open) 2016-03-13T09:58:50.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35367 #4086373 (1506 connections now open) 2016-03-13T09:58:50.269+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35368 #4086374 (1507 connections now open) 2016-03-13T09:58:51.314+0000 I NETWORK [conn4086361] end connection 10.0.216.90:33376 (1506 connections now open) 2016-03-13T09:58:51.314+0000 I NETWORK [conn4086360] end connection 10.0.216.90:33375 (1506 connections now open) 2016-03-13T09:58:51.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33384 #4086375 (1506 connections now open) 2016-03-13T09:58:51.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33385 #4086376 (1507 connections now open) 2016-03-13T09:58:52.245+0000 I NETWORK [conn4086363] end connection 10.0.20.53:55097 (1506 connections now open) 2016-03-13T09:58:52.245+0000 I NETWORK [conn4086362] end connection 10.0.20.53:55096 (1506 connections now open) 2016-03-13T09:58:52.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55111 #4086377 (1506 connections now open) 2016-03-13T09:58:52.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55112 #4086378 (1507 connections now open) 2016-03-13T09:58:54.717+0000 I NETWORK [conn4086365] end connection 10.0.179.209:44814 (1506 connections now open) 2016-03-13T09:58:54.717+0000 I NETWORK [conn4086364] end connection 10.0.179.209:44813 (1506 connections now open) 2016-03-13T09:58:54.718+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44822 #4086379 (1506 connections now open) 2016-03-13T09:58:54.718+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44821 #4086380 (1507 connections now open) 2016-03-13T09:58:55.309+0000 I NETWORK [conn4086366] end connection 10.0.26.69:46720 (1506 connections now open) 2016-03-13T09:58:55.309+0000 I NETWORK [conn4086367] end connection 10.0.26.69:46721 (1506 connections now open) 2016-03-13T09:58:55.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46741 #4086381 (1506 connections now open) 2016-03-13T09:58:55.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46740 #4086382 (1507 connections now open) 2016-03-13T09:58:58.050+0000 I NETWORK [conn4086371] end connection 10.0.27.237:40187 (1506 connections now open) 2016-03-13T09:58:58.051+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40195 #4086383 (1507 connections now open) 2016-03-13T09:58:58.051+0000 I NETWORK [conn4086372] end connection 10.0.27.237:40188 (1506 connections now open) 2016-03-13T09:58:58.052+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40196 #4086384 (1507 connections now open) 2016-03-13T09:58:58.114+0000 I NETWORK [conn4081034] end connection 10.0.179.209:40648 (1506 connections now open) 2016-03-13T09:58:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44831 #4086385 (1507 connections now open) 2016-03-13T09:59:00.281+0000 I NETWORK [conn4086373] end connection 10.0.7.113:35367 (1506 connections now open) 2016-03-13T09:59:00.281+0000 I NETWORK [conn4086374] end connection 10.0.7.113:35368 (1506 connections now open) 2016-03-13T09:59:00.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35380 #4086386 (1507 connections now open) 2016-03-13T09:59:00.282+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35381 #4086387 (1507 connections now open) 2016-03-13T09:59:01.327+0000 I NETWORK [conn4086375] end connection 10.0.216.90:33384 (1506 connections now open) 2016-03-13T09:59:01.327+0000 I NETWORK [conn4086376] end connection 10.0.216.90:33385 (1506 connections now open) 2016-03-13T09:59:01.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33396 #4086388 (1506 connections now open) 2016-03-13T09:59:01.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33397 #4086389 (1507 connections now open) 2016-03-13T09:59:02.258+0000 I NETWORK [conn4086378] end connection 10.0.20.53:55112 (1506 connections now open) 2016-03-13T09:59:02.258+0000 I NETWORK [conn4086377] end connection 10.0.20.53:55111 (1506 connections now open) 2016-03-13T09:59:02.259+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55134 #4086390 (1506 connections now open) 2016-03-13T09:59:02.259+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55133 #4086391 (1507 connections now open) 2016-03-13T09:59:04.731+0000 I NETWORK [conn4086380] end connection 10.0.179.209:44821 (1506 connections now open) 2016-03-13T09:59:04.731+0000 I NETWORK [conn4086379] end connection 10.0.179.209:44822 (1506 connections now open) 2016-03-13T09:59:04.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44839 #4086392 (1506 connections now open) 2016-03-13T09:59:04.731+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44838 #4086393 (1507 connections now open) 2016-03-13T09:59:05.321+0000 I NETWORK [conn4086382] end connection 10.0.26.69:46740 (1506 connections now open) 2016-03-13T09:59:05.321+0000 I NETWORK [conn4086381] end connection 10.0.26.69:46741 (1505 connections now open) 2016-03-13T09:59:05.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46752 #4086394 (1506 connections now open) 2016-03-13T09:59:05.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46751 #4086395 (1507 connections now open) 2016-03-13T09:59:08.063+0000 I NETWORK [conn4086383] end connection 10.0.27.237:40195 (1506 connections now open) 2016-03-13T09:59:08.064+0000 I NETWORK [conn4086384] end connection 10.0.27.237:40196 (1505 connections now open) 2016-03-13T09:59:08.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40208 #4086396 (1506 connections now open) 2016-03-13T09:59:08.073+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40209 #4086397 (1507 connections now open) 2016-03-13T09:59:10.295+0000 I NETWORK [conn4086387] end connection 10.0.7.113:35381 (1506 connections now open) 2016-03-13T09:59:10.295+0000 I NETWORK [conn4086386] end connection 10.0.7.113:35380 (1506 connections now open) 2016-03-13T09:59:10.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35403 #4086398 (1506 connections now open) 2016-03-13T09:59:10.296+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35404 #4086399 (1507 connections now open) 2016-03-13T09:59:11.340+0000 I NETWORK [conn4086389] end connection 10.0.216.90:33397 (1506 connections now open) 2016-03-13T09:59:11.340+0000 I NETWORK [conn4086388] end connection 10.0.216.90:33396 (1506 connections now open) 2016-03-13T09:59:11.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33418 #4086400 (1506 connections now open) 2016-03-13T09:59:11.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33417 #4086401 (1507 connections now open) 2016-03-13T09:59:12.271+0000 I NETWORK [conn4086391] end connection 10.0.20.53:55133 (1506 connections now open) 2016-03-13T09:59:12.271+0000 I NETWORK [conn4086390] end connection 10.0.20.53:55134 (1506 connections now open) 2016-03-13T09:59:12.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55141 #4086402 (1506 connections now open) 2016-03-13T09:59:12.272+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55142 #4086403 (1507 connections now open) 2016-03-13T09:59:14.744+0000 I NETWORK [conn4086392] end connection 10.0.179.209:44839 (1506 connections now open) 2016-03-13T09:59:14.744+0000 I NETWORK [conn4086393] end connection 10.0.179.209:44838 (1506 connections now open) 2016-03-13T09:59:14.745+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44859 #4086404 (1506 connections now open) 2016-03-13T09:59:14.745+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44858 #4086405 (1507 connections now open) 2016-03-13T09:59:15.334+0000 I NETWORK [conn4086395] end connection 10.0.26.69:46751 (1506 connections now open) 2016-03-13T09:59:15.335+0000 I NETWORK [conn4086394] end connection 10.0.26.69:46752 (1505 connections now open) 2016-03-13T09:59:15.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46766 #4086406 (1506 connections now open) 2016-03-13T09:59:15.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46767 #4086407 (1507 connections now open) 2016-03-13T09:59:15.445+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:59:15.302+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:59:16.117+0000 I NETWORK [conn4082434] end connection 10.0.179.209:41072 (1506 connections now open) 2016-03-13T09:59:16.117+0000 I NETWORK [conn4083569] end connection 10.0.26.69:43875 (1505 connections now open) 2016-03-13T09:59:16.117+0000 I NETWORK [conn4081911] end connection 10.0.27.237:36186 (1505 connections now open) 2016-03-13T09:59:16.117+0000 I NETWORK [conn4081353] end connection 10.0.7.113:59655 (1503 connections now open) 2016-03-13T09:59:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44862 #4086408 (1504 connections now open) 2016-03-13T09:59:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46769 #4086409 (1505 connections now open) 2016-03-13T09:59:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40220 #4086410 (1506 connections now open) 2016-03-13T09:59:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35409 #4086411 (1507 connections now open) 2016-03-13T09:59:18.085+0000 I NETWORK [conn4086397] end connection 10.0.27.237:40209 (1506 connections now open) 2016-03-13T09:59:18.085+0000 I NETWORK [conn4086396] end connection 10.0.27.237:40208 (1506 connections now open) 2016-03-13T09:59:18.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40232 #4086412 (1506 connections now open) 2016-03-13T09:59:18.086+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40231 #4086413 (1507 connections now open) 2016-03-13T09:59:20.308+0000 I NETWORK [conn4086398] end connection 10.0.7.113:35403 (1506 connections now open) 2016-03-13T09:59:20.308+0000 I NETWORK [conn4086399] end connection 10.0.7.113:35404 (1506 connections now open) 2016-03-13T09:59:20.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35410 #4086414 (1506 connections now open) 2016-03-13T09:59:20.309+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35411 #4086415 (1507 connections now open) 2016-03-13T09:59:21.354+0000 I NETWORK [conn4086400] end connection 10.0.216.90:33418 (1506 connections now open) 2016-03-13T09:59:21.354+0000 I NETWORK [conn4086401] end connection 10.0.216.90:33417 (1506 connections now open) 2016-03-13T09:59:21.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33427 #4086416 (1506 connections now open) 2016-03-13T09:59:21.355+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33426 #4086417 (1507 connections now open) 2016-03-13T09:59:22.284+0000 I NETWORK [conn4086402] end connection 10.0.20.53:55141 (1506 connections now open) 2016-03-13T09:59:22.285+0000 I NETWORK [conn4086403] end connection 10.0.20.53:55142 (1505 connections now open) 2016-03-13T09:59:22.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55155 #4086418 (1506 connections now open) 2016-03-13T09:59:22.285+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55156 #4086419 (1507 connections now open) 2016-03-13T09:59:24.758+0000 I NETWORK [conn4086404] end connection 10.0.179.209:44859 (1506 connections now open) 2016-03-13T09:59:24.758+0000 I NETWORK [conn4086405] end connection 10.0.179.209:44858 (1506 connections now open) 2016-03-13T09:59:24.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44870 #4086420 (1506 connections now open) 2016-03-13T09:59:24.768+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44871 #4086421 (1507 connections now open) 2016-03-13T09:59:25.347+0000 I NETWORK [conn4086406] end connection 10.0.26.69:46766 (1506 connections now open) 2016-03-13T09:59:25.347+0000 I NETWORK [conn4086407] end connection 10.0.26.69:46767 (1506 connections now open) 2016-03-13T09:59:25.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46784 #4086422 (1506 connections now open) 2016-03-13T09:59:25.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46785 #4086423 (1507 connections now open) 2016-03-13T09:59:28.097+0000 I NETWORK [conn4086412] end connection 10.0.27.237:40232 (1506 connections now open) 2016-03-13T09:59:28.098+0000 I NETWORK [conn4086413] end connection 10.0.27.237:40231 (1505 connections now open) 2016-03-13T09:59:28.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40241 #4086424 (1506 connections now open) 2016-03-13T09:59:28.098+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40242 #4086425 (1507 connections now open) 2016-03-13T09:59:28.114+0000 I NETWORK [conn4082060] end connection 10.0.20.53:51205 (1506 connections now open) 2016-03-13T09:59:28.114+0000 I NETWORK [conn4083628] end connection 10.0.216.90:58827 (1505 connections now open) 2016-03-13T09:59:28.114+0000 I NETWORK [conn4081761] end connection 10.0.7.113:59728 (1504 connections now open) 2016-03-13T09:59:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55163 #4086426 (1505 connections now open) 2016-03-13T09:59:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33439 #4086427 (1506 connections now open) 2016-03-13T09:59:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35424 #4086428 (1507 connections now open) 2016-03-13T09:59:30.321+0000 I NETWORK [conn4086414] end connection 10.0.7.113:35410 (1506 connections now open) 2016-03-13T09:59:30.321+0000 I NETWORK [conn4086415] end connection 10.0.7.113:35411 (1506 connections now open) 2016-03-13T09:59:30.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35426 #4086429 (1506 connections now open) 2016-03-13T09:59:30.322+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35427 #4086430 (1507 connections now open) 2016-03-13T09:59:31.367+0000 I NETWORK [conn4086417] end connection 10.0.216.90:33426 (1506 connections now open) 2016-03-13T09:59:31.367+0000 I NETWORK [conn4086416] end connection 10.0.216.90:33427 (1506 connections now open) 2016-03-13T09:59:31.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33446 #4086431 (1506 connections now open) 2016-03-13T09:59:31.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33445 #4086432 (1507 connections now open) 2016-03-13T09:59:32.298+0000 I NETWORK [conn4086419] end connection 10.0.20.53:55156 (1506 connections now open) 2016-03-13T09:59:32.298+0000 I NETWORK [conn4086418] end connection 10.0.20.53:55155 (1506 connections now open) 2016-03-13T09:59:32.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55179 #4086433 (1506 connections now open) 2016-03-13T09:59:32.298+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55178 #4086434 (1507 connections now open) 2016-03-13T09:59:34.779+0000 I NETWORK [conn4086420] end connection 10.0.179.209:44870 (1506 connections now open) 2016-03-13T09:59:34.779+0000 I NETWORK [conn4086421] end connection 10.0.179.209:44871 (1506 connections now open) 2016-03-13T09:59:34.780+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44886 #4086435 (1506 connections now open) 2016-03-13T09:59:34.780+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44887 #4086436 (1507 connections now open) 2016-03-13T09:59:35.361+0000 I NETWORK [conn4086423] end connection 10.0.26.69:46785 (1506 connections now open) 2016-03-13T09:59:35.361+0000 I NETWORK [conn4086422] end connection 10.0.26.69:46784 (1506 connections now open) 2016-03-13T09:59:35.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46792 #4086437 (1506 connections now open) 2016-03-13T09:59:35.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46793 #4086438 (1507 connections now open) 2016-03-13T09:59:38.110+0000 I NETWORK [conn4086424] end connection 10.0.27.237:40241 (1506 connections now open) 2016-03-13T09:59:38.110+0000 I NETWORK [conn4086425] end connection 10.0.27.237:40242 (1506 connections now open) 2016-03-13T09:59:38.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40253 #4086439 (1506 connections now open) 2016-03-13T09:59:38.111+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40254 #4086440 (1507 connections now open) 2016-03-13T09:59:40.334+0000 I NETWORK [conn4086429] end connection 10.0.7.113:35426 (1506 connections now open) 2016-03-13T09:59:40.334+0000 I NETWORK [conn4086430] end connection 10.0.7.113:35427 (1506 connections now open) 2016-03-13T09:59:40.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35443 #4086441 (1506 connections now open) 2016-03-13T09:59:40.335+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35442 #4086442 (1507 connections now open) 2016-03-13T09:59:41.380+0000 I NETWORK [conn4086432] end connection 10.0.216.90:33445 (1506 connections now open) 2016-03-13T09:59:41.380+0000 I NETWORK [conn4086431] end connection 10.0.216.90:33446 (1506 connections now open) 2016-03-13T09:59:41.381+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33464 #4086443 (1506 connections now open) 2016-03-13T09:59:41.381+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33463 #4086444 (1507 connections now open) 2016-03-13T09:59:42.311+0000 I NETWORK [conn4086434] end connection 10.0.20.53:55178 (1506 connections now open) 2016-03-13T09:59:42.311+0000 I NETWORK [conn4086433] end connection 10.0.20.53:55179 (1506 connections now open) 2016-03-13T09:59:42.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55188 #4086445 (1506 connections now open) 2016-03-13T09:59:42.320+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55187 #4086446 (1507 connections now open) 2016-03-13T09:59:44.793+0000 I NETWORK [conn4086435] end connection 10.0.179.209:44886 (1506 connections now open) 2016-03-13T09:59:44.793+0000 I NETWORK [conn4086436] end connection 10.0.179.209:44887 (1506 connections now open) 2016-03-13T09:59:44.793+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44904 #4086447 (1506 connections now open) 2016-03-13T09:59:44.793+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44905 #4086448 (1507 connections now open) 2016-03-13T09:59:45.374+0000 I NETWORK [conn4086437] end connection 10.0.26.69:46792 (1506 connections now open) 2016-03-13T09:59:45.374+0000 I NETWORK [conn4086438] end connection 10.0.26.69:46793 (1506 connections now open) 2016-03-13T09:59:45.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46805 #4086449 (1506 connections now open) 2016-03-13T09:59:45.376+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46804 #4086450 (1507 connections now open) 2016-03-13T09:59:45.606+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T09:59:45.445+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T09:59:46.117+0000 I NETWORK [conn4080110] end connection 10.0.216.90:56786 (1506 connections now open) 2016-03-13T09:59:46.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33472 #4086451 (1507 connections now open) 2016-03-13T09:59:48.123+0000 I NETWORK [conn4086439] end connection 10.0.27.237:40253 (1506 connections now open) 2016-03-13T09:59:48.123+0000 I NETWORK [conn4086440] end connection 10.0.27.237:40254 (1505 connections now open) 2016-03-13T09:59:48.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40273 #4086452 (1506 connections now open) 2016-03-13T09:59:48.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40274 #4086453 (1507 connections now open) 2016-03-13T09:59:50.347+0000 I NETWORK [conn4086442] end connection 10.0.7.113:35442 (1506 connections now open) 2016-03-13T09:59:50.347+0000 I NETWORK [conn4086441] end connection 10.0.7.113:35443 (1506 connections now open) 2016-03-13T09:59:50.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35452 #4086454 (1506 connections now open) 2016-03-13T09:59:50.348+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35453 #4086455 (1507 connections now open) 2016-03-13T09:59:51.393+0000 I NETWORK [conn4086443] end connection 10.0.216.90:33464 (1506 connections now open) 2016-03-13T09:59:51.393+0000 I NETWORK [conn4086444] end connection 10.0.216.90:33463 (1506 connections now open) 2016-03-13T09:59:51.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33475 #4086456 (1506 connections now open) 2016-03-13T09:59:51.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33474 #4086457 (1507 connections now open) 2016-03-13T09:59:52.333+0000 I NETWORK [conn4086446] end connection 10.0.20.53:55187 (1506 connections now open) 2016-03-13T09:59:52.333+0000 I NETWORK [conn4086445] end connection 10.0.20.53:55188 (1506 connections now open) 2016-03-13T09:59:52.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55202 #4086458 (1506 connections now open) 2016-03-13T09:59:52.334+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55203 #4086459 (1507 connections now open) 2016-03-13T09:59:54.807+0000 I NETWORK [conn4086448] end connection 10.0.179.209:44905 (1506 connections now open) 2016-03-13T09:59:54.807+0000 I NETWORK [conn4086447] end connection 10.0.179.209:44904 (1506 connections now open) 2016-03-13T09:59:54.807+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44914 #4086460 (1506 connections now open) 2016-03-13T09:59:54.807+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44913 #4086461 (1507 connections now open) 2016-03-13T09:59:55.388+0000 I NETWORK [conn4086450] end connection 10.0.26.69:46804 (1506 connections now open) 2016-03-13T09:59:55.388+0000 I NETWORK [conn4086449] end connection 10.0.26.69:46805 (1506 connections now open) 2016-03-13T09:59:55.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46823 #4086462 (1506 connections now open) 2016-03-13T09:59:55.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46824 #4086463 (1507 connections now open) 2016-03-13T09:59:58.114+0000 I NETWORK [conn4081309] end connection 10.0.179.209:40728 (1506 connections now open) 2016-03-13T09:59:58.114+0000 I NETWORK [conn4085513] end connection 10.0.216.90:60734 (1506 connections now open) 2016-03-13T09:59:58.114+0000 I NETWORK [conn4081990] end connection 10.0.20.53:51172 (1504 connections now open) 2016-03-13T09:59:58.114+0000 I NETWORK [conn4082606] end connection 10.0.27.237:36472 (1504 connections now open) 2016-03-13T09:59:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44922 #4086464 (1504 connections now open) 2016-03-13T09:59:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55209 #4086465 (1505 connections now open) 2016-03-13T09:59:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33486 #4086466 (1506 connections now open) 2016-03-13T09:59:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40281 #4086467 (1507 connections now open) 2016-03-13T09:59:58.136+0000 I NETWORK [conn4086452] end connection 10.0.27.237:40273 (1506 connections now open) 2016-03-13T09:59:58.136+0000 I NETWORK [conn4086453] end connection 10.0.27.237:40274 (1506 connections now open) 2016-03-13T09:59:58.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40282 #4086468 (1506 connections now open) 2016-03-13T09:59:58.137+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40283 #4086469 (1507 connections now open) 2016-03-13T10:00:00.361+0000 I NETWORK [conn4086454] end connection 10.0.7.113:35452 (1506 connections now open) 2016-03-13T10:00:00.361+0000 I NETWORK [conn4086455] end connection 10.0.7.113:35453 (1506 connections now open) 2016-03-13T10:00:00.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35466 #4086470 (1506 connections now open) 2016-03-13T10:00:00.362+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35467 #4086471 (1507 connections now open) 2016-03-13T10:00:01.406+0000 I NETWORK [conn4086457] end connection 10.0.216.90:33474 (1506 connections now open) 2016-03-13T10:00:01.406+0000 I NETWORK [conn4086456] end connection 10.0.216.90:33475 (1506 connections now open) 2016-03-13T10:00:01.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33488 #4086472 (1506 connections now open) 2016-03-13T10:00:01.407+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33489 #4086473 (1507 connections now open) 2016-03-13T10:00:02.346+0000 I NETWORK [conn4086458] end connection 10.0.20.53:55202 (1506 connections now open) 2016-03-13T10:00:02.346+0000 I NETWORK [conn4086459] end connection 10.0.20.53:55203 (1506 connections now open) 2016-03-13T10:00:02.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55223 #4086474 (1506 connections now open) 2016-03-13T10:00:02.347+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55224 #4086475 (1507 connections now open) 2016-03-13T10:00:04.820+0000 I NETWORK [conn4086461] end connection 10.0.179.209:44913 (1506 connections now open) 2016-03-13T10:00:04.820+0000 I NETWORK [conn4086460] end connection 10.0.179.209:44914 (1506 connections now open) 2016-03-13T10:00:04.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44931 #4086476 (1506 connections now open) 2016-03-13T10:00:04.821+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44932 #4086477 (1507 connections now open) 2016-03-13T10:00:05.409+0000 I NETWORK [conn4086462] end connection 10.0.26.69:46823 (1506 connections now open) 2016-03-13T10:00:05.409+0000 I NETWORK [conn4086463] end connection 10.0.26.69:46824 (1506 connections now open) 2016-03-13T10:00:05.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46836 #4086478 (1506 connections now open) 2016-03-13T10:00:05.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46837 #4086479 (1507 connections now open) 2016-03-13T10:00:08.149+0000 I NETWORK [conn4086469] end connection 10.0.27.237:40283 (1506 connections now open) 2016-03-13T10:00:08.149+0000 I NETWORK [conn4086468] end connection 10.0.27.237:40282 (1506 connections now open) 2016-03-13T10:00:08.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40297 #4086480 (1506 connections now open) 2016-03-13T10:00:08.158+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40296 #4086481 (1507 connections now open) 2016-03-13T10:00:10.374+0000 I NETWORK [conn4086470] end connection 10.0.7.113:35466 (1506 connections now open) 2016-03-13T10:00:10.374+0000 I NETWORK [conn4086471] end connection 10.0.7.113:35467 (1506 connections now open) 2016-03-13T10:00:10.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35490 #4086482 (1506 connections now open) 2016-03-13T10:00:10.375+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35489 #4086483 (1507 connections now open) 2016-03-13T10:00:11.419+0000 I NETWORK [conn4086473] end connection 10.0.216.90:33489 (1506 connections now open) 2016-03-13T10:00:11.419+0000 I NETWORK [conn4086472] end connection 10.0.216.90:33488 (1506 connections now open) 2016-03-13T10:00:11.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33508 #4086484 (1506 connections now open) 2016-03-13T10:00:11.420+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33509 #4086485 (1507 connections now open) 2016-03-13T10:00:12.360+0000 I NETWORK [conn4086475] end connection 10.0.20.53:55224 (1506 connections now open) 2016-03-13T10:00:12.360+0000 I NETWORK [conn4086474] end connection 10.0.20.53:55223 (1506 connections now open) 2016-03-13T10:00:12.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55233 #4086486 (1506 connections now open) 2016-03-13T10:00:12.360+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55234 #4086487 (1507 connections now open) 2016-03-13T10:00:14.834+0000 I NETWORK [conn4086476] end connection 10.0.179.209:44931 (1506 connections now open) 2016-03-13T10:00:14.834+0000 I NETWORK [conn4086477] end connection 10.0.179.209:44932 (1506 connections now open) 2016-03-13T10:00:14.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44951 #4086488 (1506 connections now open) 2016-03-13T10:00:14.835+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44950 #4086489 (1507 connections now open) 2016-03-13T10:00:15.422+0000 I NETWORK [conn4086479] end connection 10.0.26.69:46837 (1506 connections now open) 2016-03-13T10:00:15.422+0000 I NETWORK [conn4086478] end connection 10.0.26.69:46836 (1506 connections now open) 2016-03-13T10:00:15.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46852 #4086490 (1506 connections now open) 2016-03-13T10:00:15.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46853 #4086491 (1507 connections now open) 2016-03-13T10:00:15.875+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:00:15.606+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:00:16.116+0000 I NETWORK [conn4081697] end connection 10.0.20.53:51042 (1506 connections now open) 2016-03-13T10:00:16.116+0000 I NETWORK [conn4078547] end connection 10.0.216.90:55221 (1505 connections now open) 2016-03-13T10:00:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55239 #4086492 (1506 connections now open) 2016-03-13T10:00:16.117+0000 I NETWORK [conn4080712] end connection 10.0.7.113:59370 (1505 connections now open) 2016-03-13T10:00:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33515 #4086493 (1506 connections now open) 2016-03-13T10:00:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35498 #4086494 (1507 connections now open) 2016-03-13T10:00:18.170+0000 I NETWORK [conn4086481] end connection 10.0.27.237:40296 (1506 connections now open) 2016-03-13T10:00:18.170+0000 I NETWORK [conn4086480] end connection 10.0.27.237:40297 (1506 connections now open) 2016-03-13T10:00:18.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40321 #4086495 (1506 connections now open) 2016-03-13T10:00:18.171+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40322 #4086496 (1507 connections now open) 2016-03-13T10:00:20.387+0000 I NETWORK [conn4086482] end connection 10.0.7.113:35490 (1506 connections now open) 2016-03-13T10:00:20.387+0000 I NETWORK [conn4086483] end connection 10.0.7.113:35489 (1506 connections now open) 2016-03-13T10:00:20.388+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35500 #4086497 (1506 connections now open) 2016-03-13T10:00:20.388+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35499 #4086498 (1507 connections now open) 2016-03-13T10:00:21.433+0000 I NETWORK [conn4086484] end connection 10.0.216.90:33508 (1506 connections now open) 2016-03-13T10:00:21.433+0000 I NETWORK [conn4086485] end connection 10.0.216.90:33509 (1506 connections now open) 2016-03-13T10:00:21.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33519 #4086499 (1506 connections now open) 2016-03-13T10:00:21.433+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33518 #4086500 (1507 connections now open) 2016-03-13T10:00:22.373+0000 I NETWORK [conn4086487] end connection 10.0.20.53:55234 (1506 connections now open) 2016-03-13T10:00:22.373+0000 I NETWORK [conn4086486] end connection 10.0.20.53:55233 (1506 connections now open) 2016-03-13T10:00:22.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55246 #4086501 (1506 connections now open) 2016-03-13T10:00:22.374+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55247 #4086502 (1507 connections now open) 2016-03-13T10:00:24.847+0000 I NETWORK [conn4086489] end connection 10.0.179.209:44950 (1506 connections now open) 2016-03-13T10:00:24.847+0000 I NETWORK [conn4086488] end connection 10.0.179.209:44951 (1506 connections now open) 2016-03-13T10:00:24.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44964 #4086503 (1506 connections now open) 2016-03-13T10:00:24.857+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44963 #4086504 (1507 connections now open) 2016-03-13T10:00:25.436+0000 I NETWORK [conn4086491] end connection 10.0.26.69:46853 (1506 connections now open) 2016-03-13T10:00:25.436+0000 I NETWORK [conn4086490] end connection 10.0.26.69:46852 (1506 connections now open) 2016-03-13T10:00:25.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46871 #4086505 (1507 connections now open) 2016-03-13T10:00:25.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46872 #4086506 (1507 connections now open) 2016-03-13T10:00:28.116+0000 I NETWORK [conn4082081] end connection 10.0.20.53:51214 (1506 connections now open) 2016-03-13T10:00:28.116+0000 I NETWORK [conn4086131] end connection 10.0.216.90:33138 (1506 connections now open) 2016-03-13T10:00:28.116+0000 I NETWORK [conn4082354] end connection 10.0.179.209:41052 (1506 connections now open) 2016-03-13T10:00:28.116+0000 I NETWORK [conn4082219] end connection 10.0.27.237:36338 (1506 connections now open) 2016-03-13T10:00:28.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55254 #4086507 (1504 connections now open) 2016-03-13T10:00:28.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33528 #4086508 (1505 connections now open) 2016-03-13T10:00:28.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44972 #4086509 (1506 connections now open) 2016-03-13T10:00:28.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40329 #4086510 (1507 connections now open) 2016-03-13T10:00:28.183+0000 I NETWORK [conn4086495] end connection 10.0.27.237:40321 (1506 connections now open) 2016-03-13T10:00:28.183+0000 I NETWORK [conn4086496] end connection 10.0.27.237:40322 (1506 connections now open) 2016-03-13T10:00:28.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40330 #4086511 (1506 connections now open) 2016-03-13T10:00:28.184+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40331 #4086512 (1507 connections now open) 2016-03-13T10:00:30.401+0000 I NETWORK [conn4086497] end connection 10.0.7.113:35500 (1506 connections now open) 2016-03-13T10:00:30.401+0000 I NETWORK [conn4086498] end connection 10.0.7.113:35499 (1506 connections now open) 2016-03-13T10:00:30.401+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35514 #4086513 (1506 connections now open) 2016-03-13T10:00:30.401+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35515 #4086514 (1507 connections now open) 2016-03-13T10:00:31.446+0000 I NETWORK [conn4086500] end connection 10.0.216.90:33518 (1506 connections now open) 2016-03-13T10:00:31.446+0000 I NETWORK [conn4086499] end connection 10.0.216.90:33519 (1506 connections now open) 2016-03-13T10:00:31.446+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33534 #4086515 (1506 connections now open) 2016-03-13T10:00:31.446+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33533 #4086516 (1507 connections now open) 2016-03-13T10:00:32.386+0000 I NETWORK [conn4086502] end connection 10.0.20.53:55247 (1506 connections now open) 2016-03-13T10:00:32.386+0000 I NETWORK [conn4086501] end connection 10.0.20.53:55246 (1506 connections now open) 2016-03-13T10:00:32.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55269 #4086517 (1506 connections now open) 2016-03-13T10:00:32.387+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55270 #4086518 (1507 connections now open) 2016-03-13T10:00:34.870+0000 I NETWORK [conn4086503] end connection 10.0.179.209:44964 (1506 connections now open) 2016-03-13T10:00:34.870+0000 I NETWORK [conn4086504] end connection 10.0.179.209:44963 (1506 connections now open) 2016-03-13T10:00:34.870+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44977 #4086519 (1506 connections now open) 2016-03-13T10:00:34.870+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44978 #4086520 (1507 connections now open) 2016-03-13T10:00:35.450+0000 I NETWORK [conn4086506] end connection 10.0.26.69:46872 (1506 connections now open) 2016-03-13T10:00:35.450+0000 I NETWORK [conn4086505] end connection 10.0.26.69:46871 (1506 connections now open) 2016-03-13T10:00:35.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46883 #4086521 (1506 connections now open) 2016-03-13T10:00:35.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46882 #4086522 (1507 connections now open) 2016-03-13T10:00:38.197+0000 I NETWORK [conn4086511] end connection 10.0.27.237:40330 (1506 connections now open) 2016-03-13T10:00:38.197+0000 I NETWORK [conn4086512] end connection 10.0.27.237:40331 (1506 connections now open) 2016-03-13T10:00:38.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40343 #4086523 (1506 connections now open) 2016-03-13T10:00:38.197+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40344 #4086524 (1507 connections now open) 2016-03-13T10:00:40.414+0000 I NETWORK [conn4086514] end connection 10.0.7.113:35515 (1506 connections now open) 2016-03-13T10:00:40.414+0000 I NETWORK [conn4086513] end connection 10.0.7.113:35514 (1506 connections now open) 2016-03-13T10:00:40.415+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35532 #4086525 (1506 connections now open) 2016-03-13T10:00:40.415+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35533 #4086526 (1507 connections now open) 2016-03-13T10:00:41.458+0000 I NETWORK [conn4086516] end connection 10.0.216.90:33533 (1506 connections now open) 2016-03-13T10:00:41.458+0000 I NETWORK [conn4086515] end connection 10.0.216.90:33534 (1506 connections now open) 2016-03-13T10:00:41.459+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33550 #4086527 (1506 connections now open) 2016-03-13T10:00:41.459+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33551 #4086528 (1507 connections now open) 2016-03-13T10:00:42.400+0000 I NETWORK [conn4086517] end connection 10.0.20.53:55269 (1506 connections now open) 2016-03-13T10:00:42.400+0000 I NETWORK [conn4086518] end connection 10.0.20.53:55270 (1506 connections now open) 2016-03-13T10:00:42.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55277 #4086529 (1506 connections now open) 2016-03-13T10:00:42.409+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55276 #4086530 (1507 connections now open) 2016-03-13T10:00:44.883+0000 I NETWORK [conn4086520] end connection 10.0.179.209:44978 (1506 connections now open) 2016-03-13T10:00:44.883+0000 I NETWORK [conn4086519] end connection 10.0.179.209:44977 (1506 connections now open) 2016-03-13T10:00:44.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44996 #4086531 (1506 connections now open) 2016-03-13T10:00:44.884+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44997 #4086532 (1507 connections now open) 2016-03-13T10:00:45.463+0000 I NETWORK [conn4086522] end connection 10.0.26.69:46882 (1506 connections now open) 2016-03-13T10:00:45.463+0000 I NETWORK [conn4086521] end connection 10.0.26.69:46883 (1506 connections now open) 2016-03-13T10:00:45.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46895 #4086533 (1506 connections now open) 2016-03-13T10:00:45.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46896 #4086534 (1507 connections now open) 2016-03-13T10:00:46.008+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:00:45.875+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:00:46.116+0000 I NETWORK [conn4082010] end connection 10.0.179.209:40901 (1506 connections now open) 2016-03-13T10:00:46.116+0000 I NETWORK [conn4082654] end connection 10.0.216.90:58097 (1505 connections now open) 2016-03-13T10:00:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:44999 #4086535 (1506 connections now open) 2016-03-13T10:00:46.117+0000 I NETWORK [conn4084139] end connection 10.0.27.237:37785 (1505 connections now open) 2016-03-13T10:00:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40352 #4086536 (1506 connections now open) 2016-03-13T10:00:46.125+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33558 #4086537 (1507 connections now open) 2016-03-13T10:00:46.462+0000 I NETWORK [conn4080084] end connection 10.0.91.115:54791 (1506 connections now open) 2016-03-13T10:00:46.469+0000 I NETWORK [mongosMain] connection accepted from 10.0.91.115:57361 #4086538 (1507 connections now open) 2016-03-13T10:00:48.210+0000 I NETWORK [conn4086524] end connection 10.0.27.237:40344 (1506 connections now open) 2016-03-13T10:00:48.210+0000 I NETWORK [conn4086523] end connection 10.0.27.237:40343 (1506 connections now open) 2016-03-13T10:00:48.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40364 #4086539 (1506 connections now open) 2016-03-13T10:00:48.211+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40365 #4086540 (1507 connections now open) 2016-03-13T10:00:50.427+0000 I NETWORK [conn4086526] end connection 10.0.7.113:35533 (1506 connections now open) 2016-03-13T10:00:50.427+0000 I NETWORK [conn4086525] end connection 10.0.7.113:35532 (1506 connections now open) 2016-03-13T10:00:50.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35540 #4086541 (1506 connections now open) 2016-03-13T10:00:50.437+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35541 #4086542 (1507 connections now open) 2016-03-13T10:00:51.472+0000 I NETWORK [conn4086527] end connection 10.0.216.90:33550 (1506 connections now open) 2016-03-13T10:00:51.472+0000 I NETWORK [conn4086528] end connection 10.0.216.90:33551 (1506 connections now open) 2016-03-13T10:00:51.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33560 #4086543 (1506 connections now open) 2016-03-13T10:00:51.472+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33561 #4086544 (1507 connections now open) 2016-03-13T10:00:52.421+0000 I NETWORK [conn4086530] end connection 10.0.20.53:55276 (1506 connections now open) 2016-03-13T10:00:52.421+0000 I NETWORK [conn4086529] end connection 10.0.20.53:55277 (1506 connections now open) 2016-03-13T10:00:52.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55291 #4086545 (1506 connections now open) 2016-03-13T10:00:52.422+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55292 #4086546 (1507 connections now open) 2016-03-13T10:00:54.896+0000 I NETWORK [conn4086532] end connection 10.0.179.209:44997 (1506 connections now open) 2016-03-13T10:00:54.896+0000 I NETWORK [conn4086531] end connection 10.0.179.209:44996 (1506 connections now open) 2016-03-13T10:00:54.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45004 #4086547 (1506 connections now open) 2016-03-13T10:00:54.897+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45005 #4086548 (1507 connections now open) 2016-03-13T10:00:55.476+0000 I NETWORK [conn4086533] end connection 10.0.26.69:46895 (1506 connections now open) 2016-03-13T10:00:55.476+0000 I NETWORK [conn4086534] end connection 10.0.26.69:46896 (1505 connections now open) 2016-03-13T10:00:55.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46916 #4086549 (1506 connections now open) 2016-03-13T10:00:55.485+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46917 #4086550 (1507 connections now open) 2016-03-13T10:00:58.114+0000 I NETWORK [conn4082008] end connection 10.0.27.237:36239 (1506 connections now open) 2016-03-13T10:00:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40373 #4086551 (1507 connections now open) 2016-03-13T10:00:58.223+0000 I NETWORK [conn4086539] end connection 10.0.27.237:40364 (1506 connections now open) 2016-03-13T10:00:58.223+0000 I NETWORK [conn4086540] end connection 10.0.27.237:40365 (1506 connections now open) 2016-03-13T10:00:58.223+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40374 #4086552 (1506 connections now open) 2016-03-13T10:00:58.224+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40375 #4086553 (1507 connections now open) 2016-03-13T10:01:00.450+0000 I NETWORK [conn4086541] end connection 10.0.7.113:35540 (1506 connections now open) 2016-03-13T10:01:00.450+0000 I NETWORK [conn4086542] end connection 10.0.7.113:35541 (1506 connections now open) 2016-03-13T10:01:00.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35554 #4086554 (1506 connections now open) 2016-03-13T10:01:00.450+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35555 #4086555 (1507 connections now open) 2016-03-13T10:01:01.485+0000 I NETWORK [conn4086543] end connection 10.0.216.90:33560 (1506 connections now open) 2016-03-13T10:01:01.485+0000 I NETWORK [conn4086544] end connection 10.0.216.90:33561 (1506 connections now open) 2016-03-13T10:01:01.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33573 #4086556 (1506 connections now open) 2016-03-13T10:01:01.486+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33574 #4086557 (1507 connections now open) 2016-03-13T10:01:02.435+0000 I NETWORK [conn4086546] end connection 10.0.20.53:55292 (1506 connections now open) 2016-03-13T10:01:02.435+0000 I NETWORK [conn4086545] end connection 10.0.20.53:55291 (1506 connections now open) 2016-03-13T10:01:02.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55312 #4086558 (1506 connections now open) 2016-03-13T10:01:02.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55313 #4086559 (1507 connections now open) 2016-03-13T10:01:04.909+0000 I NETWORK [conn4086547] end connection 10.0.179.209:45004 (1506 connections now open) 2016-03-13T10:01:04.909+0000 I NETWORK [conn4086548] end connection 10.0.179.209:45005 (1506 connections now open) 2016-03-13T10:01:04.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45019 #4086560 (1506 connections now open) 2016-03-13T10:01:04.910+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45020 #4086561 (1507 connections now open) 2016-03-13T10:01:05.497+0000 I NETWORK [conn4086550] end connection 10.0.26.69:46917 (1506 connections now open) 2016-03-13T10:01:05.497+0000 I NETWORK [conn4086549] end connection 10.0.26.69:46916 (1505 connections now open) 2016-03-13T10:01:05.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46928 #4086562 (1506 connections now open) 2016-03-13T10:01:05.498+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46929 #4086563 (1507 connections now open) 2016-03-13T10:01:08.236+0000 I NETWORK [conn4086553] end connection 10.0.27.237:40375 (1506 connections now open) 2016-03-13T10:01:08.236+0000 I NETWORK [conn4086552] end connection 10.0.27.237:40374 (1506 connections now open) 2016-03-13T10:01:08.244+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40388 #4086564 (1506 connections now open) 2016-03-13T10:01:08.245+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40387 #4086565 (1507 connections now open) 2016-03-13T10:01:10.463+0000 I NETWORK [conn4086554] end connection 10.0.7.113:35554 (1506 connections now open) 2016-03-13T10:01:10.463+0000 I NETWORK [conn4086555] end connection 10.0.7.113:35555 (1506 connections now open) 2016-03-13T10:01:10.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35578 #4086566 (1506 connections now open) 2016-03-13T10:01:10.463+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35577 #4086567 (1507 connections now open) 2016-03-13T10:01:11.498+0000 I NETWORK [conn4086556] end connection 10.0.216.90:33573 (1506 connections now open) 2016-03-13T10:01:11.498+0000 I NETWORK [conn4086557] end connection 10.0.216.90:33574 (1506 connections now open) 2016-03-13T10:01:11.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33593 #4086568 (1506 connections now open) 2016-03-13T10:01:11.499+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33592 #4086569 (1507 connections now open) 2016-03-13T10:01:12.448+0000 I NETWORK [conn4086558] end connection 10.0.20.53:55312 (1506 connections now open) 2016-03-13T10:01:12.448+0000 I NETWORK [conn4086559] end connection 10.0.20.53:55313 (1506 connections now open) 2016-03-13T10:01:12.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55320 #4086570 (1506 connections now open) 2016-03-13T10:01:12.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55321 #4086571 (1507 connections now open) 2016-03-13T10:01:14.923+0000 I NETWORK [conn4086560] end connection 10.0.179.209:45019 (1506 connections now open) 2016-03-13T10:01:14.923+0000 I NETWORK [conn4086561] end connection 10.0.179.209:45020 (1506 connections now open) 2016-03-13T10:01:14.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45038 #4086572 (1506 connections now open) 2016-03-13T10:01:14.924+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45037 #4086573 (1507 connections now open) 2016-03-13T10:01:15.510+0000 I NETWORK [conn4086562] end connection 10.0.26.69:46928 (1506 connections now open) 2016-03-13T10:01:15.510+0000 I NETWORK [conn4086563] end connection 10.0.26.69:46929 (1505 connections now open) 2016-03-13T10:01:15.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46942 #4086574 (1506 connections now open) 2016-03-13T10:01:15.511+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46943 #4086575 (1507 connections now open) 2016-03-13T10:01:16.116+0000 I NETWORK [conn4081871] end connection 10.0.179.209:40852 (1506 connections now open) 2016-03-13T10:01:16.116+0000 I NETWORK [conn4082717] end connection 10.0.26.69:43220 (1505 connections now open) 2016-03-13T10:01:16.117+0000 I NETWORK [conn4081768] end connection 10.0.7.113:59729 (1504 connections now open) 2016-03-13T10:01:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45040 #4086576 (1505 connections now open) 2016-03-13T10:01:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46945 #4086577 (1506 connections now open) 2016-03-13T10:01:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35584 #4086578 (1507 connections now open) 2016-03-13T10:01:16.272+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:01:16.008+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:01:18.257+0000 I NETWORK [conn4086565] end connection 10.0.27.237:40387 (1506 connections now open) 2016-03-13T10:01:18.257+0000 I NETWORK [conn4086564] end connection 10.0.27.237:40388 (1506 connections now open) 2016-03-13T10:01:18.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40410 #4086579 (1506 connections now open) 2016-03-13T10:01:18.258+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40411 #4086580 (1507 connections now open) 2016-03-13T10:01:20.476+0000 I NETWORK [conn4086567] end connection 10.0.7.113:35577 (1506 connections now open) 2016-03-13T10:01:20.476+0000 I NETWORK [conn4086566] end connection 10.0.7.113:35578 (1506 connections now open) 2016-03-13T10:01:20.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35585 #4086581 (1506 connections now open) 2016-03-13T10:01:20.477+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35586 #4086582 (1507 connections now open) 2016-03-13T10:01:21.511+0000 I NETWORK [conn4086568] end connection 10.0.216.90:33593 (1506 connections now open) 2016-03-13T10:01:21.511+0000 I NETWORK [conn4086569] end connection 10.0.216.90:33592 (1505 connections now open) 2016-03-13T10:01:21.512+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33602 #4086583 (1506 connections now open) 2016-03-13T10:01:21.512+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33603 #4086584 (1507 connections now open) 2016-03-13T10:01:22.461+0000 I NETWORK [conn4086571] end connection 10.0.20.53:55321 (1506 connections now open) 2016-03-13T10:01:22.461+0000 I NETWORK [conn4086570] end connection 10.0.20.53:55320 (1506 connections now open) 2016-03-13T10:01:22.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55334 #4086585 (1506 connections now open) 2016-03-13T10:01:22.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55335 #4086586 (1507 connections now open) 2016-03-13T10:01:24.936+0000 I NETWORK [conn4086573] end connection 10.0.179.209:45037 (1506 connections now open) 2016-03-13T10:01:24.936+0000 I NETWORK [conn4086572] end connection 10.0.179.209:45038 (1506 connections now open) 2016-03-13T10:01:24.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45050 #4086587 (1506 connections now open) 2016-03-13T10:01:24.946+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45049 #4086588 (1507 connections now open) 2016-03-13T10:01:25.523+0000 I NETWORK [conn4086574] end connection 10.0.26.69:46942 (1506 connections now open) 2016-03-13T10:01:25.523+0000 I NETWORK [conn4086575] end connection 10.0.26.69:46943 (1506 connections now open) 2016-03-13T10:01:25.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46963 #4086589 (1508 connections now open) 2016-03-13T10:01:25.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46962 #4086590 (1507 connections now open) 2016-03-13T10:01:28.114+0000 I NETWORK [conn4082055] end connection 10.0.20.53:51202 (1506 connections now open) 2016-03-13T10:01:28.114+0000 I NETWORK [conn4081981] end connection 10.0.27.237:36231 (1505 connections now open) 2016-03-13T10:01:28.114+0000 I NETWORK [conn4082950] end connection 10.0.7.113:60177 (1504 connections now open) 2016-03-13T10:01:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55341 #4086591 (1505 connections now open) 2016-03-13T10:01:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40419 #4086592 (1506 connections now open) 2016-03-13T10:01:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35598 #4086593 (1507 connections now open) 2016-03-13T10:01:28.270+0000 I NETWORK [conn4086579] end connection 10.0.27.237:40410 (1506 connections now open) 2016-03-13T10:01:28.270+0000 I NETWORK [conn4086580] end connection 10.0.27.237:40411 (1506 connections now open) 2016-03-13T10:01:28.270+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40421 #4086594 (1506 connections now open) 2016-03-13T10:01:28.271+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40420 #4086595 (1507 connections now open) 2016-03-13T10:01:30.489+0000 I NETWORK [conn4086582] end connection 10.0.7.113:35586 (1506 connections now open) 2016-03-13T10:01:30.489+0000 I NETWORK [conn4086581] end connection 10.0.7.113:35585 (1506 connections now open) 2016-03-13T10:01:30.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35600 #4086596 (1506 connections now open) 2016-03-13T10:01:30.490+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35601 #4086597 (1507 connections now open) 2016-03-13T10:01:31.524+0000 I NETWORK [conn4086583] end connection 10.0.216.90:33602 (1506 connections now open) 2016-03-13T10:01:31.524+0000 I NETWORK [conn4086584] end connection 10.0.216.90:33603 (1505 connections now open) 2016-03-13T10:01:31.524+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33617 #4086598 (1506 connections now open) 2016-03-13T10:01:31.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33618 #4086599 (1507 connections now open) 2016-03-13T10:01:32.475+0000 I NETWORK [conn4086586] end connection 10.0.20.53:55335 (1506 connections now open) 2016-03-13T10:01:32.475+0000 I NETWORK [conn4086585] end connection 10.0.20.53:55334 (1506 connections now open) 2016-03-13T10:01:32.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55358 #4086600 (1506 connections now open) 2016-03-13T10:01:32.476+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55357 #4086601 (1507 connections now open) 2016-03-13T10:01:34.958+0000 I NETWORK [conn4086588] end connection 10.0.179.209:45049 (1506 connections now open) 2016-03-13T10:01:34.958+0000 I NETWORK [conn4086587] end connection 10.0.179.209:45050 (1506 connections now open) 2016-03-13T10:01:34.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45064 #4086602 (1506 connections now open) 2016-03-13T10:01:34.959+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45065 #4086603 (1507 connections now open) 2016-03-13T10:01:35.536+0000 I NETWORK [conn4086589] end connection 10.0.26.69:46963 (1506 connections now open) 2016-03-13T10:01:35.536+0000 I NETWORK [conn4086590] end connection 10.0.26.69:46962 (1506 connections now open) 2016-03-13T10:01:35.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46973 #4086604 (1506 connections now open) 2016-03-13T10:01:35.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46974 #4086605 (1507 connections now open) 2016-03-13T10:01:38.283+0000 I NETWORK [conn4086594] end connection 10.0.27.237:40421 (1506 connections now open) 2016-03-13T10:01:38.283+0000 I NETWORK [conn4086595] end connection 10.0.27.237:40420 (1506 connections now open) 2016-03-13T10:01:38.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40433 #4086606 (1506 connections now open) 2016-03-13T10:01:38.284+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40432 #4086607 (1507 connections now open) 2016-03-13T10:01:40.502+0000 I NETWORK [conn4086596] end connection 10.0.7.113:35600 (1506 connections now open) 2016-03-13T10:01:40.502+0000 I NETWORK [conn4086597] end connection 10.0.7.113:35601 (1506 connections now open) 2016-03-13T10:01:40.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35617 #4086608 (1506 connections now open) 2016-03-13T10:01:40.503+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35618 #4086609 (1507 connections now open) 2016-03-13T10:01:41.537+0000 I NETWORK [conn4086599] end connection 10.0.216.90:33618 (1506 connections now open) 2016-03-13T10:01:41.537+0000 I NETWORK [conn4086598] end connection 10.0.216.90:33617 (1506 connections now open) 2016-03-13T10:01:41.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33633 #4086610 (1506 connections now open) 2016-03-13T10:01:41.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33634 #4086611 (1507 connections now open) 2016-03-13T10:01:42.488+0000 I NETWORK [conn4086600] end connection 10.0.20.53:55358 (1506 connections now open) 2016-03-13T10:01:42.488+0000 I NETWORK [conn4086601] end connection 10.0.20.53:55357 (1506 connections now open) 2016-03-13T10:01:42.497+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55366 #4086612 (1506 connections now open) 2016-03-13T10:01:42.497+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55367 #4086613 (1507 connections now open) 2016-03-13T10:01:44.971+0000 I NETWORK [conn4086603] end connection 10.0.179.209:45065 (1506 connections now open) 2016-03-13T10:01:44.971+0000 I NETWORK [conn4086602] end connection 10.0.179.209:45064 (1506 connections now open) 2016-03-13T10:01:44.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45083 #4086614 (1506 connections now open) 2016-03-13T10:01:44.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45084 #4086615 (1507 connections now open) 2016-03-13T10:01:45.549+0000 I NETWORK [conn4086604] end connection 10.0.26.69:46973 (1506 connections now open) 2016-03-13T10:01:45.549+0000 I NETWORK [conn4086605] end connection 10.0.26.69:46974 (1506 connections now open) 2016-03-13T10:01:45.550+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46984 #4086616 (1506 connections now open) 2016-03-13T10:01:45.550+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46985 #4086617 (1507 connections now open) 2016-03-13T10:01:46.118+0000 I NETWORK [conn4085428] end connection 10.0.27.237:39171 (1506 connections now open) 2016-03-13T10:01:46.118+0000 I NETWORK [conn4086577] end connection 10.0.26.69:46945 (1506 connections now open) 2016-03-13T10:01:46.118+0000 I NETWORK [conn4082904] end connection 10.0.216.90:58229 (1506 connections now open) 2016-03-13T10:01:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40441 #4086618 (1505 connections now open) 2016-03-13T10:01:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:46989 #4086619 (1506 connections now open) 2016-03-13T10:01:46.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33640 #4086620 (1507 connections now open) 2016-03-13T10:01:46.540+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:01:46.272+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:01:48.296+0000 I NETWORK [conn4086607] end connection 10.0.27.237:40432 (1506 connections now open) 2016-03-13T10:01:48.296+0000 I NETWORK [conn4086606] end connection 10.0.27.237:40433 (1506 connections now open) 2016-03-13T10:01:48.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40454 #4086621 (1506 connections now open) 2016-03-13T10:01:48.297+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40453 #4086622 (1507 connections now open) 2016-03-13T10:01:50.515+0000 I NETWORK [conn4086609] end connection 10.0.7.113:35618 (1506 connections now open) 2016-03-13T10:01:50.515+0000 I NETWORK [conn4086608] end connection 10.0.7.113:35617 (1506 connections now open) 2016-03-13T10:01:50.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35627 #4086623 (1506 connections now open) 2016-03-13T10:01:50.525+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35626 #4086624 (1507 connections now open) 2016-03-13T10:01:51.550+0000 I NETWORK [conn4086610] end connection 10.0.216.90:33633 (1506 connections now open) 2016-03-13T10:01:51.550+0000 I NETWORK [conn4086611] end connection 10.0.216.90:33634 (1506 connections now open) 2016-03-13T10:01:51.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33643 #4086625 (1506 connections now open) 2016-03-13T10:01:51.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33642 #4086626 (1507 connections now open) 2016-03-13T10:01:52.509+0000 I NETWORK [conn4086612] end connection 10.0.20.53:55366 (1506 connections now open) 2016-03-13T10:01:52.509+0000 I NETWORK [conn4086613] end connection 10.0.20.53:55367 (1505 connections now open) 2016-03-13T10:01:52.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55379 #4086627 (1506 connections now open) 2016-03-13T10:01:52.510+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55380 #4086628 (1507 connections now open) 2016-03-13T10:01:54.984+0000 I NETWORK [conn4086614] end connection 10.0.179.209:45083 (1506 connections now open) 2016-03-13T10:01:54.984+0000 I NETWORK [conn4086615] end connection 10.0.179.209:45084 (1506 connections now open) 2016-03-13T10:01:54.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45092 #4086629 (1506 connections now open) 2016-03-13T10:01:54.985+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45093 #4086630 (1507 connections now open) 2016-03-13T10:01:55.562+0000 I NETWORK [conn4086617] end connection 10.0.26.69:46985 (1506 connections now open) 2016-03-13T10:01:55.562+0000 I NETWORK [conn4086616] end connection 10.0.26.69:46984 (1506 connections now open) 2016-03-13T10:01:55.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47006 #4086631 (1506 connections now open) 2016-03-13T10:01:55.572+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47005 #4086632 (1507 connections now open) 2016-03-13T10:01:58.113+0000 I NETWORK [conn4085128] end connection 10.0.179.209:43482 (1506 connections now open) 2016-03-13T10:01:58.114+0000 I NETWORK [conn4083883] end connection 10.0.216.90:59079 (1505 connections now open) 2016-03-13T10:01:58.114+0000 I NETWORK [conn4081741] end connection 10.0.26.69:42817 (1504 connections now open) 2016-03-13T10:01:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45100 #4086633 (1505 connections now open) 2016-03-13T10:01:58.114+0000 I NETWORK [conn4081429] end connection 10.0.7.113:59675 (1504 connections now open) 2016-03-13T10:01:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33650 #4086634 (1506 connections now open) 2016-03-13T10:01:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47010 #4086635 (1506 connections now open) 2016-03-13T10:01:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35639 #4086636 (1507 connections now open) 2016-03-13T10:01:58.309+0000 I NETWORK [conn4086621] end connection 10.0.27.237:40454 (1506 connections now open) 2016-03-13T10:01:58.309+0000 I NETWORK [conn4086622] end connection 10.0.27.237:40453 (1506 connections now open) 2016-03-13T10:01:58.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40465 #4086637 (1506 connections now open) 2016-03-13T10:01:58.310+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40464 #4086638 (1507 connections now open) 2016-03-13T10:02:00.537+0000 I NETWORK [conn4086624] end connection 10.0.7.113:35626 (1506 connections now open) 2016-03-13T10:02:00.537+0000 I NETWORK [conn4086623] end connection 10.0.7.113:35627 (1506 connections now open) 2016-03-13T10:02:00.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35641 #4086639 (1506 connections now open) 2016-03-13T10:02:00.538+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35642 #4086640 (1507 connections now open) 2016-03-13T10:02:01.563+0000 I NETWORK [conn4086626] end connection 10.0.216.90:33642 (1506 connections now open) 2016-03-13T10:02:01.563+0000 I NETWORK [conn4086625] end connection 10.0.216.90:33643 (1506 connections now open) 2016-03-13T10:02:01.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33656 #4086641 (1506 connections now open) 2016-03-13T10:02:01.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33657 #4086642 (1507 connections now open) 2016-03-13T10:02:02.522+0000 I NETWORK [conn4086628] end connection 10.0.20.53:55380 (1506 connections now open) 2016-03-13T10:02:02.522+0000 I NETWORK [conn4086627] end connection 10.0.20.53:55379 (1506 connections now open) 2016-03-13T10:02:02.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55400 #4086643 (1506 connections now open) 2016-03-13T10:02:02.523+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55401 #4086644 (1507 connections now open) 2016-03-13T10:02:04.997+0000 I NETWORK [conn4086629] end connection 10.0.179.209:45092 (1506 connections now open) 2016-03-13T10:02:04.997+0000 I NETWORK [conn4086630] end connection 10.0.179.209:45093 (1505 connections now open) 2016-03-13T10:02:04.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45107 #4086645 (1506 connections now open) 2016-03-13T10:02:04.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45108 #4086646 (1507 connections now open) 2016-03-13T10:02:05.585+0000 I NETWORK [conn4086631] end connection 10.0.26.69:47006 (1506 connections now open) 2016-03-13T10:02:05.585+0000 I NETWORK [conn4086632] end connection 10.0.26.69:47005 (1506 connections now open) 2016-03-13T10:02:05.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47019 #4086647 (1506 connections now open) 2016-03-13T10:02:05.586+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47018 #4086648 (1507 connections now open) 2016-03-13T10:02:08.322+0000 I NETWORK [conn4086638] end connection 10.0.27.237:40464 (1506 connections now open) 2016-03-13T10:02:08.322+0000 I NETWORK [conn4086637] end connection 10.0.27.237:40465 (1506 connections now open) 2016-03-13T10:02:08.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40480 #4086649 (1506 connections now open) 2016-03-13T10:02:08.331+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40479 #4086650 (1507 connections now open) 2016-03-13T10:02:10.550+0000 I NETWORK [conn4086639] end connection 10.0.7.113:35641 (1506 connections now open) 2016-03-13T10:02:10.550+0000 I NETWORK [conn4086640] end connection 10.0.7.113:35642 (1506 connections now open) 2016-03-13T10:02:10.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35664 #4086651 (1506 connections now open) 2016-03-13T10:02:10.551+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35665 #4086652 (1507 connections now open) 2016-03-13T10:02:11.576+0000 I NETWORK [conn4086641] end connection 10.0.216.90:33656 (1506 connections now open) 2016-03-13T10:02:11.576+0000 I NETWORK [conn4086642] end connection 10.0.216.90:33657 (1506 connections now open) 2016-03-13T10:02:11.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33677 #4086653 (1506 connections now open) 2016-03-13T10:02:11.577+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33678 #4086654 (1507 connections now open) 2016-03-13T10:02:12.536+0000 I NETWORK [conn4086644] end connection 10.0.20.53:55401 (1506 connections now open) 2016-03-13T10:02:12.536+0000 I NETWORK [conn4086643] end connection 10.0.20.53:55400 (1506 connections now open) 2016-03-13T10:02:12.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55408 #4086655 (1506 connections now open) 2016-03-13T10:02:12.536+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55407 #4086656 (1507 connections now open) 2016-03-13T10:02:15.009+0000 I NETWORK [conn4086646] end connection 10.0.179.209:45108 (1506 connections now open) 2016-03-13T10:02:15.010+0000 I NETWORK [conn4086645] end connection 10.0.179.209:45107 (1505 connections now open) 2016-03-13T10:02:15.018+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45126 #4086657 (1506 connections now open) 2016-03-13T10:02:15.018+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45127 #4086658 (1507 connections now open) 2016-03-13T10:02:15.598+0000 I NETWORK [conn4086648] end connection 10.0.26.69:47018 (1506 connections now open) 2016-03-13T10:02:15.598+0000 I NETWORK [conn4086647] end connection 10.0.26.69:47019 (1506 connections now open) 2016-03-13T10:02:15.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47033 #4086659 (1506 connections now open) 2016-03-13T10:02:15.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47034 #4086660 (1507 connections now open) 2016-03-13T10:02:16.116+0000 I NETWORK [conn4082570] end connection 10.0.20.53:51397 (1506 connections now open) 2016-03-13T10:02:16.117+0000 I NETWORK [conn4083656] end connection 10.0.7.113:60821 (1505 connections now open) 2016-03-13T10:02:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55412 #4086661 (1506 connections now open) 2016-03-13T10:02:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35672 #4086662 (1507 connections now open) 2016-03-13T10:02:16.706+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:02:16.541+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:02:18.343+0000 I NETWORK [conn4086649] end connection 10.0.27.237:40480 (1506 connections now open) 2016-03-13T10:02:18.343+0000 I NETWORK [conn4086650] end connection 10.0.27.237:40479 (1506 connections now open) 2016-03-13T10:02:18.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40502 #4086663 (1506 connections now open) 2016-03-13T10:02:18.344+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40503 #4086664 (1507 connections now open) 2016-03-13T10:02:20.563+0000 I NETWORK [conn4086651] end connection 10.0.7.113:35664 (1506 connections now open) 2016-03-13T10:02:20.563+0000 I NETWORK [conn4086652] end connection 10.0.7.113:35665 (1506 connections now open) 2016-03-13T10:02:20.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35673 #4086665 (1506 connections now open) 2016-03-13T10:02:20.564+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35674 #4086666 (1507 connections now open) 2016-03-13T10:02:21.590+0000 I NETWORK [conn4086654] end connection 10.0.216.90:33678 (1506 connections now open) 2016-03-13T10:02:21.590+0000 I NETWORK [conn4086653] end connection 10.0.216.90:33677 (1506 connections now open) 2016-03-13T10:02:21.590+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33687 #4086667 (1506 connections now open) 2016-03-13T10:02:21.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33686 #4086668 (1507 connections now open) 2016-03-13T10:02:22.549+0000 I NETWORK [conn4086655] end connection 10.0.20.53:55408 (1506 connections now open) 2016-03-13T10:02:22.549+0000 I NETWORK [conn4086656] end connection 10.0.20.53:55407 (1506 connections now open) 2016-03-13T10:02:22.550+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55421 #4086669 (1506 connections now open) 2016-03-13T10:02:22.550+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55422 #4086670 (1507 connections now open) 2016-03-13T10:02:25.030+0000 I NETWORK [conn4086657] end connection 10.0.179.209:45126 (1506 connections now open) 2016-03-13T10:02:25.030+0000 I NETWORK [conn4086658] end connection 10.0.179.209:45127 (1505 connections now open) 2016-03-13T10:02:25.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45138 #4086671 (1506 connections now open) 2016-03-13T10:02:25.040+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45139 #4086672 (1507 connections now open) 2016-03-13T10:02:25.611+0000 I NETWORK [conn4086659] end connection 10.0.26.69:47033 (1506 connections now open) 2016-03-13T10:02:25.611+0000 I NETWORK [conn4086660] end connection 10.0.26.69:47034 (1505 connections now open) 2016-03-13T10:02:25.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47054 #4086673 (1507 connections now open) 2016-03-13T10:02:25.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47055 #4086674 (1507 connections now open) 2016-03-13T10:02:28.114+0000 I NETWORK [conn4081811] end connection 10.0.20.53:51100 (1506 connections now open) 2016-03-13T10:02:28.114+0000 I NETWORK [conn4082648] end connection 10.0.27.237:36493 (1505 connections now open) 2016-03-13T10:02:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55429 #4086675 (1506 connections now open) 2016-03-13T10:02:28.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40510 #4086676 (1507 connections now open) 2016-03-13T10:02:28.357+0000 I NETWORK [conn4086663] end connection 10.0.27.237:40502 (1506 connections now open) 2016-03-13T10:02:28.357+0000 I NETWORK [conn4086664] end connection 10.0.27.237:40503 (1506 connections now open) 2016-03-13T10:02:28.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40511 #4086677 (1506 connections now open) 2016-03-13T10:02:28.358+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40512 #4086678 (1507 connections now open) 2016-03-13T10:02:30.577+0000 I NETWORK [conn4086666] end connection 10.0.7.113:35674 (1506 connections now open) 2016-03-13T10:02:30.577+0000 I NETWORK [conn4086665] end connection 10.0.7.113:35673 (1506 connections now open) 2016-03-13T10:02:30.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35688 #4086679 (1506 connections now open) 2016-03-13T10:02:30.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35687 #4086680 (1507 connections now open) 2016-03-13T10:02:31.604+0000 I NETWORK [conn4086668] end connection 10.0.216.90:33686 (1506 connections now open) 2016-03-13T10:02:31.604+0000 I NETWORK [conn4086667] end connection 10.0.216.90:33687 (1506 connections now open) 2016-03-13T10:02:31.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33704 #4086681 (1506 connections now open) 2016-03-13T10:02:31.605+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33703 #4086682 (1507 connections now open) 2016-03-13T10:02:32.562+0000 I NETWORK [conn4086670] end connection 10.0.20.53:55422 (1506 connections now open) 2016-03-13T10:02:32.562+0000 I NETWORK [conn4086669] end connection 10.0.20.53:55421 (1506 connections now open) 2016-03-13T10:02:32.563+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55445 #4086683 (1506 connections now open) 2016-03-13T10:02:32.563+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55446 #4086684 (1507 connections now open) 2016-03-13T10:02:35.052+0000 I NETWORK [conn4086672] end connection 10.0.179.209:45139 (1506 connections now open) 2016-03-13T10:02:35.052+0000 I NETWORK [conn4086671] end connection 10.0.179.209:45138 (1506 connections now open) 2016-03-13T10:02:35.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45151 #4086685 (1506 connections now open) 2016-03-13T10:02:35.053+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45152 #4086686 (1507 connections now open) 2016-03-13T10:02:35.624+0000 I NETWORK [conn4086673] end connection 10.0.26.69:47054 (1506 connections now open) 2016-03-13T10:02:35.624+0000 I NETWORK [conn4086674] end connection 10.0.26.69:47055 (1506 connections now open) 2016-03-13T10:02:35.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47063 #4086687 (1506 connections now open) 2016-03-13T10:02:35.625+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47064 #4086688 (1507 connections now open) 2016-03-13T10:02:38.370+0000 I NETWORK [conn4086677] end connection 10.0.27.237:40511 (1506 connections now open) 2016-03-13T10:02:38.370+0000 I NETWORK [conn4086678] end connection 10.0.27.237:40512 (1505 connections now open) 2016-03-13T10:02:38.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40522 #4086689 (1506 connections now open) 2016-03-13T10:02:38.370+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40523 #4086690 (1507 connections now open) 2016-03-13T10:02:40.591+0000 I NETWORK [conn4086679] end connection 10.0.7.113:35688 (1506 connections now open) 2016-03-13T10:02:40.591+0000 I NETWORK [conn4086680] end connection 10.0.7.113:35687 (1506 connections now open) 2016-03-13T10:02:40.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35705 #4086691 (1506 connections now open) 2016-03-13T10:02:40.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35706 #4086692 (1507 connections now open) 2016-03-13T10:02:41.618+0000 I NETWORK [conn4086681] end connection 10.0.216.90:33704 (1506 connections now open) 2016-03-13T10:02:41.618+0000 I NETWORK [conn4086682] end connection 10.0.216.90:33703 (1506 connections now open) 2016-03-13T10:02:41.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33721 #4086693 (1506 connections now open) 2016-03-13T10:02:41.619+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33720 #4086694 (1507 connections now open) 2016-03-13T10:02:42.576+0000 I NETWORK [conn4086684] end connection 10.0.20.53:55446 (1506 connections now open) 2016-03-13T10:02:42.576+0000 I NETWORK [conn4086683] end connection 10.0.20.53:55445 (1506 connections now open) 2016-03-13T10:02:42.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55452 #4086695 (1506 connections now open) 2016-03-13T10:02:42.585+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55453 #4086696 (1507 connections now open) 2016-03-13T10:02:45.065+0000 I NETWORK [conn4086685] end connection 10.0.179.209:45151 (1506 connections now open) 2016-03-13T10:02:45.065+0000 I NETWORK [conn4086686] end connection 10.0.179.209:45152 (1505 connections now open) 2016-03-13T10:02:45.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45170 #4086697 (1506 connections now open) 2016-03-13T10:02:45.065+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45171 #4086698 (1507 connections now open) 2016-03-13T10:02:45.637+0000 I NETWORK [conn4086687] end connection 10.0.26.69:47063 (1506 connections now open) 2016-03-13T10:02:45.637+0000 I NETWORK [conn4086688] end connection 10.0.26.69:47064 (1506 connections now open) 2016-03-13T10:02:45.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47074 #4086699 (1506 connections now open) 2016-03-13T10:02:45.638+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47075 #4086700 (1507 connections now open) 2016-03-13T10:02:46.117+0000 I NETWORK [conn4081956] end connection 10.0.179.209:40883 (1506 connections now open) 2016-03-13T10:02:46.117+0000 I NETWORK [conn4081243] end connection 10.0.216.90:57572 (1505 connections now open) 2016-03-13T10:02:46.117+0000 I NETWORK [conn4081788] end connection 10.0.27.237:36133 (1504 connections now open) 2016-03-13T10:02:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45173 #4086701 (1505 connections now open) 2016-03-13T10:02:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40531 #4086702 (1506 connections now open) 2016-03-13T10:02:46.126+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33728 #4086703 (1507 connections now open) 2016-03-13T10:02:46.950+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:02:46.706+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:02:48.383+0000 I NETWORK [conn4086690] end connection 10.0.27.237:40523 (1506 connections now open) 2016-03-13T10:02:48.383+0000 I NETWORK [conn4086689] end connection 10.0.27.237:40522 (1506 connections now open) 2016-03-13T10:02:48.383+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40542 #4086704 (1506 connections now open) 2016-03-13T10:02:48.384+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40543 #4086705 (1507 connections now open) 2016-03-13T10:02:50.604+0000 I NETWORK [conn4086692] end connection 10.0.7.113:35706 (1506 connections now open) 2016-03-13T10:02:50.604+0000 I NETWORK [conn4086691] end connection 10.0.7.113:35705 (1506 connections now open) 2016-03-13T10:02:50.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35716 #4086706 (1506 connections now open) 2016-03-13T10:02:50.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35717 #4086707 (1507 connections now open) 2016-03-13T10:02:51.632+0000 I NETWORK [conn4086693] end connection 10.0.216.90:33721 (1506 connections now open) 2016-03-13T10:02:51.632+0000 I NETWORK [conn4086694] end connection 10.0.216.90:33720 (1506 connections now open) 2016-03-13T10:02:51.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33730 #4086708 (1506 connections now open) 2016-03-13T10:02:51.632+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33731 #4086709 (1507 connections now open) 2016-03-13T10:02:52.598+0000 I NETWORK [conn4086695] end connection 10.0.20.53:55452 (1506 connections now open) 2016-03-13T10:02:52.598+0000 I NETWORK [conn4086696] end connection 10.0.20.53:55453 (1506 connections now open) 2016-03-13T10:02:52.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55465 #4086710 (1506 connections now open) 2016-03-13T10:02:52.599+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55466 #4086711 (1507 connections now open) 2016-03-13T10:02:55.078+0000 I NETWORK [conn4086698] end connection 10.0.179.209:45171 (1506 connections now open) 2016-03-13T10:02:55.078+0000 I NETWORK [conn4086697] end connection 10.0.179.209:45170 (1506 connections now open) 2016-03-13T10:02:55.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45180 #4086712 (1506 connections now open) 2016-03-13T10:02:55.078+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45179 #4086713 (1507 connections now open) 2016-03-13T10:02:55.651+0000 I NETWORK [conn4086700] end connection 10.0.26.69:47075 (1506 connections now open) 2016-03-13T10:02:55.651+0000 I NETWORK [conn4086699] end connection 10.0.26.69:47074 (1506 connections now open) 2016-03-13T10:02:55.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47097 #4086714 (1506 connections now open) 2016-03-13T10:02:55.660+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47096 #4086715 (1507 connections now open) 2016-03-13T10:02:58.113+0000 I NETWORK [conn4082061] end connection 10.0.20.53:51206 (1506 connections now open) 2016-03-13T10:02:58.113+0000 I NETWORK [conn4080949] end connection 10.0.216.90:57505 (1506 connections now open) 2016-03-13T10:02:58.113+0000 I NETWORK [conn4081763] end connection 10.0.26.69:42830 (1505 connections now open) 2016-03-13T10:02:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55473 #4086716 (1505 connections now open) 2016-03-13T10:02:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33737 #4086717 (1506 connections now open) 2016-03-13T10:02:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47100 #4086718 (1507 connections now open) 2016-03-13T10:02:58.396+0000 I NETWORK [conn4086705] end connection 10.0.27.237:40543 (1506 connections now open) 2016-03-13T10:02:58.396+0000 I NETWORK [conn4086704] end connection 10.0.27.237:40542 (1506 connections now open) 2016-03-13T10:02:58.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40552 #4086719 (1506 connections now open) 2016-03-13T10:02:58.396+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40553 #4086720 (1507 connections now open) 2016-03-13T10:03:00.626+0000 I NETWORK [conn4086706] end connection 10.0.7.113:35716 (1506 connections now open) 2016-03-13T10:03:00.626+0000 I NETWORK [conn4086707] end connection 10.0.7.113:35717 (1506 connections now open) 2016-03-13T10:03:00.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35731 #4086721 (1506 connections now open) 2016-03-13T10:03:00.627+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35732 #4086722 (1507 connections now open) 2016-03-13T10:03:01.645+0000 I NETWORK [conn4086709] end connection 10.0.216.90:33731 (1506 connections now open) 2016-03-13T10:03:01.645+0000 I NETWORK [conn4086708] end connection 10.0.216.90:33730 (1505 connections now open) 2016-03-13T10:03:01.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33743 #4086723 (1506 connections now open) 2016-03-13T10:03:01.646+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33744 #4086724 (1507 connections now open) 2016-03-13T10:03:02.612+0000 I NETWORK [conn4086710] end connection 10.0.20.53:55465 (1506 connections now open) 2016-03-13T10:03:02.612+0000 I NETWORK [conn4086711] end connection 10.0.20.53:55466 (1506 connections now open) 2016-03-13T10:03:02.612+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55488 #4086725 (1506 connections now open) 2016-03-13T10:03:02.613+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55487 #4086726 (1507 connections now open) 2016-03-13T10:03:05.091+0000 I NETWORK [conn4086713] end connection 10.0.179.209:45179 (1506 connections now open) 2016-03-13T10:03:05.091+0000 I NETWORK [conn4086712] end connection 10.0.179.209:45180 (1506 connections now open) 2016-03-13T10:03:05.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45196 #4086727 (1506 connections now open) 2016-03-13T10:03:05.092+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45197 #4086728 (1507 connections now open) 2016-03-13T10:03:05.673+0000 I NETWORK [conn4086715] end connection 10.0.26.69:47096 (1506 connections now open) 2016-03-13T10:03:05.673+0000 I NETWORK [conn4086714] end connection 10.0.26.69:47097 (1506 connections now open) 2016-03-13T10:03:05.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47109 #4086729 (1506 connections now open) 2016-03-13T10:03:05.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47110 #4086730 (1507 connections now open) 2016-03-13T10:03:08.409+0000 I NETWORK [conn4086719] end connection 10.0.27.237:40552 (1506 connections now open) 2016-03-13T10:03:08.409+0000 I NETWORK [conn4086720] end connection 10.0.27.237:40553 (1506 connections now open) 2016-03-13T10:03:08.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40563 #4086731 (1506 connections now open) 2016-03-13T10:03:08.410+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40564 #4086732 (1507 connections now open) 2016-03-13T10:03:10.639+0000 I NETWORK [conn4086721] end connection 10.0.7.113:35731 (1506 connections now open) 2016-03-13T10:03:10.639+0000 I NETWORK [conn4086722] end connection 10.0.7.113:35732 (1506 connections now open) 2016-03-13T10:03:10.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35755 #4086733 (1506 connections now open) 2016-03-13T10:03:10.640+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35754 #4086734 (1507 connections now open) 2016-03-13T10:03:11.659+0000 I NETWORK [conn4086723] end connection 10.0.216.90:33743 (1506 connections now open) 2016-03-13T10:03:11.659+0000 I NETWORK [conn4086724] end connection 10.0.216.90:33744 (1506 connections now open) 2016-03-13T10:03:11.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33763 #4086735 (1506 connections now open) 2016-03-13T10:03:11.659+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33762 #4086736 (1507 connections now open) 2016-03-13T10:03:12.625+0000 I NETWORK [conn4086726] end connection 10.0.20.53:55487 (1506 connections now open) 2016-03-13T10:03:12.626+0000 I NETWORK [conn4086725] end connection 10.0.20.53:55488 (1505 connections now open) 2016-03-13T10:03:12.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55495 #4086737 (1506 connections now open) 2016-03-13T10:03:12.626+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55496 #4086738 (1507 connections now open) 2016-03-13T10:03:15.104+0000 I NETWORK [conn4086728] end connection 10.0.179.209:45197 (1506 connections now open) 2016-03-13T10:03:15.104+0000 I NETWORK [conn4086727] end connection 10.0.179.209:45196 (1506 connections now open) 2016-03-13T10:03:15.105+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45215 #4086739 (1506 connections now open) 2016-03-13T10:03:15.105+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45216 #4086740 (1507 connections now open) 2016-03-13T10:03:15.686+0000 I NETWORK [conn4086729] end connection 10.0.26.69:47109 (1506 connections now open) 2016-03-13T10:03:15.686+0000 I NETWORK [conn4086730] end connection 10.0.26.69:47110 (1506 connections now open) 2016-03-13T10:03:15.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47123 #4086741 (1506 connections now open) 2016-03-13T10:03:15.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47124 #4086742 (1507 connections now open) 2016-03-13T10:03:16.116+0000 I NETWORK [conn4082602] end connection 10.0.179.209:41128 (1506 connections now open) 2016-03-13T10:03:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45219 #4086743 (1507 connections now open) 2016-03-13T10:03:16.117+0000 I NETWORK [conn4081897] end connection 10.0.27.237:36182 (1506 connections now open) 2016-03-13T10:03:16.117+0000 I NETWORK [conn4081982] end connection 10.0.7.113:59810 (1506 connections now open) 2016-03-13T10:03:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40575 #4086744 (1506 connections now open) 2016-03-13T10:03:16.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35762 #4086745 (1507 connections now open) 2016-03-13T10:03:17.166+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:03:16.950+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:03:18.422+0000 I NETWORK [conn4086732] end connection 10.0.27.237:40564 (1506 connections now open) 2016-03-13T10:03:18.422+0000 I NETWORK [conn4086731] end connection 10.0.27.237:40563 (1506 connections now open) 2016-03-13T10:03:18.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40585 #4086746 (1506 connections now open) 2016-03-13T10:03:18.423+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40586 #4086747 (1507 connections now open) 2016-03-13T10:03:20.652+0000 I NETWORK [conn4086734] end connection 10.0.7.113:35754 (1506 connections now open) 2016-03-13T10:03:20.653+0000 I NETWORK [conn4086733] end connection 10.0.7.113:35755 (1506 connections now open) 2016-03-13T10:03:20.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35764 #4086748 (1506 connections now open) 2016-03-13T10:03:20.653+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35765 #4086749 (1507 connections now open) 2016-03-13T10:03:21.672+0000 I NETWORK [conn4086735] end connection 10.0.216.90:33763 (1506 connections now open) 2016-03-13T10:03:21.672+0000 I NETWORK [conn4086736] end connection 10.0.216.90:33762 (1506 connections now open) 2016-03-13T10:03:21.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33772 #4086750 (1506 connections now open) 2016-03-13T10:03:21.673+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33773 #4086751 (1507 connections now open) 2016-03-13T10:03:22.638+0000 I NETWORK [conn4086737] end connection 10.0.20.53:55495 (1506 connections now open) 2016-03-13T10:03:22.638+0000 I NETWORK [conn4086738] end connection 10.0.20.53:55496 (1506 connections now open) 2016-03-13T10:03:22.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55508 #4086752 (1506 connections now open) 2016-03-13T10:03:22.639+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55507 #4086753 (1507 connections now open) 2016-03-13T10:03:25.118+0000 I NETWORK [conn4086739] end connection 10.0.179.209:45215 (1506 connections now open) 2016-03-13T10:03:25.118+0000 I NETWORK [conn4086740] end connection 10.0.179.209:45216 (1506 connections now open) 2016-03-13T10:03:25.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45228 #4086754 (1506 connections now open) 2016-03-13T10:03:25.127+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45227 #4086755 (1507 connections now open) 2016-03-13T10:03:25.699+0000 I NETWORK [conn4086741] end connection 10.0.26.69:47123 (1506 connections now open) 2016-03-13T10:03:25.699+0000 I NETWORK [conn4086742] end connection 10.0.26.69:47124 (1506 connections now open) 2016-03-13T10:03:25.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47145 #4086756 (1507 connections now open) 2016-03-13T10:03:25.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47146 #4086757 (1507 connections now open) 2016-03-13T10:03:28.115+0000 I NETWORK [conn4081367] end connection 10.0.179.209:40740 (1506 connections now open) 2016-03-13T10:03:28.115+0000 I NETWORK [conn4080952] end connection 10.0.216.90:57506 (1506 connections now open) 2016-03-13T10:03:28.115+0000 I NETWORK [conn4082145] end connection 10.0.20.53:51250 (1506 connections now open) 2016-03-13T10:03:28.115+0000 I NETWORK [conn4085847] end connection 10.0.7.113:34818 (1503 connections now open) 2016-03-13T10:03:28.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45236 #4086758 (1504 connections now open) 2016-03-13T10:03:28.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55516 #4086759 (1505 connections now open) 2016-03-13T10:03:28.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33778 #4086760 (1506 connections now open) 2016-03-13T10:03:28.116+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35777 #4086761 (1507 connections now open) 2016-03-13T10:03:28.435+0000 I NETWORK [conn4086746] end connection 10.0.27.237:40585 (1506 connections now open) 2016-03-13T10:03:28.435+0000 I NETWORK [conn4086747] end connection 10.0.27.237:40586 (1506 connections now open) 2016-03-13T10:03:28.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40595 #4086762 (1506 connections now open) 2016-03-13T10:03:28.436+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40596 #4086763 (1507 connections now open) 2016-03-13T10:03:30.666+0000 I NETWORK [conn4086748] end connection 10.0.7.113:35764 (1506 connections now open) 2016-03-13T10:03:30.666+0000 I NETWORK [conn4086749] end connection 10.0.7.113:35765 (1506 connections now open) 2016-03-13T10:03:30.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35779 #4086764 (1506 connections now open) 2016-03-13T10:03:30.666+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35780 #4086765 (1507 connections now open) 2016-03-13T10:03:31.686+0000 I NETWORK [conn4086751] end connection 10.0.216.90:33773 (1506 connections now open) 2016-03-13T10:03:31.686+0000 I NETWORK [conn4086750] end connection 10.0.216.90:33772 (1506 connections now open) 2016-03-13T10:03:31.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33787 #4086766 (1506 connections now open) 2016-03-13T10:03:31.687+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33786 #4086767 (1507 connections now open) 2016-03-13T10:03:32.652+0000 I NETWORK [conn4086752] end connection 10.0.20.53:55508 (1506 connections now open) 2016-03-13T10:03:32.652+0000 I NETWORK [conn4086753] end connection 10.0.20.53:55507 (1506 connections now open) 2016-03-13T10:03:32.652+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55533 #4086768 (1506 connections now open) 2016-03-13T10:03:32.652+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55532 #4086769 (1507 connections now open) 2016-03-13T10:03:35.140+0000 I NETWORK [conn4086755] end connection 10.0.179.209:45227 (1506 connections now open) 2016-03-13T10:03:35.140+0000 I NETWORK [conn4086754] end connection 10.0.179.209:45228 (1506 connections now open) 2016-03-13T10:03:35.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45241 #4086770 (1506 connections now open) 2016-03-13T10:03:35.140+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45242 #4086771 (1507 connections now open) 2016-03-13T10:03:35.712+0000 I NETWORK [conn4086757] end connection 10.0.26.69:47146 (1506 connections now open) 2016-03-13T10:03:35.712+0000 I NETWORK [conn4086756] end connection 10.0.26.69:47145 (1506 connections now open) 2016-03-13T10:03:35.713+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47156 #4086772 (1506 connections now open) 2016-03-13T10:03:35.713+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47157 #4086773 (1507 connections now open) 2016-03-13T10:03:38.449+0000 I NETWORK [conn4086762] end connection 10.0.27.237:40595 (1506 connections now open) 2016-03-13T10:03:38.449+0000 I NETWORK [conn4086763] end connection 10.0.27.237:40596 (1506 connections now open) 2016-03-13T10:03:38.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40606 #4086774 (1506 connections now open) 2016-03-13T10:03:38.449+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40605 #4086775 (1507 connections now open) 2016-03-13T10:03:40.679+0000 I NETWORK [conn4086765] end connection 10.0.7.113:35780 (1506 connections now open) 2016-03-13T10:03:40.679+0000 I NETWORK [conn4086764] end connection 10.0.7.113:35779 (1506 connections now open) 2016-03-13T10:03:40.679+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35797 #4086776 (1506 connections now open) 2016-03-13T10:03:40.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35798 #4086777 (1507 connections now open) 2016-03-13T10:03:41.699+0000 I NETWORK [conn4086767] end connection 10.0.216.90:33786 (1506 connections now open) 2016-03-13T10:03:41.699+0000 I NETWORK [conn4086766] end connection 10.0.216.90:33787 (1506 connections now open) 2016-03-13T10:03:41.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33805 #4086778 (1506 connections now open) 2016-03-13T10:03:41.700+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33804 #4086779 (1507 connections now open) 2016-03-13T10:03:42.665+0000 I NETWORK [conn4086768] end connection 10.0.20.53:55533 (1506 connections now open) 2016-03-13T10:03:42.665+0000 I NETWORK [conn4086769] end connection 10.0.20.53:55532 (1506 connections now open) 2016-03-13T10:03:42.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55540 #4086780 (1506 connections now open) 2016-03-13T10:03:42.675+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55541 #4086781 (1507 connections now open) 2016-03-13T10:03:45.153+0000 I NETWORK [conn4086771] end connection 10.0.179.209:45242 (1506 connections now open) 2016-03-13T10:03:45.153+0000 I NETWORK [conn4086770] end connection 10.0.179.209:45241 (1506 connections now open) 2016-03-13T10:03:45.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45260 #4086782 (1506 connections now open) 2016-03-13T10:03:45.154+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45259 #4086783 (1507 connections now open) 2016-03-13T10:03:45.726+0000 I NETWORK [conn4086772] end connection 10.0.26.69:47156 (1506 connections now open) 2016-03-13T10:03:45.726+0000 I NETWORK [conn4086773] end connection 10.0.26.69:47157 (1506 connections now open) 2016-03-13T10:03:45.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47168 #4086784 (1506 connections now open) 2016-03-13T10:03:45.726+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47167 #4086785 (1507 connections now open) 2016-03-13T10:03:46.116+0000 I NETWORK [conn4084493] end connection 10.0.216.90:59695 (1506 connections now open) 2016-03-13T10:03:46.117+0000 I NETWORK [conn4082041] end connection 10.0.27.237:36251 (1505 connections now open) 2016-03-13T10:03:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40614 #4086786 (1506 connections now open) 2016-03-13T10:03:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33811 #4086787 (1507 connections now open) 2016-03-13T10:03:47.364+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:03:47.166+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:03:48.462+0000 I NETWORK [conn4086774] end connection 10.0.27.237:40606 (1506 connections now open) 2016-03-13T10:03:48.462+0000 I NETWORK [conn4086775] end connection 10.0.27.237:40605 (1506 connections now open) 2016-03-13T10:03:48.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40615 #4086788 (1506 connections now open) 2016-03-13T10:03:48.462+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40616 #4086789 (1507 connections now open) 2016-03-13T10:03:50.692+0000 I NETWORK [conn4086776] end connection 10.0.7.113:35797 (1506 connections now open) 2016-03-13T10:03:50.692+0000 I NETWORK [conn4086777] end connection 10.0.7.113:35798 (1506 connections now open) 2016-03-13T10:03:50.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35807 #4086790 (1506 connections now open) 2016-03-13T10:03:50.702+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35806 #4086791 (1507 connections now open) 2016-03-13T10:03:51.712+0000 I NETWORK [conn4086779] end connection 10.0.216.90:33804 (1506 connections now open) 2016-03-13T10:03:51.713+0000 I NETWORK [conn4086778] end connection 10.0.216.90:33805 (1505 connections now open) 2016-03-13T10:03:51.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33814 #4086792 (1506 connections now open) 2016-03-13T10:03:51.722+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33813 #4086793 (1507 connections now open) 2016-03-13T10:03:52.687+0000 I NETWORK [conn4086781] end connection 10.0.20.53:55541 (1506 connections now open) 2016-03-13T10:03:52.687+0000 I NETWORK [conn4086780] end connection 10.0.20.53:55540 (1506 connections now open) 2016-03-13T10:03:52.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55553 #4086794 (1506 connections now open) 2016-03-13T10:03:52.688+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55554 #4086795 (1507 connections now open) 2016-03-13T10:03:55.166+0000 I NETWORK [conn4086782] end connection 10.0.179.209:45260 (1506 connections now open) 2016-03-13T10:03:55.166+0000 I NETWORK [conn4086783] end connection 10.0.179.209:45259 (1506 connections now open) 2016-03-13T10:03:55.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45270 #4086796 (1506 connections now open) 2016-03-13T10:03:55.167+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45271 #4086797 (1507 connections now open) 2016-03-13T10:03:55.739+0000 I NETWORK [conn4086785] end connection 10.0.26.69:47167 (1506 connections now open) 2016-03-13T10:03:55.739+0000 I NETWORK [conn4086784] end connection 10.0.26.69:47168 (1506 connections now open) 2016-03-13T10:03:55.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47189 #4086798 (1506 connections now open) 2016-03-13T10:03:55.748+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47190 #4086799 (1507 connections now open) 2016-03-13T10:03:58.120+0000 I NETWORK [conn4085639] end connection 10.0.179.209:44019 (1506 connections now open) 2016-03-13T10:03:58.120+0000 I NETWORK [conn4081951] end connection 10.0.27.237:36200 (1506 connections now open) 2016-03-13T10:03:58.120+0000 I NETWORK [conn4082038] end connection 10.0.7.113:59831 (1506 connections now open) 2016-03-13T10:03:58.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45278 #4086800 (1505 connections now open) 2016-03-13T10:03:58.121+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35818 #4086801 (1506 connections now open) 2016-03-13T10:03:58.129+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40633 #4086802 (1507 connections now open) 2016-03-13T10:03:58.475+0000 I NETWORK [conn4086788] end connection 10.0.27.237:40615 (1506 connections now open) 2016-03-13T10:03:58.475+0000 I NETWORK [conn4086789] end connection 10.0.27.237:40616 (1506 connections now open) 2016-03-13T10:03:58.475+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40634 #4086803 (1506 connections now open) 2016-03-13T10:03:58.475+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40635 #4086804 (1507 connections now open) 2016-03-13T10:04:00.714+0000 I NETWORK [conn4086790] end connection 10.0.7.113:35807 (1506 connections now open) 2016-03-13T10:04:00.714+0000 I NETWORK [conn4086791] end connection 10.0.7.113:35806 (1506 connections now open) 2016-03-13T10:04:00.714+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35820 #4086805 (1506 connections now open) 2016-03-13T10:04:00.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35821 #4086806 (1507 connections now open) 2016-03-13T10:04:01.734+0000 I NETWORK [conn4086792] end connection 10.0.216.90:33814 (1506 connections now open) 2016-03-13T10:04:01.734+0000 I NETWORK [conn4086793] end connection 10.0.216.90:33813 (1506 connections now open) 2016-03-13T10:04:01.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33829 #4086807 (1506 connections now open) 2016-03-13T10:04:01.735+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33830 #4086808 (1507 connections now open) 2016-03-13T10:04:02.701+0000 I NETWORK [conn4086795] end connection 10.0.20.53:55554 (1506 connections now open) 2016-03-13T10:04:02.701+0000 I NETWORK [conn4086794] end connection 10.0.20.53:55553 (1506 connections now open) 2016-03-13T10:04:02.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55575 #4086809 (1506 connections now open) 2016-03-13T10:04:02.701+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55576 #4086810 (1507 connections now open) 2016-03-13T10:04:05.179+0000 I NETWORK [conn4086796] end connection 10.0.179.209:45270 (1506 connections now open) 2016-03-13T10:04:05.179+0000 I NETWORK [conn4086797] end connection 10.0.179.209:45271 (1506 connections now open) 2016-03-13T10:04:05.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45287 #4086811 (1506 connections now open) 2016-03-13T10:04:05.180+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45286 #4086812 (1507 connections now open) 2016-03-13T10:04:05.761+0000 I NETWORK [conn4086799] end connection 10.0.26.69:47190 (1506 connections now open) 2016-03-13T10:04:05.761+0000 I NETWORK [conn4086798] end connection 10.0.26.69:47189 (1506 connections now open) 2016-03-13T10:04:05.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47201 #4086813 (1506 connections now open) 2016-03-13T10:04:05.761+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47202 #4086814 (1507 connections now open) 2016-03-13T10:04:08.488+0000 I NETWORK [conn4086803] end connection 10.0.27.237:40634 (1506 connections now open) 2016-03-13T10:04:08.488+0000 I NETWORK [conn4086804] end connection 10.0.27.237:40635 (1506 connections now open) 2016-03-13T10:04:08.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40643 #4086815 (1506 connections now open) 2016-03-13T10:04:08.488+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40642 #4086816 (1507 connections now open) 2016-03-13T10:04:10.727+0000 I NETWORK [conn4086806] end connection 10.0.7.113:35821 (1506 connections now open) 2016-03-13T10:04:10.727+0000 I NETWORK [conn4086805] end connection 10.0.7.113:35820 (1506 connections now open) 2016-03-13T10:04:10.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35842 #4086817 (1506 connections now open) 2016-03-13T10:04:10.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35843 #4086818 (1507 connections now open) 2016-03-13T10:04:11.748+0000 I NETWORK [conn4086807] end connection 10.0.216.90:33829 (1506 connections now open) 2016-03-13T10:04:11.748+0000 I NETWORK [conn4086808] end connection 10.0.216.90:33830 (1506 connections now open) 2016-03-13T10:04:11.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33849 #4086819 (1506 connections now open) 2016-03-13T10:04:11.749+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33848 #4086820 (1507 connections now open) 2016-03-13T10:04:12.714+0000 I NETWORK [conn4086810] end connection 10.0.20.53:55576 (1506 connections now open) 2016-03-13T10:04:12.714+0000 I NETWORK [conn4086809] end connection 10.0.20.53:55575 (1506 connections now open) 2016-03-13T10:04:12.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55582 #4086821 (1506 connections now open) 2016-03-13T10:04:12.715+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55583 #4086822 (1507 connections now open) 2016-03-13T10:04:15.193+0000 I NETWORK [conn4086811] end connection 10.0.179.209:45287 (1506 connections now open) 2016-03-13T10:04:15.193+0000 I NETWORK [conn4086812] end connection 10.0.179.209:45286 (1506 connections now open) 2016-03-13T10:04:15.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45304 #4086823 (1506 connections now open) 2016-03-13T10:04:15.194+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45305 #4086824 (1507 connections now open) 2016-03-13T10:04:15.773+0000 I NETWORK [conn4086813] end connection 10.0.26.69:47201 (1506 connections now open) 2016-03-13T10:04:15.773+0000 I NETWORK [conn4086814] end connection 10.0.26.69:47202 (1506 connections now open) 2016-03-13T10:04:15.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47216 #4086825 (1506 connections now open) 2016-03-13T10:04:15.774+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47215 #4086826 (1507 connections now open) 2016-03-13T10:04:16.116+0000 I NETWORK [conn4083020] end connection 10.0.26.69:43358 (1506 connections now open) 2016-03-13T10:04:16.116+0000 I NETWORK [conn4083822] end connection 10.0.216.90:59020 (1505 connections now open) 2016-03-13T10:04:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47219 #4086827 (1506 connections now open) 2016-03-13T10:04:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33855 #4086828 (1507 connections now open) 2016-03-13T10:04:17.639+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:04:17.364+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:04:18.500+0000 I NETWORK [conn4086815] end connection 10.0.27.237:40643 (1506 connections now open) 2016-03-13T10:04:18.500+0000 I NETWORK [conn4086816] end connection 10.0.27.237:40642 (1506 connections now open) 2016-03-13T10:04:18.501+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40660 #4086829 (1506 connections now open) 2016-03-13T10:04:18.501+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40659 #4086830 (1507 connections now open) 2016-03-13T10:04:20.741+0000 I NETWORK [conn4086817] end connection 10.0.7.113:35842 (1506 connections now open) 2016-03-13T10:04:20.741+0000 I NETWORK [conn4086818] end connection 10.0.7.113:35843 (1506 connections now open) 2016-03-13T10:04:20.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35853 #4086831 (1506 connections now open) 2016-03-13T10:04:20.742+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35854 #4086832 (1507 connections now open) 2016-03-13T10:04:21.762+0000 I NETWORK [conn4086819] end connection 10.0.216.90:33849 (1506 connections now open) 2016-03-13T10:04:21.762+0000 I NETWORK [conn4086820] end connection 10.0.216.90:33848 (1506 connections now open) 2016-03-13T10:04:21.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33859 #4086833 (1506 connections now open) 2016-03-13T10:04:21.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33858 #4086834 (1507 connections now open) 2016-03-13T10:04:22.727+0000 I NETWORK [conn4086821] end connection 10.0.20.53:55582 (1506 connections now open) 2016-03-13T10:04:22.727+0000 I NETWORK [conn4086822] end connection 10.0.20.53:55583 (1506 connections now open) 2016-03-13T10:04:22.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55595 #4086835 (1506 connections now open) 2016-03-13T10:04:22.728+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55596 #4086836 (1507 connections now open) 2016-03-13T10:04:25.206+0000 I NETWORK [conn4086824] end connection 10.0.179.209:45305 (1506 connections now open) 2016-03-13T10:04:25.206+0000 I NETWORK [conn4086823] end connection 10.0.179.209:45304 (1506 connections now open) 2016-03-13T10:04:25.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45316 #4086837 (1506 connections now open) 2016-03-13T10:04:25.215+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45317 #4086838 (1507 connections now open) 2016-03-13T10:04:25.787+0000 I NETWORK [conn4086825] end connection 10.0.26.69:47216 (1506 connections now open) 2016-03-13T10:04:25.787+0000 I NETWORK [conn4086826] end connection 10.0.26.69:47215 (1506 connections now open) 2016-03-13T10:04:25.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47238 #4086839 (1507 connections now open) 2016-03-13T10:04:25.787+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47237 #4086840 (1507 connections now open) 2016-03-13T10:04:28.114+0000 I NETWORK [conn4082244] end connection 10.0.27.237:36342 (1506 connections now open) 2016-03-13T10:04:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40677 #4086841 (1507 connections now open) 2016-03-13T10:04:28.513+0000 I NETWORK [conn4086829] end connection 10.0.27.237:40660 (1506 connections now open) 2016-03-13T10:04:28.513+0000 I NETWORK [conn4086830] end connection 10.0.27.237:40659 (1506 connections now open) 2016-03-13T10:04:28.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40679 #4086842 (1506 connections now open) 2016-03-13T10:04:28.514+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40678 #4086843 (1507 connections now open) 2016-03-13T10:04:30.755+0000 I NETWORK [conn4086831] end connection 10.0.7.113:35853 (1506 connections now open) 2016-03-13T10:04:30.755+0000 I NETWORK [conn4086832] end connection 10.0.7.113:35854 (1506 connections now open) 2016-03-13T10:04:30.755+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35868 #4086844 (1506 connections now open) 2016-03-13T10:04:30.756+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35869 #4086845 (1507 connections now open) 2016-03-13T10:04:31.776+0000 I NETWORK [conn4086834] end connection 10.0.216.90:33858 (1506 connections now open) 2016-03-13T10:04:31.776+0000 I NETWORK [conn4086833] end connection 10.0.216.90:33859 (1506 connections now open) 2016-03-13T10:04:31.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33874 #4086846 (1506 connections now open) 2016-03-13T10:04:31.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33875 #4086847 (1507 connections now open) 2016-03-13T10:04:32.740+0000 I NETWORK [conn4086836] end connection 10.0.20.53:55596 (1506 connections now open) 2016-03-13T10:04:32.740+0000 I NETWORK [conn4086835] end connection 10.0.20.53:55595 (1506 connections now open) 2016-03-13T10:04:32.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55621 #4086848 (1506 connections now open) 2016-03-13T10:04:32.741+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55620 #4086849 (1507 connections now open) 2016-03-13T10:04:35.227+0000 I NETWORK [conn4086838] end connection 10.0.179.209:45317 (1506 connections now open) 2016-03-13T10:04:35.227+0000 I NETWORK [conn4086837] end connection 10.0.179.209:45316 (1506 connections now open) 2016-03-13T10:04:35.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45331 #4086850 (1506 connections now open) 2016-03-13T10:04:35.228+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45330 #4086851 (1507 connections now open) 2016-03-13T10:04:35.800+0000 I NETWORK [conn4086840] end connection 10.0.26.69:47237 (1506 connections now open) 2016-03-13T10:04:35.800+0000 I NETWORK [conn4086839] end connection 10.0.26.69:47238 (1506 connections now open) 2016-03-13T10:04:35.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47247 #4086852 (1506 connections now open) 2016-03-13T10:04:35.800+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47246 #4086853 (1507 connections now open) 2016-03-13T10:04:38.526+0000 I NETWORK [conn4086843] end connection 10.0.27.237:40678 (1506 connections now open) 2016-03-13T10:04:38.526+0000 I NETWORK [conn4086842] end connection 10.0.27.237:40679 (1506 connections now open) 2016-03-13T10:04:38.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40685 #4086854 (1506 connections now open) 2016-03-13T10:04:38.526+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40686 #4086855 (1507 connections now open) 2016-03-13T10:04:40.768+0000 I NETWORK [conn4086845] end connection 10.0.7.113:35869 (1506 connections now open) 2016-03-13T10:04:40.768+0000 I NETWORK [conn4086844] end connection 10.0.7.113:35868 (1506 connections now open) 2016-03-13T10:04:40.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35886 #4086856 (1506 connections now open) 2016-03-13T10:04:40.769+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35887 #4086857 (1507 connections now open) 2016-03-13T10:04:41.790+0000 I NETWORK [conn4086846] end connection 10.0.216.90:33874 (1506 connections now open) 2016-03-13T10:04:41.790+0000 I NETWORK [conn4086847] end connection 10.0.216.90:33875 (1506 connections now open) 2016-03-13T10:04:41.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33892 #4086858 (1506 connections now open) 2016-03-13T10:04:41.791+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33891 #4086859 (1507 connections now open) 2016-03-13T10:04:42.754+0000 I NETWORK [conn4086848] end connection 10.0.20.53:55621 (1506 connections now open) 2016-03-13T10:04:42.754+0000 I NETWORK [conn4086849] end connection 10.0.20.53:55620 (1506 connections now open) 2016-03-13T10:04:42.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55629 #4086860 (1506 connections now open) 2016-03-13T10:04:42.763+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55630 #4086861 (1507 connections now open) 2016-03-13T10:04:45.241+0000 I NETWORK [conn4086850] end connection 10.0.179.209:45331 (1506 connections now open) 2016-03-13T10:04:45.241+0000 I NETWORK [conn4086851] end connection 10.0.179.209:45330 (1506 connections now open) 2016-03-13T10:04:45.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45348 #4086862 (1506 connections now open) 2016-03-13T10:04:45.241+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45349 #4086863 (1507 connections now open) 2016-03-13T10:04:45.813+0000 I NETWORK [conn4086853] end connection 10.0.26.69:47246 (1506 connections now open) 2016-03-13T10:04:45.813+0000 I NETWORK [conn4086852] end connection 10.0.26.69:47247 (1506 connections now open) 2016-03-13T10:04:45.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47259 #4086864 (1506 connections now open) 2016-03-13T10:04:45.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47260 #4086865 (1507 connections now open) 2016-03-13T10:04:46.117+0000 I NETWORK [conn4082065] end connection 10.0.216.90:57841 (1506 connections now open) 2016-03-13T10:04:46.117+0000 I NETWORK [conn4085413] end connection 10.0.7.113:34374 (1506 connections now open) 2016-03-13T10:04:46.117+0000 I NETWORK [conn4081699] end connection 10.0.20.53:51044 (1506 connections now open) 2016-03-13T10:04:46.117+0000 I NETWORK [conn4084204] end connection 10.0.26.69:44502 (1506 connections now open) 2016-03-13T10:04:46.118+0000 I NETWORK [conn4082701] end connection 10.0.27.237:36525 (1502 connections now open) 2016-03-13T10:04:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47263 #4086866 (1503 connections now open) 2016-03-13T10:04:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33899 #4086867 (1504 connections now open) 2016-03-13T10:04:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55634 #4086868 (1505 connections now open) 2016-03-13T10:04:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40697 #4086869 (1506 connections now open) 2016-03-13T10:04:46.118+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35895 #4086870 (1507 connections now open) 2016-03-13T10:04:47.782+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:04:47.639+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:04:48.539+0000 I NETWORK [conn4086855] end connection 10.0.27.237:40686 (1506 connections now open) 2016-03-13T10:04:48.539+0000 I NETWORK [conn4086854] end connection 10.0.27.237:40685 (1506 connections now open) 2016-03-13T10:04:48.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40700 #4086871 (1506 connections now open) 2016-03-13T10:04:48.539+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40699 #4086872 (1507 connections now open) 2016-03-13T10:04:50.782+0000 I NETWORK [conn4086856] end connection 10.0.7.113:35886 (1506 connections now open) 2016-03-13T10:04:50.782+0000 I NETWORK [conn4086857] end connection 10.0.7.113:35887 (1506 connections now open) 2016-03-13T10:04:50.791+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35898 #4086873 (1506 connections now open) 2016-03-13T10:04:50.791+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35897 #4086874 (1507 connections now open) 2016-03-13T10:04:51.804+0000 I NETWORK [conn4086858] end connection 10.0.216.90:33892 (1506 connections now open) 2016-03-13T10:04:51.804+0000 I NETWORK [conn4086859] end connection 10.0.216.90:33891 (1506 connections now open) 2016-03-13T10:04:51.813+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33903 #4086875 (1506 connections now open) 2016-03-13T10:04:51.814+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33902 #4086876 (1507 connections now open) 2016-03-13T10:04:52.776+0000 I NETWORK [conn4086861] end connection 10.0.20.53:55630 (1506 connections now open) 2016-03-13T10:04:52.776+0000 I NETWORK [conn4086860] end connection 10.0.20.53:55629 (1506 connections now open) 2016-03-13T10:04:52.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55642 #4086877 (1506 connections now open) 2016-03-13T10:04:52.777+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55643 #4086878 (1507 connections now open) 2016-03-13T10:04:55.254+0000 I NETWORK [conn4086862] end connection 10.0.179.209:45348 (1506 connections now open) 2016-03-13T10:04:55.254+0000 I NETWORK [conn4086863] end connection 10.0.179.209:45349 (1506 connections now open) 2016-03-13T10:04:55.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45356 #4086879 (1506 connections now open) 2016-03-13T10:04:55.255+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45357 #4086880 (1507 connections now open) 2016-03-13T10:04:55.826+0000 I NETWORK [conn4086865] end connection 10.0.26.69:47260 (1506 connections now open) 2016-03-13T10:04:55.826+0000 I NETWORK [conn4086864] end connection 10.0.26.69:47259 (1506 connections now open) 2016-03-13T10:04:55.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47281 #4086881 (1506 connections now open) 2016-03-13T10:04:55.834+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47280 #4086882 (1507 connections now open) 2016-03-13T10:04:58.114+0000 I NETWORK [conn4082478] end connection 10.0.20.53:51366 (1506 connections now open) 2016-03-13T10:04:58.114+0000 I NETWORK [conn4082638] end connection 10.0.216.90:58089 (1505 connections now open) 2016-03-13T10:04:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33909 #4086883 (1506 connections now open) 2016-03-13T10:04:58.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55649 #4086884 (1507 connections now open) 2016-03-13T10:04:58.115+0000 I NETWORK [conn4086676] end connection 10.0.27.237:40510 (1506 connections now open) 2016-03-13T10:04:58.124+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40717 #4086885 (1507 connections now open) 2016-03-13T10:04:58.551+0000 I NETWORK [conn4086871] end connection 10.0.27.237:40700 (1506 connections now open) 2016-03-13T10:04:58.551+0000 I NETWORK [conn4086872] end connection 10.0.27.237:40699 (1506 connections now open) 2016-03-13T10:04:58.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40718 #4086886 (1506 connections now open) 2016-03-13T10:04:58.552+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40719 #4086887 (1507 connections now open) 2016-03-13T10:05:00.804+0000 I NETWORK [conn4086873] end connection 10.0.7.113:35898 (1506 connections now open) 2016-03-13T10:05:00.804+0000 I NETWORK [conn4086874] end connection 10.0.7.113:35897 (1506 connections now open) 2016-03-13T10:05:00.804+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35912 #4086888 (1506 connections now open) 2016-03-13T10:05:00.804+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35913 #4086889 (1507 connections now open) 2016-03-13T10:05:01.827+0000 I NETWORK [conn4086875] end connection 10.0.216.90:33903 (1506 connections now open) 2016-03-13T10:05:01.827+0000 I NETWORK [conn4086876] end connection 10.0.216.90:33902 (1506 connections now open) 2016-03-13T10:05:01.828+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33919 #4086890 (1506 connections now open) 2016-03-13T10:05:01.828+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33918 #4086891 (1507 connections now open) 2016-03-13T10:05:02.790+0000 I NETWORK [conn4086878] end connection 10.0.20.53:55643 (1506 connections now open) 2016-03-13T10:05:02.790+0000 I NETWORK [conn4086877] end connection 10.0.20.53:55642 (1506 connections now open) 2016-03-13T10:05:02.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55664 #4086892 (1506 connections now open) 2016-03-13T10:05:02.790+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55665 #4086893 (1507 connections now open) 2016-03-13T10:05:05.267+0000 I NETWORK [conn4086879] end connection 10.0.179.209:45356 (1506 connections now open) 2016-03-13T10:05:05.267+0000 I NETWORK [conn4086880] end connection 10.0.179.209:45357 (1506 connections now open) 2016-03-13T10:05:05.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45374 #4086894 (1506 connections now open) 2016-03-13T10:05:05.268+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45373 #4086895 (1507 connections now open) 2016-03-13T10:05:05.848+0000 I NETWORK [conn4086881] end connection 10.0.26.69:47281 (1506 connections now open) 2016-03-13T10:05:05.848+0000 I NETWORK [conn4086882] end connection 10.0.26.69:47280 (1506 connections now open) 2016-03-13T10:05:05.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47294 #4086896 (1506 connections now open) 2016-03-13T10:05:05.848+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47293 #4086897 (1507 connections now open) 2016-03-13T10:05:08.564+0000 I NETWORK [conn4086886] end connection 10.0.27.237:40718 (1506 connections now open) 2016-03-13T10:05:08.564+0000 I NETWORK [conn4086887] end connection 10.0.27.237:40719 (1506 connections now open) 2016-03-13T10:05:08.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40728 #4086898 (1506 connections now open) 2016-03-13T10:05:08.565+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40729 #4086899 (1507 connections now open) 2016-03-13T10:05:10.817+0000 I NETWORK [conn4086889] end connection 10.0.7.113:35913 (1506 connections now open) 2016-03-13T10:05:10.817+0000 I NETWORK [conn4086888] end connection 10.0.7.113:35912 (1506 connections now open) 2016-03-13T10:05:10.818+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35937 #4086900 (1506 connections now open) 2016-03-13T10:05:10.818+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35938 #4086901 (1507 connections now open) 2016-03-13T10:05:11.841+0000 I NETWORK [conn4086891] end connection 10.0.216.90:33918 (1506 connections now open) 2016-03-13T10:05:11.841+0000 I NETWORK [conn4086890] end connection 10.0.216.90:33919 (1506 connections now open) 2016-03-13T10:05:11.841+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33939 #4086902 (1506 connections now open) 2016-03-13T10:05:11.842+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33940 #4086903 (1507 connections now open) 2016-03-13T10:05:12.803+0000 I NETWORK [conn4086892] end connection 10.0.20.53:55664 (1506 connections now open) 2016-03-13T10:05:12.803+0000 I NETWORK [conn4086893] end connection 10.0.20.53:55665 (1506 connections now open) 2016-03-13T10:05:12.804+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55673 #4086904 (1506 connections now open) 2016-03-13T10:05:12.804+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55674 #4086905 (1507 connections now open) 2016-03-13T10:05:15.279+0000 I NETWORK [conn4086894] end connection 10.0.179.209:45374 (1506 connections now open) 2016-03-13T10:05:15.279+0000 I NETWORK [conn4086895] end connection 10.0.179.209:45373 (1506 connections now open) 2016-03-13T10:05:15.280+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45392 #4086906 (1506 connections now open) 2016-03-13T10:05:15.280+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45391 #4086907 (1507 connections now open) 2016-03-13T10:05:15.861+0000 I NETWORK [conn4086897] end connection 10.0.26.69:47293 (1506 connections now open) 2016-03-13T10:05:15.861+0000 I NETWORK [conn4086896] end connection 10.0.26.69:47294 (1506 connections now open) 2016-03-13T10:05:15.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47309 #4086908 (1506 connections now open) 2016-03-13T10:05:15.862+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47310 #4086909 (1507 connections now open) 2016-03-13T10:05:16.116+0000 I NETWORK [conn4083612] end connection 10.0.216.90:58808 (1506 connections now open) 2016-03-13T10:05:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33947 #4086910 (1507 connections now open) 2016-03-13T10:05:16.118+0000 I NETWORK [conn4081888] end connection 10.0.7.113:59781 (1506 connections now open) 2016-03-13T10:05:16.119+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35945 #4086911 (1507 connections now open) 2016-03-13T10:05:18.048+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:05:17.783+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:05:18.577+0000 I NETWORK [conn4086898] end connection 10.0.27.237:40728 (1506 connections now open) 2016-03-13T10:05:18.577+0000 I NETWORK [conn4086899] end connection 10.0.27.237:40729 (1506 connections now open) 2016-03-13T10:05:18.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40746 #4086912 (1506 connections now open) 2016-03-13T10:05:18.578+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40747 #4086913 (1507 connections now open) 2016-03-13T10:05:20.831+0000 I NETWORK [conn4086901] end connection 10.0.7.113:35938 (1506 connections now open) 2016-03-13T10:05:20.831+0000 I NETWORK [conn4086900] end connection 10.0.7.113:35937 (1506 connections now open) 2016-03-13T10:05:20.831+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35948 #4086914 (1506 connections now open) 2016-03-13T10:05:20.831+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35947 #4086915 (1507 connections now open) 2016-03-13T10:05:21.854+0000 I NETWORK [conn4086902] end connection 10.0.216.90:33939 (1506 connections now open) 2016-03-13T10:05:21.854+0000 I NETWORK [conn4086903] end connection 10.0.216.90:33940 (1506 connections now open) 2016-03-13T10:05:21.855+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33950 #4086916 (1506 connections now open) 2016-03-13T10:05:21.855+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33951 #4086917 (1507 connections now open) 2016-03-13T10:05:22.816+0000 I NETWORK [conn4086905] end connection 10.0.20.53:55674 (1506 connections now open) 2016-03-13T10:05:22.816+0000 I NETWORK [conn4086904] end connection 10.0.20.53:55673 (1506 connections now open) 2016-03-13T10:05:22.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55685 #4086918 (1506 connections now open) 2016-03-13T10:05:22.817+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55686 #4086919 (1507 connections now open) 2016-03-13T10:05:25.292+0000 I NETWORK [conn4086906] end connection 10.0.179.209:45392 (1506 connections now open) 2016-03-13T10:05:25.292+0000 I NETWORK [conn4086907] end connection 10.0.179.209:45391 (1506 connections now open) 2016-03-13T10:05:25.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45404 #4086920 (1506 connections now open) 2016-03-13T10:05:25.302+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45405 #4086921 (1507 connections now open) 2016-03-13T10:05:25.874+0000 I NETWORK [conn4086908] end connection 10.0.26.69:47309 (1506 connections now open) 2016-03-13T10:05:25.874+0000 I NETWORK [conn4086909] end connection 10.0.26.69:47310 (1506 connections now open) 2016-03-13T10:05:25.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47330 #4086922 (1506 connections now open) 2016-03-13T10:05:25.875+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47331 #4086923 (1507 connections now open) 2016-03-13T10:05:28.114+0000 I NETWORK [conn4081862] end connection 10.0.20.53:51126 (1506 connections now open) 2016-03-13T10:05:28.114+0000 I NETWORK [conn4085254] end connection 10.0.179.209:43614 (1506 connections now open) 2016-03-13T10:05:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55694 #4086924 (1506 connections now open) 2016-03-13T10:05:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45412 #4086925 (1507 connections now open) 2016-03-13T10:05:28.590+0000 I NETWORK [conn4086913] end connection 10.0.27.237:40747 (1506 connections now open) 2016-03-13T10:05:28.590+0000 I NETWORK [conn4086912] end connection 10.0.27.237:40746 (1506 connections now open) 2016-03-13T10:05:28.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40765 #4086926 (1506 connections now open) 2016-03-13T10:05:28.591+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40766 #4086927 (1507 connections now open) 2016-03-13T10:05:30.844+0000 I NETWORK [conn4086915] end connection 10.0.7.113:35947 (1506 connections now open) 2016-03-13T10:05:30.844+0000 I NETWORK [conn4086914] end connection 10.0.7.113:35948 (1506 connections now open) 2016-03-13T10:05:30.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35962 #4086928 (1506 connections now open) 2016-03-13T10:05:30.845+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35963 #4086929 (1507 connections now open) 2016-03-13T10:05:31.867+0000 I NETWORK [conn4086917] end connection 10.0.216.90:33951 (1506 connections now open) 2016-03-13T10:05:31.867+0000 I NETWORK [conn4086916] end connection 10.0.216.90:33950 (1506 connections now open) 2016-03-13T10:05:31.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33967 #4086930 (1506 connections now open) 2016-03-13T10:05:31.868+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33966 #4086931 (1507 connections now open) 2016-03-13T10:05:32.830+0000 I NETWORK [conn4086919] end connection 10.0.20.53:55686 (1506 connections now open) 2016-03-13T10:05:32.830+0000 I NETWORK [conn4086918] end connection 10.0.20.53:55685 (1506 connections now open) 2016-03-13T10:05:32.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55709 #4086932 (1506 connections now open) 2016-03-13T10:05:32.830+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55710 #4086933 (1507 connections now open) 2016-03-13T10:05:35.314+0000 I NETWORK [conn4086921] end connection 10.0.179.209:45405 (1506 connections now open) 2016-03-13T10:05:35.314+0000 I NETWORK [conn4086920] end connection 10.0.179.209:45404 (1506 connections now open) 2016-03-13T10:05:35.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45418 #4086934 (1506 connections now open) 2016-03-13T10:05:35.315+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45419 #4086935 (1507 connections now open) 2016-03-13T10:05:35.888+0000 I NETWORK [conn4086922] end connection 10.0.26.69:47330 (1506 connections now open) 2016-03-13T10:05:35.888+0000 I NETWORK [conn4086923] end connection 10.0.26.69:47331 (1506 connections now open) 2016-03-13T10:05:35.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47341 #4086936 (1506 connections now open) 2016-03-13T10:05:35.889+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47340 #4086937 (1507 connections now open) 2016-03-13T10:05:38.603+0000 I NETWORK [conn4086927] end connection 10.0.27.237:40766 (1506 connections now open) 2016-03-13T10:05:38.603+0000 I NETWORK [conn4086926] end connection 10.0.27.237:40765 (1506 connections now open) 2016-03-13T10:05:38.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40771 #4086938 (1506 connections now open) 2016-03-13T10:05:38.603+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40770 #4086939 (1507 connections now open) 2016-03-13T10:05:40.858+0000 I NETWORK [conn4086929] end connection 10.0.7.113:35963 (1506 connections now open) 2016-03-13T10:05:40.858+0000 I NETWORK [conn4086928] end connection 10.0.7.113:35962 (1506 connections now open) 2016-03-13T10:05:40.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35980 #4086940 (1506 connections now open) 2016-03-13T10:05:40.859+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35981 #4086941 (1507 connections now open) 2016-03-13T10:05:41.881+0000 I NETWORK [conn4086931] end connection 10.0.216.90:33966 (1506 connections now open) 2016-03-13T10:05:41.881+0000 I NETWORK [conn4086930] end connection 10.0.216.90:33967 (1506 connections now open) 2016-03-13T10:05:41.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33985 #4086942 (1506 connections now open) 2016-03-13T10:05:41.882+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33986 #4086943 (1507 connections now open) 2016-03-13T10:05:42.843+0000 I NETWORK [conn4086932] end connection 10.0.20.53:55709 (1506 connections now open) 2016-03-13T10:05:42.843+0000 I NETWORK [conn4086933] end connection 10.0.20.53:55710 (1506 connections now open) 2016-03-13T10:05:42.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55718 #4086944 (1506 connections now open) 2016-03-13T10:05:42.851+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55717 #4086945 (1507 connections now open) 2016-03-13T10:05:45.327+0000 I NETWORK [conn4086935] end connection 10.0.179.209:45419 (1506 connections now open) 2016-03-13T10:05:45.327+0000 I NETWORK [conn4086934] end connection 10.0.179.209:45418 (1506 connections now open) 2016-03-13T10:05:45.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45435 #4086946 (1506 connections now open) 2016-03-13T10:05:45.328+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45436 #4086947 (1507 connections now open) 2016-03-13T10:05:45.902+0000 I NETWORK [conn4086937] end connection 10.0.26.69:47340 (1506 connections now open) 2016-03-13T10:05:45.902+0000 I NETWORK [conn4086936] end connection 10.0.26.69:47341 (1506 connections now open) 2016-03-13T10:05:45.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47353 #4086948 (1506 connections now open) 2016-03-13T10:05:45.902+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47354 #4086949 (1507 connections now open) 2016-03-13T10:05:46.116+0000 I NETWORK [conn4082460] end connection 10.0.216.90:58010 (1506 connections now open) 2016-03-13T10:05:46.116+0000 I NETWORK [conn4082177] end connection 10.0.26.69:42995 (1506 connections now open) 2016-03-13T10:05:46.116+0000 I NETWORK [conn4081890] end connection 10.0.7.113:59783 (1504 connections now open) 2016-03-13T10:05:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33993 #4086950 (1505 connections now open) 2016-03-13T10:05:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47356 #4086951 (1506 connections now open) 2016-03-13T10:05:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35988 #4086952 (1507 connections now open) 2016-03-13T10:05:48.201+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:05:48.048+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:05:48.616+0000 I NETWORK [conn4086939] end connection 10.0.27.237:40770 (1506 connections now open) 2016-03-13T10:05:48.616+0000 I NETWORK [conn4086938] end connection 10.0.27.237:40771 (1506 connections now open) 2016-03-13T10:05:48.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40785 #4086953 (1506 connections now open) 2016-03-13T10:05:48.616+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40786 #4086954 (1507 connections now open) 2016-03-13T10:05:50.871+0000 I NETWORK [conn4086940] end connection 10.0.7.113:35980 (1506 connections now open) 2016-03-13T10:05:50.871+0000 I NETWORK [conn4086941] end connection 10.0.7.113:35981 (1506 connections now open) 2016-03-13T10:05:50.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35991 #4086955 (1506 connections now open) 2016-03-13T10:05:50.881+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:35990 #4086956 (1507 connections now open) 2016-03-13T10:05:51.895+0000 I NETWORK [conn4086942] end connection 10.0.216.90:33985 (1506 connections now open) 2016-03-13T10:05:51.895+0000 I NETWORK [conn4086943] end connection 10.0.216.90:33986 (1506 connections now open) 2016-03-13T10:05:51.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33997 #4086957 (1506 connections now open) 2016-03-13T10:05:51.905+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:33996 #4086958 (1507 connections now open) 2016-03-13T10:05:52.864+0000 I NETWORK [conn4086945] end connection 10.0.20.53:55717 (1506 connections now open) 2016-03-13T10:05:52.864+0000 I NETWORK [conn4086944] end connection 10.0.20.53:55718 (1506 connections now open) 2016-03-13T10:05:52.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55731 #4086959 (1506 connections now open) 2016-03-13T10:05:52.865+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55730 #4086960 (1507 connections now open) 2016-03-13T10:05:55.340+0000 I NETWORK [conn4086946] end connection 10.0.179.209:45435 (1506 connections now open) 2016-03-13T10:05:55.340+0000 I NETWORK [conn4086947] end connection 10.0.179.209:45436 (1506 connections now open) 2016-03-13T10:05:55.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45445 #4086961 (1506 connections now open) 2016-03-13T10:05:55.341+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45446 #4086962 (1507 connections now open) 2016-03-13T10:05:55.915+0000 I NETWORK [conn4086949] end connection 10.0.26.69:47354 (1506 connections now open) 2016-03-13T10:05:55.915+0000 I NETWORK [conn4086948] end connection 10.0.26.69:47353 (1505 connections now open) 2016-03-13T10:05:55.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47374 #4086963 (1506 connections now open) 2016-03-13T10:05:55.925+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47373 #4086964 (1507 connections now open) 2016-03-13T10:05:58.114+0000 I NETWORK [conn4082402] end connection 10.0.216.90:57993 (1506 connections now open) 2016-03-13T10:05:58.114+0000 I NETWORK [conn4082096] end connection 10.0.27.237:36271 (1506 connections now open) 2016-03-13T10:05:58.114+0000 I NETWORK [conn4082018] end connection 10.0.26.69:42931 (1506 connections now open) 2016-03-13T10:05:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47378 #4086965 (1505 connections now open) 2016-03-13T10:05:58.115+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34004 #4086966 (1506 connections now open) 2016-03-13T10:05:58.123+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40801 #4086967 (1507 connections now open) 2016-03-13T10:05:58.628+0000 I NETWORK [conn4086953] end connection 10.0.27.237:40785 (1506 connections now open) 2016-03-13T10:05:58.628+0000 I NETWORK [conn4086954] end connection 10.0.27.237:40786 (1506 connections now open) 2016-03-13T10:05:58.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40802 #4086968 (1506 connections now open) 2016-03-13T10:05:58.629+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40803 #4086969 (1507 connections now open) 2016-03-13T10:06:00.894+0000 I NETWORK [conn4086956] end connection 10.0.7.113:35990 (1506 connections now open) 2016-03-13T10:06:00.894+0000 I NETWORK [conn4086955] end connection 10.0.7.113:35991 (1506 connections now open) 2016-03-13T10:06:00.894+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36004 #4086970 (1506 connections now open) 2016-03-13T10:06:00.894+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36005 #4086971 (1507 connections now open) 2016-03-13T10:06:01.919+0000 I NETWORK [conn4086957] end connection 10.0.216.90:33997 (1506 connections now open) 2016-03-13T10:06:01.919+0000 I NETWORK [conn4086958] end connection 10.0.216.90:33996 (1506 connections now open) 2016-03-13T10:06:01.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34013 #4086972 (1506 connections now open) 2016-03-13T10:06:01.919+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34012 #4086973 (1507 connections now open) 2016-03-13T10:06:02.877+0000 I NETWORK [conn4086960] end connection 10.0.20.53:55730 (1506 connections now open) 2016-03-13T10:06:02.877+0000 I NETWORK [conn4086959] end connection 10.0.20.53:55731 (1506 connections now open) 2016-03-13T10:06:02.878+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55751 #4086974 (1506 connections now open) 2016-03-13T10:06:02.878+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55752 #4086975 (1507 connections now open) 2016-03-13T10:06:05.353+0000 I NETWORK [conn4086961] end connection 10.0.179.209:45445 (1506 connections now open) 2016-03-13T10:06:05.353+0000 I NETWORK [conn4086962] end connection 10.0.179.209:45446 (1506 connections now open) 2016-03-13T10:06:05.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45462 #4086976 (1506 connections now open) 2016-03-13T10:06:05.354+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45461 #4086977 (1507 connections now open) 2016-03-13T10:06:05.937+0000 I NETWORK [conn4086963] end connection 10.0.26.69:47374 (1506 connections now open) 2016-03-13T10:06:05.937+0000 I NETWORK [conn4086964] end connection 10.0.26.69:47373 (1506 connections now open) 2016-03-13T10:06:05.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47387 #4086978 (1506 connections now open) 2016-03-13T10:06:05.938+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47386 #4086979 (1507 connections now open) 2016-03-13T10:06:08.641+0000 I NETWORK [conn4086968] end connection 10.0.27.237:40802 (1506 connections now open) 2016-03-13T10:06:08.641+0000 I NETWORK [conn4086969] end connection 10.0.27.237:40803 (1506 connections now open) 2016-03-13T10:06:08.641+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40811 #4086980 (1506 connections now open) 2016-03-13T10:06:08.642+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40812 #4086981 (1507 connections now open) 2016-03-13T10:06:10.908+0000 I NETWORK [conn4086970] end connection 10.0.7.113:36004 (1506 connections now open) 2016-03-13T10:06:10.908+0000 I NETWORK [conn4086971] end connection 10.0.7.113:36005 (1506 connections now open) 2016-03-13T10:06:10.908+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36028 #4086982 (1506 connections now open) 2016-03-13T10:06:10.909+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36027 #4086983 (1507 connections now open) 2016-03-13T10:06:11.932+0000 I NETWORK [conn4086972] end connection 10.0.216.90:34013 (1506 connections now open) 2016-03-13T10:06:11.932+0000 I NETWORK [conn4086973] end connection 10.0.216.90:34012 (1506 connections now open) 2016-03-13T10:06:11.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34032 #4086984 (1506 connections now open) 2016-03-13T10:06:11.933+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34031 #4086985 (1507 connections now open) 2016-03-13T10:06:12.890+0000 I NETWORK [conn4086974] end connection 10.0.20.53:55751 (1506 connections now open) 2016-03-13T10:06:12.890+0000 I NETWORK [conn4086975] end connection 10.0.20.53:55752 (1506 connections now open) 2016-03-13T10:06:12.891+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55759 #4086986 (1506 connections now open) 2016-03-13T10:06:12.891+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55758 #4086987 (1507 connections now open) 2016-03-13T10:06:15.366+0000 I NETWORK [conn4086976] end connection 10.0.179.209:45462 (1506 connections now open) 2016-03-13T10:06:15.366+0000 I NETWORK [conn4086977] end connection 10.0.179.209:45461 (1506 connections now open) 2016-03-13T10:06:15.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45479 #4086988 (1506 connections now open) 2016-03-13T10:06:15.367+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45480 #4086989 (1507 connections now open) 2016-03-13T10:06:15.951+0000 I NETWORK [conn4086978] end connection 10.0.26.69:47387 (1506 connections now open) 2016-03-13T10:06:15.951+0000 I NETWORK [conn4086979] end connection 10.0.26.69:47386 (1506 connections now open) 2016-03-13T10:06:15.952+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47403 #4086990 (1506 connections now open) 2016-03-13T10:06:15.952+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47402 #4086991 (1507 connections now open) 2016-03-13T10:06:16.116+0000 I NETWORK [conn4082469] end connection 10.0.20.53:51360 (1506 connections now open) 2016-03-13T10:06:16.117+0000 I NETWORK [conn4084452] end connection 10.0.26.69:44759 (1505 connections now open) 2016-03-13T10:06:16.117+0000 I NETWORK [conn4082752] end connection 10.0.27.237:36555 (1505 connections now open) 2016-03-13T10:06:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55763 #4086992 (1505 connections now open) 2016-03-13T10:06:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47405 #4086993 (1506 connections now open) 2016-03-13T10:06:16.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40825 #4086994 (1507 connections now open) 2016-03-13T10:06:18.479+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:06:18.202+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:06:18.654+0000 I NETWORK [conn4086980] end connection 10.0.27.237:40811 (1506 connections now open) 2016-03-13T10:06:18.654+0000 I NETWORK [conn4086981] end connection 10.0.27.237:40812 (1506 connections now open) 2016-03-13T10:06:18.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40828 #4086995 (1506 connections now open) 2016-03-13T10:06:18.654+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40827 #4086996 (1507 connections now open) 2016-03-13T10:06:20.922+0000 I NETWORK [conn4086983] end connection 10.0.7.113:36027 (1506 connections now open) 2016-03-13T10:06:20.922+0000 I NETWORK [conn4086982] end connection 10.0.7.113:36028 (1506 connections now open) 2016-03-13T10:06:20.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36039 #4086997 (1507 connections now open) 2016-03-13T10:06:20.922+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36038 #4086998 (1507 connections now open) 2016-03-13T10:06:21.946+0000 I NETWORK [conn4086984] end connection 10.0.216.90:34032 (1506 connections now open) 2016-03-13T10:06:21.946+0000 I NETWORK [conn4086985] end connection 10.0.216.90:34031 (1506 connections now open) 2016-03-13T10:06:21.947+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34043 #4086999 (1506 connections now open) 2016-03-13T10:06:21.947+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34042 #4087000 (1507 connections now open) 2016-03-13T10:06:22.904+0000 I NETWORK [conn4086986] end connection 10.0.20.53:55759 (1506 connections now open) 2016-03-13T10:06:22.904+0000 I NETWORK [conn4086987] end connection 10.0.20.53:55758 (1506 connections now open) 2016-03-13T10:06:22.904+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55773 #4087001 (1506 connections now open) 2016-03-13T10:06:22.904+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55774 #4087002 (1507 connections now open) 2016-03-13T10:06:25.379+0000 I NETWORK [conn4086989] end connection 10.0.179.209:45480 (1506 connections now open) 2016-03-13T10:06:25.379+0000 I NETWORK [conn4086988] end connection 10.0.179.209:45479 (1506 connections now open) 2016-03-13T10:06:25.381+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45490 #4087003 (1506 connections now open) 2016-03-13T10:06:25.381+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45489 #4087004 (1507 connections now open) 2016-03-13T10:06:25.965+0000 I NETWORK [conn4086990] end connection 10.0.26.69:47403 (1506 connections now open) 2016-03-13T10:06:25.965+0000 I NETWORK [conn4086991] end connection 10.0.26.69:47402 (1506 connections now open) 2016-03-13T10:06:25.965+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47422 #4087005 (1506 connections now open) 2016-03-13T10:06:25.965+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47423 #4087006 (1507 connections now open) 2016-03-13T10:06:28.114+0000 I NETWORK [conn4081834] end connection 10.0.179.209:40818 (1506 connections now open) 2016-03-13T10:06:28.114+0000 I NETWORK [conn4081851] end connection 10.0.20.53:51120 (1505 connections now open) 2016-03-13T10:06:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45494 #4087007 (1506 connections now open) 2016-03-13T10:06:28.114+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55781 #4087008 (1507 connections now open) 2016-03-13T10:06:28.667+0000 I NETWORK [conn4086996] end connection 10.0.27.237:40827 (1506 connections now open) 2016-03-13T10:06:28.667+0000 I NETWORK [conn4086995] end connection 10.0.27.237:40828 (1506 connections now open) 2016-03-13T10:06:28.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40846 #4087009 (1506 connections now open) 2016-03-13T10:06:28.667+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40847 #4087010 (1507 connections now open) 2016-03-13T10:06:30.935+0000 I NETWORK [conn4086998] end connection 10.0.7.113:36038 (1506 connections now open) 2016-03-13T10:06:30.935+0000 I NETWORK [conn4086997] end connection 10.0.7.113:36039 (1506 connections now open) 2016-03-13T10:06:30.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36052 #4087011 (1506 connections now open) 2016-03-13T10:06:30.936+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36053 #4087012 (1507 connections now open) 2016-03-13T10:06:31.960+0000 I NETWORK [conn4086999] end connection 10.0.216.90:34043 (1506 connections now open) 2016-03-13T10:06:31.960+0000 I NETWORK [conn4087000] end connection 10.0.216.90:34042 (1506 connections now open) 2016-03-13T10:06:31.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34059 #4087013 (1506 connections now open) 2016-03-13T10:06:31.961+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34060 #4087014 (1507 connections now open) 2016-03-13T10:06:32.917+0000 I NETWORK [conn4087002] end connection 10.0.20.53:55774 (1506 connections now open) 2016-03-13T10:06:32.917+0000 I NETWORK [conn4087001] end connection 10.0.20.53:55773 (1506 connections now open) 2016-03-13T10:06:32.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55796 #4087015 (1506 connections now open) 2016-03-13T10:06:32.917+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55797 #4087016 (1507 connections now open) 2016-03-13T10:06:35.393+0000 I NETWORK [conn4087003] end connection 10.0.179.209:45490 (1506 connections now open) 2016-03-13T10:06:35.393+0000 I NETWORK [conn4087004] end connection 10.0.179.209:45489 (1506 connections now open) 2016-03-13T10:06:35.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45504 #4087017 (1506 connections now open) 2016-03-13T10:06:35.394+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45505 #4087018 (1507 connections now open) 2016-03-13T10:06:35.978+0000 I NETWORK [conn4087006] end connection 10.0.26.69:47423 (1506 connections now open) 2016-03-13T10:06:35.978+0000 I NETWORK [conn4087005] end connection 10.0.26.69:47422 (1505 connections now open) 2016-03-13T10:06:35.978+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47433 #4087019 (1506 connections now open) 2016-03-13T10:06:35.979+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47434 #4087020 (1507 connections now open) 2016-03-13T10:06:38.679+0000 I NETWORK [conn4087010] end connection 10.0.27.237:40847 (1506 connections now open) 2016-03-13T10:06:38.679+0000 I NETWORK [conn4087009] end connection 10.0.27.237:40846 (1506 connections now open) 2016-03-13T10:06:38.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40854 #4087021 (1506 connections now open) 2016-03-13T10:06:38.680+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40853 #4087022 (1507 connections now open) 2016-03-13T10:06:40.949+0000 I NETWORK [conn4087012] end connection 10.0.7.113:36053 (1506 connections now open) 2016-03-13T10:06:40.949+0000 I NETWORK [conn4087011] end connection 10.0.7.113:36052 (1506 connections now open) 2016-03-13T10:06:40.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36071 #4087023 (1506 connections now open) 2016-03-13T10:06:40.950+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36070 #4087024 (1507 connections now open) 2016-03-13T10:06:41.974+0000 I NETWORK [conn4087014] end connection 10.0.216.90:34060 (1506 connections now open) 2016-03-13T10:06:41.974+0000 I NETWORK [conn4087013] end connection 10.0.216.90:34059 (1506 connections now open) 2016-03-13T10:06:41.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34078 #4087025 (1506 connections now open) 2016-03-13T10:06:41.975+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34079 #4087026 (1507 connections now open) 2016-03-13T10:06:42.929+0000 I NETWORK [conn4087015] end connection 10.0.20.53:55796 (1506 connections now open) 2016-03-13T10:06:42.929+0000 I NETWORK [conn4087016] end connection 10.0.20.53:55797 (1506 connections now open) 2016-03-13T10:06:42.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55804 #4087027 (1506 connections now open) 2016-03-13T10:06:42.939+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55805 #4087028 (1507 connections now open) 2016-03-13T10:06:45.406+0000 I NETWORK [conn4087017] end connection 10.0.179.209:45504 (1506 connections now open) 2016-03-13T10:06:45.406+0000 I NETWORK [conn4087018] end connection 10.0.179.209:45505 (1506 connections now open) 2016-03-13T10:06:45.406+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45522 #4087029 (1506 connections now open) 2016-03-13T10:06:45.406+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45521 #4087030 (1507 connections now open) 2016-03-13T10:06:45.991+0000 I NETWORK [conn4087019] end connection 10.0.26.69:47433 (1506 connections now open) 2016-03-13T10:06:45.991+0000 I NETWORK [conn4087020] end connection 10.0.26.69:47434 (1506 connections now open) 2016-03-13T10:06:45.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47445 #4087031 (1506 connections now open) 2016-03-13T10:06:45.992+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47446 #4087032 (1507 connections now open) 2016-03-13T10:06:46.117+0000 I NETWORK [conn4083083] end connection 10.0.7.113:60255 (1506 connections now open) 2016-03-13T10:06:46.117+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36078 #4087033 (1507 connections now open) 2016-03-13T10:06:48.692+0000 I NETWORK [conn4087022] end connection 10.0.27.237:40853 (1506 connections now open) 2016-03-13T10:06:48.692+0000 I NETWORK [conn4087021] end connection 10.0.27.237:40854 (1506 connections now open) 2016-03-13T10:06:48.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40867 #4087034 (1506 connections now open) 2016-03-13T10:06:48.693+0000 I NETWORK [mongosMain] connection accepted from 10.0.27.237:40868 #4087035 (1507 connections now open) 2016-03-13T10:06:48.737+0000 I SHARDING [LockPinger] cluster in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019 pinged successfully at 2016-03-13T10:06:48.479+0000 by distributed lock pinger 'in.dbcfg1.mongo323.test.mydomain.com:27019,in.dbcfg2.mongo323.test.mydomain.com:27019,in.dbcfg3.mongo323.test.mydomain.com:27019/ip-10-0-231-120:27017:1457539873:229444446', sleeping for 30000ms 2016-03-13T10:06:50.963+0000 I NETWORK [conn4087023] end connection 10.0.7.113:36071 (1506 connections now open) 2016-03-13T10:06:50.963+0000 I NETWORK [conn4087024] end connection 10.0.7.113:36070 (1506 connections now open) 2016-03-13T10:06:50.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36081 #4087036 (1506 connections now open) 2016-03-13T10:06:50.972+0000 I NETWORK [mongosMain] connection accepted from 10.0.7.113:36080 #4087037 (1507 connections now open) 2016-03-13T10:06:51.988+0000 I NETWORK [conn4087026] end connection 10.0.216.90:34079 (1506 connections now open) 2016-03-13T10:06:51.988+0000 I NETWORK [conn4087025] end connection 10.0.216.90:34078 (1506 connections now open) 2016-03-13T10:06:51.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34087 #4087038 (1506 connections now open) 2016-03-13T10:06:51.998+0000 I NETWORK [mongosMain] connection accepted from 10.0.216.90:34088 #4087039 (1507 connections now open) 2016-03-13T10:06:52.951+0000 I NETWORK [conn4087028] end connection 10.0.20.53:55805 (1506 connections now open) 2016-03-13T10:06:52.951+0000 I NETWORK [conn4087027] end connection 10.0.20.53:55804 (1506 connections now open) 2016-03-13T10:06:52.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55819 #4087040 (1506 connections now open) 2016-03-13T10:06:52.951+0000 I NETWORK [mongosMain] connection accepted from 10.0.20.53:55818 #4087041 (1507 connections now open) 2016-03-13T10:06:55.418+0000 I NETWORK [conn4087029] end connection 10.0.179.209:45522 (1506 connections now open) 2016-03-13T10:06:55.419+0000 I NETWORK [conn4087030] end connection 10.0.179.209:45521 (1506 connections now open) 2016-03-13T10:06:55.427+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45530 #4087042 (1506 connections now open) 2016-03-13T10:06:55.427+0000 I NETWORK [mongosMain] connection accepted from 10.0.179.209:45531 #4087043 (1507 connections now open) 2016-03-13T10:06:56.004+0000 I NETWORK [conn4087031] end connection 10.0.26.69:47445 (1506 connections now open) 2016-03-13T10:06:56.004+0000 I NETWORK [conn4087032] end connection 10.0.26.69:47446 (1505 connections now open) 2016-03-13T10:06:56.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47466 #4087044 (1506 connections now open) 2016-03-13T10:06:56.013+0000 I NETWORK [mongosMain] connection accepted from 10.0.26.69:47465 #4087045 (1507 connections now open) 2016-03-13T10:06:58.113+0000 I NETWORK [conn4081933] end connection 10.0.20.53:51150 (1506 connections now open) 2016-03-13T10:06:58.114+0000 I NETWORK [conn4082210] end connection 10.0.26.69:43021 (1505 connections now open)