2018-10-04T11:55:19.928-0400 I CONTROL  [main] 
2018-10-04T11:55:19.929-0400 I CONTROL  [main] ** WARNING: Access control is not enabled for the database.
2018-10-04T11:55:19.930-0400 I CONTROL  [main] **          Read and write access to data and configuration is unrestricted.
2018-10-04T11:55:19.930-0400 I CONTROL  [main] 
2018-10-04T11:55:19.931-0400 I SHARDING [mongosMain] mongos version v4.0.3-rc0
2018-10-04T11:55:19.932-0400 I CONTROL  [mongosMain] git version: 7ea530946fa7880364d88c8d8b6026bbc9ffa48c
2018-10-04T11:55:19.932-0400 I CONTROL  [mongosMain] allocator: system
2018-10-04T11:55:19.933-0400 I CONTROL  [mongosMain] modules: none
2018-10-04T11:55:19.933-0400 I CONTROL  [mongosMain] build environment:
2018-10-04T11:55:19.934-0400 I CONTROL  [mongosMain]     distarch: x86_64
2018-10-04T11:55:19.934-0400 I CONTROL  [mongosMain]     target_arch: x86_64
2018-10-04T11:55:19.934-0400 I CONTROL  [mongosMain] db version v4.0.3-rc0
2018-10-04T11:55:19.935-0400 I CONTROL  [mongosMain] git version: 7ea530946fa7880364d88c8d8b6026bbc9ffa48c
2018-10-04T11:55:19.935-0400 I CONTROL  [mongosMain] allocator: system
2018-10-04T11:55:19.936-0400 I CONTROL  [mongosMain] modules: none
2018-10-04T11:55:19.936-0400 I CONTROL  [mongosMain] build environment:
2018-10-04T11:55:19.937-0400 I CONTROL  [mongosMain]     distarch: x86_64
2018-10-04T11:55:19.937-0400 I CONTROL  [mongosMain]     target_arch: x86_64
2018-10-04T11:55:19.938-0400 I CONTROL  [mongosMain] options: { config: "/tmp/mms-automation/test/versions/workspace/mongos-shctl_mongosProcess.conf", net: { bindIp: "0.0.0.0", port: 9010 }, processManagement: { fork: true }, sharding: { configDB: "csrs/jazzstep-2.local:9007,jazzstep-2.local:9008,jazzstep-2.local:9009" }, systemLog: { destination: "file", path: "/tmp/mms-automation/test/logs/run9010" } }
2018-10-04T11:55:19.948-0400 I NETWORK  [mongosMain] Starting new replica set monitor for csrs/jazzstep-2.local:9007,jazzstep-2.local:9008,jazzstep-2.local:9009
2018-10-04T11:55:19.952-0400 I SHARDING [thread1] creating distributed lock ping thread for process jazzstep-2.local:9010:1538668519:-3295272312362626892 (sleeping for 30000ms)
2018-10-04T11:55:19.955-0400 I NETWORK  [mongosMain] Successfully connected to jazzstep-2.local:9007 (1 connections now open to jazzstep-2.local:9007 with a 5 second timeout)
2018-10-04T11:55:19.955-0400 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Successfully connected to jazzstep-2.local:9009 (1 connections now open to jazzstep-2.local:9009 with a 5 second timeout)
2018-10-04T11:55:19.955-0400 I NETWORK  [shard registry reload] Successfully connected to jazzstep-2.local:9008 (1 connections now open to jazzstep-2.local:9008 with a 5 second timeout)
2018-10-04T11:55:19.957-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9007
2018-10-04T11:55:19.958-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9007
2018-10-04T11:55:19.959-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9008
2018-10-04T11:55:19.962-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9007
2018-10-04T11:55:19.963-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9009
2018-10-04T11:55:19.973-0400 W SHARDING [replSetDistLockPinger] pinging failed for distributed lock pinger :: caused by :: LockStateChangeFailed: findAndModify query predicate didn't match any lock document
2018-10-04T11:55:19.974-0400 I FTDC     [mongosMain] Initializing full-time diagnostic data capture with directory '/tmp/mms-automation/test/logs/run9010.diagnostic.data'
2018-10-04T11:55:19.976-0400 I NETWORK  [mongosMain] waiting for connections on port 9010
2018-10-04T11:55:19.976-0400 I SH_REFR  [ConfigServerCatalogCacheLoader-0] Refresh for database config took 0 ms and found { _id: "config", primary: "config", partitioned: true }
2018-10-04T11:55:19.977-0400 I CONTROL  [LogicalSessionCacheRefresh] Sessions collection is not set up; waiting until next sessions refresh interval: Collection config.system.sessions is not sharded.
2018-10-04T11:55:19.979-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:58976 #9 (1 connection now open)
2018-10-04T11:55:19.980-0400 I NETWORK  [conn9] end connection 10.1.25.36:58976 (0 connections now open)
2018-10-04T11:55:19.981-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:58977 #10 (1 connection now open)
2018-10-04T11:55:19.987-0400 I ACCESS   [conn10] Supported SASL mechanisms requested for unknown user 'testuser@admin'
2018-10-04T11:55:19.992-0400 I ACCESS   [conn10] SASL SCRAM-SHA-1 authentication failed for testuser on admin from client 10.1.25.36:58977 ; UserNotFound: User "testuser@admin" not found
2018-10-04T11:55:19.997-0400 I NETWORK  [conn10] end connection 10.1.25.36:58977 (0 connections now open)
2018-10-04T11:55:19.998-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:58978 #11 (1 connection now open)
2018-10-04T11:55:20.008-0400 I FTDC     [ftdc] Unclean full-time diagnostic data capture shutdown detected, found interim file, some metrics may have been lost. OK
2018-10-04T11:55:20.008-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:58979 #12 (2 connections now open)
2018-10-04T11:55:20.014-0400 I NETWORK  [conn12] end connection 10.1.25.36:58979 (1 connection now open)
2018-10-04T11:55:20.016-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:58980 #13 (2 connections now open)
2018-10-04T11:55:20.020-0400 I ACCESS   [conn13] Supported SASL mechanisms requested for unknown user 'testuser@admin'
2018-10-04T11:55:20.023-0400 I ACCESS   [conn13] SASL SCRAM-SHA-1 authentication failed for testuser on admin from client 10.1.25.36:58980 ; UserNotFound: User "testuser@admin" not found
2018-10-04T11:55:20.025-0400 I NETWORK  [conn13] end connection 10.1.25.36:58980 (1 connection now open)
2018-10-04T11:55:20.026-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:58981 #14 (2 connections now open)
2018-10-04T11:55:20.035-0400 I ASIO     [TaskExecutorPool-0] Connecting to jazzstep-2.local:9007
2018-10-04T11:55:20.331-0400 I NETWORK  [conn14] Starting new replica set monitor for foo/jazzstep-2.local:9001,jazzstep-2.local:9002,jazzstep-2.local:9003
2018-10-04T11:55:20.335-0400 I COMMAND  [conn14] query admin.foo/jazzstep-2.local:9001,jazzstep-2.local:9002,jazzstep-2.local:9003 command: { addShard: "foo/jazzstep-2.local:9001,jazzstep-2.local:9002,jazzstep-2.local:9003", name: "shard1", $db: "admin" } numYields:0 reslen:201 280ms
2018-10-04T11:55:20.338-0400 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Successfully connected to jazzstep-2.local:9002 (1 connections now open to jazzstep-2.local:9002 with a 5 second timeout)
2018-10-04T11:55:20.347-0400 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Successfully connected to jazzstep-2.local:9001 (1 connections now open to jazzstep-2.local:9001 with a 5 second timeout)
2018-10-04T11:55:20.361-0400 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Successfully connected to jazzstep-2.local:9003 (1 connections now open to jazzstep-2.local:9003 with a 5 second timeout)
2018-10-04T11:55:20.504-0400 I COMMAND  [conn14] query a command: { enableSharding: "a", $db: "admin" } numYields:0 reslen:178 153ms
2018-10-04T11:55:20.606-0400 I SH_REFR  [ConfigServerCatalogCacheLoader-0] Refresh for database a took 0 ms and found { _id: "a", primary: "shard1", partitioned: true, version: { uuid: UUID("4ea91b7e-b5f4-4d24-b56f-6d5acdaad219"), lastMod: 1 } }
2018-10-04T11:55:20.608-0400 I ASIO     [TaskExecutorPool-0] Connecting to jazzstep-2.local:9001
2018-10-04T11:55:20.656-0400 I SHARDING [conn14] Marking cached database entry for 'a' as stale
2018-10-04T11:55:20.657-0400 I SH_REFR  [ConfigServerCatalogCacheLoader-0] Refresh for database a took 0 ms and found { _id: "a", primary: "shard1", partitioned: true, version: { uuid: UUID("4ea91b7e-b5f4-4d24-b56f-6d5acdaad219"), lastMod: 1 } }
2018-10-04T11:55:35.001-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:59339 #20 (3 connections now open)
2018-10-04T11:55:35.027-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:59344 #21 (4 connections now open)
2018-10-04T11:55:50.366-0400 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Marking host jazzstep-2.local:9001 as failed :: caused by :: Location40356: connection pool: connect failed jazzstep-2.local:9001 : couldn't connect to server jazzstep-2.local:9001, connection attempt failed: SocketException: Error connecting to jazzstep-2.local:9001 (10.1.25.36:9001) :: caused by :: Connection refused
2018-10-04T11:55:50.368-0400 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Marking host jazzstep-2.local:9003 as failed :: caused by :: Location40356: connection pool: connect failed jazzstep-2.local:9003 : couldn't connect to server jazzstep-2.local:9003, connection attempt failed: SocketException: Error connecting to jazzstep-2.local:9003 (10.1.25.36:9003) :: caused by :: Connection refused
2018-10-04T11:55:50.368-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:55:54.274-0400 I COMMAND  [conn14] query a.b command: { shardCollection: "a.b", key: { key1: 1 }, unique: false, $db: "admin" } numYields:0 reslen:321 33614ms
2018-10-04T11:55:54.275-0400 I NETWORK  [conn14] end connection 10.1.25.36:58981 (2 connections now open)
2018-10-04T11:55:54.275-0400 I NETWORK  [conn21] end connection 10.1.25.36:59344 (3 connections now open)
2018-10-04T11:56:20.372-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:56:20.661-0400 I ASIO     [TaskExecutorPool-0] Dropping all pooled connections to jazzstep-2.local:9001 due to HostUnreachable: Connection was closed
2018-10-04T11:56:49.978-0400 I ASIO     [ShardRegistry] Ending idle connection to host jazzstep-2.local:9007 because the pool meets constraints; 2 connections to that host remain open
2018-10-04T11:56:50.282-0400 I ASIO     [ShardRegistry] Ending idle connection to host jazzstep-2.local:9007 because the pool meets constraints; 1 connections to that host remain open
2018-10-04T11:56:50.377-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:57:20.381-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:57:50.389-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:58:20.392-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:58:50.404-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:59:20.029-0400 I NETWORK  [listener] connection accepted from 10.1.25.36:65214 #22 (3 connections now open)
2018-10-04T11:59:20.415-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T11:59:50.425-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:00:19.978-0400 I SH_REFR  [ConfigServerCatalogCacheLoader-1] Refresh for collection config.system.sessions took 1 ms and found the collection is not sharded
2018-10-04T12:00:19.988-0400 I CONTROL  [LogicalSessionCacheRefresh] Sessions collection is not set up; waiting until next sessions refresh interval: Collection config.system.sessions is not sharded.
2018-10-04T12:00:20.433-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:00:20.660-0400 I ASIO     [TaskExecutorPool-0] Dropping all pooled connections to jazzstep-2.local:9001 due to NetworkInterfaceExceededTimeLimit: Connection pool has been idle for longer than the host timeout
2018-10-04T12:00:20.662-0400 I ASIO     [TaskExecutorPool-0] Dropping all pooled connections to jazzstep-2.local:9007 due to NetworkInterfaceExceededTimeLimit: Connection pool has been idle for longer than the host timeout
2018-10-04T12:00:37.079-0400 I NETWORK  [conn22] end connection 10.1.25.36:65214 (1 connection now open)
2018-10-04T12:00:37.079-0400 I NETWORK  [conn20] end connection 10.1.25.36:59339 (2 connections now open)
2018-10-04T12:00:37.079-0400 I NETWORK  [conn11] end connection 10.1.25.36:58978 (0 connections now open)
2018-10-04T12:00:50.442-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:01:19.992-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9007
2018-10-04T12:01:20.450-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:01:50.456-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:02:19.997-0400 I ASIO     [ShardRegistry] Ending idle connection to host jazzstep-2.local:9007 because the pool meets constraints; 1 connections to that host remain open
2018-10-04T12:02:20.463-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:02:50.469-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:03:20.477-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:03:50.483-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:04:20.488-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:04:50.493-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:05:19.980-0400 I SH_REFR  [ConfigServerCatalogCacheLoader-2] Refresh for collection config.system.sessions took 0 ms and found the collection is not sharded
2018-10-04T12:05:19.981-0400 I CONTROL  [LogicalSessionCacheRefresh] Sessions collection is not set up; waiting until next sessions refresh interval: Collection config.system.sessions is not sharded.
2018-10-04T12:05:20.496-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:05:50.498-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:06:20.510-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:06:50.513-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:07:20.519-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:07:50.525-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:08:20.527-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:08:50.529-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:09:20.539-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:09:50.546-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:10:19.985-0400 I SH_REFR  [ConfigServerCatalogCacheLoader-3] Refresh for collection config.system.sessions took 1 ms and found the collection is not sharded
2018-10-04T12:10:19.989-0400 I CONTROL  [LogicalSessionCacheRefresh] Sessions collection is not set up; waiting until next sessions refresh interval: Collection config.system.sessions is not sharded.
2018-10-04T12:10:20.549-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:10:50.557-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:11:20.563-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:22:50.103-0400 I ASIO     [ShardRegistry] Dropping all pooled connections to jazzstep-2.local:9009 due to NetworkInterfaceExceededTimeLimit: Connection pool has been idle for longer than the host timeout
2018-10-04T12:22:50.104-0400 I ASIO     [ShardRegistry] Dropping all pooled connections to jazzstep-2.local:9008 due to NetworkInterfaceExceededTimeLimit: Connection pool has been idle for longer than the host timeout
2018-10-04T12:22:50.105-0400 I NETWORK  [ShardRegistry] Marking host jazzstep-2.local:9007 as failed :: caused by :: NotMaster: not master
2018-10-04T12:22:50.110-0400 I NETWORK  [Uptime reporter] Successfully connected to jazzstep-2.local:9007 (1 connections now open to jazzstep-2.local:9007 with a 5 second timeout)
2018-10-04T12:22:50.115-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9008
2018-10-04T12:22:50.135-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9009
2018-10-04T12:23:05.857-0400 I ACCESS   [UserCacheInvalidator] User cache generation changed from 5bb637bf290d7fa45d18bef6 to 5bb637c23d8f72c85d19042f; invalidating user cache
2018-10-04T12:23:05.870-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:23:06.370-0400 W SHARDING [replSetDistLockPinger] Lock pinger for proc: jazzstep-2.local:9010:1538668519:-3295272312362626892 was inactive for 705827ms ms
2018-10-04T12:23:35.872-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:30:03.428-0400 I ASIO     [ShardRegistry] Dropping all pooled connections to jazzstep-2.local:9007 due to NetworkInterfaceExceededTimeLimit: Connection pool has been idle for longer than the host timeout
2018-10-04T12:30:03.429-0400 I ASIO     [ShardRegistry] Dropping all pooled connections to jazzstep-2.local:9009 due to NetworkInterfaceExceededTimeLimit: Connection pool has been idle for longer than the host timeout
2018-10-04T12:30:03.430-0400 I NETWORK  [ShardRegistry] Marking host jazzstep-2.local:9008 as failed :: caused by :: NotMaster: not master
2018-10-04T12:30:03.433-0400 I NETWORK  [Uptime reporter] Successfully connected to jazzstep-2.local:9008 (1 connections now open to jazzstep-2.local:9008 with a 5 second timeout)
2018-10-04T12:30:03.434-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9007
2018-10-04T12:30:04.120-0400 I ASIO     [ShardRegistry] Connecting to jazzstep-2.local:9009
2018-10-04T12:30:19.025-0400 I ACCESS   [UserCacheInvalidator] User cache generation changed from 5bb637c23d8f72c85d19042f to 5bb637bf290d7fa45d18bef6; invalidating user cache
2018-10-04T12:30:19.041-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:30:19.583-0400 W SHARDING [replSetDistLockPinger] Lock pinger for proc: jazzstep-2.local:9010:1538668519:-3295272312362626892 was inactive for 403187ms ms
2018-10-04T12:30:49.058-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:31:19.063-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:31:49.066-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:32:19.069-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:32:48.980-0400 I SH_REFR  [ConfigServerCatalogCacheLoader-4] Refresh for collection config.system.sessions took 0 ms and found the collection is not sharded
2018-10-04T12:32:48.985-0400 I CONTROL  [LogicalSessionCacheRefresh] Sessions collection is not set up; waiting until next sessions refresh interval: Collection config.system.sessions is not sharded.
2018-10-04T12:32:49.072-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:33:19.074-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:33:49.077-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo
2018-10-04T12:34:19.150-0400 W NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set foo