2015-10-23T15:21:18.980-0400 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=9G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2015-10-23T15:21:19.222-0400 I CONTROL [initandlisten] MongoDB starting : pid=49105 port=27018 dbpath=db2 64-bit host=neurofunk.local 2015-10-23T15:21:19.222-0400 I CONTROL [initandlisten] 2015-10-23T15:21:19.222-0400 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000 2015-10-23T15:21:19.222-0400 I CONTROL [initandlisten] db version v3.2.0-rc1-pre- 2015-10-23T15:21:19.222-0400 I CONTROL [initandlisten] git version: dbbc9a2e3d8c4d7fe1748fa980ba7d01b9489dbe 2015-10-23T15:21:19.223-0400 I CONTROL [initandlisten] allocator: system 2015-10-23T15:21:19.223-0400 I CONTROL [initandlisten] modules: none 2015-10-23T15:21:19.223-0400 I CONTROL [initandlisten] build environment: 2015-10-23T15:21:19.223-0400 I CONTROL [initandlisten] distarch: x86_64 2015-10-23T15:21:19.223-0400 I CONTROL [initandlisten] target_arch: x86_64 2015-10-23T15:21:19.223-0400 I CONTROL [initandlisten] options: { net: { port: 27018 }, replication: { replSet: "rs0" }, storage: { dbPath: "db2" }, systemLog: { destination: "file", path: "db2.log" } } 2015-10-23T15:21:19.245-0400 I REPL [initandlisten] Did not find local voted for document at startup; NoMatchingDocument Did not find replica set lastVote document in local.replset.election 2015-10-23T15:21:19.245-0400 I REPL [initandlisten] Did not find local replica set configuration document at startup; NoMatchingDocument Did not find replica set configuration document in local.system.replset 2015-10-23T15:21:19.246-0400 I FTDC [initandlisten] Starting full-time diagnostic data capture with directory 'db2/diagnostic.data' 2015-10-23T15:21:19.246-0400 I NETWORK [HostnameCanonicalizationWorker] Starting hostname canonicalization worker 2015-10-23T15:21:19.247-0400 W NETWORK [HostnameCanonicalizationWorker] Failed to obtain name info for an address: nodename nor servname provided, or not known 2015-10-23T15:21:19.247-0400 W NETWORK [HostnameCanonicalizationWorker] Failed to obtain name info for an address: nodename nor servname provided, or not known 2015-10-23T15:21:19.268-0400 I NETWORK [initandlisten] waiting for connections on port 27018 2015-10-23T15:22:16.065-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60336 #1 (1 connection now open) 2015-10-23T15:22:16.065-0400 I NETWORK [conn1] end connection 10.4.126.122:60336 (0 connections now open) 2015-10-23T15:22:16.066-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60338 #2 (1 connection now open) 2015-10-23T15:22:16.067-0400 I ASIO [NetworkInterfaceASIO] Successfully connected to neurofunk.local:27017 2015-10-23T15:22:16.089-0400 I REPL [replExecDBWorker-0] Starting replication applier threads 2015-10-23T15:22:16.089-0400 W REPL [rsSync] did not receive a valid config yet 2015-10-23T15:22:16.089-0400 I REPL [ReplicationExecutor] New replica set config in use: { _id: "rs0", version: 2, protocolVersion: 1, members: [ { _id: 0, host: "neurofunk.local:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "neurofunk.local:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 5000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 } } } 2015-10-23T15:22:16.090-0400 I REPL [ReplicationExecutor] This node is neurofunk.local:27018 in the config 2015-10-23T15:22:16.090-0400 I REPL [ReplicationExecutor] transition to STARTUP2 2015-10-23T15:22:16.090-0400 I REPL [ReplicationExecutor] Member neurofunk.local:27017 is now in state PRIMARY 2015-10-23T15:22:17.090-0400 I REPL [rsSync] ****** 2015-10-23T15:22:17.090-0400 I REPL [rsSync] creating replication oplog of size: 192MB... 2015-10-23T15:22:17.101-0400 I STORAGE [rsSync] Starting WiredTigerRecordStoreThread local.oplog.rs 2015-10-23T15:22:17.101-0400 I STORAGE [rsSync] Scanning the oplog to determine where to place markers for when to truncate 2015-10-23T15:22:17.227-0400 I REPL [rsSync] ****** 2015-10-23T15:22:17.227-0400 I REPL [rsSync] initial sync pending 2015-10-23T15:22:17.251-0400 I REPL [ReplicationExecutor] syncing from: neurofunk.local:27017 2015-10-23T15:22:17.258-0400 I REPL [rsSync] initial sync drop all databases 2015-10-23T15:22:17.258-0400 I STORAGE [rsSync] dropAllDatabasesExceptLocal 1 2015-10-23T15:22:17.258-0400 I REPL [rsSync] initial sync clone all databases 2015-10-23T15:22:17.259-0400 I REPL [rsSync] initial sync data copy, starting syncup 2015-10-23T15:22:17.259-0400 I REPL [rsSync] oplog sync 1 of 3 2015-10-23T15:22:17.259-0400 I REPL [rsSync] oplog sync 2 of 3 2015-10-23T15:22:17.259-0400 I REPL [rsSync] initial sync building indexes 2015-10-23T15:22:17.259-0400 I REPL [rsSync] oplog sync 3 of 3 2015-10-23T15:22:17.260-0400 I REPL [rsSync] initial sync finishing up 2015-10-23T15:22:17.260-0400 I REPL [rsSync] set minValid=(term: 1, timestamp: Oct 23 15:22:16:1) 2015-10-23T15:22:17.265-0400 I REPL [rsSync] initial sync done 2015-10-23T15:22:17.266-0400 I REPL [ReplicationExecutor] transition to RECOVERING 2015-10-23T15:22:17.267-0400 I REPL [ReplicationExecutor] transition to SECONDARY 2015-10-23T15:22:18.095-0400 I REPL [ReplicationExecutor] could not find member to sync from 2015-10-23T15:22:19.247-0400 W NETWORK [HostnameCanonicalizationWorker] Failed to obtain name info for an address: nodename nor servname provided, or not known 2015-10-23T15:22:19.248-0400 W NETWORK [HostnameCanonicalizationWorker] Failed to obtain name info for an address: nodename nor servname provided, or not known 2015-10-23T15:22:25.752-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60344 #3 (2 connections now open) 2015-10-23T15:22:25.752-0400 I NETWORK [conn3] end connection 10.4.126.122:60344 (1 connection now open) 2015-10-23T15:22:25.756-0400 I REPL [ReplicationExecutor] New replica set config in use: { _id: "rs0", version: 3, protocolVersion: 1, members: [ { _id: 0, host: "neurofunk.local:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "neurofunk.local:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "neurofunk.local:27019", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 5000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 } } } 2015-10-23T15:22:25.756-0400 I REPL [ReplicationExecutor] This node is neurofunk.local:27018 in the config 2015-10-23T15:22:25.757-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60355 #4 (2 connections now open) 2015-10-23T15:22:25.757-0400 I NETWORK [conn4] end connection 10.4.126.122:60355 (1 connection now open) 2015-10-23T15:22:25.757-0400 I ASIO [NetworkInterfaceASIO] Successfully connected to neurofunk.local:27019 2015-10-23T15:22:25.758-0400 I REPL [ReplicationExecutor] Member neurofunk.local:27019 is now in state STARTUP 2015-10-23T15:22:25.758-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60359 #5 (2 connections now open) 2015-10-23T15:22:26.122-0400 I REPL [ReplicationExecutor] syncing from: neurofunk.local:27017 2015-10-23T15:22:26.124-0400 I REPL [SyncSourceFeedback] setting syncSourceFeedback to neurofunk.local:27017 2015-10-23T15:22:26.125-0400 I ASIO [NetworkInterfaceASIO] Successfully connected to neurofunk.local:27017 2015-10-23T15:22:26.906-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60364 #6 (3 connections now open) 2015-10-23T15:22:26.919-0400 I NETWORK [conn6] end connection 10.4.126.122:60364 (2 connections now open) 2015-10-23T15:22:28.258-0400 I REPL [ReplicationExecutor] Member neurofunk.local:27019 is now in state SECONDARY 2015-10-23T15:22:43.878-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60365 #7 (3 connections now open) 2015-10-23T15:22:43.879-0400 I NETWORK [conn7] end connection 10.4.126.122:60365 (2 connections now open) 2015-10-23T15:22:43.881-0400 I REPL [SyncSourceFeedback] SyncSourceFeedback error sending update, response: { configVersion: 4, ok: 0.0, errmsg: "Received replSetUpdatePosition for node with memberId 1 whose config version of 3 doesn't match our config version of 4", code: 93 } 2015-10-23T15:22:43.881-0400 I REPL [SyncSourceFeedback] updateUpstream failed: InvalidReplicaSetConfig Received replSetUpdatePosition for node with memberId 1 whose config version of 3 doesn't match our config version of 4, will retry 2015-10-23T15:22:43.883-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60369 #8 (3 connections now open) 2015-10-23T15:22:43.883-0400 I NETWORK [conn8] end connection 10.4.126.122:60369 (2 connections now open) 2015-10-23T15:22:43.884-0400 I REPL [ReplicationExecutor] New replica set config in use: { _id: "rs0", version: 4, members: [ { _id: 0, host: "neurofunk.local:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "neurofunk.local:27018", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "neurofunk.local:27019", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 5000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 } } } 2015-10-23T15:22:43.884-0400 I REPL [ReplicationExecutor] This node is neurofunk.local:27018 in the config 2015-10-23T15:22:44.250-0400 I NETWORK [conn2] end connection 10.4.126.122:60338 (1 connection now open) 2015-10-23T15:22:44.250-0400 I REPL [ReplicationExecutor] could not find member to sync from 2015-10-23T15:22:44.250-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection reset by peer 2015-10-23T15:22:44.251-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:44.251-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:44.252-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:44.252-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:44.252-0400 I REPL [ReplicationExecutor] Standing for election 2015-10-23T15:22:44.252-0400 I REPL [ReplicationExecutor] not electing self, neurofunk.local:27019 would veto with 'neurofunk.local:27018 is trying to elect itself but neurofunk.local:27017 is already primary and more up-to-date' 2015-10-23T15:22:44.252-0400 I REPL [ReplicationExecutor] not electing self, we are not freshest 2015-10-23T15:22:44.830-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60375 #9 (2 connections now open) 2015-10-23T15:22:44.831-0400 I NETWORK [conn9] end connection 10.4.126.122:60375 (1 connection now open) 2015-10-23T15:22:44.831-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60376 #10 (2 connections now open) 2015-10-23T15:22:44.832-0400 I NETWORK [initandlisten] connection accepted from 10.4.126.122:60378 #11 (3 connections now open) 2015-10-23T15:22:46.255-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:46.255-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:46.255-0400 I REPL [ReplicationExecutor] Standing for election 2015-10-23T15:22:46.256-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:46.257-0400 I ASIO [NetworkInterfaceASIO] Successfully connected to neurofunk.local:27019 2015-10-23T15:22:46.257-0400 I ASIO [NetworkInterfaceASIO] Successfully connected to neurofunk.local:27019 2015-10-23T15:22:46.257-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:46.257-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:46.257-0400 I REPL [ReplicationExecutor] possible election tie; sleeping 525ms until 2015-10-23T15:22:46.782-0400 2015-10-23T15:22:46.258-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:46.258-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:46.783-0400 I REPL [ReplicationExecutor] Standing for election 2015-10-23T15:22:46.783-0400 I REPL [ReplicationExecutor] running for election 2015-10-23T15:22:46.784-0400 I REPL [ReplicationExecutor] received vote: 1 votes from neurofunk.local:27019 2015-10-23T15:22:46.784-0400 I REPL [ReplicationExecutor] election succeeded, assuming primary role 2015-10-23T15:22:46.784-0400 I REPL [ReplicationExecutor] transition to PRIMARY 2015-10-23T15:22:46.785-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:46.785-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:46.786-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:46.786-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:46.787-0400 I ASIO [NetworkInterfaceASIO] Failed to connect to neurofunk.local:27017 - HostUnreachable Connection refused 2015-10-23T15:22:46.787-0400 I REPL [ReplicationExecutor] Error in heartbeat request to neurofunk.local:27017; HostUnreachable Connection refused 2015-10-23T15:22:47.259-0400 I REPL [rsSync] transition to primary complete; database writes are now permitted 2015-10-23T15:22:48.596-0400 I CONTROL [signalProcessingThread] got signal 2 (Interrupt: 2), will terminate after current cmd ends 2015-10-23T15:22:48.597-0400 I FTDC [signalProcessingThread] Stopping full-time diagnostic data capture 2015-10-23T15:22:48.599-0400 I REPL [signalProcessingThread] Stopping replication applier threads 2015-10-23T15:22:49.268-0400 I CONTROL [signalProcessingThread] now exiting 2015-10-23T15:22:49.268-0400 I STORAGE [conn5] got request after shutdown() 2015-10-23T15:22:49.268-0400 I NETWORK [signalProcessingThread] shutdown: going to close listening sockets... 2015-10-23T15:22:49.268-0400 I NETWORK [signalProcessingThread] closing listening socket: 6 2015-10-23T15:22:49.269-0400 I NETWORK [signalProcessingThread] closing listening socket: 7 2015-10-23T15:22:49.269-0400 I NETWORK [signalProcessingThread] removing socket file: /tmp/mongodb-27018.sock 2015-10-23T15:22:49.269-0400 I NETWORK [signalProcessingThread] shutdown: going to flush diaglog... 2015-10-23T15:22:49.269-0400 I NETWORK [signalProcessingThread] shutdown: going to close sockets... 2015-10-23T15:22:49.269-0400 I STORAGE [signalProcessingThread] WiredTigerKVEngine shutting down 2015-10-23T15:22:49.269-0400 I NETWORK [conn10] end connection 10.4.126.122:60376 (0 connections now open) 2015-10-23T15:22:49.416-0400 I STORAGE [signalProcessingThread] shutdown: removing fs lock... 2015-10-23T15:22:49.417-0400 I CONTROL [signalProcessingThread] dbexit: rc: 0