When the following sequence of events happen, mongos seems to retry reading the chunk data infinitely from the first config server:
1. Initialize the sharded cluster
2. Enable sharding on the "test" database
3. Create a sharded collection called "server16691" with shard key {_id: 1}
4. Configure the mongobridge corresponding to the first config server to discard messages from the mongos
5. Configure the mongobridges corresponding to the other config servers to delay messages from the mongos
6. Run the "flushRouterConfig" command to clear the routing table of the mongos
7. Try to run db.server16691.find({}).itcount() to trigger read operations of the chunk data
[js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:35:57.238-0500 b20014| 2015-11-04T14:35:57.229-0500 I - [thread2] Received "discardMessagesFrom" command with arguments { discardMessagesFrom: 1.0, $forBridge: true, host: "hanamizu:20020", loss: 1.0 } from <unknown> [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:35:57.240-0500 b20014| 2015-11-04T14:35:57.231-0500 I - [thread12] Received "find" command with arguments { find: "databases", filter: { _id: "test" }, readConcern: { level: "majority", afterOpTime: { ts: Timestamp 1446665757000|20, t: 1 } }, limit: 1, maxTimeMS: 30000 } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:35:57.240-0500 b20014| 2015-11-04T14:35:57.231-0500 I - [thread12] Discarding "find" command with arguments { find: "databases", filter: { _id: "test" }, readConcern: { level: "majority", afterOpTime: { ts: Timestamp 1446665757000|20, t: 1 } }, limit: 1, maxTimeMS: 30000 } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:07.007-0500 b20014| 2015-11-04T14:36:07.007-0500 I - [thread24] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:07.008-0500 b20014| 2015-11-04T14:36:07.007-0500 I - [thread24] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:13.883-0500 b20014| 2015-11-04T14:36:13.883-0500 I - [thread11] Received "ismaster" command with arguments { ismaster: 1 } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:13.883-0500 b20014| 2015-11-04T14:36:13.883-0500 I - [thread11] Discarding "ismaster" command with arguments { ismaster: 1 } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:26.807-0500 b20014| 2015-11-04T14:36:26.807-0500 I - [thread25] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:26.808-0500 b20014| 2015-11-04T14:36:26.807-0500 I - [thread25] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:41.806-0500 b20014| 2015-11-04T14:36:41.806-0500 I - [thread26] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:41.807-0500 b20014| 2015-11-04T14:36:41.806-0500 I - [thread26] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:57.008-0500 b20014| 2015-11-04T14:36:57.007-0500 I - [thread27] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:36:57.008-0500 b20014| 2015-11-04T14:36:57.007-0500 I - [thread27] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 ... [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:38:57.608-0500 b20014| 2015-11-04T14:38:57.607-0500 I - [thread38] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:38:57.608-0500 b20014| 2015-11-04T14:38:57.607-0500 I - [thread38] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:12.808-0500 b20014| 2015-11-04T14:39:12.808-0500 I - [thread39] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:12.808-0500 b20014| 2015-11-04T14:39:12.808-0500 I - [thread39] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:22.808-0500 b20014| 2015-11-04T14:39:22.807-0500 I - [thread40] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:22.808-0500 b20014| 2015-11-04T14:39:22.807-0500 I - [thread40] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:28.008-0500 b20014| 2015-11-04T14:39:28.007-0500 I - [thread41] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:28.008-0500 b20014| 2015-11-04T14:39:28.007-0500 I - [thread41] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:43.207-0500 b20014| 2015-11-04T14:39:43.207-0500 I - [thread42] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:43.208-0500 b20014| 2015-11-04T14:39:43.207-0500 I - [thread42] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:58.008-0500 b20014| 2015-11-04T14:39:58.007-0500 I - [thread43] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:39:58.008-0500 b20014| 2015-11-04T14:39:58.007-0500 I - [thread43] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:40:13.006-0500 b20014| 2015-11-04T14:40:13.006-0500 I - [thread44] Received "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020 [js_test:blackhole_first_config_server_from_mongos] 2015-11-04T14:40:13.006-0500 b20014| 2015-11-04T14:40:13.006-0500 I - [thread44] Discarding "isMaster" command with arguments { isMaster: 1, hostInfo: "hanamizu:20020" } from hanamizu:20020
- is duplicated by
-
SERVER-21400 Insert takes 90 seconds when first config server (SCCC) is in TCP blackhole from mongos triggering autosplit
- Closed
- is related to
-
SERVER-22486 query to the router never done when the ip of the config1 lxc is down
- Closed