[2015/10/07 14:59:35.703] Task logger initialized (agent revision: b8b69288a6af9aa593b25c57c1693815400e6de0) [2015/10/07 14:59:35.703] Running agent without a https certificate. [2015/10/07 14:59:38.827] Running command 'shell.track' (step 1 of 2) [2015/10/07 14:59:38.827] Running command 'shell.exec' (step 2 of 2) [2015/10/07 14:59:39.356] Running command 's3.get' in "do setup" (step 1.1 of 2) [2015/10/07 14:59:39.356] ['s3.get' in "do setup"] Fetching mongodb-mongo-master/linux-64/0cf1be838e229b4385ac4313e994040a230857e5/artifacts/mongodb_mongo_master_linux_64_0cf1be838e229b4385ac4313e994040a230857e5_15_10_07_18_34_55.tgz from s3 bucket mciuploads [2015/10/07 14:59:59.104] Running command 's3.get' in "do setup" (step 1.2 of 2) [2015/10/07 14:59:59.104] ['s3.get' in "do setup"] Fetching mongodb-mongo-master/linux-64/0cf1be838e229b4385ac4313e994040a230857e5/binaries/mongo-mongodb_mongo_master_linux_64_0cf1be838e229b4385ac4313e994040a230857e5_15_10_07_18_34_55.tgz from s3 bucket mciuploads [2015/10/07 14:59:59.829] Running command 'shell.exec' in "do setup" (step 1.3 of 2) [2015/10/07 14:59:59.841] mongodb-linux-x86_64-3.1.10-pre-/README [2015/10/07 14:59:59.842] mongodb-linux-x86_64-3.1.10-pre-/THIRD-PARTY-NOTICES [2015/10/07 14:59:59.842] mongodb-linux-x86_64-3.1.10-pre-/MPL-2 [2015/10/07 14:59:59.842] mongodb-linux-x86_64-3.1.10-pre-/GNU-AGPL-3.0 [2015/10/07 14:59:59.842] mongodb-linux-x86_64-3.1.10-pre-/bin/mongodump [2015/10/07 14:59:59.915] mongodb-linux-x86_64-3.1.10-pre-/bin/mongorestore [2015/10/07 15:00:00.002] mongodb-linux-x86_64-3.1.10-pre-/bin/mongoexport [2015/10/07 15:00:00.058] mongodb-linux-x86_64-3.1.10-pre-/bin/mongoimport [2015/10/07 15:00:00.108] mongodb-linux-x86_64-3.1.10-pre-/bin/mongostat [2015/10/07 15:00:00.156] mongodb-linux-x86_64-3.1.10-pre-/bin/mongotop [2015/10/07 15:00:00.203] mongodb-linux-x86_64-3.1.10-pre-/bin/bsondump [2015/10/07 15:00:00.239] mongodb-linux-x86_64-3.1.10-pre-/bin/mongofiles [2015/10/07 15:00:00.288] mongodb-linux-x86_64-3.1.10-pre-/bin/mongooplog [2015/10/07 15:00:00.335] mongodb-linux-x86_64-3.1.10-pre-/bin/mongoperf [2015/10/07 15:00:00.624] mongodb-linux-x86_64-3.1.10-pre-/bin/mongod [2015/10/07 15:00:00.916] mongodb-linux-x86_64-3.1.10-pre-/bin/mongos [2015/10/07 15:00:01.038] mongodb-linux-x86_64-3.1.10-pre-/bin/mongo [2015/10/07 15:00:01.194] Running command 'keyval.inc' in "do setup" (step 1.4 of 2) [2015/10/07 15:00:01.198] Running command 'shell.exec' in "do setup" (step 1.5 of 2) [2015/10/07 15:00:01.208] Running command 'shell.exec' in "run tests" (step 2 of 2) [2015/10/07 15:00:01.222] # check if virtualenv is set up [2015/10/07 15:00:01.222] if [ -d "venv" ]; then [2015/10/07 15:00:01.222] if [ "Windows_NT" = "$OS" ]; then [2015/10/07 15:00:01.222] . ./venv/Scripts/activate [2015/10/07 15:00:01.222] else [2015/10/07 15:00:01.222] . ./venv/bin/activate [2015/10/07 15:00:01.222] fi [2015/10/07 15:00:01.222] fi [2015/10/07 15:00:01.222] cp mongodb*/bin/* . [2015/10/07 15:00:01.520] if [ false = true ]; then [2015/10/07 15:00:01.520] python buildscripts/resmoke.py --suites=sharding_legacy --jobs=$(grep -c ^processor /proc/cpuinfo) --continueOnFailure --log=buildlogger --reportFile=report.json [2015/10/07 15:00:01.520] else [2015/10/07 15:00:01.520] python buildscripts/resmoke.py --suites=sharding_legacy --jobs=1 --continueOnFailure --log=buildlogger --reportFile=report.json [2015/10/07 15:00:01.520] fi [2015/10/07 15:00:01.753] [resmoke] 2015-10-07T19:00:01.753+0000 YAML configuration of suite sharding_legacy [2015/10/07 15:00:01.753] selector: [2015/10/07 15:00:01.753] js_test: [2015/10/07 15:00:01.753] exclude_files: [2015/10/07 15:00:01.753] - jstests/sharding/csrs_upgrade.js [2015/10/07 15:00:01.753] - jstests/sharding/csrs_upgrade_during_migrate.js [2015/10/07 15:00:01.753] - jstests/sharding/multi_mongos2.js [2015/10/07 15:00:01.753] roots: [2015/10/07 15:00:01.753] - jstests/sharding/*.js [2015/10/07 15:00:01.753] - jstests/sharding/sync_cluster_config/*.js [2015/10/07 15:00:01.753] executor: [2015/10/07 15:00:01.753] js_test: [2015/10/07 15:00:01.753] config: [2015/10/07 15:00:01.753] shell_options: [2015/10/07 15:00:01.753] global_vars: [2015/10/07 15:00:01.753] TestData: [2015/10/07 15:00:01.753] useLegacyConfigServers: true [2015/10/07 15:00:01.753] nodb: '' [2015/10/07 15:00:01.753] readMode: legacy [2015/10/07 15:00:01.753] logging: [2015/10/07 15:00:01.753] executor: [2015/10/07 15:00:01.753] format: '[%(name)s] %(asctime)s %(message)s' [2015/10/07 15:00:01.753] handlers: [2015/10/07 15:00:01.753] - class: logging.StreamHandler [2015/10/07 15:00:01.753] fixture: [2015/10/07 15:00:01.753] format: '[%(name)s] %(message)s' [2015/10/07 15:00:01.753] handlers: [2015/10/07 15:00:01.753] - class: buildlogger [2015/10/07 15:00:01.753] tests: [2015/10/07 15:00:01.753] format: '[%(name)s] %(asctime)s %(message)s' [2015/10/07 15:00:01.754] handlers: [2015/10/07 15:00:01.754] - class: buildlogger [2015/10/07 15:00:01.834] [executor:js_test:job0] 2015-10-07T19:00:01.833+0000 Writing output of job #0 to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/. [2015/10/07 15:00:01.834] [executor:js_test] 2015-10-07T19:00:01.834+0000 Starting execution of js_tests... [2015/10/07 15:00:01.837] [executor:js_test:job0] 2015-10-07T19:00:01.837+0000 Running addshard1.js... [2015/10/07 15:00:01.837] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "addshard1"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/addshard1.js [2015/10/07 15:00:01.903] [executor:js_test:job0] 2015-10-07T19:00:01.902+0000 Writing output of JSTest jstests/sharding/addshard1.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156bb2be07c4135fd7d49d/. [2015/10/07 15:00:11.827] [executor:js_test:job0] 2015-10-07T19:00:11.826+0000 addshard1.js ran in 9.99 seconds. [2015/10/07 15:00:11.841] [executor:js_test:job0] 2015-10-07T19:00:11.841+0000 Running addshard2.js... [2015/10/07 15:00:11.841] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "addshard2"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/addshard2.js [2015/10/07 15:00:11.908] [executor:js_test:job0] 2015-10-07T19:00:11.908+0000 Writing output of JSTest jstests/sharding/addshard2.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156bbcbe07c4135fd7d626/. [2015/10/07 15:01:40.552] [executor:js_test:job0] 2015-10-07T19:01:40.552+0000 addshard2.js ran in 88.71 seconds. [2015/10/07 15:01:40.559] [executor:js_test:job0] 2015-10-07T19:01:40.559+0000 Running addshard3.js... [2015/10/07 15:01:40.559] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "addshard3"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/addshard3.js [2015/10/07 15:01:40.627] [executor:js_test:job0] 2015-10-07T19:01:40.627+0000 Writing output of JSTest jstests/sharding/addshard3.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156c15be07c4135fd7e00f/. [2015/10/07 15:01:46.974] [executor:js_test:job0] 2015-10-07T19:01:46.974+0000 addshard3.js ran in 6.42 seconds. [2015/10/07 15:01:46.999] [executor:js_test:job0] 2015-10-07T19:01:46.999+0000 Running addshard4.js... [2015/10/07 15:01:46.999] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "addshard4"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/addshard4.js [2015/10/07 15:01:47.070] [executor:js_test:job0] 2015-10-07T19:01:47.070+0000 Writing output of JSTest jstests/sharding/addshard4.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156c1bbe07c4135fd7e0c6/. [2015/10/07 15:02:34.173] [executor:js_test:job0] 2015-10-07T19:02:34.173+0000 addshard4.js ran in 47.17 seconds. [2015/10/07 15:02:34.201] [executor:js_test:job0] 2015-10-07T19:02:34.201+0000 Running addshard5.js... [2015/10/07 15:02:34.201] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "addshard5"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/addshard5.js [2015/10/07 15:02:34.266] [executor:js_test:job0] 2015-10-07T19:02:34.266+0000 Writing output of JSTest jstests/sharding/addshard5.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156c4a90413021b1d7f07d/. [2015/10/07 15:02:34.323] [executor:js_test:job0] 2015-10-07T19:02:34.323+0000 addshard5.js ran in 0.12 seconds. [2015/10/07 15:02:34.325] [executor:js_test:job0] 2015-10-07T19:02:34.324+0000 Running array_shard_key.js... [2015/10/07 15:02:34.325] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "array_shard_key"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/array_shard_key.js [2015/10/07 15:02:34.402] [executor:js_test:job0] 2015-10-07T19:02:34.402+0000 Writing output of JSTest jstests/sharding/array_shard_key.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156c4a90413021b1d7f07f/. [2015/10/07 15:02:43.811] [executor:js_test:job0] 2015-10-07T19:02:43.811+0000 array_shard_key.js ran in 9.49 seconds. [2015/10/07 15:02:43.823] [executor:js_test:job0] 2015-10-07T19:02:43.823+0000 Running auth.js... [2015/10/07 15:02:43.823] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auth"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auth.js [2015/10/07 15:02:43.887] [executor:js_test:job0] 2015-10-07T19:02:43.887+0000 Writing output of JSTest jstests/sharding/auth.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156c54be07c4135fd7e565/. [2015/10/07 15:03:54.240] [executor:js_test:job0] 2015-10-07T19:03:54.240+0000 auth.js ran in 70.42 seconds. [2015/10/07 15:03:54.287] [executor:js_test:job0] 2015-10-07T19:03:54.287+0000 Running auth2.js... [2015/10/07 15:03:54.287] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auth2"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auth2.js [2015/10/07 15:03:54.351] [executor:js_test:job0] 2015-10-07T19:03:54.351+0000 Writing output of JSTest jstests/sharding/auth2.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156c9abe07c4135fd7eb18/. [2015/10/07 15:04:04.071] [executor:js_test:job0] 2015-10-07T19:04:04.071+0000 auth2.js ran in 9.78 seconds. [2015/10/07 15:04:04.077] [executor:js_test:job0] 2015-10-07T19:04:04.077+0000 Running auth_add_shard.js... [2015/10/07 15:04:04.077] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auth_add_shard"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auth_add_shard.js [2015/10/07 15:04:04.145] [executor:js_test:job0] 2015-10-07T19:04:04.145+0000 Writing output of JSTest jstests/sharding/auth_add_shard.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156ca490413021b1d7f744/. [2015/10/07 15:04:24.369] [executor:js_test:job0] 2015-10-07T19:04:24.369+0000 auth_add_shard.js ran in 20.29 seconds. [2015/10/07 15:04:24.372] [executor:js_test:job0] 2015-10-07T19:04:24.372+0000 Running auth_copydb.js... [2015/10/07 15:04:24.372] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auth_copydb"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auth_copydb.js [2015/10/07 15:04:24.437] [executor:js_test:job0] 2015-10-07T19:04:24.437+0000 Writing output of JSTest jstests/sharding/auth_copydb.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156cb890413021b1d7f8a9/. [2015/10/07 15:04:32.585] [executor:js_test:job0] 2015-10-07T19:04:32.584+0000 auth_copydb.js ran in 8.21 seconds. [2015/10/07 15:04:32.599] [executor:js_test:job0] 2015-10-07T19:04:32.599+0000 Running auth_repl.js... [2015/10/07 15:04:32.599] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auth_repl"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auth_repl.js [2015/10/07 15:04:32.662] [executor:js_test:job0] 2015-10-07T19:04:32.662+0000 Writing output of JSTest jstests/sharding/auth_repl.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156cc190413021b1d7f949/. [2015/10/07 15:04:59.082] [executor:js_test:job0] 2015-10-07T19:04:59.082+0000 auth_repl.js ran in 26.48 seconds. [2015/10/07 15:04:59.086] [executor:js_test:job0] 2015-10-07T19:04:59.086+0000 Running auth_slaveok_routing.js... [2015/10/07 15:04:59.086] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auth_slaveok_routing"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auth_slaveok_routing.js [2015/10/07 15:04:59.145] [executor:js_test:job0] 2015-10-07T19:04:59.145+0000 Writing output of JSTest jstests/sharding/auth_slaveok_routing.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156cdb90413021b1d7fb41/. [2015/10/07 15:05:53.745] [executor:js_test:job0] 2015-10-07T19:05:53.745+0000 auth_slaveok_routing.js ran in 54.66 seconds. [2015/10/07 15:05:53.747] [executor:js_test:job0] 2015-10-07T19:05:53.747+0000 Running authCommands.js... [2015/10/07 15:05:53.747] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "authCommands"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/authCommands.js [2015/10/07 15:05:53.831] [executor:js_test:job0] 2015-10-07T19:05:53.831+0000 Writing output of JSTest jstests/sharding/authCommands.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156d12be07c4135fd7f40f/. [2015/10/07 15:07:14.018] [executor:js_test:job0] 2015-10-07T19:07:14.018+0000 authCommands.js ran in 80.27 seconds. [2015/10/07 15:07:14.020] [executor:js_test:job0] 2015-10-07T19:07:14.020+0000 Running authConnectionHook.js... [2015/10/07 15:07:14.020] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "authConnectionHook"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/authConnectionHook.js [2015/10/07 15:07:14.089] [executor:js_test:job0] 2015-10-07T19:07:14.089+0000 Writing output of JSTest jstests/sharding/authConnectionHook.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156d6290413021b1d806c5/. [2015/10/07 15:07:26.439] [executor:js_test:job0] 2015-10-07T19:07:26.439+0000 authConnectionHook.js ran in 12.42 seconds. [2015/10/07 15:07:26.452] [executor:js_test:job0] 2015-10-07T19:07:26.452+0000 Running authmr.js... [2015/10/07 15:07:26.452] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "authmr"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/authmr.js [2015/10/07 15:07:26.518] [executor:js_test:job0] 2015-10-07T19:07:26.517+0000 Writing output of JSTest jstests/sharding/authmr.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156d6fbe07c4135fd7fd27/. [2015/10/07 15:07:33.590] [executor:js_test:job0] 2015-10-07T19:07:33.590+0000 authmr.js ran in 7.14 seconds. [2015/10/07 15:07:33.625] [executor:js_test:job0] 2015-10-07T19:07:33.625+0000 Running authwhere.js... [2015/10/07 15:07:33.625] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "authwhere"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/authwhere.js [2015/10/07 15:07:33.693] [executor:js_test:job0] 2015-10-07T19:07:33.693+0000 Writing output of JSTest jstests/sharding/authwhere.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156d7690413021b1d80942/. [2015/10/07 15:07:40.741] [executor:js_test:job0] 2015-10-07T19:07:40.741+0000 authwhere.js ran in 7.12 seconds. [2015/10/07 15:07:40.794] [executor:js_test:job0] 2015-10-07T19:07:40.794+0000 Running auto1.js... [2015/10/07 15:07:40.794] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auto1"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auto1.js [2015/10/07 15:07:40.869] [executor:js_test:job0] 2015-10-07T19:07:40.869+0000 Writing output of JSTest jstests/sharding/auto1.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156d7d90413021b1d80a0c/. [2015/10/07 15:07:49.670] [executor:js_test:job0] 2015-10-07T19:07:49.670+0000 auto1.js ran in 8.88 seconds. [2015/10/07 15:07:49.693] [executor:js_test:job0] 2015-10-07T19:07:49.693+0000 Running auto2.js... [2015/10/07 15:07:49.693] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auto2"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auto2.js [2015/10/07 15:07:49.768] [executor:js_test:job0] 2015-10-07T19:07:49.767+0000 Writing output of JSTest jstests/sharding/auto2.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156d8690413021b1d80b40/. [2015/10/07 15:08:11.854] [executor:js_test:job0] 2015-10-07T19:08:11.854+0000 auto2.js ran in 22.16 seconds. [2015/10/07 15:08:11.856] [executor:js_test:job0] 2015-10-07T19:08:11.856+0000 Running auto_rebalance.js... [2015/10/07 15:08:11.856] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "auto_rebalance"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/auto_rebalance.js [2015/10/07 15:08:11.922] [executor:js_test:job0] 2015-10-07T19:08:11.922+0000 Writing output of JSTest jstests/sharding/auto_rebalance.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156d9cbe07c4135fd802df/. [2015/10/07 15:09:05.882] [executor:js_test:job0] 2015-10-07T19:09:05.882+0000 auto_rebalance.js ran in 54.03 seconds. [2015/10/07 15:09:05.885] [executor:js_test:job0] 2015-10-07T19:09:05.885+0000 Running balance_tags1.js... [2015/10/07 15:09:05.885] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "balance_tags1"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/balance_tags1.js [2015/10/07 15:09:05.951] [executor:js_test:job0] 2015-10-07T19:09:05.951+0000 Writing output of JSTest jstests/sharding/balance_tags1.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156dd290413021b1d814fd/. [2015/10/07 15:09:57.016] [executor:js_test:job0] 2015-10-07T19:09:57.015+0000 balance_tags1.js ran in 51.13 seconds. [2015/10/07 15:09:57.017] [executor:js_test:job0] 2015-10-07T19:09:57.017+0000 Running balance_tags2.js... [2015/10/07 15:09:57.017] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "balance_tags2"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/balance_tags2.js [2015/10/07 15:09:57.083] [executor:js_test:job0] 2015-10-07T19:09:57.083+0000 Writing output of JSTest jstests/sharding/balance_tags2.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e05be07c4135fd80e3d/. [2015/10/07 15:10:56.138] [executor:js_test:job0] 2015-10-07T19:10:56.138+0000 balance_tags2.js ran in 59.12 seconds. [2015/10/07 15:10:56.143] [executor:js_test:job0] 2015-10-07T19:10:56.143+0000 Running bouncing_count.js... [2015/10/07 15:10:56.143] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "bouncing_count"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/bouncing_count.js [2015/10/07 15:10:56.220] [executor:js_test:job0] 2015-10-07T19:10:56.219+0000 Writing output of JSTest jstests/sharding/bouncing_count.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e4090413021b1d81e67/. [2015/10/07 15:11:18.229] [executor:js_test:job0] 2015-10-07T19:11:18.229+0000 bouncing_count.js ran in 22.09 seconds. [2015/10/07 15:11:18.232] [executor:js_test:job0] 2015-10-07T19:11:18.232+0000 Running bulk_insert.js... [2015/10/07 15:11:18.232] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "bulk_insert"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/bulk_insert.js [2015/10/07 15:11:18.320] [executor:js_test:job0] 2015-10-07T19:11:18.320+0000 Writing output of JSTest jstests/sharding/bulk_insert.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e5690413021b1d81ff2/. [2015/10/07 15:11:28.592] [executor:js_test:job0] 2015-10-07T19:11:28.592+0000 bulk_insert.js ran in 10.36 seconds. [2015/10/07 15:11:28.594] [executor:js_test:job0] 2015-10-07T19:11:28.594+0000 Running cleanup_orphaned_cmd.js... [2015/10/07 15:11:28.594] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "cleanup_orphaned_cmd"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/cleanup_orphaned_cmd.js [2015/10/07 15:11:28.653] [executor:js_test:job0] 2015-10-07T19:11:28.653+0000 Writing output of JSTest jstests/sharding/cleanup_orphaned_cmd.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e6190413021b1d82067/. [2015/10/07 15:11:36.745] [executor:js_test:job0] 2015-10-07T19:11:36.745+0000 cleanup_orphaned_cmd.js ran in 8.15 seconds. [2015/10/07 15:11:36.763] [executor:js_test:job0] 2015-10-07T19:11:36.763+0000 Running cleanup_orphaned_cmd_hashed.js... [2015/10/07 15:11:36.763] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "cleanup_orphaned_cmd_hashed"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/cleanup_orphaned_cmd_hashed.js [2015/10/07 15:11:36.832] [executor:js_test:job0] 2015-10-07T19:11:36.832+0000 Writing output of JSTest jstests/sharding/cleanup_orphaned_cmd_hashed.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e6990413021b1d820ed/. [2015/10/07 15:11:45.218] [executor:js_test:job0] 2015-10-07T19:11:45.218+0000 cleanup_orphaned_cmd_hashed.js ran in 8.46 seconds. [2015/10/07 15:11:45.232] [executor:js_test:job0] 2015-10-07T19:11:45.232+0000 Running cleanup_orphaned_cmd_prereload.js... [2015/10/07 15:11:45.232] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "cleanup_orphaned_cmd_prereload"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/cleanup_orphaned_cmd_prereload.js [2015/10/07 15:11:45.305] [executor:js_test:job0] 2015-10-07T19:11:45.305+0000 Writing output of JSTest jstests/sharding/cleanup_orphaned_cmd_prereload.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e7190413021b1d82153/. [2015/10/07 15:12:03.686] [executor:js_test:job0] 2015-10-07T19:12:03.686+0000 cleanup_orphaned_cmd_prereload.js ran in 18.45 seconds. [2015/10/07 15:12:03.697] [executor:js_test:job0] 2015-10-07T19:12:03.696+0000 Running coll_epoch_test0.js... [2015/10/07 15:12:03.697] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "coll_epoch_test0"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/coll_epoch_test0.js [2015/10/07 15:12:03.757] [executor:js_test:job0] 2015-10-07T19:12:03.756+0000 Writing output of JSTest jstests/sharding/coll_epoch_test0.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e84be07c4135fd816bd/. [2015/10/07 15:12:11.600] [executor:js_test:job0] 2015-10-07T19:12:11.600+0000 coll_epoch_test0.js ran in 7.90 seconds. [2015/10/07 15:12:11.611] [executor:js_test:job0] 2015-10-07T19:12:11.611+0000 Running coll_epoch_test1.js... [2015/10/07 15:12:11.611] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "coll_epoch_test1"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/coll_epoch_test1.js [2015/10/07 15:12:11.691] [executor:js_test:job0] 2015-10-07T19:12:11.691+0000 Writing output of JSTest jstests/sharding/coll_epoch_test1.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e8c90413021b1d822c4/. [2015/10/07 15:12:23.604] [executor:js_test:job0] 2015-10-07T19:12:23.604+0000 coll_epoch_test1.js ran in 11.99 seconds. [2015/10/07 15:12:23.606] [executor:js_test:job0] 2015-10-07T19:12:23.606+0000 Running coll_epoch_test2.js... [2015/10/07 15:12:23.606] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "coll_epoch_test2"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/coll_epoch_test2.js [2015/10/07 15:12:23.680] [executor:js_test:job0] 2015-10-07T19:12:23.680+0000 Writing output of JSTest jstests/sharding/coll_epoch_test2.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156e98be07c4135fd81820/. [2015/10/07 15:12:36.758] [executor:js_test:job0] 2015-10-07T19:12:36.758+0000 coll_epoch_test2.js ran in 13.15 seconds. [2015/10/07 15:12:36.760] [executor:js_test:job0] 2015-10-07T19:12:36.760+0000 Running conf_server_write_concern.js... [2015/10/07 15:12:36.760] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "conf_server_write_concern"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/conf_server_write_concern.js [2015/10/07 15:12:36.843] [executor:js_test:job0] 2015-10-07T19:12:36.843+0000 Writing output of JSTest jstests/sharding/conf_server_write_concern.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56156ea590413021b1d82444/. [2015/10/07 17:12:36.848] Task timed out: ''shell.exec' in "run tests"' [2015/10/07 17:12:36.848] Running task-timeout commands. [2015/10/07 17:12:36.848] Running command 'shell.exec' (step 1 of 1) [2015/10/07 17:12:36.856] Calling the hang analyzer... [2015/10/07 17:12:36.904] Python Version: 2.6.8 (unknown, Nov 7 2012, 14:47:45) [2015/10/07 17:12:36.904] [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] [2015/10/07 17:12:36.904] OSLinux-2.6.18-194.el5xen-x86_64-with-redhat-5.5-Tikanga [2015/10/07 17:12:36.904] Linux Distribution: ('Red Hat Enterprise Linux Server', '5.5', 'Tikanga') [2015/10/07 17:12:36.904] Current User: 0 [2015/10/07 17:12:36.904] Cannot determine Unix Current Login [2015/10/07 17:12:36.904] INFO: Getting list of processes using /bin/ps [2015/10/07 17:12:36.904] ['/bin/ps', '--version'] [2015/10/07 17:12:36.906] procps version 3.2.7 [2015/10/07 17:12:36.907] ['/bin/ps', '-eo', 'pid,args'] [2015/10/07 17:12:36.915] INFO: Done analyzing process [2015/10/07 17:12:36.915] Found 9 interesting processes [2015/10/07 17:12:36.915] Dumping process 3087 of mongo [2015/10/07 17:12:36.915] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3087 [2015/10/07 17:12:36.915] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:37.020] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:37.020] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:37.020] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:37.020] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:37.020] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:37.020] and "show warranty" for details. [2015/10/07 17:12:37.020] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:37.020] For bug reporting instructions, please see: [2015/10/07 17:12:37.020] . [2015/10/07 17:12:37.020] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3087', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:37.040] Attaching to process 3087 [2015/10/07 17:12:37.141] Reading symbols from /data/mci/src/mongo...(no debugging symbols found)...done. [2015/10/07 17:12:37.155] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:37.155] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:37.156] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.156] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:37.166] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.166] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:37.168] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.168] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:37.168] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:37.168] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:37.169] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:37.176] [New LWP 3708] [2015/10/07 17:12:37.176] [New LWP 3682] [2015/10/07 17:12:37.176] [New LWP 3662] [2015/10/07 17:12:37.176] [New LWP 3642] [2015/10/07 17:12:37.176] [New LWP 3636] [2015/10/07 17:12:37.176] [New LWP 3342] [2015/10/07 17:12:37.176] [New LWP 3319] [2015/10/07 17:12:37.176] [New LWP 3296] [2015/10/07 17:12:37.176] [New LWP 3099] [2015/10/07 17:12:37.176] [New LWP 3098] [2015/10/07 17:12:37.176] [New LWP 3097] [2015/10/07 17:12:37.176] [New LWP 3096] [2015/10/07 17:12:37.176] [New LWP 3095] [2015/10/07 17:12:37.176] [New LWP 3094] [2015/10/07 17:12:37.176] [New LWP 3093] [2015/10/07 17:12:37.176] [New LWP 3092] [2015/10/07 17:12:37.176] [New LWP 3091] [2015/10/07 17:12:37.176] [New LWP 3090] [2015/10/07 17:12:37.176] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:37.176] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:37.176] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:37.183] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.183] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:37.184] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.184] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:37.184] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.184] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:37.185] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.185] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:37.185] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.193] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:37.193] 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.193] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.195] Thread 19 (Thread 0x2b92b124b940 (LWP 3090)): [2015/10/07 17:12:37.195] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.195] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.195] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.195] #2 0x0000000000833198 in mongo::DeadlineMonitor::deadlineMonitorThread() () [2015/10/07 17:12:37.195] #3 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.195] #4 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.196] #5 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.196] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.196] Thread 18 (Thread 0x2b92b134c940 (LWP 3091)): [2015/10/07 17:12:37.196] #0 0x00002b92b02e0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.196] #1 0x00000000008b7a6b in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.196] #2 0x00000000008b7ab1 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.196] #3 0x00000000008b7b0d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.197] #4 0x00000000008b0c12 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.197] #5 0x00000000008b1b22 in mongo::MessagingPort::recv(mongo::Message const&, mongo::Message&) () [2015/10/07 17:12:37.197] #6 0x00000000007359af in mongo::DBClientConnection::call(mongo::Message&, mongo::Message&, bool, std::string*) () [2015/10/07 17:12:37.197] #7 0x00000000007484ca in mongo::DBClientCursor::init() () [2015/10/07 17:12:37.197] #8 0x0000000000728b29 in mongo::DBClientBase::query(std::string const&, mongo::Query, int, int, mongo::BSONObj const*, int, int) () [2015/10/07 17:12:37.197] #9 0x000000000073a3bf in mongo::DBClientConnection::query(std::string const&, mongo::Query, int, int, mongo::BSONObj const*, int, int) () [2015/10/07 17:12:37.197] #10 0x00000000008498a8 in mongo::mozjs::MongoBase::Functions::find::call(JSContext*, JS::CallArgs) () [2015/10/07 17:12:37.197] #11 0x000000000084d2a2 in bool mongo::mozjs::smUtils::wrapConstrainedMethod(JSContext*, unsigned int, JS::Value*) () [2015/10/07 17:12:37.197] #12 0x0000000000ad5578 in js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct) () [2015/10/07 17:12:37.197] #13 0x0000000000ad0a47 in Interpret(JSContext*, js::RunState&) () [2015/10/07 17:12:37.197] #14 0x0000000000ad534e in js::RunScript(JSContext*, js::RunState&) () [2015/10/07 17:12:37.197] #15 0x0000000000ae4ad5 in js::ExecuteKernel(JSContext*, JS::Handle, JSObject&, JS::Value const&, js::ExecuteType, js::AbstractFramePtr, JS::Value*) () [2015/10/07 17:12:37.197] #16 0x0000000000ae4e24 in js::Execute(JSContext*, JS::Handle, JSObject&, JS::Value*) () [2015/10/07 17:12:37.197] #17 0x0000000000dbd3e3 in ExecuteScript(JSContext*, JS::Handle, JS::Handle, JS::Value*) () [2015/10/07 17:12:37.198] #18 0x000000000083e668 in mongo::mozjs::MozJSImplScope::exec(mongo::StringData, std::string const&, bool, bool, bool, int) () [2015/10/07 17:12:37.198] #19 0x000000000085be14 in std::_Function_handler::_M_invoke(std::_Any_data const&) () [2015/10/07 17:12:37.198] #20 0x000000000085bf64 in mongo::mozjs::MozJSProxyScope::implThread(void*) () [2015/10/07 17:12:37.198] #21 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.198] #22 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.198] #23 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.198] #24 0x0000000000000000 in ?? () [2015/10/07 17:12:37.198] Thread 17 (Thread 0x2b92b13cd940 (LWP 3092)): [2015/10/07 17:12:37.198] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.198] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.198] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.198] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.198] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.198] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.198] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.198] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.198] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.199] Thread 16 (Thread 0x2b92b144e940 (LWP 3093)): [2015/10/07 17:12:37.199] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.199] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.199] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.199] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.199] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.199] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.199] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.199] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.199] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.199] Thread 15 (Thread 0x2b92b14cf940 (LWP 3094)): [2015/10/07 17:12:37.199] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.199] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.199] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.199] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.199] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.199] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.199] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.199] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.200] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.200] Thread 14 (Thread 0x2b92b1550940 (LWP 3095)): [2015/10/07 17:12:37.200] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.200] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.200] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.200] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.200] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.200] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.200] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.200] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.200] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.200] Thread 13 (Thread 0x2b92b15d1940 (LWP 3096)): [2015/10/07 17:12:37.200] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.200] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.200] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.200] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.200] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.200] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.200] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.200] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.201] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.201] Thread 12 (Thread 0x2b92b1652940 (LWP 3097)): [2015/10/07 17:12:37.201] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.201] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.201] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.201] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.201] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.201] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.201] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.201] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.201] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.201] Thread 11 (Thread 0x2b92b16d3940 (LWP 3098)): [2015/10/07 17:12:37.201] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.201] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.201] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.201] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.201] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.201] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.201] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.202] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.202] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.202] Thread 10 (Thread 0x2b92b1754940 (LWP 3099)): [2015/10/07 17:12:37.202] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.202] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.202] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.202] #2 0x0000000000827fbc in PR_WaitCondVar(nspr::CondVar*, unsigned int) () [2015/10/07 17:12:37.202] #3 0x0000000000aa7a92 in js::HelperThread::threadLoop() () [2015/10/07 17:12:37.202] #4 0x0000000000828036 in nspr::Thread::ThreadRoutine(void*) () [2015/10/07 17:12:37.202] #5 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.202] #6 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.202] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.202] Thread 9 (Thread 0x2b92b6505940 (LWP 3296)): [2015/10/07 17:12:37.202] #0 0x00002b92b02e0aab in read () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.202] #1 0x0000000000892fc8 in mongo::shell_utils::ProgramRunner::operator()() () [2015/10/07 17:12:37.202] #2 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.202] #3 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.202] #4 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.203] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.203] Thread 8 (Thread 0x2b92b5b04940 (LWP 3319)): [2015/10/07 17:12:37.203] #0 0x00002b92b02e0aab in read () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.203] #1 0x0000000000892fc8 in mongo::shell_utils::ProgramRunner::operator()() () [2015/10/07 17:12:37.203] #2 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.203] #3 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.203] #4 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.203] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.203] Thread 7 (Thread 0x2b92b3d01940 (LWP 3342)): [2015/10/07 17:12:37.203] #0 0x00002b92b02e0aab in read () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.203] #1 0x0000000000892fc8 in mongo::shell_utils::ProgramRunner::operator()() () [2015/10/07 17:12:37.203] #2 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.203] #3 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.203] #4 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.203] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.203] Thread 6 (Thread 0x2b92b3300940 (LWP 3636)): [2015/10/07 17:12:37.204] #0 0x00002b92b02de280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.204] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.204] #1 0x00000000007556cd in mongo::(anonymous namespace)::ReplicaSetMonitorWatcher::run() () [2015/10/07 17:12:37.204] #2 0x000000000089c9a7 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.204] #3 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.204] #4 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.204] #5 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.204] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.204] Thread 5 (Thread 0x2b92b4702940 (LWP 3642)): [2015/10/07 17:12:37.204] #0 0x00002b92b02e0aab in read () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.204] #1 0x0000000000892fc8 in mongo::shell_utils::ProgramRunner::operator()() () [2015/10/07 17:12:37.204] #2 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.204] #3 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.204] #4 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.204] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.204] Thread 4 (Thread 0x2b92b5103940 (LWP 3662)): [2015/10/07 17:12:37.204] #0 0x00002b92b02e0aab in read () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.204] #1 0x0000000000892fc8 in mongo::shell_utils::ProgramRunner::operator()() () [2015/10/07 17:12:37.205] #2 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.205] #3 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.205] #4 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.205] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.205] Thread 3 (Thread 0x2b92b6f06940 (LWP 3682)): [2015/10/07 17:12:37.205] #0 0x00002b92b02e0aab in read () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.205] #1 0x0000000000892fc8 in mongo::shell_utils::ProgramRunner::operator()() () [2015/10/07 17:12:37.205] #2 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.205] #3 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.205] #4 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.205] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.205] Thread 2 (Thread 0x2b92b8100940 (LWP 3708)): [2015/10/07 17:12:37.205] #0 0x00002b92b02e0aab in read () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.205] #1 0x0000000000892fc8 in mongo::shell_utils::ProgramRunner::operator()() () [2015/10/07 17:12:37.205] #2 0x0000000000f5a750 in execute_native_thread_routine () [2015/10/07 17:12:37.205] #3 0x00002b92b02d983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.205] #4 0x00002b92b05c4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.206] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.206] Thread 1 (Thread 0x2b92b0849e40 (LWP 3087)): [2015/10/07 17:12:37.206] #0 0x00002b92b02de019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.206] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.206] #1 0x0000000000f59edc in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.206] #2 0x0000000000859e5b in mongo::mozjs::MozJSProxyScope::runOnImplThread(std::function) () [2015/10/07 17:12:37.206] #3 0x000000000085b5b9 in mongo::mozjs::MozJSProxyScope::exec(mongo::StringData, std::string const&, bool, bool, bool, int) () [2015/10/07 17:12:37.206] #4 0x00000000008255c3 in mongo::Scope::execFile(std::string const&, bool, bool, int) () [2015/10/07 17:12:37.206] #5 0x00000000006ca17a in _main(int, char**, char**) () [2015/10/07 17:12:37.206] #6 0x00000000006cf15b in main () [2015/10/07 17:12:37.206] Detaching from program: /data/mci/src/mongo, process 3087 [2015/10/07 17:12:37.211] INFO: Done analyzing process [2015/10/07 17:12:37.211] Dumping process 3295 of mongod [2015/10/07 17:12:37.211] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3295 [2015/10/07 17:12:37.211] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:37.231] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:37.231] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:37.231] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:37.231] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:37.231] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:37.231] and "show warranty" for details. [2015/10/07 17:12:37.231] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:37.231] For bug reporting instructions, please see: [2015/10/07 17:12:37.231] . [2015/10/07 17:12:37.231] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3295', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:37.251] Attaching to process 3295 [2015/10/07 17:12:37.441] Reading symbols from /data/mci/src/mongod...(no debugging symbols found)...done. [2015/10/07 17:12:37.456] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:37.457] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:37.457] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.457] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:37.468] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.468] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:37.469] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.469] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:37.470] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:37.470] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:37.471] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:37.494] [New LWP 3831] [2015/10/07 17:12:37.494] [New LWP 3819] [2015/10/07 17:12:37.494] [New LWP 3818] [2015/10/07 17:12:37.494] [New LWP 3817] [2015/10/07 17:12:37.494] [New LWP 3810] [2015/10/07 17:12:37.494] [New LWP 3808] [2015/10/07 17:12:37.494] [New LWP 3806] [2015/10/07 17:12:37.494] [New LWP 3804] [2015/10/07 17:12:37.494] [New LWP 3797] [2015/10/07 17:12:37.494] [New LWP 3796] [2015/10/07 17:12:37.494] [New LWP 3795] [2015/10/07 17:12:37.494] [New LWP 3794] [2015/10/07 17:12:37.494] [New LWP 3789] [2015/10/07 17:12:37.494] [New LWP 3640] [2015/10/07 17:12:37.495] [New LWP 3436] [2015/10/07 17:12:37.495] [New LWP 3435] [2015/10/07 17:12:37.495] [New LWP 3434] [2015/10/07 17:12:37.495] [New LWP 3433] [2015/10/07 17:12:37.495] [New LWP 3432] [2015/10/07 17:12:37.495] [New LWP 3431] [2015/10/07 17:12:37.495] [New LWP 3430] [2015/10/07 17:12:37.495] [New LWP 3429] [2015/10/07 17:12:37.495] [New LWP 3428] [2015/10/07 17:12:37.495] [New LWP 3427] [2015/10/07 17:12:37.495] [New LWP 3426] [2015/10/07 17:12:37.495] [New LWP 3425] [2015/10/07 17:12:37.495] [New LWP 3424] [2015/10/07 17:12:37.495] [New LWP 3423] [2015/10/07 17:12:37.495] [New LWP 3422] [2015/10/07 17:12:37.495] [New LWP 3421] [2015/10/07 17:12:37.495] [New LWP 3420] [2015/10/07 17:12:37.495] [New LWP 3419] [2015/10/07 17:12:37.495] [New LWP 3418] [2015/10/07 17:12:37.495] [New LWP 3417] [2015/10/07 17:12:37.495] [New LWP 3416] [2015/10/07 17:12:37.495] [New LWP 3415] [2015/10/07 17:12:37.495] [New LWP 3414] [2015/10/07 17:12:37.495] [New LWP 3413] [2015/10/07 17:12:37.495] [New LWP 3412] [2015/10/07 17:12:37.495] [New LWP 3411] [2015/10/07 17:12:37.495] [New LWP 3410] [2015/10/07 17:12:37.495] [New LWP 3409] [2015/10/07 17:12:37.495] [New LWP 3408] [2015/10/07 17:12:37.495] [New LWP 3407] [2015/10/07 17:12:37.495] [New LWP 3406] [2015/10/07 17:12:37.495] [New LWP 3405] [2015/10/07 17:12:37.495] [New LWP 3404] [2015/10/07 17:12:37.495] [New LWP 3403] [2015/10/07 17:12:37.495] [New LWP 3402] [2015/10/07 17:12:37.495] [New LWP 3401] [2015/10/07 17:12:37.495] [New LWP 3400] [2015/10/07 17:12:37.495] [New LWP 3399] [2015/10/07 17:12:37.495] [New LWP 3384] [2015/10/07 17:12:37.495] [New LWP 3374] [2015/10/07 17:12:37.495] [New LWP 3373] [2015/10/07 17:12:37.495] [New LWP 3368] [2015/10/07 17:12:37.495] [New LWP 3317] [2015/10/07 17:12:37.495] [New LWP 3315] [2015/10/07 17:12:37.495] [New LWP 3314] [2015/10/07 17:12:37.495] [New LWP 3313] [2015/10/07 17:12:37.495] [New LWP 3312] [2015/10/07 17:12:37.495] [New LWP 3311] [2015/10/07 17:12:37.495] [New LWP 3310] [2015/10/07 17:12:37.495] [New LWP 3309] [2015/10/07 17:12:37.495] [New LWP 3308] [2015/10/07 17:12:37.495] [New LWP 3307] [2015/10/07 17:12:37.495] [New LWP 3306] [2015/10/07 17:12:37.495] [New LWP 3305] [2015/10/07 17:12:37.495] [New LWP 3304] [2015/10/07 17:12:37.495] [New LWP 3303] [2015/10/07 17:12:37.495] [New LWP 3302] [2015/10/07 17:12:37.495] [New LWP 3301] [2015/10/07 17:12:37.495] [New LWP 3300] [2015/10/07 17:12:37.495] [New LWP 3299] [2015/10/07 17:12:37.495] [New LWP 3298] [2015/10/07 17:12:37.495] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:37.495] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:37.495] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:37.502] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.502] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:37.502] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.502] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:37.503] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.503] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:37.503] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.503] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:37.504] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.514] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:37.514] 0x00002b6b5555e172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:37.521] Thread 76 (Thread 0x2b6b561ec940 (LWP 3298)): [2015/10/07 17:12:37.521] #0 0x00002b6b552819c8 in do_sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.521] #1 0x00002b6b55281a6d in sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.521] #2 0x0000000001265316 in mongo::(anonymous namespace)::signalProcessingThread() () [2015/10/07 17:12:37.521] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.521] #4 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.521] #5 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.521] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.521] Thread 75 (Thread 0x2b6b56bed940 (LWP 3299)): [2015/10/07 17:12:37.521] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.521] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.521] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.522] #2 0x00000000018ea002 in __evict_server () [2015/10/07 17:12:37.522] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.522] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.522] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.522] Thread 74 (Thread 0x2b6b575ee940 (LWP 3300)): [2015/10/07 17:12:37.522] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.522] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.522] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.522] #2 0x00000000018cce33 in __sweep_server () [2015/10/07 17:12:37.522] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.522] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.522] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.522] Thread 73 (Thread 0x2b6b57fef940 (LWP 3301)): [2015/10/07 17:12:37.522] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.522] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.522] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.522] #2 0x00000000018c9b90 in __log_file_server () [2015/10/07 17:12:37.522] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.522] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.523] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.523] Thread 72 (Thread 0x2b6b589f0940 (LWP 3302)): [2015/10/07 17:12:37.523] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.523] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.523] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.523] #2 0x00000000018ca2f4 in __log_wrlsn_server () [2015/10/07 17:12:37.523] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.523] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.523] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.523] Thread 71 (Thread 0x2b6b593f1940 (LWP 3303)): [2015/10/07 17:12:37.523] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.523] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.523] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.523] #2 0x00000000018c96b7 in __log_server () [2015/10/07 17:12:37.523] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.523] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.523] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.523] Thread 70 (Thread 0x2b6b59df2940 (LWP 3304)): [2015/10/07 17:12:37.523] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.523] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.523] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.524] #2 0x00000000018c7239 in __ckpt_server () [2015/10/07 17:12:37.524] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.524] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.524] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.524] Thread 69 (Thread 0x2b6b5a7f3940 (LWP 3305)): [2015/10/07 17:12:37.524] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.524] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.524] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.524] #2 0x00000000011ced98 in mongo::DeadlineMonitor::deadlineMonitorThread() () [2015/10/07 17:12:37.524] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.524] #4 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.524] #5 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.524] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.524] Thread 68 (Thread 0x2b6b5b1f4940 (LWP 3306)): [2015/10/07 17:12:37.524] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.524] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.524] #1 0x0000000000dba6e3 in mongo::RangeDeleter::doWork() () [2015/10/07 17:12:37.524] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.524] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.524] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.525] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.525] Thread 67 (Thread 0x2b6b5bbf5940 (LWP 3307)): [2015/10/07 17:12:37.525] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.525] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.525] #1 0x000000000102d164 in mongo::executor::NetworkInterfaceASIO::waitForWorkUntil(mongo::Date_t) () [2015/10/07 17:12:37.525] #2 0x0000000000e8609d in mongo::repl::ReplicationExecutor::getWork() () [2015/10/07 17:12:37.525] #3 0x0000000000e88bbd in mongo::repl::ReplicationExecutor::run() () [2015/10/07 17:12:37.525] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.525] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.525] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.525] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.525] Thread 66 (Thread 0x2b6b5c5f6940 (LWP 3308)): [2015/10/07 17:12:37.525] #0 0x00002b6b55281351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.525] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:37.525] #2 0x00000000010137eb in mongo::TTLMonitor::run() () [2015/10/07 17:12:37.525] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.525] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.525] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.526] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.526] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.526] Thread 65 (Thread 0x2b6b5cff7940 (LWP 3309)): [2015/10/07 17:12:37.526] #0 0x00002b6b555653c8 in epoll_wait () from /lib64/libc.so.6 [2015/10/07 17:12:37.526] #1 0x0000000001282e35 in asio::detail::epoll_reactor::run(bool, asio::detail::op_queue&) () [2015/10/07 17:12:37.526] #2 0x0000000001283988 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.526] #3 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.526] #4 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.526] #5 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:37.526] #6 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.526] #7 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.526] #8 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.526] #9 0x0000000000000000 in ?? () [2015/10/07 17:12:37.526] Thread 64 (Thread 0x2b6b5d9f8940 (LWP 3310)): [2015/10/07 17:12:37.526] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.526] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.526] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.526] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.527] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.527] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.527] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.527] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.527] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.527] Thread 63 (Thread 0x2b6b5e3f9940 (LWP 3311)): [2015/10/07 17:12:37.527] #0 0x00002b6b55281351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.527] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:37.527] #2 0x0000000000a61378 in mongo::ClientCursorMonitor::run() () [2015/10/07 17:12:37.527] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.527] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.527] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.527] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.527] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.527] Thread 62 (Thread 0x2b6b5edfa940 (LWP 3312)): [2015/10/07 17:12:37.527] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.527] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.527] #1 0x00000000012055ce in mongo::(anonymous namespace)::PeriodicTaskRunner::run() () [2015/10/07 17:12:37.528] #2 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.528] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.528] #4 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.528] #5 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.528] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.528] Thread 61 (Thread 0x2b6b5f7fb940 (LWP 3313)): [2015/10/07 17:12:37.528] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.528] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.528] #1 0x0000000000b95a66 in mongo::FTDCController::doLoop() () [2015/10/07 17:12:37.528] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.528] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.528] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.528] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.528] Thread 60 (Thread 0x2b6b601fc940 (LWP 3314)): [2015/10/07 17:12:37.528] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.528] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.528] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.528] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.528] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.529] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.529] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.529] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.529] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.529] Thread 59 (Thread 0x2b6b60bfd940 (LWP 3315)): [2015/10/07 17:12:37.529] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.529] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.529] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.529] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.529] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.529] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.529] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.529] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.529] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.529] Thread 58 (Thread 0x2b6b60cff940 (LWP 3317)): [2015/10/07 17:12:37.529] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.529] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.529] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.530] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.530] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.530] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.530] () [2015/10/07 17:12:37.530] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.530] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.530] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.530] Thread 57 (Thread 0x2b6b61d31940 (LWP 3368)): [2015/10/07 17:12:37.530] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.530] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.530] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.530] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.530] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.530] #4 0x000000000127b8a0 in asio_detail_posix_thread_function () [2015/10/07 17:12:37.530] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.530] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.530] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.530] Thread 56 (Thread 0x2b6b61e32940 (LWP 3373)): [2015/10/07 17:12:37.531] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.531] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.531] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.531] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.531] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.531] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.531] () [2015/10/07 17:12:37.531] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.531] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.531] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.531] Thread 55 (Thread 0x2b6b61f33940 (LWP 3374)): [2015/10/07 17:12:37.531] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.531] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.531] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.531] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.531] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.531] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.531] () [2015/10/07 17:12:37.531] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.532] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.532] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.532] Thread 54 (Thread 0x2b6b62a35940 (LWP 3384)): [2015/10/07 17:12:37.532] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.532] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.532] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.532] #2 0x0000000000ff96e1 in mongo::WiredTigerRecordStore::OplogStones::awaitHasExcessStonesOrDead() () [2015/10/07 17:12:37.532] #3 0x0000000000ffa460 in mongo::WiredTigerRecordStore::yieldAndAwaitOplogDeletionRequest(mongo::OperationContext*) () [2015/10/07 17:12:37.532] #4 0x0000000000fffd27 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::_deleteExcessDocuments() () [2015/10/07 17:12:37.532] #5 0x0000000001000238 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::run() () [2015/10/07 17:12:37.532] #6 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.532] #7 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.532] #8 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.532] #9 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.532] #10 0x0000000000000000 in ?? () [2015/10/07 17:12:37.532] Thread 53 (Thread 0x2b6b63436940 (LWP 3399)): [2015/10/07 17:12:37.532] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.532] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.532] #1 0x0000000000dc5888 in mongo::repl::BackgroundSync::waitForMore() () [2015/10/07 17:12:37.533] #2 0x0000000000eb52b9 in mongo::repl::SyncTail::tryPopAndWaitForMore(mongo::OperationContext*, mongo::repl::SyncTail::OpQueue*, mongo::repl::ReplicationCoordinator*) () [2015/10/07 17:12:37.533] #3 0x0000000000eb621e in mongo::repl::SyncTail::oplogApplication() () [2015/10/07 17:12:37.533] #4 0x0000000000ead424 in mongo::repl::runSyncThread() () [2015/10/07 17:12:37.533] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.533] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.533] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.533] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.533] Thread 52 (Thread 0x2b6b63e37940 (LWP 3400)): [2015/10/07 17:12:37.533] #0 0x00002b6b55281351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.533] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:37.533] #2 0x0000000000dc9212 in mongo::repl::BackgroundSync::_producerThread() () [2015/10/07 17:12:37.533] #3 0x0000000000dc9305 in mongo::repl::BackgroundSync::producerThread() () [2015/10/07 17:12:37.533] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.533] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.533] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.533] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.534] Thread 51 (Thread 0x2b6b64838940 (LWP 3401)): [2015/10/07 17:12:37.534] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.534] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.534] #1 0x0000000000eb0dbd in mongo::repl::SyncSourceFeedback::run() () [2015/10/07 17:12:37.534] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.534] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.534] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.534] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.534] Thread 50 (Thread 0x2b6b65239940 (LWP 3402)): [2015/10/07 17:12:37.534] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.534] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.534] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.534] #2 0x0000000000e1f69d in mongo::repl::SnapshotThread::run() () [2015/10/07 17:12:37.534] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.534] #4 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.534] #5 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.534] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.534] Thread 49 (Thread 0x2b6b65c3a940 (LWP 3403)): [2015/10/07 17:12:37.534] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.534] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.534] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.535] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.535] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.535] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.535] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.535] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.535] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.535] Thread 48 (Thread 0x2b6b6663b940 (LWP 3404)): [2015/10/07 17:12:37.535] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.535] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.535] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.535] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.535] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.535] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.535] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.535] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.535] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.535] Thread 47 (Thread 0x2b6b6703c940 (LWP 3405)): [2015/10/07 17:12:37.535] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.535] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.535] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.536] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.536] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.536] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.536] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.536] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.536] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.536] Thread 46 (Thread 0x2b6b67a3d940 (LWP 3406)): [2015/10/07 17:12:37.536] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.536] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.536] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.536] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.536] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.536] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:37.536] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.536] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.536] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.536] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.537] Thread 45 (Thread 0x2b6b6843e940 (LWP 3407)): [2015/10/07 17:12:37.537] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.537] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.537] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.537] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.537] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.537] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.537] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.537] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.537] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.537] Thread 44 (Thread 0x2b6b68e3f940 (LWP 3408)): [2015/10/07 17:12:37.537] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.537] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.537] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.537] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.537] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.537] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.537] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.537] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.537] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.538] Thread 43 (Thread 0x2b6b69840940 (LWP 3409)): [2015/10/07 17:12:37.538] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.538] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.538] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.538] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.538] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.538] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.538] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.538] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.538] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.538] Thread 42 (Thread 0x2b6b6a241940 (LWP 3410)): [2015/10/07 17:12:37.538] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.538] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.538] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.538] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.538] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.538] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.538] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.538] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.539] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.539] Thread 41 (Thread 0x2b6b6ac42940 (LWP 3411)): [2015/10/07 17:12:37.539] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.539] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.539] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.539] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.539] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.539] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.539] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.539] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.539] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.539] Thread 40 (Thread 0x2b6b6b643940 (LWP 3412)): [2015/10/07 17:12:37.539] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.539] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.539] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.539] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.539] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.539] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.539] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.539] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.540] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.540] Thread 39 (Thread 0x2b6b6c044940 (LWP 3413)): [2015/10/07 17:12:37.540] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.540] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.540] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.540] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.540] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.540] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.540] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.540] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.540] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.540] Thread 38 (Thread 0x2b6b6ca45940 (LWP 3414)): [2015/10/07 17:12:37.540] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.540] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.540] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.540] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.540] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.540] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.541] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.541] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.541] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.541] Thread 37 (Thread 0x2b6b6d446940 (LWP 3415)): [2015/10/07 17:12:37.541] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.541] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.541] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.541] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.541] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.541] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.541] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.541] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.541] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.541] Thread 36 (Thread 0x2b6b6de47940 (LWP 3416)): [2015/10/07 17:12:37.541] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.541] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.541] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.541] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.541] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.541] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.542] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.542] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.542] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.542] Thread 35 (Thread 0x2b6b6e848940 (LWP 3417)): [2015/10/07 17:12:37.542] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.542] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.542] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.542] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.542] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.542] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.542] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.542] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.542] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.542] Thread 34 (Thread 0x2b6b6f249940 (LWP 3418)): [2015/10/07 17:12:37.542] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.542] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.542] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.542] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.542] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.543] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.543] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.543] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.543] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.543] Thread 33 (Thread 0x2b6b6fc4a940 (LWP 3419)): [2015/10/07 17:12:37.543] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.543] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.543] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.543] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.543] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.543] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.543] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.543] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.543] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.543] Thread 32 (Thread 0x2b6b7064b940 (LWP 3420)): [2015/10/07 17:12:37.543] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.543] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.543] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.543] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.544] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.544] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.544] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.544] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.544] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.544] Thread 31 (Thread 0x2b6b7104c940 (LWP 3421)): [2015/10/07 17:12:37.544] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.544] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.544] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.544] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.544] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.544] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.544] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.544] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.544] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.544] Thread 30 (Thread 0x2b6b71a4d940 (LWP 3422)): [2015/10/07 17:12:37.544] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.544] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.544] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.545] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.545] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.545] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.545] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.545] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.545] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.545] Thread 29 (Thread 0x2b6b7244e940 (LWP 3423)): [2015/10/07 17:12:37.545] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.545] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.545] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.545] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.545] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.545] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.545] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.545] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.545] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.545] Thread 28 (Thread 0x2b6b72e4f940 (LWP 3424)): [2015/10/07 17:12:37.545] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.545] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.545] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.546] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.546] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.546] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.546] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.546] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.546] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.546] Thread 27 (Thread 0x2b6b73850940 (LWP 3425)): [2015/10/07 17:12:37.546] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.546] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.546] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.546] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.546] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.546] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.546] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.546] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.546] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.546] Thread 26 (Thread 0x2b6b74251940 (LWP 3426)): [2015/10/07 17:12:37.546] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.546] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.547] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.547] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.547] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.547] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.547] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.547] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.547] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.547] Thread 25 (Thread 0x2b6b74c52940 (LWP 3427)): [2015/10/07 17:12:37.547] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.547] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.547] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.547] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.547] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.547] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.547] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.547] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.547] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.548] Thread 24 (Thread 0x2b6b75653940 (LWP 3428)): [2015/10/07 17:12:37.548] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.548] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.548] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.548] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.548] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.548] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.548] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.548] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.548] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.548] Thread 23 (Thread 0x2b6b76054940 (LWP 3429)): [2015/10/07 17:12:37.548] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.548] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.548] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.548] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.548] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.548] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.548] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.548] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.548] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.549] Thread 22 (Thread 0x2b6b76a55940 (LWP 3430)): [2015/10/07 17:12:37.549] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.549] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.549] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.549] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.549] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.549] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.549] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.549] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.549] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.549] Thread 21 (Thread 0x2b6b77456940 (LWP 3431)): [2015/10/07 17:12:37.549] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.549] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.549] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.549] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.549] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.549] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.549] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.549] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.549] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.550] Thread 20 (Thread 0x2b6b77e57940 (LWP 3432)): [2015/10/07 17:12:37.550] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.550] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.550] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.550] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.550] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.550] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.550] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.550] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.550] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.550] Thread 19 (Thread 0x2b6b78858940 (LWP 3433)): [2015/10/07 17:12:37.550] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.550] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.550] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.550] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.550] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.550] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.550] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.551] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.551] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.551] Thread 18 (Thread 0x2b6b79259940 (LWP 3434)): [2015/10/07 17:12:37.551] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.551] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.551] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.551] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.551] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.551] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.551] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.551] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.551] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.551] Thread 17 (Thread 0x2b6b79c5a940 (LWP 3435)): [2015/10/07 17:12:37.551] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.551] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.551] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.551] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.551] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.551] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.552] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.552] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.552] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.552] Thread 16 (Thread 0x2b6b7a65b940 (LWP 3436)): [2015/10/07 17:12:37.552] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.552] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.552] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.552] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.552] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.552] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.552] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.552] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.552] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.552] Thread 15 (Thread 0x2b6b62034940 (LWP 3640)): [2015/10/07 17:12:37.552] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.552] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.552] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.552] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.552] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.553] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.553] () [2015/10/07 17:12:37.553] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.553] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.553] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.553] Thread 14 (Thread 0x2b6b7a75c940 (LWP 3789)): [2015/10/07 17:12:37.553] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.553] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.553] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.553] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.553] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.553] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.553] () [2015/10/07 17:12:37.553] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.553] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.553] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.553] Thread 13 (Thread 0x2b6b7b35f940 (LWP 3794)): [2015/10/07 17:12:37.553] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.553] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.554] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.554] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.554] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.554] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:37.554] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.554] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.554] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.554] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.554] Thread 12 (Thread 0x2b6b7bd60940 (LWP 3795)): [2015/10/07 17:12:37.554] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.554] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.554] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.554] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.554] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.554] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.554] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.554] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.554] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.555] Thread 11 (Thread 0x2b6b7c761940 (LWP 3796)): [2015/10/07 17:12:37.555] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.555] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.555] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.555] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.555] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.555] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:37.555] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.555] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.555] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.555] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.555] Thread 10 (Thread 0x2b6b7d162940 (LWP 3797)): [2015/10/07 17:12:37.555] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.555] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.555] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.555] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.555] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.555] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.555] #5 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.556] #6 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.556] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.556] Thread 9 (Thread 0x2b6b7db63940 (LWP 3804)): [2015/10/07 17:12:37.556] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.556] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.556] #1 0x00000000009a3e3d in mongo::(anonymous namespace)::ReplicaSetMonitorWatcher::run() () [2015/10/07 17:12:37.556] #2 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.556] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.556] #4 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.556] #5 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.556] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.556] Thread 8 (Thread 0x2b6b7dc64940 (LWP 3806)): [2015/10/07 17:12:37.556] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.556] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.556] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.556] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.556] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.557] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.557] () [2015/10/07 17:12:37.557] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.557] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.557] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.557] Thread 7 (Thread 0x2b6b7dd65940 (LWP 3808)): [2015/10/07 17:12:37.557] #0 0x00002b6b5527e019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.557] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.557] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.557] #2 0x0000000000e60f03 in mongo::repl::ReplicationCoordinatorImpl::_awaitReplication_inlock(mongo::Timer const*, std::unique_lock*, mongo::OperationContext*, mongo::repl::OpTime const&, mongo::SnapshotName, mongo::WriteConcernOptions const&) () [2015/10/07 17:12:37.557] #3 0x0000000000e613e6 in mongo::repl::ReplicationCoordinatorImpl::awaitReplication(mongo::OperationContext*, mongo::repl::OpTime const&, mongo::WriteConcernOptions const&) () [2015/10/07 17:12:37.557] #4 0x0000000001015117 in mongo::waitForWriteConcern(mongo::OperationContext*, mongo::repl::OpTime const&, mongo::WriteConcernResult*) () [2015/10/07 17:12:37.557] #5 0x0000000000afdac7 in mongo::WriteBatchExecutor::executeBatch(mongo::BatchedCommandRequest const&, mongo::BatchedCommandResponse*) () [2015/10/07 17:12:37.557] #6 0x0000000000b00a50 in mongo::WriteCmd::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&) () [2015/10/07 17:12:37.557] #7 0x0000000000b1a047 in mongo::Command::run(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) () [2015/10/07 17:12:37.557] #8 0x0000000000b1aeae in mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) () [2015/10/07 17:12:37.557] #9 0x0000000000a7073d in mongo::runCommands(mongo::OperationContext*, mongo::rpc::RequestInterface const&, mongo::rpc::ReplyBuilderInterface*) () [2015/10/07 17:12:37.557] #10 0x0000000000c31576 in mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) () [2015/10/07 17:12:37.558] #11 0x0000000000902a2d in mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*) () [2015/10/07 17:12:37.558] #12 0x0000000001221535 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.558] () [2015/10/07 17:12:37.558] #13 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.558] #14 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.558] #15 0x0000000000000000 in ?? () [2015/10/07 17:12:37.558] Thread 6 (Thread 0x2b6b7de66940 (LWP 3810)): [2015/10/07 17:12:37.558] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.558] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.558] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.558] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.558] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.558] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.558] () [2015/10/07 17:12:37.558] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.558] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.558] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.559] Thread 5 (Thread 0x2b6b7e867940 (LWP 3817)): [2015/10/07 17:12:37.559] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.559] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.559] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.559] #2 0x00000000018e92ab in __evict_worker () [2015/10/07 17:12:37.559] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.559] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.559] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.559] Thread 4 (Thread 0x2b6b7f268940 (LWP 3818)): [2015/10/07 17:12:37.559] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.559] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.559] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.559] #2 0x00000000018e92ab in __evict_worker () [2015/10/07 17:12:37.559] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.559] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.559] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.559] Thread 3 (Thread 0x2b6b7fc69940 (LWP 3819)): [2015/10/07 17:12:37.559] #0 0x00002b6b5527e280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.559] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.559] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.559] #2 0x00000000018e92ab in __evict_worker () [2015/10/07 17:12:37.560] #3 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.560] #4 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.560] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.560] Thread 2 (Thread 0x2b6b7fd6a940 (LWP 3831)): [2015/10/07 17:12:37.560] #0 0x00002b6b55280e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.560] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.560] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.560] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.560] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.560] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.560] () [2015/10/07 17:12:37.560] #6 0x00002b6b5527983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.560] #7 0x00002b6b55564fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.560] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.560] Thread 1 (Thread 0x2b6b557ea260 (LWP 3295)): [2015/10/07 17:12:37.560] #0 0x00002b6b5555e172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:37.560] #1 0x000000000121b0be in mongo::Listener::initAndListen() () [2015/10/07 17:12:37.561] #2 0x00000000008fef12 in mongo::initAndListen(int) () [2015/10/07 17:12:37.561] #3 0x0000000000902714 in main () [2015/10/07 17:12:37.561] Detaching from program: /data/mci/src/mongod, process 3295 [2015/10/07 17:12:37.566] INFO: Done analyzing process [2015/10/07 17:12:37.566] Dumping process 3318 of mongod [2015/10/07 17:12:37.566] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3318 [2015/10/07 17:12:37.566] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:37.587] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:37.587] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:37.587] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:37.587] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:37.587] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:37.587] and "show warranty" for details. [2015/10/07 17:12:37.587] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:37.587] For bug reporting instructions, please see: [2015/10/07 17:12:37.587] . [2015/10/07 17:12:37.588] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3318', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:37.607] Attaching to process 3318 [2015/10/07 17:12:37.802] Reading symbols from /data/mci/src/mongod...(no debugging symbols found)...done. [2015/10/07 17:12:37.817] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:37.817] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:37.817] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.817] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:37.829] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.829] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:37.830] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.830] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:37.831] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:37.831] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:37.832] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:37.853] [New LWP 3833] [2015/10/07 17:12:37.853] [New LWP 3827] [2015/10/07 17:12:37.853] [New LWP 3825] [2015/10/07 17:12:37.853] [New LWP 3635] [2015/10/07 17:12:37.853] [New LWP 3633] [2015/10/07 17:12:37.853] [New LWP 3632] [2015/10/07 17:12:37.853] [New LWP 3631] [2015/10/07 17:12:37.853] [New LWP 3630] [2015/10/07 17:12:37.853] [New LWP 3629] [2015/10/07 17:12:37.853] [New LWP 3628] [2015/10/07 17:12:37.853] [New LWP 3627] [2015/10/07 17:12:37.853] [New LWP 3626] [2015/10/07 17:12:37.853] [New LWP 3625] [2015/10/07 17:12:37.853] [New LWP 3624] [2015/10/07 17:12:37.853] [New LWP 3623] [2015/10/07 17:12:37.853] [New LWP 3622] [2015/10/07 17:12:37.853] [New LWP 3621] [2015/10/07 17:12:37.854] [New LWP 3620] [2015/10/07 17:12:37.854] [New LWP 3619] [2015/10/07 17:12:37.854] [New LWP 3618] [2015/10/07 17:12:37.854] [New LWP 3617] [2015/10/07 17:12:37.854] [New LWP 3616] [2015/10/07 17:12:37.854] [New LWP 3615] [2015/10/07 17:12:37.854] [New LWP 3614] [2015/10/07 17:12:37.854] [New LWP 3613] [2015/10/07 17:12:37.854] [New LWP 3612] [2015/10/07 17:12:37.854] [New LWP 3611] [2015/10/07 17:12:37.854] [New LWP 3610] [2015/10/07 17:12:37.854] [New LWP 3609] [2015/10/07 17:12:37.854] [New LWP 3608] [2015/10/07 17:12:37.854] [New LWP 3607] [2015/10/07 17:12:37.854] [New LWP 3606] [2015/10/07 17:12:37.854] [New LWP 3605] [2015/10/07 17:12:37.854] [New LWP 3604] [2015/10/07 17:12:37.854] [New LWP 3603] [2015/10/07 17:12:37.854] [New LWP 3602] [2015/10/07 17:12:37.854] [New LWP 3437] [2015/10/07 17:12:37.854] [New LWP 3387] [2015/10/07 17:12:37.854] [New LWP 3386] [2015/10/07 17:12:37.854] [New LWP 3383] [2015/10/07 17:12:37.854] [New LWP 3382] [2015/10/07 17:12:37.854] [New LWP 3381] [2015/10/07 17:12:37.854] [New LWP 3380] [2015/10/07 17:12:37.854] [New LWP 3379] [2015/10/07 17:12:37.854] [New LWP 3371] [2015/10/07 17:12:37.854] [New LWP 3340] [2015/10/07 17:12:37.854] [New LWP 3338] [2015/10/07 17:12:37.854] [New LWP 3337] [2015/10/07 17:12:37.854] [New LWP 3336] [2015/10/07 17:12:37.854] [New LWP 3335] [2015/10/07 17:12:37.854] [New LWP 3334] [2015/10/07 17:12:37.854] [New LWP 3333] [2015/10/07 17:12:37.854] [New LWP 3332] [2015/10/07 17:12:37.854] [New LWP 3331] [2015/10/07 17:12:37.854] [New LWP 3330] [2015/10/07 17:12:37.854] [New LWP 3329] [2015/10/07 17:12:37.854] [New LWP 3328] [2015/10/07 17:12:37.854] [New LWP 3327] [2015/10/07 17:12:37.854] [New LWP 3326] [2015/10/07 17:12:37.854] [New LWP 3325] [2015/10/07 17:12:37.854] [New LWP 3324] [2015/10/07 17:12:37.854] [New LWP 3323] [2015/10/07 17:12:37.854] [New LWP 3322] [2015/10/07 17:12:37.854] [New LWP 3321] [2015/10/07 17:12:37.854] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:37.854] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:37.854] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:37.861] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:37.861] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:37.861] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.861] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:37.862] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.862] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:37.862] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.862] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:37.863] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:37.873] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:37.873] 0x00002b2bf25ce172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:37.879] Thread 65 (Thread 0x2b2bf325c940 (LWP 3321)): [2015/10/07 17:12:37.879] #0 0x00002b2bf22f19c8 in do_sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.879] #1 0x00002b2bf22f1a6d in sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.879] #2 0x0000000001265316 in mongo::(anonymous namespace)::signalProcessingThread() () [2015/10/07 17:12:37.879] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.879] #4 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.879] #5 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.879] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.879] Thread 64 (Thread 0x2b2bf3c5d940 (LWP 3322)): [2015/10/07 17:12:37.879] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.880] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.880] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.880] #2 0x00000000018ea002 in __evict_server () [2015/10/07 17:12:37.880] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.880] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.880] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.880] Thread 63 (Thread 0x2b2bf465e940 (LWP 3323)): [2015/10/07 17:12:37.880] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.880] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.880] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.880] #2 0x00000000018cce33 in __sweep_server () [2015/10/07 17:12:37.880] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.880] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.880] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.880] Thread 62 (Thread 0x2b2bf505f940 (LWP 3324)): [2015/10/07 17:12:37.880] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.880] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.880] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.880] #2 0x00000000018c9b90 in __log_file_server () [2015/10/07 17:12:37.881] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.881] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.881] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.881] Thread 61 (Thread 0x2b2bf5a60940 (LWP 3325)): [2015/10/07 17:12:37.881] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.881] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.881] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.881] #2 0x00000000018ca2f4 in __log_wrlsn_server () [2015/10/07 17:12:37.881] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.881] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.881] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.881] Thread 60 (Thread 0x2b2bf6461940 (LWP 3326)): [2015/10/07 17:12:37.881] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.881] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.881] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.881] #2 0x00000000018c96b7 in __log_server () [2015/10/07 17:12:37.881] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.881] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.881] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.882] Thread 59 (Thread 0x2b2bf6e62940 (LWP 3327)): [2015/10/07 17:12:37.882] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.882] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.882] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:37.882] #2 0x00000000018c7239 in __ckpt_server () [2015/10/07 17:12:37.882] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.882] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.882] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.882] Thread 58 (Thread 0x2b2bf7863940 (LWP 3328)): [2015/10/07 17:12:37.882] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.882] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.882] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.882] #2 0x00000000011ced98 in mongo::DeadlineMonitor::deadlineMonitorThread() () [2015/10/07 17:12:37.882] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.882] #4 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.882] #5 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.882] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.882] Thread 57 (Thread 0x2b2bf8264940 (LWP 3329)): [2015/10/07 17:12:37.882] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.882] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.883] #1 0x0000000000dba6e3 in mongo::RangeDeleter::doWork() () [2015/10/07 17:12:37.883] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.883] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.883] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.883] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.883] Thread 56 (Thread 0x2b2bf8c65940 (LWP 3330)): [2015/10/07 17:12:37.883] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.883] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.883] #1 0x000000000102d164 in mongo::executor::NetworkInterfaceASIO::waitForWorkUntil(mongo::Date_t) () [2015/10/07 17:12:37.883] #2 0x0000000000e8609d in mongo::repl::ReplicationExecutor::getWork() () [2015/10/07 17:12:37.883] #3 0x0000000000e88bbd in mongo::repl::ReplicationExecutor::run() () [2015/10/07 17:12:37.883] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.883] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.883] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.883] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.883] Thread 55 (Thread 0x2b2bf9666940 (LWP 3331)): [2015/10/07 17:12:37.883] #0 0x00002b2bf25d53c8 in epoll_wait () from /lib64/libc.so.6 [2015/10/07 17:12:37.883] #1 0x0000000001282e35 in asio::detail::epoll_reactor::run(bool, asio::detail::op_queue&) () [2015/10/07 17:12:37.884] #2 0x0000000001283988 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.884] #3 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.884] #4 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.884] #5 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:37.884] #6 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.884] #7 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.884] #8 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.884] #9 0x0000000000000000 in ?? () [2015/10/07 17:12:37.884] Thread 54 (Thread 0x2b2bfa067940 (LWP 3332)): [2015/10/07 17:12:37.884] #0 0x00002b2bf22f1351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.884] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:37.884] #2 0x00000000010137eb in mongo::TTLMonitor::run() () [2015/10/07 17:12:37.884] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.884] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.884] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.884] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.884] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.885] Thread 53 (Thread 0x2b2bfaa68940 (LWP 3333)): [2015/10/07 17:12:37.885] #0 0x00002b2bf22f1351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.885] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:37.885] #2 0x0000000000a61378 in mongo::ClientCursorMonitor::run() () [2015/10/07 17:12:37.885] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.885] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.885] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.885] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.885] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.885] Thread 52 (Thread 0x2b2bfb469940 (LWP 3334)): [2015/10/07 17:12:37.885] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.885] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.885] #1 0x00000000012055ce in mongo::(anonymous namespace)::PeriodicTaskRunner::run() () [2015/10/07 17:12:37.885] #2 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.885] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.885] #4 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.885] #5 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.885] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.886] Thread 51 (Thread 0x2b2bfbe6a940 (LWP 3335)): [2015/10/07 17:12:37.886] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.886] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.886] #1 0x0000000000b95a66 in mongo::FTDCController::doLoop() () [2015/10/07 17:12:37.886] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.886] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.886] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.886] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.886] Thread 50 (Thread 0x2b2bfc86b940 (LWP 3336)): [2015/10/07 17:12:37.886] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.886] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.886] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.886] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.886] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.886] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.886] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.886] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.886] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.887] Thread 49 (Thread 0x2b2bfd26c940 (LWP 3337)): [2015/10/07 17:12:37.887] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.887] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.887] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.887] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.887] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.887] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.887] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.887] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.887] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.887] Thread 48 (Thread 0x2b2bfdc6d940 (LWP 3338)): [2015/10/07 17:12:37.887] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.887] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.887] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.887] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.887] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.887] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.887] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.887] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.887] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.888] Thread 47 (Thread 0x2b2bfdd6f940 (LWP 3340)): [2015/10/07 17:12:37.888] #0 0x00002b2bf22f0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.888] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.888] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.888] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.888] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.888] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.888] () [2015/10/07 17:12:37.888] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.888] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.888] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.888] Thread 46 (Thread 0x2b2bfe871940 (LWP 3371)): [2015/10/07 17:12:37.888] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.888] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.888] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.888] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.888] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.888] #4 0x000000000127b8a0 in asio_detail_posix_thread_function () [2015/10/07 17:12:37.888] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.889] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.889] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.889] Thread 45 (Thread 0x2b2bff8a3940 (LWP 3379)): [2015/10/07 17:12:37.889] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.889] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.889] #1 0x0000000000dc5888 in mongo::repl::BackgroundSync::waitForMore() () [2015/10/07 17:12:37.889] #2 0x0000000000eb52b9 in mongo::repl::SyncTail::tryPopAndWaitForMore(mongo::OperationContext*, mongo::repl::SyncTail::OpQueue*, mongo::repl::ReplicationCoordinator*) () [2015/10/07 17:12:37.889] #3 0x0000000000eb621e in mongo::repl::SyncTail::oplogApplication() () [2015/10/07 17:12:37.889] #4 0x0000000000ead424 in mongo::repl::runSyncThread() () [2015/10/07 17:12:37.889] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.889] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.889] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.889] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.889] Thread 44 (Thread 0x2b2c002a4940 (LWP 3380)): [2015/10/07 17:12:37.889] #0 0x00002b2bf22f1351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.889] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:37.889] #2 0x0000000000dc8132 in mongo::repl::BackgroundSync::_produce(mongo::OperationContext*) () [2015/10/07 17:12:37.890] #3 0x0000000000dc91d2 in mongo::repl::BackgroundSync::_producerThread() () [2015/10/07 17:12:37.890] #4 0x0000000000dc9305 in mongo::repl::BackgroundSync::producerThread() () [2015/10/07 17:12:37.890] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.890] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.890] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.890] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.890] Thread 43 (Thread 0x2b2c00ca5940 (LWP 3381)): [2015/10/07 17:12:37.890] #0 0x00002b2bf22ee280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.890] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.890] #1 0x0000000000eb0dbd in mongo::repl::SyncSourceFeedback::run() () [2015/10/07 17:12:37.890] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.890] #3 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.890] #4 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.890] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:37.890] Thread 42 (Thread 0x2b2c016a6940 (LWP 3382)): [2015/10/07 17:12:37.890] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.890] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.890] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.890] #2 0x0000000000e1f69d in mongo::repl::SnapshotThread::run() () [2015/10/07 17:12:37.890] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.891] #4 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.891] #5 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.891] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:37.891] Thread 41 (Thread 0x2b2c020a7940 (LWP 3383)): [2015/10/07 17:12:37.891] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.891] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.891] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:37.891] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:37.891] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:37.891] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:37.891] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.891] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.891] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.891] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.891] Thread 40 (Thread 0x2b2c02aa8940 (LWP 3386)): [2015/10/07 17:12:37.891] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.891] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.891] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.892] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.892] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.892] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.892] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.892] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.892] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.892] Thread 39 (Thread 0x2b2c02ba9940 (LWP 3387)): [2015/10/07 17:12:37.892] #0 0x00002b2bf22f0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.892] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.892] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.892] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.892] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.892] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.892] () [2015/10/07 17:12:37.892] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.892] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.892] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.893] Thread 38 (Thread 0x2b2c036ab940 (LWP 3437)): [2015/10/07 17:12:37.893] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.893] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.893] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.893] #2 0x0000000000ff96e1 in mongo::WiredTigerRecordStore::OplogStones::awaitHasExcessStonesOrDead() () [2015/10/07 17:12:37.893] #3 0x0000000000ffa460 in mongo::WiredTigerRecordStore::yieldAndAwaitOplogDeletionRequest(mongo::OperationContext*) () [2015/10/07 17:12:37.893] #4 0x0000000000fffd27 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::_deleteExcessDocuments() () [2015/10/07 17:12:37.893] #5 0x0000000001000238 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::run() () [2015/10/07 17:12:37.893] #6 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:37.893] #7 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.893] #8 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.893] #9 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.893] #10 0x0000000000000000 in ?? () [2015/10/07 17:12:37.893] Thread 37 (Thread 0x2b2c0d6bb940 (LWP 3602)): [2015/10/07 17:12:37.893] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.893] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.893] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.893] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.893] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.894] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.894] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.894] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.894] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.894] Thread 36 (Thread 0x2b2c0ccba940 (LWP 3603)): [2015/10/07 17:12:37.894] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.894] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.894] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.894] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.894] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.894] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.894] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.894] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.894] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.894] Thread 35 (Thread 0x2b2c0c2b9940 (LWP 3604)): [2015/10/07 17:12:37.894] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.894] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.894] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.894] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.895] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.895] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.895] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.895] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.895] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.895] Thread 34 (Thread 0x2b2c040ac940 (LWP 3605)): [2015/10/07 17:12:37.895] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.895] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.895] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.895] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.895] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.895] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.895] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.895] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.895] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.895] Thread 33 (Thread 0x2b2c04aad940 (LWP 3606)): [2015/10/07 17:12:37.895] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.895] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.895] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.896] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.896] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.896] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.896] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.896] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.896] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.896] Thread 32 (Thread 0x2b2c054ae940 (LWP 3607)): [2015/10/07 17:12:37.896] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.896] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.896] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.896] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.896] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.896] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.896] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.896] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.896] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.896] Thread 31 (Thread 0x2b2c05eaf940 (LWP 3608)): [2015/10/07 17:12:37.896] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.896] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.896] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.897] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.897] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.897] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.897] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.897] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.897] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.897] Thread 30 (Thread 0x2b2c068b0940 (LWP 3609)): [2015/10/07 17:12:37.897] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.897] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.897] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.897] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.897] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.897] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.897] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.897] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.897] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.897] Thread 29 (Thread 0x2b2c072b1940 (LWP 3610)): [2015/10/07 17:12:37.897] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.897] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.898] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.898] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.898] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.898] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.898] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.898] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.898] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.898] Thread 28 (Thread 0x2b2c07cb2940 (LWP 3611)): [2015/10/07 17:12:37.898] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.898] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.898] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.898] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.898] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.898] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.898] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.898] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.898] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.899] Thread 27 (Thread 0x2b2c086b3940 (LWP 3612)): [2015/10/07 17:12:37.899] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.899] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.899] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.899] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.899] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.899] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.899] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.899] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.899] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.899] Thread 26 (Thread 0x2b2c090b4940 (LWP 3613)): [2015/10/07 17:12:37.899] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.899] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.899] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.899] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.899] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.899] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.899] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.899] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.899] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.900] Thread 25 (Thread 0x2b2c09ab5940 (LWP 3614)): [2015/10/07 17:12:37.900] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.900] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.900] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.900] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.900] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.900] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.900] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.900] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.900] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.900] Thread 24 (Thread 0x2b2c0a4b6940 (LWP 3615)): [2015/10/07 17:12:37.900] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.900] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.900] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.900] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.900] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.900] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.900] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.900] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.900] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.901] Thread 23 (Thread 0x2b2c0aeb7940 (LWP 3616)): [2015/10/07 17:12:37.901] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.901] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.901] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.901] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.901] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.901] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.901] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.901] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.901] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.901] Thread 22 (Thread 0x2b2c0b8b8940 (LWP 3617)): [2015/10/07 17:12:37.901] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.901] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.901] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.901] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.901] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.901] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.901] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.901] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.902] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.902] Thread 21 (Thread 0x2b2c0e0bc940 (LWP 3618)): [2015/10/07 17:12:37.902] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.902] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.902] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.902] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.902] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.902] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.902] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.902] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.902] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.902] Thread 20 (Thread 0x2b2c0eabd940 (LWP 3619)): [2015/10/07 17:12:37.902] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.902] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.902] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.902] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.902] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.902] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.903] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.903] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.903] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.903] Thread 19 (Thread 0x2b2c0f4be940 (LWP 3620)): [2015/10/07 17:12:37.903] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.903] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.903] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.903] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.903] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.903] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.903] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.903] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.903] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.903] Thread 18 (Thread 0x2b2c0febf940 (LWP 3621)): [2015/10/07 17:12:37.903] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.903] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.903] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.903] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.903] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.904] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.904] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.904] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.904] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.904] Thread 17 (Thread 0x2b2c108c0940 (LWP 3622)): [2015/10/07 17:12:37.904] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.904] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.904] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.904] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.904] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.904] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.904] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.904] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.904] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.904] Thread 16 (Thread 0x2b2c112c1940 (LWP 3623)): [2015/10/07 17:12:37.904] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.904] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.904] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.904] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.905] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.905] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.905] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.905] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.905] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.905] Thread 15 (Thread 0x2b2c11cc2940 (LWP 3624)): [2015/10/07 17:12:37.905] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.905] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.905] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.905] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.905] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.905] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.905] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.905] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.905] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.905] Thread 14 (Thread 0x2b2c126c3940 (LWP 3625)): [2015/10/07 17:12:37.905] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.905] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.905] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.906] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.906] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.906] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.906] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.906] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.906] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.906] Thread 13 (Thread 0x2b2c130c4940 (LWP 3626)): [2015/10/07 17:12:37.906] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.906] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.906] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.906] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.906] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.906] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.906] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.906] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.906] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.906] Thread 12 (Thread 0x2b2c13ac5940 (LWP 3627)): [2015/10/07 17:12:37.906] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.906] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.906] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.907] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.907] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.907] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.907] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.907] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.907] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.907] Thread 11 (Thread 0x2b2c144c6940 (LWP 3628)): [2015/10/07 17:12:37.907] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.907] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.907] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.907] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.907] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.907] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.907] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.907] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.907] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.907] Thread 10 (Thread 0x2b2c14ec7940 (LWP 3629)): [2015/10/07 17:12:37.907] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.908] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.908] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.908] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.908] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.908] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.908] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.908] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.908] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.908] Thread 9 (Thread 0x2b2c158c8940 (LWP 3630)): [2015/10/07 17:12:37.908] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.908] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.908] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.908] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.908] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.908] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.908] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.908] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.908] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.909] Thread 8 (Thread 0x2b2c162c9940 (LWP 3631)): [2015/10/07 17:12:37.909] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.909] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.909] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.909] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.909] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.909] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.909] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.909] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.909] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.909] Thread 7 (Thread 0x2b2c16cca940 (LWP 3632)): [2015/10/07 17:12:37.909] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.909] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.909] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.909] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.909] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.909] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.909] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.909] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.909] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.910] Thread 6 (Thread 0x2b2c176cb940 (LWP 3633)): [2015/10/07 17:12:37.910] #0 0x00002b2bf22ee019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:37.910] from /lib64/libpthread.so.0 [2015/10/07 17:12:37.910] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:37.910] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:37.910] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:37.910] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:37.910] #5 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.910] #6 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.910] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:37.910] Thread 5 (Thread 0x2b2c177cc940 (LWP 3635)): [2015/10/07 17:12:37.910] #0 0x00002b2bf22f0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.910] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.910] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.910] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.910] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.910] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.910] () [2015/10/07 17:12:37.911] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.911] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.911] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.911] Thread 4 (Thread 0x2b2c182ce940 (LWP 3825)): [2015/10/07 17:12:37.911] #0 0x00002b2bf22f0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.911] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.911] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.911] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.911] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.911] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.911] () [2015/10/07 17:12:37.911] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.911] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.911] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.911] Thread 3 (Thread 0x2b2c183cf940 (LWP 3827)): [2015/10/07 17:12:37.911] #0 0x00002b2bf22f0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.911] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.911] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.912] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.912] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.912] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.912] () [2015/10/07 17:12:37.912] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.912] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.912] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.912] Thread 2 (Thread 0x2b2c184d0940 (LWP 3833)): [2015/10/07 17:12:37.912] #0 0x00002b2bf22f0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.912] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:37.912] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:37.912] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:37.912] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:37.912] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:37.912] () [2015/10/07 17:12:37.912] #6 0x00002b2bf22e983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:37.912] #7 0x00002b2bf25d4fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:37.912] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:37.913] Thread 1 (Thread 0x2b2bf285a260 (LWP 3318)): [2015/10/07 17:12:37.913] #0 0x00002b2bf25ce172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:37.913] #1 0x000000000121b0be in mongo::Listener::initAndListen() () [2015/10/07 17:12:37.913] #2 0x00000000008fef12 in mongo::initAndListen(int) () [2015/10/07 17:12:37.913] #3 0x0000000000902714 in main () [2015/10/07 17:12:37.914] Detaching from program: /data/mci/src/mongod, process 3318 [2015/10/07 17:12:37.920] INFO: Done analyzing process [2015/10/07 17:12:37.920] Dumping process 3341 of mongod [2015/10/07 17:12:37.920] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3341 [2015/10/07 17:12:37.920] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:37.939] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:37.939] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:37.939] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:37.939] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:37.939] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:37.939] and "show warranty" for details. [2015/10/07 17:12:37.939] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:37.939] For bug reporting instructions, please see: [2015/10/07 17:12:37.939] . [2015/10/07 17:12:37.939] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3341', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:37.959] Attaching to process 3341 [2015/10/07 17:12:38.154] Reading symbols from /data/mci/src/mongod...(no debugging symbols found)...done. [2015/10/07 17:12:38.169] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:38.170] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:38.170] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.170] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:38.181] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.181] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:38.183] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.183] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:38.183] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:38.183] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:38.184] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:38.204] [New LWP 3829] [2015/10/07 17:12:38.205] [New LWP 3787] [2015/10/07 17:12:38.205] [New LWP 3638] [2015/10/07 17:12:38.205] [New LWP 3535] [2015/10/07 17:12:38.205] [New LWP 3534] [2015/10/07 17:12:38.205] [New LWP 3533] [2015/10/07 17:12:38.205] [New LWP 3532] [2015/10/07 17:12:38.205] [New LWP 3531] [2015/10/07 17:12:38.205] [New LWP 3530] [2015/10/07 17:12:38.205] [New LWP 3529] [2015/10/07 17:12:38.205] [New LWP 3528] [2015/10/07 17:12:38.205] [New LWP 3527] [2015/10/07 17:12:38.205] [New LWP 3526] [2015/10/07 17:12:38.205] [New LWP 3525] [2015/10/07 17:12:38.205] [New LWP 3524] [2015/10/07 17:12:38.205] [New LWP 3523] [2015/10/07 17:12:38.205] [New LWP 3522] [2015/10/07 17:12:38.205] [New LWP 3521] [2015/10/07 17:12:38.205] [New LWP 3520] [2015/10/07 17:12:38.205] [New LWP 3519] [2015/10/07 17:12:38.205] [New LWP 3518] [2015/10/07 17:12:38.205] [New LWP 3517] [2015/10/07 17:12:38.205] [New LWP 3516] [2015/10/07 17:12:38.205] [New LWP 3515] [2015/10/07 17:12:38.205] [New LWP 3514] [2015/10/07 17:12:38.205] [New LWP 3513] [2015/10/07 17:12:38.205] [New LWP 3512] [2015/10/07 17:12:38.205] [New LWP 3511] [2015/10/07 17:12:38.205] [New LWP 3510] [2015/10/07 17:12:38.205] [New LWP 3509] [2015/10/07 17:12:38.205] [New LWP 3508] [2015/10/07 17:12:38.205] [New LWP 3507] [2015/10/07 17:12:38.205] [New LWP 3506] [2015/10/07 17:12:38.205] [New LWP 3505] [2015/10/07 17:12:38.205] [New LWP 3504] [2015/10/07 17:12:38.205] [New LWP 3398] [2015/10/07 17:12:38.205] [New LWP 3397] [2015/10/07 17:12:38.205] [New LWP 3396] [2015/10/07 17:12:38.205] [New LWP 3395] [2015/10/07 17:12:38.205] [New LWP 3394] [2015/10/07 17:12:38.205] [New LWP 3393] [2015/10/07 17:12:38.205] [New LWP 3392] [2015/10/07 17:12:38.205] [New LWP 3385] [2015/10/07 17:12:38.205] [New LWP 3372] [2015/10/07 17:12:38.205] [New LWP 3370] [2015/10/07 17:12:38.205] [New LWP 3363] [2015/10/07 17:12:38.205] [New LWP 3361] [2015/10/07 17:12:38.205] [New LWP 3360] [2015/10/07 17:12:38.205] [New LWP 3359] [2015/10/07 17:12:38.205] [New LWP 3358] [2015/10/07 17:12:38.205] [New LWP 3357] [2015/10/07 17:12:38.206] [New LWP 3356] [2015/10/07 17:12:38.206] [New LWP 3355] [2015/10/07 17:12:38.206] [New LWP 3354] [2015/10/07 17:12:38.206] [New LWP 3353] [2015/10/07 17:12:38.206] [New LWP 3352] [2015/10/07 17:12:38.206] [New LWP 3351] [2015/10/07 17:12:38.206] [New LWP 3350] [2015/10/07 17:12:38.206] [New LWP 3349] [2015/10/07 17:12:38.206] [New LWP 3348] [2015/10/07 17:12:38.206] [New LWP 3347] [2015/10/07 17:12:38.206] [New LWP 3346] [2015/10/07 17:12:38.206] [New LWP 3345] [2015/10/07 17:12:38.206] [New LWP 3344] [2015/10/07 17:12:38.206] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:38.206] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:38.206] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:38.212] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.213] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:38.213] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.213] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:38.214] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.214] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:38.214] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.214] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:38.215] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.225] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:38.225] 0x00002b6be4faa172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:38.231] Thread 65 (Thread 0x2b6be5c38940 (LWP 3344)): [2015/10/07 17:12:38.231] #0 0x00002b6be4ccd9c8 in do_sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.231] #1 0x00002b6be4ccda6d in sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.231] #2 0x0000000001265316 in mongo::(anonymous namespace)::signalProcessingThread() () [2015/10/07 17:12:38.231] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.231] #4 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.231] #5 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.231] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.232] Thread 64 (Thread 0x2b6be6639940 (LWP 3345)): [2015/10/07 17:12:38.232] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.232] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.232] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.232] #2 0x00000000018ea002 in __evict_server () [2015/10/07 17:12:38.232] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.232] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.232] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.232] Thread 63 (Thread 0x2b6be703a940 (LWP 3346)): [2015/10/07 17:12:38.232] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.232] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.232] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.232] #2 0x00000000018cce33 in __sweep_server () [2015/10/07 17:12:38.232] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.232] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.233] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.233] Thread 62 (Thread 0x2b6be7a3b940 (LWP 3347)): [2015/10/07 17:12:38.233] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.233] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.233] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.233] #2 0x00000000018c9b90 in __log_file_server () [2015/10/07 17:12:38.233] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.233] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.233] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.233] Thread 61 (Thread 0x2b6be843c940 (LWP 3348)): [2015/10/07 17:12:38.233] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.233] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.233] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.233] #2 0x00000000018ca2f4 in __log_wrlsn_server () [2015/10/07 17:12:38.234] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.234] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.234] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.234] Thread 60 (Thread 0x2b6be8e3d940 (LWP 3349)): [2015/10/07 17:12:38.234] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.234] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.234] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.234] #2 0x00000000018c96b7 in __log_server () [2015/10/07 17:12:38.234] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.234] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.234] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.234] Thread 59 (Thread 0x2b6be983e940 (LWP 3350)): [2015/10/07 17:12:38.234] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.234] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.234] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.235] #2 0x00000000018c7239 in __ckpt_server () [2015/10/07 17:12:38.235] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.235] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.235] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.235] Thread 58 (Thread 0x2b6bea23f940 (LWP 3351)): [2015/10/07 17:12:38.235] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.235] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.235] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.235] #2 0x00000000011ced98 in mongo::DeadlineMonitor::deadlineMonitorThread() () [2015/10/07 17:12:38.235] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.235] #4 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.235] #5 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.235] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.236] Thread 57 (Thread 0x2b6beac40940 (LWP 3352)): [2015/10/07 17:12:38.236] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.236] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.236] #1 0x0000000000dba6e3 in mongo::RangeDeleter::doWork() () [2015/10/07 17:12:38.236] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.236] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.236] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.236] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.236] Thread 56 (Thread 0x2b6beb641940 (LWP 3353)): [2015/10/07 17:12:38.236] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.236] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.236] #1 0x000000000102d164 in mongo::executor::NetworkInterfaceASIO::waitForWorkUntil(mongo::Date_t) () [2015/10/07 17:12:38.236] #2 0x0000000000e8609d in mongo::repl::ReplicationExecutor::getWork() () [2015/10/07 17:12:38.236] #3 0x0000000000e88bbd in mongo::repl::ReplicationExecutor::run() () [2015/10/07 17:12:38.236] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.237] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.237] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.237] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.237] Thread 55 (Thread 0x2b6bec042940 (LWP 3354)): [2015/10/07 17:12:38.237] #0 0x00002b6be4ccd351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.237] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.237] #2 0x00000000010137eb in mongo::TTLMonitor::run() () [2015/10/07 17:12:38.237] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.237] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.237] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.237] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.237] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.238] Thread 54 (Thread 0x2b6beca43940 (LWP 3355)): [2015/10/07 17:12:38.238] #0 0x00002b6be4ccd351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.238] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.238] #2 0x0000000000a61378 in mongo::ClientCursorMonitor::run() () [2015/10/07 17:12:38.238] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.238] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.238] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.238] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.238] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.238] Thread 53 (Thread 0x2b6bed444940 (LWP 3356)): [2015/10/07 17:12:38.238] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.238] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.238] #1 0x00000000012055ce in mongo::(anonymous namespace)::PeriodicTaskRunner::run() () [2015/10/07 17:12:38.238] #2 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.238] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.239] #4 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.239] #5 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.239] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.239] Thread 52 (Thread 0x2b6bede45940 (LWP 3357)): [2015/10/07 17:12:38.239] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.239] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.239] #1 0x0000000000b95a66 in mongo::FTDCController::doLoop() () [2015/10/07 17:12:38.239] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.239] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.239] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.239] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.239] Thread 51 (Thread 0x2b6bee846940 (LWP 3358)): [2015/10/07 17:12:38.239] #0 0x00002b6be4fb13c8 in epoll_wait () from /lib64/libc.so.6 [2015/10/07 17:12:38.240] #1 0x0000000001282e35 in asio::detail::epoll_reactor::run(bool, asio::detail::op_queue&) () [2015/10/07 17:12:38.240] #2 0x0000000001283988 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:38.240] #3 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:38.240] #4 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:38.240] #5 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:38.240] #6 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.240] #7 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.240] #8 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.240] #9 0x0000000000000000 in ?? () [2015/10/07 17:12:38.240] Thread 50 (Thread 0x2b6bef247940 (LWP 3359)): [2015/10/07 17:12:38.240] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.240] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.240] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.241] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.241] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.241] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.241] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.241] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.241] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.241] Thread 49 (Thread 0x2b6befc49940 (LWP 3360)): [2015/10/07 17:12:38.241] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.241] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.241] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.241] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.241] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.241] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.242] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.242] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.242] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.242] Thread 48 (Thread 0x2b6bf064a940 (LWP 3361)): [2015/10/07 17:12:38.242] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.242] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.242] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.242] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.242] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.242] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.242] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.242] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.242] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.243] Thread 47 (Thread 0x2b6bf074b940 (LWP 3363)): [2015/10/07 17:12:38.243] #0 0x00002b6be4ccce11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.243] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.243] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.243] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.243] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.243] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.243] () [2015/10/07 17:12:38.243] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.243] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.243] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.243] Thread 46 (Thread 0x2b6bf084c940 (LWP 3370)): [2015/10/07 17:12:38.243] #0 0x00002b6be4ccce11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.243] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.244] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.244] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.244] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.244] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.244] () [2015/10/07 17:12:38.244] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.244] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.244] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.244] Thread 45 (Thread 0x2b6bf124d940 (LWP 3372)): [2015/10/07 17:12:38.244] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.244] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.244] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:38.244] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:38.244] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:38.245] #4 0x000000000127b8a0 in asio_detail_posix_thread_function () [2015/10/07 17:12:38.245] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.245] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.245] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.245] Thread 44 (Thread 0x2b6bf197f940 (LWP 3385)): [2015/10/07 17:12:38.245] #0 0x00002b6be4ccce11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.245] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.245] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.245] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.245] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.245] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.245] () [2015/10/07 17:12:38.245] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.246] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.246] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.246] Thread 43 (Thread 0x2b6bf2380940 (LWP 3392)): [2015/10/07 17:12:38.246] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.246] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.246] #1 0x0000000000dc5888 in mongo::repl::BackgroundSync::waitForMore() () [2015/10/07 17:12:38.246] #2 0x0000000000eb52b9 in mongo::repl::SyncTail::tryPopAndWaitForMore(mongo::OperationContext*, mongo::repl::SyncTail::OpQueue*, mongo::repl::ReplicationCoordinator*) () [2015/10/07 17:12:38.246] #3 0x0000000000eb621e in mongo::repl::SyncTail::oplogApplication() () [2015/10/07 17:12:38.246] #4 0x0000000000ead424 in mongo::repl::runSyncThread() () [2015/10/07 17:12:38.246] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.246] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.246] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.246] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.247] Thread 42 (Thread 0x2b6bf2d81940 (LWP 3393)): [2015/10/07 17:12:38.247] #0 0x00002b6be4ccd351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.247] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.247] #2 0x0000000000dc8132 in mongo::repl::BackgroundSync::_produce(mongo::OperationContext*) () [2015/10/07 17:12:38.247] #3 0x0000000000dc91d2 in mongo::repl::BackgroundSync::_producerThread() () [2015/10/07 17:12:38.247] #4 0x0000000000dc9305 in mongo::repl::BackgroundSync::producerThread() () [2015/10/07 17:12:38.247] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.247] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.247] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.247] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.247] Thread 41 (Thread 0x2b6bf3782940 (LWP 3394)): [2015/10/07 17:12:38.247] #0 0x00002b6be4cca280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.247] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.247] #1 0x0000000000eb0dbd in mongo::repl::SyncSourceFeedback::run() () [2015/10/07 17:12:38.248] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.248] #3 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.248] #4 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.248] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.248] Thread 40 (Thread 0x2b6bf4183940 (LWP 3395)): [2015/10/07 17:12:38.248] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.248] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.248] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.248] #2 0x0000000000e1f69d in mongo::repl::SnapshotThread::run() () [2015/10/07 17:12:38.248] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.248] #4 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.248] #5 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.248] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.249] Thread 39 (Thread 0x2b6bf4b84940 (LWP 3396)): [2015/10/07 17:12:38.249] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.249] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.249] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:38.249] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:38.249] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:38.249] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:38.249] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.249] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.249] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.249] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.249] Thread 38 (Thread 0x2b6bf5585940 (LWP 3397)): [2015/10/07 17:12:38.250] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.250] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.250] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.250] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.250] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.250] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.250] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.250] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.250] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.250] Thread 37 (Thread 0x2b6bf5f86940 (LWP 3398)): [2015/10/07 17:12:38.250] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.250] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.250] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.250] #2 0x0000000000ff96e1 in mongo::WiredTigerRecordStore::OplogStones::awaitHasExcessStonesOrDead() () [2015/10/07 17:12:38.251] #3 0x0000000000ffa460 in mongo::WiredTigerRecordStore::yieldAndAwaitOplogDeletionRequest(mongo::OperationContext*) () [2015/10/07 17:12:38.251] #4 0x0000000000fffd27 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::_deleteExcessDocuments() () [2015/10/07 17:12:38.251] #5 0x0000000001000238 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::run() () [2015/10/07 17:12:38.251] #6 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.251] #7 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.251] #8 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.251] #9 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.251] #10 0x0000000000000000 in ?? () [2015/10/07 17:12:38.251] Thread 36 (Thread 0x2b6bfff96940 (LWP 3504)): [2015/10/07 17:12:38.251] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.251] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.251] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.251] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.252] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.252] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.252] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.252] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.252] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.252] Thread 35 (Thread 0x2b6bff595940 (LWP 3505)): [2015/10/07 17:12:38.252] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.252] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.252] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.252] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.252] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.252] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.253] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.253] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.253] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.253] Thread 34 (Thread 0x2b6bfeb94940 (LWP 3506)): [2015/10/07 17:12:38.253] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.253] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.253] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.253] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.253] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.253] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.253] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.253] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.253] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.254] Thread 33 (Thread 0x2b6bf6987940 (LWP 3507)): [2015/10/07 17:12:38.254] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.254] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.254] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.254] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.254] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.254] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.254] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.254] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.254] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.254] Thread 32 (Thread 0x2b6bf7388940 (LWP 3508)): [2015/10/07 17:12:38.254] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.254] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.254] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.255] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.255] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.255] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.255] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.255] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.255] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.255] Thread 31 (Thread 0x2b6bf7d89940 (LWP 3509)): [2015/10/07 17:12:38.255] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.255] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.255] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.255] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.255] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.256] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.256] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.256] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.256] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.256] Thread 30 (Thread 0x2b6bf878a940 (LWP 3510)): [2015/10/07 17:12:38.256] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.256] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.256] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.256] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.256] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.256] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.256] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.256] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.257] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.257] Thread 29 (Thread 0x2b6bf918b940 (LWP 3511)): [2015/10/07 17:12:38.257] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.257] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.257] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.257] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.257] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.257] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.257] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.257] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.257] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.257] Thread 28 (Thread 0x2b6bf9b8c940 (LWP 3512)): [2015/10/07 17:12:38.257] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.258] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.258] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.258] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.258] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.258] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.258] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.258] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.258] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.258] Thread 27 (Thread 0x2b6bfa58d940 (LWP 3513)): [2015/10/07 17:12:38.258] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.258] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.258] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.258] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.259] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.259] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.259] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.259] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.259] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.259] Thread 26 (Thread 0x2b6bfaf8e940 (LWP 3514)): [2015/10/07 17:12:38.259] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.259] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.259] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.259] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.259] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.259] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.259] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.260] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.260] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.260] Thread 25 (Thread 0x2b6bfb98f940 (LWP 3515)): [2015/10/07 17:12:38.260] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.260] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.260] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.260] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.260] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.260] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.260] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.260] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.260] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.261] Thread 24 (Thread 0x2b6bfc390940 (LWP 3516)): [2015/10/07 17:12:38.261] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.261] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.261] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.261] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.261] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.261] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.261] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.261] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.261] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.261] Thread 23 (Thread 0x2b6bfcd91940 (LWP 3517)): [2015/10/07 17:12:38.261] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.261] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.261] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.261] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.262] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.262] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.262] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.262] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.262] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.262] Thread 22 (Thread 0x2b6bfd792940 (LWP 3518)): [2015/10/07 17:12:38.262] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.262] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.262] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.262] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.262] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.262] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.263] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.263] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.263] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.263] Thread 21 (Thread 0x2b6bfe193940 (LWP 3519)): [2015/10/07 17:12:38.263] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.263] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.263] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.263] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.263] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.263] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.263] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.263] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.263] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.264] Thread 20 (Thread 0x2b6c00997940 (LWP 3520)): [2015/10/07 17:12:38.264] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.264] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.264] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.264] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.264] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.264] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.264] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.264] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.264] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.264] Thread 19 (Thread 0x2b6c01398940 (LWP 3521)): [2015/10/07 17:12:38.264] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.264] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.265] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.265] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.265] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.265] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.265] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.265] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.265] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.265] Thread 18 (Thread 0x2b6c01d99940 (LWP 3522)): [2015/10/07 17:12:38.265] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.265] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.265] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.265] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.265] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.266] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.266] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.266] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.266] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.266] Thread 17 (Thread 0x2b6c0279a940 (LWP 3523)): [2015/10/07 17:12:38.266] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.266] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.266] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.266] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.266] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.266] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.266] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.266] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.267] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.267] Thread 16 (Thread 0x2b6c0319b940 (LWP 3524)): [2015/10/07 17:12:38.267] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.267] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.267] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.267] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.267] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.267] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.267] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.267] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.267] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.267] Thread 15 (Thread 0x2b6c03b9c940 (LWP 3525)): [2015/10/07 17:12:38.267] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.268] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.268] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.268] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.268] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.268] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.268] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.268] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.268] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.271] Thread 14 (Thread 0x2b6c0459d940 (LWP 3526)): [2015/10/07 17:12:38.271] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.271] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.271] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.271] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.271] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.271] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.271] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.271] Thread 13 (Thread 0x2b6c04f9e940 (LWP 3527)): [2015/10/07 17:12:38.271] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.271] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.271] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.271] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.271] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.271] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.271] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.271] Thread 12 (Thread 0x2b6c0599f940 (LWP 3528)): [2015/10/07 17:12:38.271] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.271] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.271] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.271] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.271] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.271] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.271] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.271] Thread 11 (Thread 0x2b6c063a0940 (LWP 3529)): [2015/10/07 17:12:38.271] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.271] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.271] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.271] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.271] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.271] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.271] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.271] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.271] Thread 10 (Thread 0x2b6c06da1940 (LWP 3530)): [2015/10/07 17:12:38.272] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.272] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.272] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.272] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.272] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.272] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.272] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.272] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.272] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.272] Thread 9 (Thread 0x2b6c077a2940 (LWP 3531)): [2015/10/07 17:12:38.272] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.272] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.272] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.272] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.272] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.272] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.273] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.273] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.273] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.273] Thread 8 (Thread 0x2b6c081a3940 (LWP 3532)): [2015/10/07 17:12:38.273] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.273] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.273] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.273] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.273] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.273] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.273] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.273] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.273] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.274] Thread 7 (Thread 0x2b6c08ba4940 (LWP 3533)): [2015/10/07 17:12:38.274] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.274] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.274] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.274] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.274] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.274] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.274] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.274] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.275] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.275] Thread 6 (Thread 0x2b6c095a5940 (LWP 3534)): [2015/10/07 17:12:38.275] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.275] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.276] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.276] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.276] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.276] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.276] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.276] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.276] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.276] Thread 5 (Thread 0x2b6c09fa6940 (LWP 3535)): [2015/10/07 17:12:38.276] #0 0x00002b6be4cca019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.276] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.276] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.276] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.276] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.276] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.276] #5 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.276] #6 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.276] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.276] Thread 4 (Thread 0x2b6c0a0a7940 (LWP 3638)): [2015/10/07 17:12:38.277] #0 0x00002b6be4ccce11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.277] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.277] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.277] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.277] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.277] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.277] () [2015/10/07 17:12:38.277] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.277] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.277] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.277] Thread 3 (Thread 0x2b6c0a1a8940 (LWP 3787)): [2015/10/07 17:12:38.277] #0 0x00002b6be4ccce11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.277] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.277] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.278] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.278] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.278] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.278] () [2015/10/07 17:12:38.278] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.278] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.278] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.278] Thread 2 (Thread 0x2b6c0acaa940 (LWP 3829)): [2015/10/07 17:12:38.278] #0 0x00002b6be4ccce11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.278] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.278] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.278] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.278] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.279] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.279] () [2015/10/07 17:12:38.279] #6 0x00002b6be4cc583d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.279] #7 0x00002b6be4fb0fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.279] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.279] Thread 1 (Thread 0x2b6be5236260 (LWP 3341)): [2015/10/07 17:12:38.279] #0 0x00002b6be4faa172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:38.279] #1 0x000000000121b0be in mongo::Listener::initAndListen() () [2015/10/07 17:12:38.279] #2 0x00000000008fef12 in mongo::initAndListen(int) () [2015/10/07 17:12:38.279] #3 0x0000000000902714 in main () [2015/10/07 17:12:38.280] Detaching from program: /data/mci/src/mongod, process 3341 [2015/10/07 17:12:38.285] INFO: Done analyzing process [2015/10/07 17:12:38.285] Dumping process 3641 of mongod [2015/10/07 17:12:38.285] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3641 [2015/10/07 17:12:38.285] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:38.305] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:38.305] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:38.305] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:38.305] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:38.305] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:38.305] and "show warranty" for details. [2015/10/07 17:12:38.305] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:38.305] For bug reporting instructions, please see: [2015/10/07 17:12:38.305] . [2015/10/07 17:12:38.306] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3641', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:38.325] Attaching to process 3641 [2015/10/07 17:12:38.518] Reading symbols from /data/mci/src/mongod...(no debugging symbols found)...done. [2015/10/07 17:12:38.533] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:38.533] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:38.534] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.534] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:38.545] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.545] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:38.546] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.547] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:38.547] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:38.547] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:38.548] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:38.558] [New LWP 3812] [2015/10/07 17:12:38.558] [New LWP 3799] [2015/10/07 17:12:38.558] [New LWP 3780] [2015/10/07 17:12:38.558] [New LWP 3772] [2015/10/07 17:12:38.559] [New LWP 3760] [2015/10/07 17:12:38.559] [New LWP 3750] [2015/10/07 17:12:38.559] [New LWP 3749] [2015/10/07 17:12:38.559] [New LWP 3748] [2015/10/07 17:12:38.559] [New LWP 3747] [2015/10/07 17:12:38.559] [New LWP 3742] [2015/10/07 17:12:38.559] [New LWP 3736] [2015/10/07 17:12:38.559] [New LWP 3729] [2015/10/07 17:12:38.559] [New LWP 3723] [2015/10/07 17:12:38.559] [New LWP 3716] [2015/10/07 17:12:38.559] [New LWP 3702] [2015/10/07 17:12:38.559] [New LWP 3660] [2015/10/07 17:12:38.559] [New LWP 3658] [2015/10/07 17:12:38.559] [New LWP 3657] [2015/10/07 17:12:38.559] [New LWP 3656] [2015/10/07 17:12:38.559] [New LWP 3655] [2015/10/07 17:12:38.559] [New LWP 3654] [2015/10/07 17:12:38.559] [New LWP 3653] [2015/10/07 17:12:38.559] [New LWP 3652] [2015/10/07 17:12:38.559] [New LWP 3651] [2015/10/07 17:12:38.559] [New LWP 3650] [2015/10/07 17:12:38.559] [New LWP 3649] [2015/10/07 17:12:38.559] [New LWP 3648] [2015/10/07 17:12:38.559] [New LWP 3647] [2015/10/07 17:12:38.559] [New LWP 3646] [2015/10/07 17:12:38.559] [New LWP 3645] [2015/10/07 17:12:38.559] [New LWP 3644] [2015/10/07 17:12:38.559] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:38.559] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:38.559] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:38.566] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.566] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:38.567] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.567] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:38.567] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.568] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:38.568] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.568] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:38.569] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.579] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:38.579] 0x00002b0e611d6172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:38.582] Thread 32 (Thread 0x2b0e61e64940 (LWP 3644)): [2015/10/07 17:12:38.582] #0 0x00002b0e60ef99c8 in do_sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.582] #1 0x00002b0e60ef9a6d in sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.582] #2 0x0000000001265316 in mongo::(anonymous namespace)::signalProcessingThread() () [2015/10/07 17:12:38.582] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.583] #4 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.583] #5 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.583] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.583] Thread 31 (Thread 0x2b0e62866940 (LWP 3645)): [2015/10/07 17:12:38.583] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.583] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.583] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.583] #2 0x00000000018ea002 in __evict_server () [2015/10/07 17:12:38.583] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.584] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.584] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.584] Thread 30 (Thread 0x2b0e63267940 (LWP 3646)): [2015/10/07 17:12:38.584] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.584] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.584] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.584] #2 0x00000000018cce33 in __sweep_server () [2015/10/07 17:12:38.584] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.584] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.584] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.584] Thread 29 (Thread 0x2b0e63c68940 (LWP 3647)): [2015/10/07 17:12:38.584] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.584] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.585] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.585] #2 0x00000000018c9b90 in __log_file_server () [2015/10/07 17:12:38.585] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.585] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.585] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.585] Thread 28 (Thread 0x2b0e64669940 (LWP 3648)): [2015/10/07 17:12:38.585] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.585] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.585] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.585] #2 0x00000000018ca2f4 in __log_wrlsn_server () [2015/10/07 17:12:38.585] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.586] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.586] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.586] Thread 27 (Thread 0x2b0e6506a940 (LWP 3649)): [2015/10/07 17:12:38.586] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.586] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.586] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.586] #2 0x00000000018c96b7 in __log_server () [2015/10/07 17:12:38.586] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.586] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.586] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.586] Thread 26 (Thread 0x2b0e65a6b940 (LWP 3650)): [2015/10/07 17:12:38.586] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.586] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.587] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.587] #2 0x00000000018c7239 in __ckpt_server () [2015/10/07 17:12:38.587] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.587] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.587] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.587] Thread 25 (Thread 0x2b0e6646c940 (LWP 3651)): [2015/10/07 17:12:38.587] #0 0x00002b0e60ef6019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.587] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.587] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.587] #2 0x00000000011ced98 in mongo::DeadlineMonitor::deadlineMonitorThread() () [2015/10/07 17:12:38.587] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.588] #4 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.588] #5 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.588] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.588] Thread 24 (Thread 0x2b0e66e6d940 (LWP 3652)): [2015/10/07 17:12:38.588] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.588] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.588] #1 0x0000000000dba6e3 in mongo::RangeDeleter::doWork() () [2015/10/07 17:12:38.588] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.588] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.588] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.588] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.589] Thread 23 (Thread 0x2b0e6786e940 (LWP 3653)): [2015/10/07 17:12:38.589] #0 0x00002b0e60ef6019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.589] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.589] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.589] #2 0x0000000000ff96e1 in mongo::WiredTigerRecordStore::OplogStones::awaitHasExcessStonesOrDead() () [2015/10/07 17:12:38.589] #3 0x0000000000ffa460 in mongo::WiredTigerRecordStore::yieldAndAwaitOplogDeletionRequest(mongo::OperationContext*) () [2015/10/07 17:12:38.589] #4 0x0000000000fffd27 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::_deleteExcessDocuments() () [2015/10/07 17:12:38.589] #5 0x0000000001000238 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::run() () [2015/10/07 17:12:38.589] #6 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.589] #7 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.589] #8 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.590] #9 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.590] #10 0x0000000000000000 in ?? () [2015/10/07 17:12:38.590] Thread 22 (Thread 0x2b0e6826f940 (LWP 3654)): [2015/10/07 17:12:38.590] #0 0x00002b0e60ef9351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.590] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.590] #2 0x0000000000e08233 in mongo::repl::replMasterThread() () [2015/10/07 17:12:38.590] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.590] #4 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.590] #5 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.590] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.591] Thread 21 (Thread 0x2b0e68c70940 (LWP 3655)): [2015/10/07 17:12:38.591] #0 0x00002b0e60ef9351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.591] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.591] #2 0x00000000010137eb in mongo::TTLMonitor::run() () [2015/10/07 17:12:38.591] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.591] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.591] #5 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.591] #6 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.591] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.591] Thread 20 (Thread 0x2b0e69671940 (LWP 3656)): [2015/10/07 17:12:38.591] #0 0x00002b0e60ef9351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.591] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.592] #2 0x0000000000a61378 in mongo::ClientCursorMonitor::run() () [2015/10/07 17:12:38.592] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.592] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.592] #5 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.592] #6 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.592] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.592] Thread 19 (Thread 0x2b0e6a072940 (LWP 3657)): [2015/10/07 17:12:38.592] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.592] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.592] #1 0x00000000012055ce in mongo::(anonymous namespace)::PeriodicTaskRunner::run() () [2015/10/07 17:12:38.592] #2 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.593] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.593] #4 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.593] #5 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.593] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.593] Thread 18 (Thread 0x2b0e6aa73940 (LWP 3658)): [2015/10/07 17:12:38.593] #0 0x00002b0e60ef6280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.593] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.593] #1 0x0000000000b95a66 in mongo::FTDCController::doLoop() () [2015/10/07 17:12:38.593] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.593] #3 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.593] #4 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.593] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.594] Thread 17 (Thread 0x2b0e6ab75940 (LWP 3660)): [2015/10/07 17:12:38.594] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.594] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.594] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.594] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.594] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.594] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.594] () [2015/10/07 17:12:38.594] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.594] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.594] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.595] Thread 16 (Thread 0x2b0e6ac76940 (LWP 3702)): [2015/10/07 17:12:38.595] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.595] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.595] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.595] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.595] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.595] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.595] () [2015/10/07 17:12:38.595] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.595] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.595] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.596] Thread 15 (Thread 0x2b0e6ad77940 (LWP 3716)): [2015/10/07 17:12:38.596] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.596] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.596] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.596] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.596] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.596] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.596] () [2015/10/07 17:12:38.596] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.596] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.596] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.597] Thread 14 (Thread 0x2b0e6ae78940 (LWP 3723)): [2015/10/07 17:12:38.597] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.597] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.597] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.597] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.597] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.597] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.597] () [2015/10/07 17:12:38.597] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.597] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.597] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.598] Thread 13 (Thread 0x2b0e6af79940 (LWP 3729)): [2015/10/07 17:12:38.598] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.598] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.598] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.598] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.598] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.598] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.598] () [2015/10/07 17:12:38.598] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.598] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.598] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.599] Thread 12 (Thread 0x2b0e6b07a940 (LWP 3736)): [2015/10/07 17:12:38.599] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.599] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.599] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.599] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.599] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.599] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.599] () [2015/10/07 17:12:38.599] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.599] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.599] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.600] Thread 11 (Thread 0x2b0e6b17b940 (LWP 3742)): [2015/10/07 17:12:38.600] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.600] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.600] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.600] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.600] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.600] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.600] () [2015/10/07 17:12:38.600] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.600] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.600] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.601] Thread 10 (Thread 0x2b0e6c1ad940 (LWP 3747)): [2015/10/07 17:12:38.601] #0 0x00002b0e60ef6019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.601] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.601] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:38.601] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:38.601] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:38.601] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:38.601] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.601] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.601] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.601] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.602] Thread 9 (Thread 0x2b0e6cbae940 (LWP 3748)): [2015/10/07 17:12:38.602] #0 0x00002b0e60ef6019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.602] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.602] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.602] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.602] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.602] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.602] #5 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.602] #6 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.602] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.602] Thread 8 (Thread 0x2b0e6d5af940 (LWP 3749)): [2015/10/07 17:12:38.602] #0 0x00002b0e60ef6019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.602] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.603] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:38.603] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:38.603] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:38.603] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:38.603] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.603] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.603] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.603] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.603] Thread 7 (Thread 0x2b0e6dfb0940 (LWP 3750)): [2015/10/07 17:12:38.603] #0 0x00002b0e60ef6019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.603] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.603] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.604] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.604] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.604] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.604] #5 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.604] #6 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.604] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.604] Thread 6 (Thread 0x2b0e6e0b1940 (LWP 3760)): [2015/10/07 17:12:38.604] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.604] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.604] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.605] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.605] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.605] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.605] () [2015/10/07 17:12:38.605] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.605] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.605] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.605] Thread 5 (Thread 0x2b0e6e1b2940 (LWP 3772)): [2015/10/07 17:12:38.605] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.605] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.605] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.606] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.606] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.606] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.606] () [2015/10/07 17:12:38.606] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.606] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.606] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.606] Thread 4 (Thread 0x2b0e6e2b3940 (LWP 3780)): [2015/10/07 17:12:38.606] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.606] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.606] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.607] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.607] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.607] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.607] () [2015/10/07 17:12:38.607] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.607] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.607] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.607] Thread 3 (Thread 0x2b0e6e3b4940 (LWP 3799)): [2015/10/07 17:12:38.607] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.607] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.607] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.607] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.608] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.608] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.608] () [2015/10/07 17:12:38.608] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.608] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.608] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.608] Thread 2 (Thread 0x2b0e6e4b5940 (LWP 3812)): [2015/10/07 17:12:38.608] #0 0x00002b0e60ef8e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.608] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.608] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.609] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.609] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.609] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.609] () [2015/10/07 17:12:38.609] #6 0x00002b0e60ef183d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.609] #7 0x00002b0e611dcfdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.609] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.609] Thread 1 (Thread 0x2b0e61462260 (LWP 3641)): [2015/10/07 17:12:38.609] #0 0x00002b0e611d6172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:38.609] #1 0x000000000121b0be in mongo::Listener::initAndListen() () [2015/10/07 17:12:38.609] #2 0x00000000008fef12 in mongo::initAndListen(int) () [2015/10/07 17:12:38.610] #3 0x0000000000902714 in main () [2015/10/07 17:12:38.610] Detaching from program: /data/mci/src/mongod, process 3641 [2015/10/07 17:12:38.615] INFO: Done analyzing process [2015/10/07 17:12:38.615] Dumping process 3661 of mongod [2015/10/07 17:12:38.615] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3661 [2015/10/07 17:12:38.615] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:38.636] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:38.636] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:38.636] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:38.636] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:38.636] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:38.636] and "show warranty" for details. [2015/10/07 17:12:38.636] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:38.636] For bug reporting instructions, please see: [2015/10/07 17:12:38.636] . [2015/10/07 17:12:38.636] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3661', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:38.656] Attaching to process 3661 [2015/10/07 17:12:38.852] Reading symbols from /data/mci/src/mongod...(no debugging symbols found)...done. [2015/10/07 17:12:38.867] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:38.867] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:38.867] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.868] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:38.878] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.878] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:38.880] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.880] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:38.880] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:38.881] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:38.882] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:38.892] [New LWP 3814] [2015/10/07 17:12:38.892] [New LWP 3801] [2015/10/07 17:12:38.892] [New LWP 3782] [2015/10/07 17:12:38.892] [New LWP 3774] [2015/10/07 17:12:38.892] [New LWP 3762] [2015/10/07 17:12:38.892] [New LWP 3754] [2015/10/07 17:12:38.892] [New LWP 3753] [2015/10/07 17:12:38.892] [New LWP 3752] [2015/10/07 17:12:38.892] [New LWP 3751] [2015/10/07 17:12:38.892] [New LWP 3744] [2015/10/07 17:12:38.892] [New LWP 3738] [2015/10/07 17:12:38.892] [New LWP 3731] [2015/10/07 17:12:38.892] [New LWP 3725] [2015/10/07 17:12:38.892] [New LWP 3718] [2015/10/07 17:12:38.892] [New LWP 3704] [2015/10/07 17:12:38.892] [New LWP 3680] [2015/10/07 17:12:38.892] [New LWP 3678] [2015/10/07 17:12:38.892] [New LWP 3677] [2015/10/07 17:12:38.892] [New LWP 3676] [2015/10/07 17:12:38.892] [New LWP 3675] [2015/10/07 17:12:38.892] [New LWP 3674] [2015/10/07 17:12:38.892] [New LWP 3673] [2015/10/07 17:12:38.892] [New LWP 3672] [2015/10/07 17:12:38.892] [New LWP 3671] [2015/10/07 17:12:38.892] [New LWP 3670] [2015/10/07 17:12:38.892] [New LWP 3669] [2015/10/07 17:12:38.892] [New LWP 3668] [2015/10/07 17:12:38.892] [New LWP 3667] [2015/10/07 17:12:38.892] [New LWP 3666] [2015/10/07 17:12:38.893] [New LWP 3665] [2015/10/07 17:12:38.893] [New LWP 3664] [2015/10/07 17:12:38.893] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:38.893] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:38.893] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:38.900] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:38.900] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:38.900] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.900] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:38.901] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.901] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:38.901] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.901] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:38.902] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:38.912] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:38.913] 0x00002b4d5ae8e172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:38.915] Thread 32 (Thread 0x2b4d5bb1c940 (LWP 3664)): [2015/10/07 17:12:38.915] #0 0x00002b4d5abb19c8 in do_sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.915] #1 0x00002b4d5abb1a6d in sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.915] #2 0x0000000001265316 in mongo::(anonymous namespace)::signalProcessingThread() () [2015/10/07 17:12:38.915] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.916] #4 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.916] #5 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.916] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.916] Thread 31 (Thread 0x2b4d5c51e940 (LWP 3665)): [2015/10/07 17:12:38.916] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.916] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.916] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.916] #2 0x00000000018ea002 in __evict_server () [2015/10/07 17:12:38.916] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.916] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.916] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.917] Thread 30 (Thread 0x2b4d5cf1f940 (LWP 3666)): [2015/10/07 17:12:38.917] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.917] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.917] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.917] #2 0x00000000018cce33 in __sweep_server () [2015/10/07 17:12:38.917] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.917] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.917] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.917] Thread 29 (Thread 0x2b4d5d920940 (LWP 3667)): [2015/10/07 17:12:38.917] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.917] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.917] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.917] #2 0x00000000018c9b90 in __log_file_server () [2015/10/07 17:12:38.917] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.917] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.918] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.918] Thread 28 (Thread 0x2b4d5e321940 (LWP 3668)): [2015/10/07 17:12:38.918] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.918] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.918] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.918] #2 0x00000000018ca2f4 in __log_wrlsn_server () [2015/10/07 17:12:38.918] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.918] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.918] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.918] Thread 27 (Thread 0x2b4d5ed22940 (LWP 3669)): [2015/10/07 17:12:38.918] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.918] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.918] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.918] #2 0x00000000018c96b7 in __log_server () [2015/10/07 17:12:38.919] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.919] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.919] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.919] Thread 26 (Thread 0x2b4d5f723940 (LWP 3670)): [2015/10/07 17:12:38.919] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.919] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.919] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:38.919] #2 0x00000000018c7239 in __ckpt_server () [2015/10/07 17:12:38.919] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.919] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.919] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.919] Thread 25 (Thread 0x2b4d60124940 (LWP 3671)): [2015/10/07 17:12:38.919] #0 0x00002b4d5abae019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.919] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.919] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.920] #2 0x00000000011ced98 in mongo::DeadlineMonitor::deadlineMonitorThread() () [2015/10/07 17:12:38.920] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.920] #4 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.920] #5 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.920] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.920] Thread 24 (Thread 0x2b4d60b25940 (LWP 3672)): [2015/10/07 17:12:38.920] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.920] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.920] #1 0x0000000000dba6e3 in mongo::RangeDeleter::doWork() () [2015/10/07 17:12:38.920] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.920] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.920] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.920] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.921] Thread 23 (Thread 0x2b4d61526940 (LWP 3673)): [2015/10/07 17:12:38.921] #0 0x00002b4d5abae019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.921] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.921] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.921] #2 0x0000000000ff96e1 in mongo::WiredTigerRecordStore::OplogStones::awaitHasExcessStonesOrDead() () [2015/10/07 17:12:38.921] #3 0x0000000000ffa460 in mongo::WiredTigerRecordStore::yieldAndAwaitOplogDeletionRequest(mongo::OperationContext*) () [2015/10/07 17:12:38.921] #4 0x0000000000fffd27 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::_deleteExcessDocuments() () [2015/10/07 17:12:38.921] #5 0x0000000001000238 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::run() () [2015/10/07 17:12:38.921] #6 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.921] #7 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.921] #8 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.921] #9 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.921] #10 0x0000000000000000 in ?? () [2015/10/07 17:12:38.922] Thread 22 (Thread 0x2b4d61f27940 (LWP 3674)): [2015/10/07 17:12:38.922] #0 0x00002b4d5abb1351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.922] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.922] #2 0x0000000000e08233 in mongo::repl::replMasterThread() () [2015/10/07 17:12:38.922] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.922] #4 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.922] #5 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.922] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.922] Thread 21 (Thread 0x2b4d62928940 (LWP 3675)): [2015/10/07 17:12:38.922] #0 0x00002b4d5abb1351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.922] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.922] #2 0x00000000010137eb in mongo::TTLMonitor::run() () [2015/10/07 17:12:38.922] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.922] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.923] #5 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.923] #6 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.923] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.923] Thread 20 (Thread 0x2b4d63329940 (LWP 3676)): [2015/10/07 17:12:38.923] #0 0x00002b4d5abb1351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.923] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:38.923] #2 0x0000000000a61378 in mongo::ClientCursorMonitor::run() () [2015/10/07 17:12:38.923] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.923] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.923] #5 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.923] #6 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.923] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.924] Thread 19 (Thread 0x2b4d63d2a940 (LWP 3677)): [2015/10/07 17:12:38.924] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.924] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.924] #1 0x00000000012055ce in mongo::(anonymous namespace)::PeriodicTaskRunner::run() () [2015/10/07 17:12:38.924] #2 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:38.924] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.924] #4 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.924] #5 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.924] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:38.924] Thread 18 (Thread 0x2b4d6472b940 (LWP 3678)): [2015/10/07 17:12:38.924] #0 0x00002b4d5abae280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.924] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.924] #1 0x0000000000b95a66 in mongo::FTDCController::doLoop() () [2015/10/07 17:12:38.924] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.925] #3 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.925] #4 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.925] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:38.925] Thread 17 (Thread 0x2b4d6482d940 (LWP 3680)): [2015/10/07 17:12:38.925] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.925] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.925] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.925] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.925] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.925] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.925] () [2015/10/07 17:12:38.925] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.925] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.925] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.926] Thread 16 (Thread 0x2b4d6492e940 (LWP 3704)): [2015/10/07 17:12:38.926] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.926] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.926] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.926] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.926] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.926] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.926] () [2015/10/07 17:12:38.926] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.926] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.926] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.926] Thread 15 (Thread 0x2b4d64a2f940 (LWP 3718)): [2015/10/07 17:12:38.926] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.927] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.927] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.927] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.927] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.927] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.927] () [2015/10/07 17:12:38.927] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.927] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.927] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.927] Thread 14 (Thread 0x2b4d64b30940 (LWP 3725)): [2015/10/07 17:12:38.927] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.927] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.927] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.928] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.928] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.928] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.928] () [2015/10/07 17:12:38.928] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.928] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.928] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.928] Thread 13 (Thread 0x2b4d64c31940 (LWP 3731)): [2015/10/07 17:12:38.928] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.928] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.928] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.928] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.928] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.929] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.929] () [2015/10/07 17:12:38.929] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.929] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.929] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.929] Thread 12 (Thread 0x2b4d64d32940 (LWP 3738)): [2015/10/07 17:12:38.929] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.929] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.929] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.929] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.929] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.929] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.929] () [2015/10/07 17:12:38.929] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.929] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.930] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.930] Thread 11 (Thread 0x2b4d64e33940 (LWP 3744)): [2015/10/07 17:12:38.930] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.930] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.930] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.930] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.930] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.930] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.930] () [2015/10/07 17:12:38.930] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.930] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.930] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.930] Thread 10 (Thread 0x2b4d65e65940 (LWP 3751)): [2015/10/07 17:12:38.931] #0 0x00002b4d5abae019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.931] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.931] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:38.931] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:38.931] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:38.931] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:38.931] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.931] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.931] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.931] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.931] Thread 9 (Thread 0x2b4d66866940 (LWP 3752)): [2015/10/07 17:12:38.931] #0 0x00002b4d5abae019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.931] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.931] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.932] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.932] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.932] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.932] #5 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.932] #6 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.932] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.932] Thread 8 (Thread 0x2b4d67267940 (LWP 3753)): [2015/10/07 17:12:38.932] #0 0x00002b4d5abae019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.932] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.932] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:38.932] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:38.932] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:38.932] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:38.933] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.933] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.933] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.933] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.933] Thread 7 (Thread 0x2b4d67c68940 (LWP 3754)): [2015/10/07 17:12:38.933] #0 0x00002b4d5abae019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:38.933] from /lib64/libpthread.so.0 [2015/10/07 17:12:38.933] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:38.933] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:38.933] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:38.933] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:38.933] #5 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.933] #6 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.933] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:38.934] Thread 6 (Thread 0x2b4d67d69940 (LWP 3762)): [2015/10/07 17:12:38.934] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.934] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.934] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.934] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.934] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.934] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.934] () [2015/10/07 17:12:38.934] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.934] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.934] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.934] Thread 5 (Thread 0x2b4d67e6a940 (LWP 3774)): [2015/10/07 17:12:38.934] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.935] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.935] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.935] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.935] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.935] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.935] () [2015/10/07 17:12:38.935] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.935] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.935] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.935] Thread 4 (Thread 0x2b4d67f6b940 (LWP 3782)): [2015/10/07 17:12:38.935] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.935] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.935] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.935] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.936] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.936] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.936] () [2015/10/07 17:12:38.936] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.936] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.936] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.936] Thread 3 (Thread 0x2b4d6806c940 (LWP 3801)): [2015/10/07 17:12:38.936] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.936] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.936] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.936] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.936] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.936] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.937] () [2015/10/07 17:12:38.937] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.937] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.937] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.937] Thread 2 (Thread 0x2b4d6816d940 (LWP 3814)): [2015/10/07 17:12:38.937] #0 0x00002b4d5abb0e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.937] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:38.937] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:38.937] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:38.937] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:38.937] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:38.937] () [2015/10/07 17:12:38.937] #6 0x00002b4d5aba983d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:38.937] #7 0x00002b4d5ae94fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:38.938] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:38.938] Thread 1 (Thread 0x2b4d5b11a260 (LWP 3661)): [2015/10/07 17:12:38.938] #0 0x00002b4d5ae8e172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:38.938] #1 0x000000000121b0be in mongo::Listener::initAndListen() () [2015/10/07 17:12:38.938] #2 0x00000000008fef12 in mongo::initAndListen(int) () [2015/10/07 17:12:38.938] #3 0x0000000000902714 in main () [2015/10/07 17:12:38.938] Detaching from program: /data/mci/src/mongod, process 3661 [2015/10/07 17:12:38.943] INFO: Done analyzing process [2015/10/07 17:12:38.943] Dumping process 3681 of mongod [2015/10/07 17:12:38.943] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3681 [2015/10/07 17:12:38.943] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:38.963] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:38.963] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:38.963] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:38.963] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:38.963] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:38.963] and "show warranty" for details. [2015/10/07 17:12:38.964] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:38.964] For bug reporting instructions, please see: [2015/10/07 17:12:38.964] . [2015/10/07 17:12:38.964] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3681', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:38.984] Attaching to process 3681 [2015/10/07 17:12:39.183] Reading symbols from /data/mci/src/mongod...(no debugging symbols found)...done. [2015/10/07 17:12:39.198] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:39.199] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:39.199] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.199] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:39.210] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:39.210] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:39.212] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:39.212] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:39.212] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:39.212] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:39.213] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:39.224] [New LWP 3816] [2015/10/07 17:12:39.224] [New LWP 3803] [2015/10/07 17:12:39.224] [New LWP 3784] [2015/10/07 17:12:39.224] [New LWP 3776] [2015/10/07 17:12:39.224] [New LWP 3764] [2015/10/07 17:12:39.224] [New LWP 3758] [2015/10/07 17:12:39.224] [New LWP 3757] [2015/10/07 17:12:39.224] [New LWP 3756] [2015/10/07 17:12:39.224] [New LWP 3755] [2015/10/07 17:12:39.224] [New LWP 3746] [2015/10/07 17:12:39.224] [New LWP 3740] [2015/10/07 17:12:39.224] [New LWP 3733] [2015/10/07 17:12:39.224] [New LWP 3727] [2015/10/07 17:12:39.224] [New LWP 3720] [2015/10/07 17:12:39.224] [New LWP 3706] [2015/10/07 17:12:39.224] [New LWP 3700] [2015/10/07 17:12:39.224] [New LWP 3698] [2015/10/07 17:12:39.224] [New LWP 3697] [2015/10/07 17:12:39.224] [New LWP 3696] [2015/10/07 17:12:39.224] [New LWP 3695] [2015/10/07 17:12:39.224] [New LWP 3694] [2015/10/07 17:12:39.224] [New LWP 3693] [2015/10/07 17:12:39.224] [New LWP 3692] [2015/10/07 17:12:39.224] [New LWP 3691] [2015/10/07 17:12:39.224] [New LWP 3690] [2015/10/07 17:12:39.224] [New LWP 3689] [2015/10/07 17:12:39.224] [New LWP 3688] [2015/10/07 17:12:39.224] [New LWP 3687] [2015/10/07 17:12:39.224] [New LWP 3686] [2015/10/07 17:12:39.224] [New LWP 3685] [2015/10/07 17:12:39.224] [New LWP 3684] [2015/10/07 17:12:39.224] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:39.224] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:39.224] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:39.231] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:39.232] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:39.232] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.232] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:39.233] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.233] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:39.233] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.233] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:39.234] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.245] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:39.245] 0x00002b57bf3b1172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:39.247] Thread 32 (Thread 0x2b57c003f940 (LWP 3684)): [2015/10/07 17:12:39.247] #0 0x00002b57bf0d49c8 in do_sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.247] #1 0x00002b57bf0d4a6d in sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.248] #2 0x0000000001265316 in mongo::(anonymous namespace)::signalProcessingThread() () [2015/10/07 17:12:39.248] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.248] #4 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.248] #5 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.248] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.248] Thread 31 (Thread 0x2b57c0a41940 (LWP 3685)): [2015/10/07 17:12:39.248] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.248] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.248] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:39.248] #2 0x00000000018ea002 in __evict_server () [2015/10/07 17:12:39.248] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.249] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.249] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.249] Thread 30 (Thread 0x2b57c1442940 (LWP 3686)): [2015/10/07 17:12:39.249] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.249] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.249] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:39.249] #2 0x00000000018cce33 in __sweep_server () [2015/10/07 17:12:39.249] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.249] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.249] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.249] Thread 29 (Thread 0x2b57c1e43940 (LWP 3687)): [2015/10/07 17:12:39.249] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.249] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.249] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:39.249] #2 0x00000000018c9b90 in __log_file_server () [2015/10/07 17:12:39.250] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.250] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.250] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.250] Thread 28 (Thread 0x2b57c2844940 (LWP 3688)): [2015/10/07 17:12:39.250] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.250] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.250] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:39.250] #2 0x00000000018ca2f4 in __log_wrlsn_server () [2015/10/07 17:12:39.250] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.250] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.250] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.250] Thread 27 (Thread 0x2b57c3245940 (LWP 3689)): [2015/10/07 17:12:39.250] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.250] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.250] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:39.251] #2 0x00000000018c96b7 in __log_server () [2015/10/07 17:12:39.251] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.251] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.251] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.251] Thread 26 (Thread 0x2b57c3c46940 (LWP 3690)): [2015/10/07 17:12:39.251] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.251] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.251] #1 0x0000000001905c2c in __wt_cond_wait_signal () [2015/10/07 17:12:39.251] #2 0x00000000018c7239 in __ckpt_server () [2015/10/07 17:12:39.251] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.251] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.251] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.251] Thread 25 (Thread 0x2b57c4647940 (LWP 3691)): [2015/10/07 17:12:39.251] #0 0x00002b57bf0d1019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.251] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.252] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:39.252] #2 0x00000000011ced98 in mongo::DeadlineMonitor::deadlineMonitorThread() () [2015/10/07 17:12:39.252] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.252] #4 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.252] #5 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.252] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.252] Thread 24 (Thread 0x2b57c5048940 (LWP 3692)): [2015/10/07 17:12:39.252] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.252] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.252] #1 0x0000000000dba6e3 in mongo::RangeDeleter::doWork() () [2015/10/07 17:12:39.252] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.252] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.252] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.252] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.253] Thread 23 (Thread 0x2b57c5a49940 (LWP 3693)): [2015/10/07 17:12:39.253] #0 0x00002b57bf0d1019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.253] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.253] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:39.253] #2 0x0000000000ff96e1 in mongo::WiredTigerRecordStore::OplogStones::awaitHasExcessStonesOrDead() () [2015/10/07 17:12:39.253] #3 0x0000000000ffa460 in mongo::WiredTigerRecordStore::yieldAndAwaitOplogDeletionRequest(mongo::OperationContext*) () [2015/10/07 17:12:39.253] #4 0x0000000000fffd27 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::_deleteExcessDocuments() () [2015/10/07 17:12:39.253] #5 0x0000000001000238 in mongo::(anonymous namespace)::WiredTigerRecordStoreThread::run() () [2015/10/07 17:12:39.253] #6 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.253] #7 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.253] #8 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.253] #9 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.253] #10 0x0000000000000000 in ?? () [2015/10/07 17:12:39.254] Thread 22 (Thread 0x2b57c644a940 (LWP 3694)): [2015/10/07 17:12:39.254] #0 0x00002b57bf0d4351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.254] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:39.254] #2 0x0000000000e08233 in mongo::repl::replMasterThread() () [2015/10/07 17:12:39.254] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.254] #4 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.254] #5 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.254] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.254] Thread 21 (Thread 0x2b57c6e4b940 (LWP 3695)): [2015/10/07 17:12:39.254] #0 0x00002b57bf0d4351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.254] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:39.254] #2 0x00000000010137eb in mongo::TTLMonitor::run() () [2015/10/07 17:12:39.254] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.255] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.255] #5 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.255] #6 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.255] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.255] Thread 20 (Thread 0x2b57c784c940 (LWP 3696)): [2015/10/07 17:12:39.255] #0 0x00002b57bf0d4351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.255] #1 0x0000000001271325 in mongo::sleepsecs(int) () [2015/10/07 17:12:39.255] #2 0x0000000000a61378 in mongo::ClientCursorMonitor::run() () [2015/10/07 17:12:39.255] #3 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.255] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.255] #5 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.255] #6 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.255] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.256] Thread 19 (Thread 0x2b57c824d940 (LWP 3697)): [2015/10/07 17:12:39.256] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.256] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.256] #1 0x00000000012055ce in mongo::(anonymous namespace)::PeriodicTaskRunner::run() () [2015/10/07 17:12:39.256] #2 0x0000000001204b17 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.256] #3 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.256] #4 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.256] #5 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.256] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.256] Thread 18 (Thread 0x2b57c8c4e940 (LWP 3698)): [2015/10/07 17:12:39.256] #0 0x00002b57bf0d1280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.256] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.256] #1 0x0000000000b95a66 in mongo::FTDCController::doLoop() () [2015/10/07 17:12:39.256] #2 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.257] #3 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.257] #4 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.257] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.257] Thread 17 (Thread 0x2b57c8d50940 (LWP 3700)): [2015/10/07 17:12:39.257] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.257] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.257] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.257] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.257] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.257] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.257] () [2015/10/07 17:12:39.257] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.257] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.257] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.258] Thread 16 (Thread 0x2b57c8e51940 (LWP 3706)): [2015/10/07 17:12:39.258] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.258] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.258] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.258] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.258] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.258] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.258] () [2015/10/07 17:12:39.258] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.258] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.258] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.258] Thread 15 (Thread 0x2b57c8f52940 (LWP 3720)): [2015/10/07 17:12:39.258] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.259] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.259] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.259] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.259] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.259] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.259] () [2015/10/07 17:12:39.259] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.259] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.259] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.259] Thread 14 (Thread 0x2b57c9053940 (LWP 3727)): [2015/10/07 17:12:39.259] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.259] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.259] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.259] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.260] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.260] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.260] () [2015/10/07 17:12:39.260] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.260] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.260] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.260] Thread 13 (Thread 0x2b57c9154940 (LWP 3733)): [2015/10/07 17:12:39.260] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.260] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.260] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.260] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.260] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.260] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.260] () [2015/10/07 17:12:39.261] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.261] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.261] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.261] Thread 12 (Thread 0x2b57c9255940 (LWP 3740)): [2015/10/07 17:12:39.261] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.261] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.261] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.261] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.261] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.261] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.261] () [2015/10/07 17:12:39.261] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.261] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.262] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.262] Thread 11 (Thread 0x2b57c9356940 (LWP 3746)): [2015/10/07 17:12:39.262] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.262] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.262] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.262] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.262] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.262] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.262] () [2015/10/07 17:12:39.262] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.262] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.262] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.262] Thread 10 (Thread 0x2b57ca388940 (LWP 3755)): [2015/10/07 17:12:39.262] #0 0x00002b57bf0d1019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.262] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.263] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:39.263] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:39.263] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:39.263] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:39.263] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.263] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.263] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.263] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.263] Thread 9 (Thread 0x2b57cad89940 (LWP 3756)): [2015/10/07 17:12:39.263] #0 0x00002b57bf0d1019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.263] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.263] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:39.263] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:39.264] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:39.264] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.264] #5 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.264] #6 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.264] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.264] Thread 8 (Thread 0x2b57cb78a940 (LWP 3757)): [2015/10/07 17:12:39.264] #0 0x00002b57bf0d1019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.264] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.264] #1 0x0000000001283aa0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:39.264] #2 0x0000000001283d51 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:39.264] #3 0x0000000001287edf in asio::io_service::run() () [2015/10/07 17:12:39.264] #4 0x000000000102cdff in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:39.264] #5 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.265] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.265] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.265] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.265] Thread 7 (Thread 0x2b57cc18b940 (LWP 3758)): [2015/10/07 17:12:39.265] #0 0x00002b57bf0d1019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.265] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.265] #1 0x00000000019a9b0c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:39.265] #2 0x000000000120bc3e in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:39.265] #3 0x000000000120c3f0 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:39.265] #4 0x00000000019aa2b0 in execute_native_thread_routine () [2015/10/07 17:12:39.265] #5 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.265] #6 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.265] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.265] Thread 6 (Thread 0x2b57cc28c940 (LWP 3764)): [2015/10/07 17:12:39.266] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.266] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.266] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.266] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.266] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.266] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.266] () [2015/10/07 17:12:39.266] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.266] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.266] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.266] Thread 5 (Thread 0x2b57cc38d940 (LWP 3776)): [2015/10/07 17:12:39.266] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.266] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.266] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.267] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.267] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.267] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.267] () [2015/10/07 17:12:39.267] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.267] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.267] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.267] Thread 4 (Thread 0x2b57cc48e940 (LWP 3784)): [2015/10/07 17:12:39.267] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.267] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.267] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.267] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.267] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.268] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.268] () [2015/10/07 17:12:39.268] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.268] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.268] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.268] Thread 3 (Thread 0x2b57cc58f940 (LWP 3803)): [2015/10/07 17:12:39.268] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.268] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.268] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.268] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.268] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.268] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.268] () [2015/10/07 17:12:39.268] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.269] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.269] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.269] Thread 2 (Thread 0x2b57cc690940 (LWP 3816)): [2015/10/07 17:12:39.269] #0 0x00002b57bf0d3e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.269] #1 0x000000000122abeb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.269] #2 0x000000000122ac31 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.269] #3 0x000000000122ac8d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.269] #4 0x000000000121f952 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.269] #5 0x0000000001221517 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.269] () [2015/10/07 17:12:39.269] #6 0x00002b57bf0cc83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.269] #7 0x00002b57bf3b7fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.269] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.270] Thread 1 (Thread 0x2b57bf63d260 (LWP 3681)): [2015/10/07 17:12:39.270] #0 0x00002b57bf3b1172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:39.270] #1 0x000000000121b0be in mongo::Listener::initAndListen() () [2015/10/07 17:12:39.270] #2 0x00000000008fef12 in mongo::initAndListen(int) () [2015/10/07 17:12:39.270] #3 0x0000000000902714 in main () [2015/10/07 17:12:39.270] Detaching from program: /data/mci/src/mongod, process 3681 [2015/10/07 17:12:39.275] INFO: Done analyzing process [2015/10/07 17:12:39.275] Dumping process 3707 of mongos [2015/10/07 17:12:39.275] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 3707 [2015/10/07 17:12:39.275] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:39.296] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:39.296] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:39.297] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:39.297] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:39.297] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:39.297] and "show warranty" for details. [2015/10/07 17:12:39.297] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:39.297] For bug reporting instructions, please see: [2015/10/07 17:12:39.297] . [2015/10/07 17:12:39.298] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 3707', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:39.317] Attaching to process 3707 [2015/10/07 17:12:39.402] Reading symbols from /data/mci/src/mongos...(no debugging symbols found)...done. [2015/10/07 17:12:39.412] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:39.412] Loaded symbols for /lib64/librt.so.1 [2015/10/07 17:12:39.412] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.412] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:39.422] Reading symbols from /usr/lib64/libstdc++.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:39.422] Loaded symbols for /usr/lib64/libstdc++.so.6 [2015/10/07 17:12:39.423] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:39.423] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:39.424] Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:39.424] Loaded symbols for /lib64/libgcc_s.so.1 [2015/10/07 17:12:39.425] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:39.431] [New LWP 3792] [2015/10/07 17:12:39.431] [New LWP 3790] [2015/10/07 17:12:39.431] [New LWP 3785] [2015/10/07 17:12:39.431] [New LWP 3778] [2015/10/07 17:12:39.431] [New LWP 3770] [2015/10/07 17:12:39.431] [New LWP 3769] [2015/10/07 17:12:39.431] [New LWP 3768] [2015/10/07 17:12:39.431] [New LWP 3767] [2015/10/07 17:12:39.431] [New LWP 3766] [2015/10/07 17:12:39.431] [New LWP 3734] [2015/10/07 17:12:39.431] [New LWP 3721] [2015/10/07 17:12:39.431] [New LWP 3714] [2015/10/07 17:12:39.431] [New LWP 3713] [2015/10/07 17:12:39.431] [New LWP 3712] [2015/10/07 17:12:39.431] [New LWP 3711] [2015/10/07 17:12:39.431] [New LWP 3710] [2015/10/07 17:12:39.431] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:39.431] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:39.431] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:39.438] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:39.438] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:39.439] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.439] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:39.439] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.439] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:39.440] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.440] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:39.440] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:39.446] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:39.446] 0x00002b9e6143f172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:39.447] Thread 17 (Thread 0x2b9e620cc940 (LWP 3710)): [2015/10/07 17:12:39.447] #0 0x00002b9e611629c8 in do_sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.447] #1 0x00002b9e61162a6d in sigwait () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.447] #2 0x0000000000ba6eb6 in mongo::(anonymous namespace)::signalProcessingThread() () [2015/10/07 17:12:39.447] #3 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.447] #4 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.448] #5 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.448] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.448] Thread 16 (Thread 0x2b9e630fe940 (LWP 3711)): [2015/10/07 17:12:39.448] #0 0x00002b9e614463c8 in epoll_wait () from /lib64/libc.so.6 [2015/10/07 17:12:39.448] #1 0x0000000000bc4b65 in asio::detail::epoll_reactor::run(bool, asio::detail::op_queue&) () [2015/10/07 17:12:39.448] #2 0x0000000000bc56b8 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:39.448] #3 0x0000000000bc5a81 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:39.448] #4 0x0000000000bc9c0f in asio::io_service::run() () [2015/10/07 17:12:39.448] #5 0x000000000095253f in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:39.448] #6 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.448] #7 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.448] #8 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.448] #9 0x0000000000000000 in ?? () [2015/10/07 17:12:39.448] Thread 15 (Thread 0x2b9e63aff940 (LWP 3712)): [2015/10/07 17:12:39.448] #0 0x00002b9e6115f019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.448] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.449] #1 0x0000000000d0c59c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:39.449] #2 0x0000000000b4dace in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:39.449] #3 0x0000000000b4e280 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:39.449] #4 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.449] #5 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.449] #6 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.449] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.449] Thread 14 (Thread 0x2b9e64500940 (LWP 3713)): [2015/10/07 17:12:39.449] #0 0x00002b9e614463c8 in epoll_wait () from /lib64/libc.so.6 [2015/10/07 17:12:39.449] #1 0x0000000000bc4b65 in asio::detail::epoll_reactor::run(bool, asio::detail::op_queue&) () [2015/10/07 17:12:39.449] #2 0x0000000000bc56b8 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:39.449] #3 0x0000000000bc5a81 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:39.449] #4 0x0000000000bc9c0f in asio::io_service::run() () [2015/10/07 17:12:39.449] #5 0x000000000095253f in std::thread::_Impl >::_M_run() () [2015/10/07 17:12:39.449] #6 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.449] #7 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.449] #8 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.450] #9 0x0000000000000000 in ?? () [2015/10/07 17:12:39.450] Thread 13 (Thread 0x2b9e64f01940 (LWP 3714)): [2015/10/07 17:12:39.450] #0 0x00002b9e6115f019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.450] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.450] #1 0x0000000000d0c59c in std::condition_variable::wait(std::unique_lock&) () [2015/10/07 17:12:39.450] #2 0x0000000000b4dace in mongo::ThreadPool::_consumeTasks() () [2015/10/07 17:12:39.450] #3 0x0000000000b4e280 in mongo::ThreadPool::_workerThreadBody(mongo::ThreadPool*, std::string const&) () [2015/10/07 17:12:39.450] #4 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.450] #5 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.450] #6 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.450] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.450] Thread 12 (Thread 0x2b9e66303940 (LWP 3721)): [2015/10/07 17:12:39.450] #0 0x00002b9e6115f280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.450] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.450] #1 0x00000000009c30b0 in mongo::CatalogManagerLegacy::_consistencyChecker() [2015/10/07 17:12:39.450] () [2015/10/07 17:12:39.450] #2 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.450] #3 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.450] #4 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.451] #5 0x0000000000000000 in ?? () [2015/10/07 17:12:39.451] Thread 11 (Thread 0x2b9e65902940 (LWP 3734)): [2015/10/07 17:12:39.451] #0 0x00002b9e6115f280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.451] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.451] #1 0x00000000009ee9d1 in mongo::LegacyDistLockPinger::waitTillNextPingTime(std::chrono::duration >) () [2015/10/07 17:12:39.451] #2 0x00000000009f30a4 in mongo::LegacyDistLockPinger::_distLockPingThread(mongo::ConnectionString, std::string const&, std::chrono::duration >) () [2015/10/07 17:12:39.451] #3 0x00000000009f450f in mongo::LegacyDistLockPinger::distLockPingThread(mongo::ConnectionString, long long, std::string const&, std::chrono::duration >) () [2015/10/07 17:12:39.451] #4 0x00000000009f51fa in std::thread::_Impl >)> (mongo::LegacyDistLockPinger*, mongo::ConnectionString, long long, std::string, std::chrono::duration >)> ()> >::_M_run() () [2015/10/07 17:12:39.451] #5 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.451] #6 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.451] #7 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.451] #8 0x0000000000000000 in ?? () [2015/10/07 17:12:39.451] Thread 10 (Thread 0x2b9e66d04940 (LWP 3766)): [2015/10/07 17:12:39.451] #0 0x00002b9e61162351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.451] #1 0x0000000000bb3055 in mongo::sleepsecs(int) () [2015/10/07 17:12:39.451] #2 0x000000000099e12b in mongo::Balancer::run() () [2015/10/07 17:12:39.451] #3 0x0000000000b46817 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.452] #4 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.452] #5 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.452] #6 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.452] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.452] Thread 9 (Thread 0x2b9e67705940 (LWP 3767)): [2015/10/07 17:12:39.452] #0 0x00002b9e61162351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.452] #1 0x0000000000bb3147 in mongo::sleepmicros(long long) () [2015/10/07 17:12:39.452] #2 0x0000000000b4c38c in mongo::task::Task::run() () [2015/10/07 17:12:39.452] #3 0x0000000000b46817 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.452] #4 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.452] #5 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.452] #6 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.452] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.452] Thread 8 (Thread 0x2b9e68106940 (LWP 3768)): [2015/10/07 17:12:39.452] #0 0x00002b9e61162351 in nanosleep () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.452] #1 0x0000000000bb3147 in mongo::sleepmicros(long long) () [2015/10/07 17:12:39.452] #2 0x0000000000ae0b35 in mongo::ClusterCursorCleanupJob::run() () [2015/10/07 17:12:39.453] #3 0x0000000000b46817 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.453] #4 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.453] #5 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.453] #6 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.453] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.453] Thread 7 (Thread 0x2b9e68b07940 (LWP 3769)): [2015/10/07 17:12:39.453] #0 0x00002b9e6115f280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.453] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.453] #1 0x0000000000728114 in mongo::UserCacheInvalidator::run() () [2015/10/07 17:12:39.453] #2 0x0000000000b46817 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.453] #3 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.453] #4 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.453] #5 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.453] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.453] Thread 6 (Thread 0x2b9e69508940 (LWP 3770)): [2015/10/07 17:12:39.453] #0 0x00002b9e6115f280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.453] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.453] #1 0x0000000000b472ce in mongo::(anonymous namespace)::PeriodicTaskRunner::run() () [2015/10/07 17:12:39.453] #2 0x0000000000b46817 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.454] #3 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.454] #4 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.454] #5 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.454] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.454] Thread 5 (Thread 0x2b9e6a00a940 (LWP 3778)): [2015/10/07 17:12:39.454] #0 0x00002b9e61161e11 in recv () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.454] #1 0x0000000000b6c9cb in mongo::Socket::_recv(char*, int) () [2015/10/07 17:12:39.454] #2 0x0000000000b6ca11 in mongo::Socket::unsafe_recv(char*, int) () [2015/10/07 17:12:39.454] #3 0x0000000000b6ca6d in mongo::Socket::recv(char*, int) () [2015/10/07 17:12:39.454] #4 0x0000000000b61732 in mongo::MessagingPort::recv(mongo::Message&) () [2015/10/07 17:12:39.454] #5 0x000000000066e3fd in mongo::DBClientConnection::recv(mongo::Message&) () [2015/10/07 17:12:39.454] #6 0x0000000000a60186 in mongo::DBClientMultiCommand::recvAny(mongo::ConnectionString*, mongo::BSONSerializable*) () [2015/10/07 17:12:39.454] #7 0x0000000000b0dbe2 in mongo::BatchWriteExec::executeBatch(mongo::OperationContext*, mongo::BatchedCommandRequest const&, mongo::BatchedCommandResponse*) [2015/10/07 17:12:39.454] () [2015/10/07 17:12:39.454] #8 0x0000000000a80033 in mongo::ClusterWriter::write(mongo::OperationContext*, mongo::BatchedCommandRequest const&, mongo::BatchedCommandResponse*) () [2015/10/07 17:12:39.455] #9 0x0000000000abb71c in mongo::(anonymous namespace)::ClusterWriteCmd::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&) () [2015/10/07 17:12:39.455] #10 0x0000000000aed701 in mongo::Command::execCommandClientBasic(mongo::OperationContext*, mongo::Command*, mongo::ClientBasic&, int, char const*, mongo::BSONObj&, mongo::BSONObjBuilder&) () [2015/10/07 17:12:39.455] #11 0x0000000000aee3a9 in mongo::Command::runAgainstRegistered(mongo::OperationContext*, char const*, mongo::BSONObj&, mongo::BSONObjBuilder&, int) () [2015/10/07 17:12:39.455] #12 0x0000000000afbe26 in mongo::Strategy::clientCommandOp(mongo::OperationContext*, mongo::Request&) () [2015/10/07 17:12:39.455] #13 0x0000000000aecaf8 in mongo::Request::process(mongo::OperationContext*, int) () [2015/10/07 17:12:39.455] #14 0x000000000060bcf5 in mongo::ShardedMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*) () [2015/10/07 17:12:39.455] #15 0x0000000000b63315 in mongo::PortMessageServer::handleIncomingMsg(void*) [2015/10/07 17:12:39.455] () [2015/10/07 17:12:39.455] #16 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.455] #17 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.455] #18 0x0000000000000000 in ?? () [2015/10/07 17:12:39.455] Thread 4 (Thread 0x2b9e69f09940 (LWP 3785)): [2015/10/07 17:12:39.455] #0 0x00002b9e6115f280 in pthread_cond_timedwait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.455] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.455] #1 0x00000000006b2e1d in mongo::(anonymous namespace)::ReplicaSetMonitorWatcher::run() () [2015/10/07 17:12:39.455] #2 0x0000000000b46817 in mongo::BackgroundJob::jobBody() () [2015/10/07 17:12:39.455] #3 0x0000000000d0cd40 in execute_native_thread_routine () [2015/10/07 17:12:39.456] #4 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.456] #5 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.456] #6 0x0000000000000000 in ?? () [2015/10/07 17:12:39.456] Thread 3 (Thread 0x2b9e6aa0b940 (LWP 3790)): [2015/10/07 17:12:39.456] #0 0x00002b9e6115f019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.456] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.456] #1 0x0000000000bc57d0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:39.456] #2 0x0000000000bc5a81 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:39.456] #3 0x0000000000bc9c0f in asio::io_service::run() () [2015/10/07 17:12:39.456] #4 0x0000000000bbd5d0 in asio_detail_posix_thread_function () [2015/10/07 17:12:39.456] #5 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.456] #6 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.456] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.456] Thread 2 (Thread 0x2b9e6b40c940 (LWP 3792)): [2015/10/07 17:12:39.456] #0 0x00002b9e6115f019 in pthread_cond_wait@@GLIBC_2.3.2 () [2015/10/07 17:12:39.456] from /lib64/libpthread.so.0 [2015/10/07 17:12:39.456] #1 0x0000000000bc57d0 in asio::detail::scheduler::do_run_one(asio::detail::scoped_lock&, asio::detail::scheduler_thread_info&, std::error_code const&) () [2015/10/07 17:12:39.456] #2 0x0000000000bc5a81 in asio::detail::scheduler::run(std::error_code&) () [2015/10/07 17:12:39.456] #3 0x0000000000bc9c0f in asio::io_service::run() () [2015/10/07 17:12:39.457] #4 0x0000000000bbd5d0 in asio_detail_posix_thread_function () [2015/10/07 17:12:39.457] #5 0x00002b9e6115a83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:39.457] #6 0x00002b9e61445fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:39.457] #7 0x0000000000000000 in ?? () [2015/10/07 17:12:39.457] Thread 1 (Thread 0x2b9e616cad40 (LWP 3707)): [2015/10/07 17:12:39.457] #0 0x00002b9e6143f172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:39.457] #1 0x0000000000b5ce9e in mongo::Listener::initAndListen() () [2015/10/07 17:12:39.457] #2 0x00000000006092ad in mongoSMain(int, char**, char**) () [2015/10/07 17:12:39.457] #3 0x000000000060b239 in main () [2015/10/07 17:12:39.457] Detaching from program: /data/mci/src/mongos, process 3707 [2015/10/07 17:12:44.460] INFO: Done analyzing process [2015/10/07 17:12:44.460] Waiting for python process to report [2015/10/07 17:12:44.460] INFO: Debugger /opt/mongodbtoolchain/bin/gdb, analyzing 19760 [2015/10/07 17:12:44.461] ['/opt/mongodbtoolchain/bin/gdb', '--version'] [2015/10/07 17:12:44.490] GNU gdb (GDB) 7.6.1 [2015/10/07 17:12:44.491] Copyright (C) 2013 Free Software Foundation, Inc. [2015/10/07 17:12:44.491] License GPLv3+: GNU GPL version 3 or later [2015/10/07 17:12:44.491] This is free software: you are free to change and redistribute it. [2015/10/07 17:12:44.491] There is NO WARRANTY, to the extent permitted by law. Type "show copying" [2015/10/07 17:12:44.491] and "show warranty" for details. [2015/10/07 17:12:44.491] This GDB was configured as "x86_64-unknown-linux-gnu". [2015/10/07 17:12:44.491] For bug reporting instructions, please see: [2015/10/07 17:12:44.491] . [2015/10/07 17:12:44.491] ['/opt/mongodbtoolchain/bin/gdb', '--quiet', '-ex', 'attach 19760', '-ex', 'thread apply all bt', '-ex', 'set confirm off', '-ex', 'quit'] [2015/10/07 17:12:44.510] Attaching to process 19760 [2015/10/07 17:12:44.511] Reading symbols from /usr/bin/python2.6...(no debugging symbols found)...done. [2015/10/07 17:12:44.525] Reading symbols from /usr/lib64/libpython2.6.so.1.0...(no debugging symbols found)...done. [2015/10/07 17:12:44.525] Loaded symbols for /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.526] Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:44.528] [New LWP 3089] [2015/10/07 17:12:44.528] [New LWP 3088] [2015/10/07 17:12:44.528] [New LWP 3086] [2015/10/07 17:12:44.528] [New LWP 19766] [2015/10/07 17:12:44.528] [New LWP 19765] [2015/10/07 17:12:44.528] [Thread debugging using libthread_db enabled] [2015/10/07 17:12:44.528] Using host libthread_db library "/lib64/libthread_db.so.1". [2015/10/07 17:12:44.528] Loaded symbols for /lib64/libpthread.so.0 [2015/10/07 17:12:44.528] Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:44.528] Loaded symbols for /lib64/libdl.so.2 [2015/10/07 17:12:44.529] Reading symbols from /lib64/libutil.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:44.529] Loaded symbols for /lib64/libutil.so.1 [2015/10/07 17:12:44.530] Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:44.530] Loaded symbols for /lib64/libm.so.6 [2015/10/07 17:12:44.538] Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:44.538] Loaded symbols for /lib64/libc.so.6 [2015/10/07 17:12:44.539] Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:44.539] Loaded symbols for /lib64/ld-linux-x86-64.so.2 [2015/10/07 17:12:44.539] Reading symbols from /usr/lib64/python2.6/lib-dynload/_json.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.539] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_json.so [2015/10/07 17:12:44.539] Reading symbols from /usr/lib64/python2.6/lib-dynload/mathmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.539] Loaded symbols for /usr/lib64/python2.6/lib-dynload/mathmodule.so [2015/10/07 17:12:44.540] Reading symbols from /usr/lib64/python2.6/lib-dynload/binascii.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.540] Loaded symbols for /usr/lib64/python2.6/lib-dynload/binascii.so [2015/10/07 17:12:44.540] Reading symbols from /usr/lib64/python2.6/lib-dynload/_randommodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.540] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_randommodule.so [2015/10/07 17:12:44.540] Reading symbols from /usr/lib64/python2.6/lib-dynload/timemodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.540] Loaded symbols for /usr/lib64/python2.6/lib-dynload/timemodule.so [2015/10/07 17:12:44.541] Reading symbols from /usr/lib64/python2.6/lib-dynload/stropmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.541] Loaded symbols for /usr/lib64/python2.6/lib-dynload/stropmodule.so [2015/10/07 17:12:44.541] Reading symbols from /usr/lib64/python2.6/lib-dynload/cStringIO.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.541] Loaded symbols for /usr/lib64/python2.6/lib-dynload/cStringIO.so [2015/10/07 17:12:44.541] Reading symbols from /usr/lib64/python2.6/lib-dynload/_functoolsmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.541] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_functoolsmodule.so [2015/10/07 17:12:44.542] Reading symbols from /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.542] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so [2015/10/07 17:12:44.542] Reading symbols from /usr/lib64/python2.6/lib-dynload/operator.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.542] Loaded symbols for /usr/lib64/python2.6/lib-dynload/operator.so [2015/10/07 17:12:44.542] Reading symbols from /usr/lib64/python2.6/lib-dynload/_struct.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.543] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_struct.so [2015/10/07 17:12:44.543] Reading symbols from /usr/lib64/python2.6/lib-dynload/_hashlib.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.543] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_hashlib.so [2015/10/07 17:12:44.545] Reading symbols from /lib64/libssl.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:44.545] Loaded symbols for /lib64/libssl.so.6 [2015/10/07 17:12:44.549] Reading symbols from /lib64/libcrypto.so.6...(no debugging symbols found)...done. [2015/10/07 17:12:44.549] Loaded symbols for /lib64/libcrypto.so.6 [2015/10/07 17:12:44.550] Reading symbols from /usr/lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:44.550] Loaded symbols for /usr/lib64/libgssapi_krb5.so.2 [2015/10/07 17:12:44.554] Reading symbols from /usr/lib64/libkrb5.so.3...(no debugging symbols found)...done. [2015/10/07 17:12:44.554] Loaded symbols for /usr/lib64/libkrb5.so.3 [2015/10/07 17:12:44.554] Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:44.554] Loaded symbols for /lib64/libcom_err.so.2 [2015/10/07 17:12:44.555] Reading symbols from /usr/lib64/libk5crypto.so.3...(no debugging symbols found)...done. [2015/10/07 17:12:44.555] Loaded symbols for /usr/lib64/libk5crypto.so.3 [2015/10/07 17:12:44.555] Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:44.555] Loaded symbols for /lib64/libz.so.1 [2015/10/07 17:12:44.555] Reading symbols from /usr/lib64/libkrb5support.so.0...(no debugging symbols found)...done. [2015/10/07 17:12:44.556] Loaded symbols for /usr/lib64/libkrb5support.so.0 [2015/10/07 17:12:44.556] Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:44.556] Loaded symbols for /lib64/libkeyutils.so.1 [2015/10/07 17:12:44.556] Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:44.557] Loaded symbols for /lib64/libresolv.so.2 [2015/10/07 17:12:44.557] Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:44.557] Loaded symbols for /lib64/libselinux.so.1 [2015/10/07 17:12:44.558] Reading symbols from /lib64/libsepol.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:44.558] Loaded symbols for /lib64/libsepol.so.1 [2015/10/07 17:12:44.558] Reading symbols from /usr/lib64/python2.6/lib-dynload/arraymodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.558] Loaded symbols for /usr/lib64/python2.6/lib-dynload/arraymodule.so [2015/10/07 17:12:44.559] Reading symbols from /usr/lib64/python2.6/lib-dynload/_socketmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.559] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_socketmodule.so [2015/10/07 17:12:44.559] Reading symbols from /usr/lib64/python2.6/lib-dynload/_ssl.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.559] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_ssl.so [2015/10/07 17:12:44.560] Reading symbols from /usr/lib64/python2.6/lib-dynload/fcntlmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.560] Loaded symbols for /usr/lib64/python2.6/lib-dynload/fcntlmodule.so [2015/10/07 17:12:44.560] Reading symbols from /usr/lib64/python2.6/lib-dynload/_bisectmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.560] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_bisectmodule.so [2015/10/07 17:12:44.560] Reading symbols from /usr/lib64/python2.6/lib-dynload/datetime.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.561] Loaded symbols for /usr/lib64/python2.6/lib-dynload/datetime.so [2015/10/07 17:12:44.561] Reading symbols from /usr/lib64/python2.6/lib-dynload/_localemodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.561] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_localemodule.so [2015/10/07 17:12:44.561] Reading symbols from /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.561] Loaded symbols for /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so [2015/10/07 17:12:44.562] Reading symbols from /usr/lib64/python2.6/lib-dynload/_ctypes.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.562] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_ctypes.so [2015/10/07 17:12:44.562] Reading symbols from /usr/lib64/libffi.so.5...(no debugging symbols found)...done. [2015/10/07 17:12:44.563] Loaded symbols for /usr/lib64/libffi.so.5 [2015/10/07 17:12:44.563] Reading symbols from /lib64/libuuid.so.1...(no debugging symbols found)...done. [2015/10/07 17:12:44.563] Loaded symbols for /lib64/libuuid.so.1 [2015/10/07 17:12:44.565] Reading symbols from /usr/lib64/python2.6/site-packages/bson/_cbson.so...done. [2015/10/07 17:12:44.565] Loaded symbols for /usr/lib64/python2.6/site-packages/bson/_cbson.so [2015/10/07 17:12:44.565] Reading symbols from /usr/lib64/python2.6/site-packages/pymongo/_cmessage.so...done. [2015/10/07 17:12:44.565] Loaded symbols for /usr/lib64/python2.6/site-packages/pymongo/_cmessage.so [2015/10/07 17:12:44.566] Reading symbols from /usr/lib64/python2.6/lib-dynload/_weakref.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.566] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_weakref.so [2015/10/07 17:12:44.566] Reading symbols from /usr/lib64/python2.6/lib-dynload/selectmodule.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.566] Loaded symbols for /usr/lib64/python2.6/lib-dynload/selectmodule.so [2015/10/07 17:12:44.566] Reading symbols from /usr/lib64/python2.6/lib-dynload/unicodedata.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.566] Loaded symbols for /usr/lib64/python2.6/lib-dynload/unicodedata.so [2015/10/07 17:12:44.567] Reading symbols from /usr/lib64/python2.6/lib-dynload/_heapq.so...(no debugging symbols found)...done. [2015/10/07 17:12:44.567] Loaded symbols for /usr/lib64/python2.6/lib-dynload/_heapq.so [2015/10/07 17:12:44.567] Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:44.567] Loaded symbols for /lib64/libnss_files.so.2 [2015/10/07 17:12:44.568] Reading symbols from /lib64/libnss_dns.so.2...(no debugging symbols found)...done. [2015/10/07 17:12:44.570] Loaded symbols for /lib64/libnss_dns.so.2 [2015/10/07 17:12:44.570] 0x00002b89d16fd172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:44.571] Thread 6 (Thread 0x2b89d7b72940 (LWP 19765)): [2015/10/07 17:12:44.571] #0 0x00002b89d0d94e51 in sem_wait () from /lib64/libpthread.so.0 [2015/10/07 17:12:44.571] #1 0x00002b89d0ade8d8 in PyThread_acquire_lock () [2015/10/07 17:12:44.571] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.571] #2 0x00002b89d0ae2782 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.571] #3 0x00002b89d0ab74c1 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.571] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.571] #4 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.571] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #5 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #6 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #7 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #8 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #9 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #10 0x00002b89d0ab7284 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #11 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #12 0x00002b89d0a4b3cd in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #13 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #14 0x00002b89d0a3281f in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #15 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #16 0x00002b89d0ab1af6 in PyEval_CallObjectWithKeywords () [2015/10/07 17:12:44.572] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #17 0x00002b89d0ae2c1a in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.572] #18 0x00002b89d0d8e83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:44.576] #19 0x00002b89d1703fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:44.576] #20 0x0000000000000000 in ?? () [2015/10/07 17:12:44.576] Thread 5 (Thread 0x2b89d8990940 (LWP 19766)): [2015/10/07 17:12:44.576] #0 0x00002b89d0d9681f in waitpid () from /lib64/libpthread.so.0 [2015/10/07 17:12:44.576] #1 0x00002b89d0ae6b01 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.576] #2 0x00002b89d0ab6a43 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.576] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.576] #3 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #4 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #5 0x00002b89d0ab7284 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #6 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #7 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #8 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #9 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #10 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #11 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #12 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #13 0x00002b89d0a4b4cc in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #14 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #15 0x00002b89d0ab364b in PyEval_EvalFrameEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #16 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.577] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.577] #17 0x00002b89d0a4b3cd in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #18 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #19 0x00002b89d0a3281f in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #20 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #21 0x00002b89d0a7ad4a in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #22 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #23 0x00002b89d0ab3a04 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.578] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #24 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.578] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #25 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.578] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #26 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.578] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #27 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.578] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #28 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.578] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #29 0x00002b89d0a4b4cc in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.578] #30 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #31 0x00002b89d0a3281f in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #32 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #33 0x00002b89d0a7ad4a in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #34 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #35 0x00002b89d0ab364b in PyEval_EvalFrameEx () [2015/10/07 17:12:44.579] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #36 0x00002b89d0ab7284 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.579] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #37 0x00002b89d0ab7284 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.579] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #38 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.579] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #39 0x00002b89d0a4b3cd in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #40 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #41 0x00002b89d0a3281f in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #42 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #43 0x00002b89d0ab1af6 in PyEval_CallObjectWithKeywords () [2015/10/07 17:12:44.579] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.579] #44 0x00002b89d0ae2c1a in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #45 0x00002b89d0d8e83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:44.580] #46 0x00002b89d1703fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:44.580] #47 0x0000000000000000 in ?? () [2015/10/07 17:12:44.580] Thread 4 (Thread 0x2b89d9391940 (LWP 3086)): [2015/10/07 17:12:44.580] #0 0x00002b89d16fd172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:44.580] #1 0x00002b89d219e44f in ?? () [2015/10/07 17:12:44.580] from /usr/lib64/python2.6/lib-dynload/timemodule.so [2015/10/07 17:12:44.580] #2 0x00002b89d0ab74c1 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.580] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #3 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.580] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #4 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.580] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #5 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.580] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #6 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.580] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #7 0x00002b89d0ab7284 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.580] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #8 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.580] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.580] #9 0x00002b89d0a4b3cd in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #10 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #11 0x00002b89d0a3281f in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #12 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #13 0x00002b89d0ab1af6 in PyEval_CallObjectWithKeywords () [2015/10/07 17:12:44.581] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #14 0x00002b89d0ae2c1a in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #15 0x00002b89d0d8e83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:44.581] #16 0x00002b89d1703fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:44.581] #17 0x0000000000000000 in ?? () [2015/10/07 17:12:44.581] Thread 3 (Thread 0x2b89d9d92940 (LWP 3088)): [2015/10/07 17:12:44.581] #0 0x00002b89d16f61fb in read () from /lib64/libc.so.6 [2015/10/07 17:12:44.581] #1 0x00002b89d169ad07 in _IO_new_file_underflow () from /lib64/libc.so.6 [2015/10/07 17:12:44.581] #2 0x00002b89d169b6ce in _IO_default_uflow_internal () from /lib64/libc.so.6 [2015/10/07 17:12:44.581] #3 0x00002b89d0a41ec2 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #4 0x00002b89d0a42072 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #5 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.581] #6 0x00002b89d0a4db83 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #7 0x00002b89d0ab2fac in PyEval_EvalFrameEx () [2015/10/07 17:12:44.582] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #8 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.582] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #9 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.582] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #10 0x00002b89d0ab7284 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.582] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #11 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.582] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #12 0x00002b89d0a4b3cd in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #13 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #14 0x00002b89d0a3281f in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #15 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #16 0x00002b89d0ab1af6 in PyEval_CallObjectWithKeywords () [2015/10/07 17:12:44.582] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #17 0x00002b89d0ae2c1a in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.582] #18 0x00002b89d0d8e83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:44.582] #19 0x00002b89d1703fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:44.582] #20 0x0000000000000000 in ?? () [2015/10/07 17:12:44.583] Thread 2 (Thread 0x2b89db194940 (LWP 3089)): [2015/10/07 17:12:44.583] #0 0x00002b89d16f61fb in read () from /lib64/libc.so.6 [2015/10/07 17:12:44.583] #1 0x00002b89d169ad07 in _IO_new_file_underflow () from /lib64/libc.so.6 [2015/10/07 17:12:44.583] #2 0x00002b89d169b6ce in _IO_default_uflow_internal () from /lib64/libc.so.6 [2015/10/07 17:12:44.583] #3 0x00002b89d0a41ec2 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #4 0x00002b89d0a42072 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #5 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #6 0x00002b89d0a4db83 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #7 0x00002b89d0ab2fac in PyEval_EvalFrameEx () [2015/10/07 17:12:44.583] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #8 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.583] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #9 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.583] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #10 0x00002b89d0ab7284 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.583] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #11 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.583] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #12 0x00002b89d0a4b3cd in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #13 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.583] #14 0x00002b89d0a3281f in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #15 0x00002b89d0a21d28 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #16 0x00002b89d0ab1af6 in PyEval_CallObjectWithKeywords () [2015/10/07 17:12:44.584] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #17 0x00002b89d0ae2c1a in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #18 0x00002b89d0d8e83d in start_thread () from /lib64/libpthread.so.0 [2015/10/07 17:12:44.584] #19 0x00002b89d1703fdd in clone () from /lib64/libc.so.6 [2015/10/07 17:12:44.584] #20 0x0000000000000000 in ?? () [2015/10/07 17:12:44.584] Thread 1 (Thread 0x2b89d1989190 (LWP 19760)): [2015/10/07 17:12:44.584] #0 0x00002b89d16fd172 in select () from /lib64/libc.so.6 [2015/10/07 17:12:44.584] #1 0x00002b89d219e44f in ?? () [2015/10/07 17:12:44.584] from /usr/lib64/python2.6/lib-dynload/timemodule.so [2015/10/07 17:12:44.584] #2 0x00002b89d0ab74c1 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.584] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #3 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.584] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #4 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.584] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #5 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.584] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #6 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.584] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.584] #7 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.584] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #8 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #9 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #10 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #11 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #12 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #13 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #14 0x00002b89d0ab6f51 in PyEval_EvalFrameEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #15 0x00002b89d0ab8891 in PyEval_EvalCodeEx () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #16 0x00002b89d0ab8912 in PyEval_EvalCode () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #17 0x00002b89d0ad31f2 in ?? () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #18 0x00002b89d0ad32c6 in PyRun_FileExFlags () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #19 0x00002b89d0ad4817 in PyRun_SimpleFileExFlags () [2015/10/07 17:12:44.585] from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #20 0x00002b89d0ae0b73 in Py_Main () from /usr/lib64/libpython2.6.so.1.0 [2015/10/07 17:12:44.585] #21 0x00002b89d164c9f4 in __libc_start_main () from /lib64/libc.so.6 [2015/10/07 17:12:44.585] #22 0x0000000000400629 in _start () [2015/10/07 17:12:44.586] Detaching from program: /usr/bin/python2.6, process 19760 [2015/10/07 17:12:44.600] INFO: Done analyzing process [2015/10/07 17:12:44.600] Done analyzing processes for hangs [2015/10/07 17:12:44.601] Finished running task-timeout commands in 7.752419s. [2015/10/07 17:12:44.601] Command failed: Shell command interrupted. [2015/10/07 17:12:44.601] Task completed - FAILURE. [2015/10/07 17:12:44.601] Running post-task commands. [2015/10/07 17:12:44.601] Running command 'attach.results' (step 1 of 5) [2015/10/07 17:12:44.601] Command failed: Couldn't open report file: 'open /data/mci/src/report.json: no such file or directory' [2015/10/07 17:12:44.601] Running command 'shell.exec' (step 2 of 5) [2015/10/07 17:12:44.641] ==================================== [2015/10/07 17:12:44.641] Dumping stacks due to SIGUSR1 signal [2015/10/07 17:12:44.641] ==================================== [2015/10/07 17:12:44.641] Total threads: 6 [2015/10/07 17:12:44.641] Thread 47871086381376: [2015/10/07 17:12:44.641] File "/usr/lib64/python2.6/threading.py", line 504, in __bootstrap [2015/10/07 17:12:44.641] self.__bootstrap_inner() [2015/10/07 17:12:44.641] File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner [2015/10/07 17:12:44.641] self.run() [2015/10/07 17:12:44.641] File "/data/mci/src/buildscripts/resmokelib/core/pipe.py", line 60, in run [2015/10/07 17:12:44.641] for line in iter(self.__pipe_out.readline, b""): [2015/10/07 17:12:44.641] Thread 47871065401664: [2015/10/07 17:12:44.641] File "/usr/lib64/python2.6/threading.py", line 504, in __bootstrap [2015/10/07 17:12:44.641] self.__bootstrap_inner() [2015/10/07 17:12:44.641] File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner [2015/10/07 17:12:44.641] self.run() [2015/10/07 17:12:44.641] File "/data/mci/src/buildscripts/resmokelib/core/pipe.py", line 60, in run [2015/10/07 17:12:44.641] for line in iter(self.__pipe_out.readline, b""): [2015/10/07 17:12:44.641] Thread 47870926950800: [2015/10/07 17:12:44.641] File "buildscripts/resmoke.py", line 209, in [2015/10/07 17:12:44.641] main() [2015/10/07 17:12:44.641] File "buildscripts/resmoke.py", line 155, in main [2015/10/07 17:12:44.641] _execute_suite(suite, logging_config) [2015/10/07 17:12:44.641] File "buildscripts/resmoke.py", line 64, in _execute_suite [2015/10/07 17:12:44.641] executor.run() [2015/10/07 17:12:44.641] File "/data/mci/src/buildscripts/resmokelib/testing/executor.py", line 82, in run [2015/10/07 17:12:44.641] (report, interrupted) = self._run_tests(test_queue) [2015/10/07 17:12:44.641] File "/data/mci/src/buildscripts/resmokelib/testing/executor.py", line 156, in _run_tests [2015/10/07 17:12:44.641] joined = test_queue.join(TestGroupExecutor._TIMEOUT) [2015/10/07 17:12:44.641] File "/data/mci/src/buildscripts/resmokelib/utils/queue.py", line 51, in join [2015/10/07 17:12:44.641] self.all_tasks_done.wait(remaining) [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 258, in wait [2015/10/07 17:12:44.642] _sleep(delay) [2015/10/07 17:12:44.642] File "buildscripts/resmoke.py", line 199, in _dump_stacks [2015/10/07 17:12:44.642] sb.append("".join(traceback.format_stack(stack))) [2015/10/07 17:12:44.642] Thread 47871044421952: [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 504, in __bootstrap [2015/10/07 17:12:44.642] self.__bootstrap_inner() [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner [2015/10/07 17:12:44.642] self.run() [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 484, in run [2015/10/07 17:12:44.642] self.__target(*self.__args, **self.__kwargs) [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/testing/job.py", line 39, in __call__ [2015/10/07 17:12:44.642] self._run(queue, interrupt_flag) [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/testing/job.py", line 70, in _run [2015/10/07 17:12:44.642] self._execute_test(test) [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/testing/job.py", line 85, in _execute_test [2015/10/07 17:12:44.642] test(self.report) [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/unittest.py", line 299, in __call__ [2015/10/07 17:12:44.642] return self.run(*args, **kwds) [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/unittest.py", line 278, in run [2015/10/07 17:12:44.642] testMethod() [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/testing/testcases.py", line 343, in run_test [2015/10/07 17:12:44.642] self._execute(shell) [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/testing/testcases.py", line 107, in _execute [2015/10/07 17:12:44.642] self.return_code = process.wait() [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/core/process.py", line 159, in wait [2015/10/07 17:12:44.642] return_code = self._process.wait() [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/subprocess.py", line 1182, in wait [2015/10/07 17:12:44.642] pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0) [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/subprocess.py", line 455, in _eintr_retry_call [2015/10/07 17:12:44.642] return func(*args) [2015/10/07 17:12:44.642] Thread 47871054911808: [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 504, in __bootstrap [2015/10/07 17:12:44.642] self.__bootstrap_inner() [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner [2015/10/07 17:12:44.642] self.run() [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/utils/timer.py", line 98, in run [2015/10/07 17:12:44.642] self.cond.wait(self.interval) [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 258, in wait [2015/10/07 17:12:44.642] _sleep(delay) [2015/10/07 17:12:44.642] Thread 47871029619008: [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 504, in __bootstrap [2015/10/07 17:12:44.642] self.__bootstrap_inner() [2015/10/07 17:12:44.642] File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner [2015/10/07 17:12:44.642] self.run() [2015/10/07 17:12:44.642] File "/data/mci/src/buildscripts/resmokelib/logging/flush.py", line 56, in run [2015/10/07 17:12:44.642] logger = _LOGGER_QUEUE.get() [2015/10/07 17:12:44.643] File "/usr/lib64/python2.6/Queue.py", line 168, in get [2015/10/07 17:12:44.643] self.not_empty.wait() [2015/10/07 17:12:44.643] File "/usr/lib64/python2.6/threading.py", line 239, in wait [2015/10/07 17:12:44.643] waiter.acquire() [2015/10/07 17:12:44.643] ==================================== [2015/10/07 17:12:44.643] [executor:js_test:job0] 2015-10-07T21:12:44.640+0000 conf_server_write_concern.js ran in 7207.88 seconds. [2015/10/07 17:12:44.673] [executor:js_test:job0] 2015-10-07T21:12:44.673+0000 Running copydb_from_mongos.js... [2015/10/07 17:12:44.673] ./mongo --eval MongoRunner.dataDir = "/data/db/job0/mongorunner"; TestData = new Object(); TestData.wiredTigerEngineConfigString = ""; TestData.maxPort = 20249; TestData.wiredTigerIndexConfigString = ""; TestData.noJournal = false; TestData.testName = "copydb_from_mongos"; TestData.useLegacyConfigServers = true; TestData.storageEngine = ""; TestData.minPort = 20010; TestData.noJournalPrealloc = true; TestData.wiredTigerCollectionConfigString = ""; MongoRunner.dataPath = "/data/db/job0/mongorunner/" --readMode legacy --nodb jstests/sharding/copydb_from_mongos.js [2015/10/07 17:12:44.743] [executor:js_test:job0] 2015-10-07T21:12:44.742+0000 Writing output of JSTest jstests/sharding/copydb_from_mongos.js to https://logkeeper.mongodb.org/build/56156bb290413021b1d7e01f/test/56158acdbe07c4135fd9b043/. [2015/10/07 17:12:44.795] Running command 'archive.targz_pack' (step 3 of 5) [2015/10/07 17:12:44.809] Running command 's3.put' (step 4 of 5) [2015/10/07 17:12:44.809] ['s3.put'] Putting /data/mci/mongo-coredumps.tgz into path mongodb-mongo-master/linux-64/0cf1be838e229b4385ac4313e994040a230857e5/coredumps/mongo-coredumps-mongodb_mongo_master_linux_64_0cf1be838e229b4385ac4313e994040a230857e5_15_10_07_18_34_55-sharding_legacy-0.tgz in s3 bucket mciuploads [2015/10/07 17:12:44.809] Running command 'shell.cleanup' (step 5 of 5) [2015/10/07 17:12:44.811] ['shell.cleanup'] Killed process 4865 [2015/10/07 17:12:44.811] Finished running post-task commands in 210.279ms.