[Errno 111] Connection refused 2014-11-14T15:24:07.562-0500 I CONTROL [initandlisten] MongoDB starting : pid=949 port=27999 dbpath=/home/dpercy/data/smoke/db/data/db/sconsTests/ 64-bit host=beefsupreme 2014-11-14T15:24:07.562-0500 I CONTROL [initandlisten] _DEBUG build (which is slower) 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] ** WARNING: You are running on a NUMA machine. 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] ** We suggest launching mongod like this to avoid performance problems: 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] ** numactl --interleave=all mongod [other options] 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] db version v2.8.0-rc1-pre- 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] git version: 7d1099afb4c5392500154b05394c42f6256b24e2 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] build info: Linux beefsupreme 3.8.0-38-generic #56~precise1-Ubuntu SMP Thu Mar 13 16:22:48 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] allocator: tcmalloc 2014-11-14T15:24:07.563-0500 I CONTROL [initandlisten] options: { net: { http: { enabled: true }, port: 27999 }, setParameter: { enableTestCommands: "1" }, storage: { dbPath: "/home/dpercy/data/smoke/db/data/db/sconsTests/", engine: "wiredtiger" } } 2014-11-14T15:24:07.563-0500 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=15G,session_max=20000,extensions=[local=(entry=index_collator_extension)],statistics=(all),log=(enabled=true,archive=true,path=journal),checkpoint=(wait=60,log_size=2GB), 2014-11-14T15:24:07.681-0500 I NETWORK [websvr] admin web console waiting for connections on port 28999 2014-11-14T15:24:07.713-0500 I NETWORK [initandlisten] waiting for connections on port 27999 2014-11-14T15:24:08.487-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60032 #1 (1 connection now open) clean_dbroot: /home/dpercy/data/smoke/db/data/db/sconsTests/ num procs:376 running /data01/home/dpercy/beef/10gen/mongo/mongod --port 27999 --dbpath /home/dpercy/data/smoke/db/data/db/sconsTests/ --setParameter enableTestCommands=1 --httpinterface --storageEngine wiredtiger ******************************************* Test : fsm_all.js ... 2014-11-14T15:24:08.487-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60033 #2 (2 connections now open) Command : /data01/home/dpercy/beef/10gen/mongo/mongo --port 27999 --authenticationMechanism SCRAM-SHA-1 --writeMode commands /data01/home/dpercy/beef/10gen/mongo/fsm_all.js --eval TestData = new Object();TestData.storageEngine = "wiredtiger";TestData.wiredTigerEngineConfig = "";TestData.testPath = "/data01/home/dpercy/beef/10gen/mongo/fsm_all.js";TestData.testFile = "fsm_all.js";TestData.testName = "fsm_all";TestData.setParameters = "";TestData.setParametersMongos = "";TestData.noJournal = false;TestData.noJournalPrealloc = false;TestData.auth = false;TestData.keyFile = null;TestData.keyFileData = null;TestData.authMechanism = "SCRAM-SHA-1";TestData.useSSL = false;TestData.useX509 = false;MongoRunner.dataDir = "/home/dpercy/data/smoke/db/data/db";MongoRunner.dataPath = MongoRunner.dataDir + "/"; Date : Fri Nov 14 15:24:08 2014 2014-11-14T15:24:08.488-0500 I NETWORK [conn1] end connection 127.0.0.1:60032 (1 connection now open) 2014-11-14T15:24:08.488-0500 I NETWORK [conn2] end connection 127.0.0.1:60033 (1 connection now open) MongoDB shell version: 2.8.0-rc1-pre- connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:09.044-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60034 #3 (1 connection now open) /home/dpercy/data/smoke/db/data/db/ 2014-11-14T15:24:09.109-0500 I COMMANDS [conn3] CMD: drop db0.coll0 2014-11-14T15:24:09.205-0500 I INDEXING [conn3] build index on: db0.coll0 properties: { v: 1, key: { x: 1.0 }, name: "x_1", ns: "db0.coll0" } 2014-11-14T15:24:09.205-0500 I INDEXING [conn3] building index using bulk method 2014-11-14T15:24:09.205-0500 I INDEXING [conn3] build index done. scanned 0 total records. 0 secs 2014-11-14T15:24:09.233-0500 I INDEXING [conn3] build index on: db0.coll0 properties: { v: 1, key: { y: 1.0 }, name: "y_1", ns: "db0.coll0" } 2014-11-14T15:24:09.233-0500 I INDEXING [conn3] building index using bulk method 2014-11-14T15:24:09.233-0500 I INDEXING [conn3] build index done. scanned 0 total records. 0 secs 2014-11-14T15:24:09.258-0500 I INDEXING [conn3] build index on: db0.coll0 properties: { v: 1, key: { z: 1.0 }, name: "z_1", ns: "db0.coll0" } 2014-11-14T15:24:09.258-0500 I INDEXING [conn3] building index using bulk method 2014-11-14T15:24:09.258-0500 I INDEXING [conn3] build index done. scanned 0 total records. 0 secs 2014-11-14T15:24:09.283-0500 I INDEXING [conn3] build index on: db0.coll0 properties: { v: 1, key: { x: 1.0, y: 1.0, z: 1.0 }, name: "x_1_y_1_z_1", ns: "db0.coll0" } 2014-11-14T15:24:09.284-0500 I INDEXING [conn3] building index using bulk method 2014-11-14T15:24:09.284-0500 I INDEXING [conn3] build index done. scanned 0 total records. 0 secs using num threads: 50 ---- jstests/parallel/fsm_workloads/update_multifield.js ---- setting random seed: 1415996649455 connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.631-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60035 #4 (2 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.643-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60036 #5 (3 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.662-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60037 #6 (4 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.665-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60038 #7 (5 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.675-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60039 #8 (6 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.691-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60040 #9 (7 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.765-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60041 #10 (8 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.769-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60042 #11 (9 connections now open) connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.785-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60043 #12 (10 connections now open) 2014-11-14T15:24:10.785-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60044 #13 (11 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.788-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60045 #14 (12 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.791-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60046 #15 (13 connections now open) connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.793-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60047 #16 (14 connections now open) 2014-11-14T15:24:10.793-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60048 #17 (15 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.812-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60049 #18 (16 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.815-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60050 #19 (17 connections now open) connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.831-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60051 #20 (18 connections now open) 2014-11-14T15:24:10.831-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60052 #21 (19 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.847-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60053 #22 (20 connections now open) 2014-11-14T15:24:10.847-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60054 #23 (21 connections now open) 2014-11-14T15:24:10.848-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60055 #24 (22 connections now open) connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.872-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60056 #25 (23 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.877-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60057 #26 (24 connections now open) 2014-11-14T15:24:10.879-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60058 #27 (25 connections now open) 2014-11-14T15:24:10.883-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60059 #28 (26 connections now open) 2014-11-14T15:24:10.887-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60060 #29 (27 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.899-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60061 #30 (28 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.918-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60062 #31 (29 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.923-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60063 #32 (30 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.925-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60064 #33 (31 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.926-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60065 #34 (32 connections now open) connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.936-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60066 #35 (33 connections now open) 2014-11-14T15:24:10.937-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60067 #36 (34 connections now open) connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.943-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60068 #37 (35 connections now open) 2014-11-14T15:24:10.943-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60069 #38 (36 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.947-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60070 #39 (37 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.957-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60071 #40 (38 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.958-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60072 #41 (39 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.965-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60073 #42 (40 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.968-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60074 #43 (41 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.969-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60075 #44 (42 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.973-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60076 #45 (43 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.983-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60077 #46 (44 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.987-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60078 #47 (45 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:10.993-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60079 #48 (46 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:11.002-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60080 #49 (47 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:11.006-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60081 #50 (48 connections now open) connecting to: 127.0.0.1:27999/test connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:11.010-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60082 #51 (49 connections now open) 2014-11-14T15:24:11.010-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60083 #52 (50 connections now open) connecting to: 127.0.0.1:27999/test 2014-11-14T15:24:11.011-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60084 #53 (51 connections now open) setting random seed: 4556420696899 setting random seed: 6980524379760 setting random seed: 8645318192429 setting random seed: 2477481132373 setting random seed: 231276135891 setting random seed: 4770106859505 setting random seed: 2310746288858 setting random seed: 8420955445617 setting random seed: 3357711327262 setting random seed: 3102109623141 setting random seed: 189484851434 setting random seed: 4747796142473 setting random seed: 2181837647221 setting random seed: 6299890158697 setting random seed: 8235559375025 setting random seed: 3573119929060 setting random seed: 7723304629325 setting random seed: 567676294595 setting random seed: 6733485632576 setting random seed: 5231399410404 setting random seed: 5955064902082 setting random seed: 8855578484945 setting random seed: 443302085623 setting random seed: 3326955735683 setting random seed: 3661768068559 setting random seed: 3287525353953 setting random seed: 361830233596 setting random seed: 5021795560605 setting random seed: 5878803911618 setting random seed: 2007651072926 setting random seed: 90899728238 setting random seed: 9983920161612 setting random seed: 1543492167256 setting random seed: 7883912501856 setting random seed: 234888754785 setting random seed: 7963563990779 setting random seed: 9303652113303 setting random seed: 9518718924373 setting random seed: 6521812314167 setting random seed: 8947472758591 setting random seed: 8823601738549 setting random seed: 1978489216417 setting random seed: 4379733218811 setting random seed: 8100666664540 setting random seed: 5984390904195 setting random seed: 8971419315785 setting random seed: 1057215360924 setting random seed: 6873255427926 setting random seed: 6916465996764 setting random seed: 4640662376768 2014-11-14T15:24:11.240-0500 I WRITES [conn13] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.243-0500 I WRITES [conn16] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.340-0500 I - [conn35] Deadlock found: Locker 132 waits for resource {17591420565866845252: Collection, 2198927570733355656, db0.coll0} held by [135, ] Locker 135 waits for resource {17591420565866845252: Collection, 2198927570733355656, db0.coll0} held by [132, ] 2014-11-14T15:24:11.342-0500 I WRITES [conn28] update db0.coll0 query: { n: 9.0 } update: { $unset: { x: 4.0 }, $push: { y: 1.0 }, $inc: { z: 2.0 } } nscanned:0 nscannedObjects:10 nMatched:1 nModified:1 keyUpdates:5 103ms 2014-11-14T15:24:11.343-0500 I QUERY [conn28] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 9.0 }, u: { $unset: { x: 4.0 }, $push: { y: 1.0 }, $inc: { z: 2.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 103ms 2014-11-14T15:24:11.345-0500 I - [conn35] Invariant failure _isMultikey == _catalogIsMultikey( txn ) src/mongo/db/catalog/index_catalog_entry.cpp 112 2014-11-14T15:24:11.352-0500 I - [conn16] Invariant failure _isMultikey == _catalogIsMultikey( txn ) src/mongo/db/catalog/index_catalog_entry.cpp 112 2014-11-14T15:24:11.352-0500 I - [conn13] Invariant failure _isMultikey == _catalogIsMultikey( txn ) src/mongo/db/catalog/index_catalog_entry.cpp 112 2014-11-14T15:24:11.372-0500 I WRITES [conn27] update db0.coll0 query: { n: 3.0 } update: { $unset: { x: 1.0 }, $push: { y: 1.0 }, $inc: { z: 0.0 } } nscanned:0 nscannedObjects:4 nMatched:1 nModified:1 keyUpdates:3 130ms 2014-11-14T15:24:11.372-0500 I WRITES [conn14] update db0.coll0 query: { n: 1.0 } update: { $unset: { x: 2.0 }, $push: { y: 2.0 }, $inc: { z: 0.0 } } nscanned:0 nscannedObjects:2 nMatched:1 nModified:1 keyUpdates:2 120ms 2014-11-14T15:24:11.375-0500 I QUERY [conn27] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 3.0 }, u: { $unset: { x: 1.0 }, $push: { y: 1.0 }, $inc: { z: 0.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 132ms 2014-11-14T15:24:11.376-0500 I QUERY [conn14] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 1.0 }, u: { $unset: { x: 2.0 }, $push: { y: 2.0 }, $inc: { z: 0.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 122ms 2014-11-14T15:24:11.381-0500 I WRITES [conn51] update db0.coll0 query: { n: 9.0 } update: { $unset: { x: 4.0 }, $push: { y: 4.0 }, $inc: { z: 2.0 } } nscanned:0 nscannedObjects:10 nMatched:1 nModified:1 keyUpdates:5 115ms 2014-11-14T15:24:11.383-0500 I QUERY [conn51] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 9.0 }, u: { $unset: { x: 4.0 }, $push: { y: 4.0 }, $inc: { z: 2.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 117ms 2014-11-14T15:24:11.392-0500 I WRITES [conn39] update db0.coll0 query: { n: 0.0 } update: { $set: { x: 3.0 }, $pull: { y: 2.0 }, $inc: { z: 0.0 } } nscanned:0 nscannedObjects:1 nMatched:1 nModified:0 keyUpdates:0 150ms 2014-11-14T15:24:11.394-0500 I QUERY [conn39] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 0.0 }, u: { $set: { x: 3.0 }, $pull: { y: 2.0 }, $inc: { z: 0.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 152ms 2014-11-14T15:24:11.399-0500 I WRITES [conn37] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.400-0500 I WRITES [conn48] update db0.coll0 query: { n: 4.0 } update: { $set: { x: 3.0 }, $push: { y: 0.0 }, $inc: { z: 2.0 } } nscanned:0 nscannedObjects:5 nMatched:1 nModified:1 keyUpdates:5 111ms 2014-11-14T15:24:11.402-0500 I QUERY [conn48] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 4.0 }, u: { $set: { x: 3.0 }, $push: { y: 0.0 }, $inc: { z: 2.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 113ms 2014-11-14T15:24:11.402-0500 I WRITES [conn11] update db0.coll0 query: { n: 2.0 } update: { $unset: { x: 0.0 }, $push: { y: 2.0 }, $inc: { z: 2.0 } } nscanned:0 nscannedObjects:3 nMatched:1 nModified:1 keyUpdates:4 111ms 2014-11-14T15:24:11.403-0500 I WRITES [conn19] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.404-0500 I WRITES [conn38] update db0.coll0 query: { n: 6.0 } update: { $unset: { x: 4.0 }, $push: { y: 0.0 }, $inc: { z: 0.0 } } nscanned:0 nscannedObjects:7 nMatched:1 nModified:1 keyUpdates:3 123ms 2014-11-14T15:24:11.405-0500 I QUERY [conn11] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 2.0 }, u: { $unset: { x: 0.0 }, $push: { y: 2.0 }, $inc: { z: 2.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 113ms 2014-11-14T15:24:11.405-0500 I WRITES [conn5] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.405-0500 I WRITES [conn42] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.407-0500 I WRITES [conn53] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.409-0500 I QUERY [conn38] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 6.0 }, u: { $unset: { x: 4.0 }, $push: { y: 0.0 }, $inc: { z: 0.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 128ms 2014-11-14T15:24:11.410-0500 I WRITES [conn8] update db0.coll0 query: { n: 7.0 } update: { $set: { x: 3.0 }, $push: { y: 4.0 }, $inc: { z: 3.0 } } nscanned:0 nscannedObjects:8 nMatched:1 nModified:1 keyUpdates:5 134ms 2014-11-14T15:24:11.410-0500 I WRITES [conn4] update db0.coll0 query: { n: 8.0 } update: { $unset: { x: 3.0 }, $push: { y: 0.0 }, $inc: { z: 1.0 } } nscanned:0 nscannedObjects:9 nMatched:1 nModified:1 keyUpdates:3 123ms 2014-11-14T15:24:11.412-0500 I QUERY [conn4] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 8.0 }, u: { $unset: { x: 3.0 }, $push: { y: 0.0 }, $inc: { z: 1.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 125ms 2014-11-14T15:24:11.414-0500 I WRITES [conn33] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.415-0500 I QUERY [conn8] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 7.0 }, u: { $set: { x: 3.0 }, $push: { y: 4.0 }, $inc: { z: 3.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 137ms 2014-11-14T15:24:11.416-0500 I WRITES [conn18] update db0.coll0 query: { n: 0.0 } update: { $set: { x: 1.0 }, $push: { y: 4.0 }, $inc: { z: 2.0 } } nscanned:0 nscannedObjects:1 nMatched:1 nModified:1 keyUpdates:5 125ms 2014-11-14T15:24:11.418-0500 I QUERY [conn18] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 0.0 }, u: { $set: { x: 1.0 }, $push: { y: 4.0 }, $inc: { z: 2.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 127ms 2014-11-14T15:24:11.421-0500 I WRITES [conn46] update db0.coll0 query: { n: 5.0 } update: { $unset: { x: 4.0 }, $pull: { y: 1.0 }, $inc: { z: 3.0 } } nscanned:0 nscannedObjects:6 nMatched:1 nModified:1 keyUpdates:4 152ms 2014-11-14T15:24:11.415-0500 I WRITES [conn23] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.422-0500 I WRITES [conn17] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.422-0500 I WRITES [conn32] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.424-0500 I WRITES [conn49] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.424-0500 I WRITES [conn47] update db0.coll0 query: { n: 1.0 } update: { $unset: { x: 0.0 }, $push: { y: 4.0 }, $inc: { z: 0.0 } } nscanned:0 nscannedObjects:2 nMatched:1 nModified:1 keyUpdates:2 157ms 2014-11-14T15:24:11.425-0500 I QUERY [conn46] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 5.0 }, u: { $unset: { x: 4.0 }, $pull: { y: 1.0 }, $inc: { z: 3.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 156ms 2014-11-14T15:24:11.426-0500 I QUERY [conn47] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 1.0 }, u: { $unset: { x: 0.0 }, $push: { y: 4.0 }, $inc: { z: 0.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 158ms 2014-11-14T15:24:11.427-0500 I WRITES [conn20] update db0.coll0 query: { n: 9.0 } update: { $set: { x: 3.0 }, $push: { y: 3.0 }, $inc: { z: 4.0 } } nscanned:0 nscannedObjects:10 nMatched:1 nModified:1 keyUpdates:7 134ms 2014-11-14T15:24:11.429-0500 I QUERY [conn20] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 9.0 }, u: { $set: { x: 3.0 }, $push: { y: 3.0 }, $inc: { z: 4.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 135ms 2014-11-14T15:24:11.429-0500 I WRITES [conn52] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.430-0500 I WRITES [conn24] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.430-0500 I WRITES [conn26] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.432-0500 I WRITES [conn10] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.434-0500 I WRITES [conn50] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.437-0500 I WRITES [conn31] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.439-0500 I WRITES [conn33] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.440-0500 I WRITES [conn42] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.442-0500 I WRITES [conn6] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.443-0500 I WRITES [conn19] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.444-0500 I CONTROL [conn35] 0x19f4f63 0x19a68df 0x1993c0b 0x13103ca 0x130a856 0x130c9ff 0x15db24c 0x15db82b 0x15de3d8 0x1540648 0x15409fe 0x13a7054 0x13a4bd8 0x13a454e 0x13a1fee 0x13ac16e 0x13c6913 0x13c78dd 0x13c826c 0x15f3de4 0x15f5abc 0x14d93d9 0x14da539 0x11c9296 0x19bb043 0x7fc94c76ce9a 0x7fc94b8773fd ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"15F4F63"},{"b":"400000","o":"15A68DF"},{"b":"400000","o":"1593C0B"},{"b":"400000","o":"F103CA"},{"b":"400000","o":"F0A856"},{"b":"400000","o":"F0C9FF"},{"b":"400000","o":"11DB24C"},{"b":"400000","o":"11DB82B"},{"b":"400000","o":"11DE3D8"},{"b":"400000","o":"1140648"},{"b":"400000","o":"11409FE"},{"b":"400000","o":"FA7054"},{"b":"400000","o":"FA4BD8"},{"b":"400000","o":"FA454E"},{"b":"400000","o":"FA1FEE"},{"b":"400000","o":"FAC16E"},{"b":"400000","o":"FC6913"},{"b":"400000","o":"FC78DD"},{"b":"400000","o":"FC826C"},{"b":"400000","o":"11F3DE4"},{"b":"400000","o":"11F5ABC"},{"b":"400000","o":"10D93D9"},{"b":"400000","o":"10DA539"},{"b":"400000","o":"DC9296"},{"b":"400000","o":"15BB043"},{"b":"7FC94C765000","o":"7E9A"},{"b":"7FC94B783000","o":"F43FD"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc1-pre-", "gitVersion" : "7d1099afb4c5392500154b05394c42f6256b24e2", "uname" : { "sysname" : "Linux", "release" : "3.8.0-38-generic", "version" : "#56~precise1-Ubuntu SMP Thu Mar 13 16:22:48 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "BC8433F588A329146FBD677B343AE871064F3442" }, { "b" : "7FFFD4052000", "elfType" : 3, "buildId" : "CB7F92627725102C17126BB9F42F9EB0E15D19D7" }, { "b" : "7FC94C765000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7FC94C55D000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7FC94C359000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7FC94C055000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "88FBC3C8675A12143E0C6EE228D355F665BA367E" }, { "b" : "7FC94BD59000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7FC94BB43000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "5C4DD7ADFA943E5A18AD415D91D5E50945FAE97A" }, { "b" : "7FC94B783000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "53514F7C9AD6614703AC01FD7CBCC9D3C6F621CD" }, { "b" : "7FC94C982000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "83B40B93BD2FD266F12E008E61309FC03EB406A5" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x27) [0x19f4f63] mongod(_ZN5mongo10logContextEPKc+0x71) [0x19a68df] mongod(_ZN5mongo15invariantFailedEPKcS1_j+0xB2) [0x1993c0b] mongod(_ZNK5mongo17IndexCatalogEntry10isMultikeyEPNS_16OperationContextE+0x56) [0x13103ca] mongod(_ZN5mongo12IndexCatalog10isMultikeyEPNS_16OperationContextEPKNS_15IndexDescriptorE+0x6A) [0x130a856] mongod(_ZNK5mongo15IndexDescriptor10isMultikeyEPNS_16OperationContextE+0x3F) [0x130c9ff] mongod(_ZN5mongo20fillOutPlannerParamsEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPNS_18QueryPlannerParamsE+0xC8) [0x15db24c] mongod(+0x11DB82B) [0x15db82b] mongod(_ZN5mongo17getExecutorUpdateEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPKNS_13UpdateRequestEPNS_12UpdateDriverEPNS_7OpDebugENS_12PlanExecutor11YieldPolicyEPPSD_+0xCE) [0x15de3d8] mongod(_ZN5mongo14UpdateExecutor13prepareInLockEPNS_8DatabaseE+0x980) [0x1540648] mongod(_ZN5mongo14UpdateExecutor7executeEPNS_8DatabaseE+0x112) [0x15409fe] mongod(+0xFA7054) [0x13a7054] mongod(_ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE+0x92) [0x13a4bd8] mongod(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0xB8) [0x13a454e] mongod(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x512) [0x13a1fee] mongod(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x266) [0x13ac16e] mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x96) [0x13c6913] mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xBD1) [0x13c78dd] mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x4F8) [0x13c826c] mongod(+0x11F3DE4) [0x15f3de4] mongod(_ZN5mongo11newRunQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERNS_5CurOpES3_b+0x39E) [0x15f5abc] mongod(+0x10D93D9) [0x14d93d9] mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0x470) [0x14da539] mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x124) [0x11c9296] mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4D6) [0x19bb043] libpthread.so.0(+0x7E9A) [0x7fc94c76ce9a] libc.so.6(clone+0x6D) [0x7fc94b8773fd] ----- END BACKTRACE ----- 2014-11-14T15:24:11.444-0500 I - [conn35] ***aborting after invariant() failure 2014-11-14T15:24:11.444-0500 I WRITES [conn21] update db0.coll0 query: { n: 3.0 } update: { $unset: { x: 2.0 }, $push: { y: 4.0 }, $inc: { z: 2.0 } } nscanned:0 nscannedObjects:4 nMatched:1 nModified:1 keyUpdates:4 186ms 2014-11-14T15:24:11.447-0500 I WRITES [conn5] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.447-0500 I WRITES [conn15] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.447-0500 I QUERY [conn21] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 3.0 }, u: { $unset: { x: 2.0 }, $push: { y: 4.0 }, $inc: { z: 2.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 190ms 2014-11-14T15:24:11.447-0500 I WRITES [conn37] update db0.coll0 query: { n: 4.0 } update: { $set: { x: 4.0 }, $pull: { y: 3.0 }, $inc: { z: 0.0 } } nscanned:0 nscannedObjects:5 nMatched:1 nModified:1 keyUpdates:3 196ms 2014-11-14T15:24:11.448-0500 I WRITES [conn34] update db0.coll0 query: { n: 6.0 } update: { $unset: { x: 2.0 }, $push: { y: 2.0 }, $inc: { z: 1.0 } } nscanned:0 nscannedObjects:7 nMatched:1 nModified:1 keyUpdates:4 178ms 2014-11-14T15:24:11.449-0500 I WRITES [conn44] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.451-0500 I QUERY [conn34] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 6.0 }, u: { $unset: { x: 2.0 }, $push: { y: 2.0 }, $inc: { z: 1.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 181ms 2014-11-14T15:24:11.453-0500 I WRITES [conn43] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.454-0500 I WRITES [conn53] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.455-0500 I WRITES [conn9] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.457-0500 I QUERY [conn37] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 4.0 }, u: { $set: { x: 4.0 }, $pull: { y: 3.0 }, $inc: { z: 0.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 204ms 2014-11-14T15:24:11.458-0500 I WRITES [conn7] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.460-0500 I WRITES [conn30] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.460-0500 I WRITES [conn28] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.464-0500 I WRITES [conn22] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.465-0500 I WRITES [conn41] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.468-0500 I WRITES [conn40] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.471-0500 I WRITES [conn17] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.471-0500 I WRITES [conn29] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.471-0500 I WRITES [conn23] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.472-0500 I WRITES [conn12] update db0.coll0 query: { n: 2.0 } update: { $unset: { x: 0.0 }, $push: { y: 2.0 }, $inc: { z: 3.0 } } nscanned:0 nscannedObjects:3 nMatched:1 nModified:1 keyUpdates:3 174ms 2014-11-14T15:24:11.474-0500 I WRITES [conn32] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.475-0500 I WRITES [conn25] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.476-0500 I QUERY [conn12] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 2.0 }, u: { $unset: { x: 0.0 }, $push: { y: 2.0 }, $inc: { z: 3.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 177ms 2014-11-14T15:24:11.477-0500 I WRITES [conn36] update db0.coll0 query: { n: 7.0 } update: { $set: { x: 4.0 }, $push: { y: 3.0 }, $inc: { z: 4.0 } } nscanned:0 nscannedObjects:8 nMatched:1 nModified:1 keyUpdates:6 205ms 2014-11-14T15:24:11.478-0500 I WRITES [conn49] update db0.coll0 query: { n: 0.0 } update: { $unset: { x: 3.0 }, $push: { y: 1.0 }, $inc: { z: 3.0 } } nscanned:0 nscannedObjects:1 nMatched:1 nModified:1 keyUpdates:6 206ms 2014-11-14T15:24:11.479-0500 I QUERY [conn36] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 7.0 }, u: { $set: { x: 4.0 }, $push: { y: 3.0 }, $inc: { z: 4.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 208ms 2014-11-14T15:24:11.479-0500 I WRITES [conn26] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.482-0500 I QUERY [conn49] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 0.0 }, u: { $unset: { x: 3.0 }, $push: { y: 1.0 }, $inc: { z: 3.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 211ms 2014-11-14T15:24:11.483-0500 I WRITES [conn45] update db0.coll0 query: { n: 8.0 } update: { $set: { x: 0.0 }, $push: { y: 3.0 }, $inc: { z: 3.0 } } nscanned:0 nscannedObjects:9 nMatched:1 nModified:1 keyUpdates:5 213ms 2014-11-14T15:24:11.484-0500 I WRITES [conn15] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.484-0500 I WRITES [conn31] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.484-0500 I WRITES [conn33] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.485-0500 I QUERY [conn45] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 8.0 }, u: { $set: { x: 0.0 }, $push: { y: 3.0 }, $inc: { z: 3.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 215ms 2014-11-14T15:24:11.486-0500 I WRITES [conn10] update db0.coll0 query: { n: 9.0 } update: { $unset: { x: 1.0 }, $pull: { y: 1.0 }, $inc: { z: 0.0 } } nscanned:0 nscannedObjects:10 nMatched:1 nModified:1 keyUpdates:4 197ms 2014-11-14T15:24:11.488-0500 I WRITES [conn24] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.488-0500 I WRITES [conn52] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.488-0500 I QUERY [conn10] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 9.0 }, u: { $unset: { x: 1.0 }, $pull: { y: 1.0 }, $inc: { z: 0.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 198ms 2014-11-14T15:24:11.489-0500 I WRITES [conn44] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.490-0500 I WRITES [conn50] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.494-0500 I WRITES [conn42] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.496-0500 I WRITES [conn5] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.496-0500 I WRITES [conn19] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.497-0500 I CONTROL [conn13] 0x19f4f63 0x19a68df 0x1993c0b 0x13103ca 0x130a856 0x130c9ff 0x15db24c 0x15db82b 0x15de3d8 0x1540648 0x15409fe 0x13a7054 0x13a4bd8 0x13a454e 0x13a1fee 0x13ac16e 0x13c6913 0x13c78dd 0x13c826c 0x15f3de4 0x15f5abc 0x14d93d9 0x14da539 0x11c9296 0x19bb043 0x7fc94c76ce9a 0x7fc94b8773fd ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"15F4F63"},{"b":"400000","o":"15A68DF"},{"b":"400000","o":"1593C0B"},{"b":"400000","o":"F103CA"},{"b":"400000","o":"F0A856"},{"b":"400000","o":"F0C9FF"},{"b":"400000","o":"11DB24C"},{"b":"400000","o":"11DB82B"},{"b":"400000","o":"11DE3D8"},{"b":"400000","o":"1140648"},{"b":"400000","o":"11409FE"},{"b":"400000","o":"FA7054"},{"b":"400000","o":"FA4BD8"},{"b":"400000","o":"FA454E"},{"b":"400000","o":"FA1FEE"},{"b":"400000","o":"FAC16E"},{"b":"400000","o":"FC6913"},{"b":"400000","o":"FC78DD"},{"b":"400000","o":"FC826C"},{"b":"400000","o":"11F3DE4"},{"b":"400000","o":"11F5ABC"},{"b":"400000","o":"10D93D9"},{"b":"400000","o":"10DA539"},{"b":"400000","o":"DC9296"},{"b":"400000","o":"15BB043"},{"b":"7FC94C765000","o":"7E9A"},{"b":"7FC94B783000","o":"F43FD"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc1-pre-", "gitVersion" : "7d1099afb4c5392500154b05394c42f6256b24e2", "uname" : { "sysname" : "Linux", "release" : "3.8.0-38-generic", "version" : "#56~precise1-Ubuntu SMP Thu Mar 13 16:22:48 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "BC8433F588A329146FBD677B343AE871064F3442" }, { "b" : "7FFFD4052000", "elfType" : 3, "buildId" : "CB7F92627725102C17126BB9F42F9EB0E15D19D7" }, { "b" : "7FC94C765000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7FC94C55D000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7FC94C359000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7FC94C055000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "88FBC3C8675A12143E0C6EE228D355F665BA367E" }, { "b" : "7FC94BD59000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7FC94BB43000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "5C4DD7ADFA943E5A18AD415D91D5E50945FAE97A" }, { "b" : "7FC94B783000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "53514F7C9AD6614703AC01FD7CBCC9D3C6F621CD" }, { "b" : "7FC94C982000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "83B40B93BD2FD266F12E008E61309FC03EB406A5" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x27) [0x19f4f63] mongod(_ZN5mongo10logContextEPKc+0x71) [0x19a68df] mongod(_ZN5mongo15invariantFailedEPKcS1_j+0xB2) [0x1993c0b] mongod(_ZNK5mongo17IndexCatalogEntry10isMultikeyEPNS_16OperationContextE+0x56) [0x13103ca] mongod(_ZN5mongo12IndexCatalog10isMultikeyEPNS_16OperationContextEPKNS_15IndexDescriptorE+0x6A) [0x130a856] mongod(_ZNK5mongo15IndexDescriptor10isMultikeyEPNS_16OperationContextE+0x3F) [0x130c9ff] mongod(_ZN5mongo20fillOutPlannerParamsEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPNS_18QueryPlannerParamsE+0xC8) [0x15db24c] mongod(+0x11DB82B) [0x15db82b] mongod(_ZN5mongo17getExecutorUpdateEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPKNS_13UpdateRequestEPNS_12UpdateDriverEPNS_7OpDebugENS_12PlanExecutor11YieldPolicyEPPSD_+0xCE) [0x15de3d8] mongod(_ZN5mongo14UpdateExecutor13prepareInLockEPNS_8DatabaseE+0x980) [0x1540648] mongod(_ZN5mongo14UpdateExecutor7executeEPNS_8DatabaseE+0x112) [0x15409fe] mongod(+0xFA7054) [0x13a7054] mongod(_ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE+0x92) [0x13a4bd8] mongod(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0xB8) [0x13a454e] mongod(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x512) [0x13a1fee] mongod(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x266) [0x13ac16e] mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x96) [0x13c6913] mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xBD1) [0x13c78dd] mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x4F8) [0x13c826c] mongod(+0x11F3DE4) [0x15f3de4] mongod(_ZN5mongo11newRunQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERNS_5CurOpES3_b+0x39E) [0x15f5abc] mongod(+0x10D93D9) [0x14d93d9] mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0x470) [0x14da539] mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x124) [0x11c9296] mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4D6) [0x19bb043] libpthread.so.0(+0x7E9A) [0x7fc94c76ce9a] libc.so.6(clone+0x6D) [0x7fc94b8773fd] ----- END BACKTRACE ----- 2014-11-14T15:24:11.497-0500 I - [conn13] ***aborting after invariant() failure 2014-11-14T15:24:11.502-0500 I WRITES [conn6] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.502-0500 I WRITES [conn53] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.507-0500 I WRITES [conn28] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.508-0500 I WRITES [conn30] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.509-0500 I WRITES [conn9] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.511-0500 I WRITES [conn41] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.513-0500 I WRITES [conn22] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.514-0500 I WRITES [conn7] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.517-0500 I WRITES [conn11] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.517-0500 I WRITES [conn14] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.517-0500 I CONTROL [conn16] 0x19f4f63 0x19a68df 0x1993c0b 0x13103ca 0x130a856 0x130c9ff 0x15db24c 0x15db82b 0x15de3d8 0x1540648 0x15409fe 0x13a7054 0x13a4bd8 0x13a454e 0x13a1fee 0x13ac16e 0x13c6913 0x13c78dd 0x13c826c 0x15f3de4 0x15f5abc 0x14d93d9 0x14da539 0x11c9296 0x19bb043 0x7fc94c76ce9a 0x7fc94b8773fd ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"15F4F63"},{"b":"400000","o":"15A68DF"},{"b":"400000","o":"1593C0B"},{"b":"400000","o":"F103CA"},{"b":"400000","o":"F0A856"},{"b":"400000","o":"F0C9FF"},{"b":"400000","o":"11DB24C"},{"b":"400000","o":"11DB82B"},{"b":"400000","o":"11DE3D8"},{"b":"400000","o":"1140648"},{"b":"400000","o":"11409FE"},{"b":"400000","o":"FA7054"},{"b":"400000","o":"FA4BD8"},{"b":"400000","o":"FA454E"},{"b":"400000","o":"FA1FEE"},{"b":"400000","o":"FAC16E"},{"b":"400000","o":"FC6913"},{"b":"400000","o":"FC78DD"},{"b":"400000","o":"FC826C"},{"b":"400000","o":"11F3DE4"},{"b":"400000","o":"11F5ABC"},{"b":"400000","o":"10D93D9"},{"b":"400000","o":"10DA539"},{"b":"400000","o":"DC9296"},{"b":"400000","o":"15BB043"},{"b":"7FC94C765000","o":"7E9A"},{"b":"7FC94B783000","o":"F43FD"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc1-pre-", "gitVersion" : "7d1099afb4c5392500154b05394c42f6256b24e2", "uname" : { "sysname" : "Linux", "release" : "3.8.0-38-generic", "version" : "#56~precise1-Ubuntu SMP Thu Mar 13 16:22:48 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "BC8433F588A329146FBD677B343AE871064F3442" }, { "b" : "7FFFD4052000", "elfType" : 3, "buildId" : "CB7F92627725102C17126BB9F42F9EB0E15D19D7" }, { "b" : "7FC94C765000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7FC94C55D000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7FC94C359000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7FC94C055000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "88FBC3C8675A12143E0C6EE228D355F665BA367E" }, { "b" : "7FC94BD59000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7FC94BB43000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "5C4DD7ADFA943E5A18AD415D91D5E50945FAE97A" }, { "b" : "7FC94B783000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "53514F7C9AD6614703AC01FD7CBCC9D3C6F621CD" }, { "b" : "7FC94C982000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "83B40B93BD2FD266F12E008E61309FC03EB406A5" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x27) [0x19f4f63] mongod(_ZN5mongo10logContextEPKc+0x71) [0x19a68df] mongod(_ZN5mongo15invariantFailedEPKcS1_j+0xB2) [0x1993c0b] mongod(_ZNK5mongo17IndexCatalogEntry10isMultikeyEPNS_16OperationContextE+0x56) [0x13103ca] mongod(_ZN5mongo12IndexCatalog10isMultikeyEPNS_16OperationContextEPKNS_15IndexDescriptorE+0x6A) [0x130a856] mongod(_ZNK5mongo15IndexDescriptor10isMultikeyEPNS_16OperationContextE+0x3F) [0x130c9ff] mongod(_ZN5mongo20fillOutPlannerParamsEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPNS_18QueryPlannerParamsE+0xC8) [0x15db24c] mongod(+0x11DB82B) [0x15db82b] mongod(_ZN5mongo17getExecutorUpdateEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPKNS_13UpdateRequestEPNS_12UpdateDriverEPNS_7OpDebugENS_12PlanExecutor11YieldPolicyEPPSD_+0xCE) [0x15de3d8] mongod(_ZN5mongo14UpdateExecutor13prepareInLockEPNS_8DatabaseE+0x980) [0x1540648] mongod(_ZN5mongo14UpdateExecutor7executeEPNS_8DatabaseE+0x112) [0x15409fe] mongod(+0xFA7054) [0x13a7054] mongod(_ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE+0x92) [0x13a4bd8] mongod(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0xB8) [0x13a454e] mongod(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x512) [0x13a1fee] mongod(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x266) [0x13ac16e] mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x96) [0x13c6913] mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xBD1) [0x13c78dd] mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x4F8) [0x13c826c] mongod(+0x11F3DE4) [0x15f3de4] mongod(_ZN5mongo11newRunQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERNS_5CurOpES3_b+0x39E) [0x15f5abc] mongod(+0x10D93D9) [0x14d93d9] mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0x470) [0x14da539] mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x124) [0x11c9296] mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4D6) [0x19bb043] libpthread.so.0(+0x7E9A) [0x7fc94c76ce9a] libc.so.6(clone+0x6D) [0x7fc94b8773fd] ----- END BACKTRACE ----- 2014-11-14T15:24:11.517-0500 I - [conn16] ***aborting after invariant() failure 2014-11-14T15:24:11.518-0500 I WRITES [conn48] Had WriteConflict doing update on db0.coll0, attempt: 2 retrying 2014-11-14T15:24:11.518-0500 I WRITES [conn17] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.518-0500 I WRITES [conn43] update db0.coll0 query: { n: 3.0 } update: { $unset: { x: 1.0 }, $push: { y: 3.0 }, $inc: { z: 4.0 } } nscanned:0 nscannedObjects:4 nMatched:1 nModified:1 keyUpdates:5 278ms 2014-11-14T15:24:11.520-0500 I QUERY [conn43] command db0.$cmd command: update { update: "coll0", updates: [ { q: { n: 3.0 }, u: { $unset: { x: 1.0 }, $push: { y: 3.0 }, $inc: { z: 4.0 } }, multi: false, upsert: false } ], ordered: true } keyUpdates:0 reslen:55 280ms 2014-11-14T15:24:11.523-0500 I WRITES [conn23] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.523-0500 I WRITES [conn25] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.525-0500 I WRITES [conn26] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.526-0500 I WRITES [conn40] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.527-0500 I WRITES [conn52] Had WriteConflict doing update on db0.coll0, attempt: 4 retrying 2014-11-14T15:24:11.528-0500 I WRITES [conn29] Had WriteConflict doing update on db0.coll0, attempt: 3 retrying 2014-11-14T15:24:11.529-0500 F - [conn35] Got signal: 6 (Aborted). 0x19f4f63 0x19f4779 0x19f4810 0x7fc94b7b94a0 0x7fc94b7b9425 0x7fc94b7bcb8b 0x1993c52 0x13103ca 0x130a856 0x130c9ff 0x15db24c 0x15db82b 0x15de3d8 0x1540648 0x15409fe 0x13a7054 0x13a4bd8 0x13a454e 0x13a1fee 0x13ac16e 0x13c6913 0x13c78dd 0x13c826c 0x15f3de4 0x15f5abc 0x14d93d9 0x14da539 0x11c9296 0x19bb043 0x7fc94c76ce9a 0x7fc94b8773fd ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"15F4F63"},{"b":"400000","o":"15F4779"},{"b":"400000","o":"15F4810"},{"b":"7FC94B783000","o":"364A0"},{"b":"7FC94B783000","o":"36425"},{"b":"7FC94B783000","o":"39B8B"},{"b":"400000","o":"1593C52"},{"b":"400000","o":"F103CA"},{"b":"400000","o":"F0A856"},{"b":"400000","o":"F0C9FF"},{"b":"400000","o":"11DB24C"},{"b":"400000","o":"11DB82B"},{"b":"400000","o":"11DE3D8"},{"b":"400000","o":"1140648"},{"b":"400000","o":"11409FE"},{"b":"400000","o":"FA7054"},{"b":"400000","o":"FA4BD8"},{"b":"400000","o":"FA454E"},{"b":"400000","o":"FA1FEE"},{"b":"400000","o":"FAC16E"},{"b":"400000","o":"FC6913"},{"b":"400000","o":"FC78DD"},{"b":"400000","o":"FC826C"},{"b":"400000","o":"11F3DE4"},{"b":"400000","o":"11F5ABC"},{"b":"400000","o":"10D93D9"},{"b":"400000","o":"10DA539"},{"b":"400000","o":"DC9296"},{"b":"400000","o":"15BB043"},{"b":"7FC94C765000","o":"7E9A"},{"b":"7FC94B783000","o":"F43FD"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc1-pre-", "gitVersion" : "7d1099afb4c5392500154b05394c42f6256b24e2", "uname" : { "sysname" : "Linux", "release" : "3.8.0-38-generic", "version" : "#56~precise1-Ubuntu SMP Thu Mar 13 16:22:48 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "BC8433F588A329146FBD677B343AE871064F3442" }, { "b" : "7FFFD4052000", "elfType" : 3, "buildId" : "CB7F92627725102C17126BB9F42F9EB0E15D19D7" }, { "b" : "7FC94C765000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7FC94C55D000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7FC94C359000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7FC94C055000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "88FBC3C8675A12143E0C6EE228D355F665BA367E" }, { "b" : "7FC94BD59000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7FC94BB43000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "5C4DD7ADFA943E5A18AD415D91D5E50945FAE97A" }, { "b" : "7FC94B783000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "53514F7C9AD6614703AC01FD7CBCC9D3C6F621CD" }, { "b" : "7FC94C982000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "83B40B93BD2FD266F12E008E61309FC03EB406A5" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x27) [0x19f4f63] mongod(+0x15F4779) [0x19f4779] mongod(+0x15F4810) [0x19f4810] libc.so.6(+0x364A0) [0x7fc94b7b94a0] libc.so.6(gsignal+0x35) [0x7fc94b7b9425] libc.so.6(abort+0x17B) [0x7fc94b7bcb8b] mongod(_ZN5mongo15invariantFailedEPKcS1_j+0xF9) [0x1993c52] mongod(_ZNK5mongo17IndexCatalogEntry10isMultikeyEPNS_16OperationContextE+0x56) [0x13103ca] mongod(_ZN5mongo12IndexCatalog10isMultikeyEPNS_16OperationContextEPKNS_15IndexDescriptorE+0x6A) [0x130a856] mongod(_ZNK5mongo15IndexDescriptor10isMultikeyEPNS_16OperationContextE+0x3F) [0x130c9ff] mongod(_ZN5mongo20fillOutPlannerParamsEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPNS_18QueryPlannerParamsE+0xC8) [0x15db24c] mongod(+0x11DB82B) [0x15db82b] mongod(_ZN5mongo17getExecutorUpdateEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryEPKNS_13UpdateRequestEPNS_12UpdateDriverEPNS_7OpDebugENS_12PlanExecutor11YieldPolicyEPPSD_+0xCE) [0x15de3d8] mongod(_ZN5mongo14UpdateExecutor13prepareInLockEPNS_8DatabaseE+0x980) [0x1540648] mongod(_ZN5mongo14UpdateExecutor7executeEPNS_8DatabaseE+0x112) [0x15409fe] mongod(+0xFA7054) [0x13a7054] mongod(_ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE+0x92) [0x13a4bd8] mongod(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0xB8) [0x13a454e] mongod(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x512) [0x13a1fee] mongod(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x266) [0x13ac16e] mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x96) [0x13c6913] mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xBD1) [0x13c78dd] mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x4F8) [0x13c826c] mongod(+0x11F3DE4) [0x15f3de4] mongod(_ZN5mongo11newRunQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERNS_5CurOpES3_b+0x39E) [0x15f5abc] mongod(+0x10D93D9) [0x14d93d9] mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortEb+0x470) [0x14da539] mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x124) [0x11c9296] mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4D6) [0x19bb043] libpthread.so.0(+0x7E9A) [0x7fc94c76ce9a] libc.so.6(clone+0x6D) [0x7fc94b8773fd] ----- END BACKTRACE ----- 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.539-0500 I NETWORK Socket recv() errno:104 Connection reset by peer 127.0.0.1:27999 2014-11-14T15:24:11.540-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.540-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.540-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.540-0500 I NETWORK SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 2014-11-14T15:24:11.540-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.540-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.540-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.540-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.543-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.547-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.551-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.551-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.554-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.554-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.554-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.558-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.558-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.558-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.558-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.558-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.559-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.560-0500 I NETWORK Socket recv() errno:104 Connection reset by peer 127.0.0.1:27999 2014-11-14T15:24:11.564-0500 I NETWORK SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 2014-11-14T15:24:11.564-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.576-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.576-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.588-0500 I NETWORK Socket recv() errno:104 Connection reset by peer 127.0.0.1:27999 2014-11-14T15:24:11.588-0500 I NETWORK Socket recv() errno:104 Connection reset by peer 127.0.0.1:27999 2014-11-14T15:24:11.588-0500 I NETWORK Socket recv() errno:104 Connection reset by peer 127.0.0.1:27999 2014-11-14T15:24:11.588-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.588-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.588-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.589-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.589-0500 I NETWORK SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 2014-11-14T15:24:11.589-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.589-0500 I NETWORK SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 2014-11-14T15:24:11.589-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.589-0500 I NETWORK SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 2014-11-14T15:24:11.589-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.590-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.590-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.589-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:11.617-0500 I NETWORK DBClientCursor::init call() failed 2014-11-14T15:24:12.549-0500 I QUERY Error: 50 threads threw Error: Error: error doing query: failed at Error () at DBCollection.update (src/mongo/shell/collection.js:465:23) at Object.update (jstests/parallel/fsm_workloads/update_multifield.js:41:36) at Object.runFSM [as run] (jstests/parallel/fsm_libs/fsm.js:15:39) at :8:13 at Object.main (jstests/parallel/fsm_libs/worker_thread.js:52:13) at ____MongoToV8_newFunction_temp (:5:25) at throwError (jstests/parallel/fsm_libs/runner.js:498:19) at _runAllWorkloads (jstests/parallel/fsm_libs/runner.js:353:5) at _runWorkload (jstests/parallel/fsm_libs/runner.js:253:5) at jstests/parallel/fsm_libs/runner.js:39:9 at Array.forEach (native) at runWorkloadsSerially (jstests/parallel/fsm_libs/runner.js:34:15) at /data01/home/dpercy/beef/10gen/mongo/fsm_all.js:26:1 at jstests/parallel/fsm_libs/runner.js:510 failed to load: /data01/home/dpercy/beef/10gen/mongo/fsm_all.js 4117.5311 ms [Errno 111] Connection refused mongod is not running after test mongod not running after executing test /data01/home/dpercy/beef/10gen/mongo/fsm_all.js [Errno 111] Connection refused 0 tests succeeded The following tests failed (with exit code): /data01/home/dpercy/beef/10gen/mongo/fsm_all.js -1 Traceback (most recent call last): File "buildscripts/smoke.py", line 1412, in main() File "buildscripts/smoke.py", line 1409, in main report() File "buildscripts/smoke.py", line 870, in report raise Exception("Test failures") Exception: Test failures real 0m5.341s user 1m23.952s sys 0m3.752s