Tue Apr 27 14:05:18 connection accepted from 10.250.74.65:36173 #129660 Tue Apr 27 14:05:18 connection accepted from 10.250.74.65:36178 #129661 Tue Apr 27 14:05:18 connection accepted from 127.0.0.1:40673 #129662 Tue Apr 27 14:05:19 query admin.$cmd ntoreturn:1 command reslen:994 350ms Tue Apr 27 14:05:19 end connection 127.0.0.1:40673 Tue Apr 27 14:05:20 end connection 10.250.74.65:36106 Tue Apr 27 14:05:20 end connection 10.250.74.65:36178 Tue Apr 27 14:05:20 repl: applied 24 operations Tue Apr 27 14:05:20 repl: end sync_pullOpLog syncedTo: Tue Apr 27 14:05:18 2010 4bd6ef1e:2 Tue Apr 27 14:05:20 repl: sleep 1sec before next pass Tue Apr 27 14:05:21 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:21 repl: sleep 2sec before next pass Tue Apr 27 14:05:21 end connection 10.211.55.96:55071 Tue Apr 27 14:05:23 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:23 repl: sleep 2sec before next pass Tue Apr 27 14:05:25 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:26 end connection 10.192.229.187:40063 Tue Apr 27 14:05:26 Assertion: 10276:DBClientBase::findOne: transport error 0x4fe0e4 0x510495 0x597cd4 0x5a5260 0x5a55bc 0x5a5bfe 0x5a630b 0x6bbf90 0x330440729a 0x3303ce439d /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x204) [0x4fe0e4] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo17DBClientInterface7findOneERKSsNS_5QueryEPKNS_7BSONObjEi+0x1c5) [0x510495] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo8ReplPair9negotiateEPNS_18DBClientConnectionESs+0x344) [0x597cd4] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo10ReplSource4syncERi+0x290) [0x5a5260] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo9_replMainERSt6vectorIN5boost10shared_ptrINS_10ReplSourceEEESaIS4_EERi+0xec) [0x5a55bc] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo8replMainEv+0xce) [0x5a5bfe] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo15replSlaveThreadEv+0x1bb) [0x5a630b] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(thread_proxy+0x80) [0x6bbf90] /lib64/libpthread.so.0 [0x330440729a] /lib64/libc.so.6(clone+0x6d) [0x3303ce439d] Tue Apr 27 14:05:26 repl: AssertionException DBClientBase::findOne: transport error Tue Apr 27 14:05:26 repl: sleep 2sec before next pass Tue Apr 27 14:05:27 end connection 10.192.229.187:51838 Tue Apr 27 14:05:27 connection accepted from 10.211.55.96:55081 #129663 Tue Apr 27 14:05:28 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:28 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:28 pair: setting master=1 was 0 Tue Apr 27 14:05:28 repl: sleep 3sec before next pass Tue Apr 27 14:05:29 connection accepted from 10.250.118.178:44832 #129664 Tue Apr 27 14:05:29 connection accepted from 10.250.118.178:44833 #129665 Tue Apr 27 14:05:31 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:31 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:31 repl: sleep 3sec before next pass Tue Apr 27 14:05:33 connection accepted from 10.250.74.240:55226 #129666 Tue Apr 27 14:05:33 connection accepted from 10.250.74.240:55227 #129667 Tue Apr 27 14:05:33 end connection 10.250.118.178:44833 Tue Apr 27 14:05:33 connection accepted from 10.250.74.240:55228 #129668 Tue Apr 27 14:05:33 connection accepted from 10.250.74.240:55229 #129669 Tue Apr 27 14:05:34 query random_db_name.$cmd ntoreturn:1 command reslen:64 bytes:48 197ms Tue Apr 27 14:05:34 CMD: drop random_db_name.tmp.mr.mapreduce_1272377134_1 Tue Apr 27 14:05:34 CMD: drop random_db_name.tmp.mr.mapreduce_1272377134_1_inc Tue Apr 27 14:05:34 allocating new datafile /mongo/master/random_db_name.222, filling with zeroes... Tue Apr 27 14:05:34 query random_db_name.$cmd ntoreturn:1 command reslen:64 bytes:48 267ms Tue Apr 27 14:05:35 query random_db_name.random_collection_name ntoreturn:0 reslen:440749 nscanned:101 query: { people.pid: { $in: { 0: "a9B3DmTxir3P-veJe6aitH", 1: "a9B3y-Txir3P-veJe6aitH", 2: "a9zM20Txir3P-veJe6aitH", 3: "aCU4l8Txir3P-veJe6aitH", 4: "aCU4tsTxir3P-veJe6aitH", 5: "aDzn7qTxOr3P-veJe6aitH", 6: "aDzn_OTxOr3P-veJe6aitH", 7: "aEM6_kTxir3P-veJe6aitH", 8: "aESE0uTxOr3P-veJe6aitH", 9: "aEkarYTyar3P-veJe6aitH", 10: "aFd3N6TwGr3P-veJe6aitH", 11: "aFd3NiTwGr3P-veJe6aitH", 12: "aGKQbMW7er3P-veJe6aitH", 13: "aIiQlETxir3P-veJe6aitH", 14: "aMQ5LATxir3P-veJe6aitH", 15: "aOxSUot5qr37X2eJe5buIH", 16: "aTHpp-TxOr3P-veJe6aitH", 17: "aZ-2KancOr35CEeJe5buIH", 18: "a_R0cETxKr3P-veJe6aitH", 19: "abluNAUUar3P-veJe6aitH", 20: "ah7fnUTYer3P-veJe6aitH", 21: "ah7fq2TYer3P-veJe6aitH", 22: "anMKjoUnqr3P-veJe6aitH", 23: "ang3U8Tw4r3P-veJe6aitH", 24: "ang3X6Tw4r3P-veJe6aitH", 25: "aq6eQ2YY0r3Q9CeJe6aitH", 26: "ar7YJMTw4r3P-veJe6aitH", 27: "aroH3-Tyar3P-veJe6aitH", 28: "awHyWWTyar3P-veJe6aitH", 29: "ax77RmTxir3P-veJe6aitH", 30: "azSDPuTxOr3P-veJe6aitH", 31: "azpCJkTyar3P-veJe6aitH", 32: "b3lGZ0Txmr3P-veJe6aitH", 33: "b4dVpUTymr3P-veJe6aitH", 34: "b4fjZKTxur3P-veJe6aitH", 35: "b8MamiTxir3P-veJe6aitH", 36: "bBmE_0Txyr3P-veJe6aitH", 37: "bDEsDwTxur3P-veJe6aitH", 38: "bHDJ2kUnSr3P-veJe6aitH", 39: "bHHqVyTxir3P-veJe6aitH", 40: "bHHqW8Txir3P-veJe6aitH", 41: "bHHqX4Txir3P-veJe6aitH", 42: "bHHqYQTxir3P-veJe6aitH", 43: "bLsvFW1pqr3Q9CeJe6aitH", 44: "bSYt-eTxur3P-veJe6aitH", 45: "bSoG9Aswer37X2eJe5buIH", 46: "bSoWDoTxir3P-veJe6aitH", 47: "bUy5wkTx0r3P-veJe6aitH", 48: "bVRmlCTxir3P-veJe6aitH", 49: "b_RiHa-EOr3Q9CeJe6aitH", 50: "bcDqhmTxyr3P-veJe6aitH", 51: "beneFSTxir3P-veJe6aitH", 52: "bh-N82Txir3P-veJe6aitH", 53: "bhLcMOXbCr3P-veJe6aitH", 54: "bhZ5ZqTxir3P-veJe6aitH", 55: "bhqDGKTxyr3P-veJe6aitH", 56: "bl8RlQ6jWr3Q9CeJe6aitH", 57: "blU4CUTxer3P-veJe6aitH", 58: "blk2mkTxir3P-veJe6aitH", 59: "blk2nUTxir3P-veJe6aitH", 60: "blk2tQTxir3P-veJe6aitH", 61: "brTTW2Tw0r3P-veJe6aitH", 62: "bv4-SOTxir3P-veJe6aitH", 63: "bv4-XuTxir3P-veJe6aitH", 64: "bxYVQaTxir3P-veJe6aitH", 65: "bxYVRKTxir3P-veJe6aitH", 66: "bxYVSwTxir3P-veJe6aitH", 67: "bxYVVOTxir3P-veJe6aitH", 68: "byO3VcTyar3P-veJe6aitH", 69: "bz-P_ITxur3P-veJe6aitH", 70: "c2r6NgTxKr3P-veJe6aitH", 71: "c2r6O-TxKr3P-veJe6aitH", 72: "c5OQXaTxer3P-veJe6aitH", 73: "c5OQYUTxer3P-veJe6aitH", 74: "c5kpayYt4r3P-veJe6aitH", 75: "c8Rs_-iMKr369CeJe6aitH", 76: "cC1VswTxar3P-veJe6aitH", 77: "cIoN3IbSeqlBDyeJe6akwI", 78: "cJsgnQTxKr3P-veJe6aitH", 79: "cQF4K4TxKr3P-veJe6aitH", 80: "cQF4NyTxKr3P-veJe6aitH", 81: "cctpjCTymr3P-veJe6aitH", 82: "ch3DKKTxur3P-veJe6aitH", 83: "ciJQhSTymr3P-veJe6aitH", 84: "ck6-gSZIqr3Q9CeJe6aitH", 85: "ckKQAMiMar369CeJe6aitH", 86: "cl41o86j4r3Q9CeJe6aitH", 87: "cnXSAQkcir369CeJe6aitH", 88: "cox5UGTxir3P-veJe6aitH", 89: "cspaZwTxir3P-veJe6aitH", 90: "cubcsAXvGr3P-veJe6aitH", 91: "cx3NTeTxar3P-veJe6aitH", 92: "d2YMVqTyar3P-veJe6aitH", 93: "d5KA3ITxer3P-veJe6aitH", 94: "d5KBcETxer3P-veJe6aitH", 95: "d99bo-Tw4r3P-veJe6aitH", 96: "d99br8Tw4r3P-veJe6aitH", 97: "d99bziTw4r3P-veJe6aitH", 98: "d9IaMsTx8r3P-veJe6aitH", 99: "dAmRvoTxWr3P-veJe6aitH", 100: "dEiDwqTxWr3P-veJe6aitH", 101: "dGexfYTAGr3P-veJe6aitH", 102: "dKLDO8Tx4r3P-veJe6aitH", 103: "dKeeY8Txyr3P-veJe6aitH", 104: "dKeeYaTxyr3P-veJe6aitH", 105: "dKeeZ4Txyr3P-veJe6aitH", 106: "dSQsEYTxWr3P-veJe6aitH", 107: "dSbWq4Txer3P-veJe6aitH", 108: "dT0DmUT2ur3P-veJe6aitH", 109: "dVSObWTYar3P-veJe6aitH", 110: "dWIm32fS8r369CeJe6aitH", 111: "dd0zlUaUWr369CeJe6aitH", 112: "dd4uUY8Kmr3Q9CeJe6aitH", 113: "duiDGETyir3P-veJe6aitH" } } } nreturned:101 bytes:440733 335ms Tue Apr 27 14:05:35 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:35 building new index on { 0: 1 } for random_db_name.tmp.mr.mapreduce_1272377134_1_inc Tue Apr 27 14:05:35 Buildindex random_db_name.tmp.mr.mapreduce_1272377134_1_inc idxNo:0 { ns: "random_db_name.tmp.mr.mapreduce_1272377134_1_inc", key: { 0: 1 }, name: "0_1" } Tue Apr 27 14:05:35 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:35 end connection 10.250.74.240:55227 Tue Apr 27 14:05:35 done for 2 records 0.192secs Tue Apr 27 14:05:35 insert random_db_name.system.indexes 192ms Tue Apr 27 14:05:35 building new index on { _id: 1 } for random_db_name.tmp.mr.mapreduce_1272377134_1 Tue Apr 27 14:05:35 Buildindex random_db_name.tmp.mr.mapreduce_1272377134_1 idxNo:0 { name: "_id_", ns: "random_db_name.tmp.mr.mapreduce_1272377134_1", key: { _id: 1 } } Tue Apr 27 14:05:35 end connection 10.192.203.32:37156 Tue Apr 27 14:05:35 done for 0 records 0.183secs Tue Apr 27 14:05:35 CMD: drop random_db_name.tmp.mr.mapreduce_1272377134_1_inc Tue Apr 27 14:05:35 query random_db_name.$cmd ntoreturn:1 command reslen:176 bytes:160 232ms Tue Apr 27 14:05:35 repl: sleep 3sec before next pass Tue Apr 27 14:05:35 query random_db_name.$cmd ntoreturn:1 command reslen:157 bytes:141 1996ms Tue Apr 27 14:05:38 done allocating datafile /mongo/master/random_db_name.222, size: 2047MB, took 3.712 secs Tue Apr 27 14:05:38 CMD: drop random_db_name.tmp.mr.mapreduce_1272377138_2 Tue Apr 27 14:05:39 CMD: drop random_db_name.tmp.mr.mapreduce_1272377138_2_inc Tue Apr 27 14:05:39 query random_db_name.$cmd ntoreturn:1 command reslen:78 bytes:62 510ms Tue Apr 27 14:05:39 building new index on { 0: 1 } for random_db_name.tmp.mr.mapreduce_1272377138_2_inc Tue Apr 27 14:05:39 Buildindex random_db_name.tmp.mr.mapreduce_1272377138_2_inc idxNo:0 { ns: "random_db_name.tmp.mr.mapreduce_1272377138_2_inc", key: { 0: 1 }, name: "0_1" } Tue Apr 27 14:05:39 done for 2 records 0.638secs Tue Apr 27 14:05:39 insert random_db_name.system.indexes 638ms Tue Apr 27 14:05:39 building new index on { _id: 1 } for random_db_name.tmp.mr.mapreduce_1272377138_2 Tue Apr 27 14:05:39 Buildindex random_db_name.tmp.mr.mapreduce_1272377138_2 idxNo:0 { name: "_id_", ns: "random_db_name.tmp.mr.mapreduce_1272377138_2", key: { _id: 1 } } Tue Apr 27 14:05:40 done for 0 records 0.226secs Tue Apr 27 14:05:40 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:40 CMD: drop random_db_name.tmp.mr.mapreduce_1272377138_2_inc Tue Apr 27 14:05:40 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:40 query random_db_name.$cmd ntoreturn:1 command reslen:176 bytes:160 171ms Tue Apr 27 14:05:40 repl: sleep 3sec before next pass Tue Apr 27 14:05:40 query random_db_name.$cmd ntoreturn:1 command reslen:157 bytes:141 1657ms Tue Apr 27 14:05:41 connection accepted from 10.250.74.65:36195 #129670 Tue Apr 27 14:05:41 connection accepted from 10.250.74.65:36196 #129671 Tue Apr 27 14:05:44 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:44 update random_db_name.random_collection_name_2 query: { _id: "4bd6ef1a57006601df0000d4" } nscanned:1 18 key updates 3082ms Tue Apr 27 14:05:44 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:44 repl: sleep 3sec before next pass Tue Apr 27 14:05:44 connection accepted from 10.250.118.178:44838 #129672 Tue Apr 27 14:05:44 connection accepted from 10.250.118.178:44839 #129673 Tue Apr 27 14:05:47 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:47 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:47 repl: sleep 3sec before next pass Tue Apr 27 14:05:47 end connection 10.250.74.65:36196 Tue Apr 27 14:05:48 connection accepted from 10.250.74.240:55238 #129674 Tue Apr 27 14:05:48 connection accepted from 10.250.74.240:55239 #129675 Tue Apr 27 14:05:50 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:50 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:50 repl: sleep 3sec before next pass Tue Apr 27 14:05:50 end connection 10.250.118.178:44839 Tue Apr 27 14:05:51 end connection 10.250.74.65:36094 Tue Apr 27 14:05:51 end connection 10.250.74.65:36113 Tue Apr 27 14:05:52 end connection 10.250.74.240:55239 Tue Apr 27 14:05:53 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:53 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:53 repl: sleep 3sec before next pass Tue Apr 27 14:05:56 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:56 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:56 repl: sleep 3sec before next pass Tue Apr 27 14:05:57 end connection 10.250.74.65:36173 Tue Apr 27 14:05:57 end connection 10.250.74.65:36195 Tue Apr 27 14:05:57 end connection 10.250.118.178:44815 Tue Apr 27 14:05:57 end connection 10.250.118.178:44832 Tue Apr 27 14:05:57 end connection 10.250.118.178:44838 Tue Apr 27 14:05:58 end connection 10.250.74.240:55226 Tue Apr 27 14:05:58 end connection 10.250.74.240:55238 Tue Apr 27 14:05:58 CMD: drop random_db_name.tmp.mr.mapreduce_1272377158_3 Tue Apr 27 14:05:58 CMD: drop random_db_name.tmp.mr.mapreduce_1272377158_3_inc Tue Apr 27 14:05:58 building new index on { 0: 1 } for random_db_name.tmp.mr.mapreduce_1272377158_3_inc Tue Apr 27 14:05:58 Buildindex random_db_name.tmp.mr.mapreduce_1272377158_3_inc idxNo:0 { ns: "random_db_name.tmp.mr.mapreduce_1272377158_3_inc", key: { 0: 1 }, name: "0_1" } Tue Apr 27 14:05:58 done for 2 records 0.088secs Tue Apr 27 14:05:58 building new index on { _id: 1 } for random_db_name.tmp.mr.mapreduce_1272377158_3 Tue Apr 27 14:05:58 Buildindex random_db_name.tmp.mr.mapreduce_1272377158_3 idxNo:0 { name: "_id_", ns: "random_db_name.tmp.mr.mapreduce_1272377158_3", key: { _id: 1 } } Tue Apr 27 14:05:58 done for 0 records 0.078secs Tue Apr 27 14:05:58 CMD: drop random_db_name.tmp.mr.mapreduce_1272377158_3_inc Tue Apr 27 14:05:59 query random_db_name.$cmd ntoreturn:1 command reslen:176 bytes:160 112ms Tue Apr 27 14:05:59 query random_db_name.$cmd ntoreturn:1 command reslen:157 bytes:141 475ms Tue Apr 27 14:05:59 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:05:59 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:05:59 repl: sleep 3sec before next pass Tue Apr 27 14:05:59 connection accepted from 10.250.74.240:55243 #129676 Tue Apr 27 14:05:59 connection accepted from 10.250.74.240:55244 #129677 Tue Apr 27 14:06:01 end connection 10.250.74.65:36076 Tue Apr 27 14:06:01 end connection 10.250.74.65:36127 Tue Apr 27 14:06:02 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:02 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:02 repl: sleep 3sec before next pass Tue Apr 27 14:06:04 connection accepted from 10.250.118.178:44853 #129678 Tue Apr 27 14:06:04 connection accepted from 10.250.118.178:44854 #129679 Tue Apr 27 14:06:05 end connection 10.250.74.65:36119 Tue Apr 27 14:06:05 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:05 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:05 repl: sleep 3sec before next pass Tue Apr 27 14:06:05 end connection 10.250.74.240:55244 Tue Apr 27 14:06:07 end connection 10.250.118.178:44854 Tue Apr 27 14:06:08 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:08 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:08 repl: sleep 3sec before next pass Tue Apr 27 14:06:09 connection accepted from 10.214.251.128:44783 #129680 Tue Apr 27 14:06:11 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:11 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:11 repl: sleep 3sec before next pass Tue Apr 27 14:06:13 connection accepted from 10.211.67.79:50060 #129681 Tue Apr 27 14:06:14 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:14 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:14 repl: sleep 3sec before next pass Tue Apr 27 14:06:14 connection accepted from 10.250.74.240:55249 #129682 Tue Apr 27 14:06:14 connection accepted from 10.250.74.240:55250 #129683 Tue Apr 27 14:06:17 end connection 10.211.67.79:49955 Tue Apr 27 14:06:17 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:17 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:17 repl: sleep 3sec before next pass Tue Apr 27 14:06:20 connection accepted from 10.250.118.178:44863 #129684 Tue Apr 27 14:06:20 connection accepted from 10.250.118.178:44864 #129685 Tue Apr 27 14:06:20 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:20 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:20 repl: sleep 3sec before next pass Tue Apr 27 14:06:20 connection accepted from 10.192.203.32:37320 #129686 Tue Apr 27 14:06:20 end connection 10.250.74.240:55250 Tue Apr 27 14:06:22 end connection 10.250.118.178:44864 Tue Apr 27 14:06:23 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:23 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:23 repl: sleep 3sec before next pass Tue Apr 27 14:06:24 CMD: drop random_db_name.tmp.mr.mapreduce_1272377184_4 Tue Apr 27 14:06:24 CMD: drop random_db_name.tmp.mr.mapreduce_1272377184_4_inc Tue Apr 27 14:06:24 query random_db_name.$cmd ntoreturn:1 command reslen:78 bytes:62 145ms Tue Apr 27 14:06:24 building new index on { 0: 1 } for random_db_name.tmp.mr.mapreduce_1272377184_4_inc Tue Apr 27 14:06:24 Buildindex random_db_name.tmp.mr.mapreduce_1272377184_4_inc idxNo:0 { ns: "random_db_name.tmp.mr.mapreduce_1272377184_4_inc", key: { 0: 1 }, name: "0_1" } Tue Apr 27 14:06:24 done for 2 records 0.04secs Tue Apr 27 14:06:24 building new index on { _id: 1 } for random_db_name.tmp.mr.mapreduce_1272377184_4 Tue Apr 27 14:06:24 Buildindex random_db_name.tmp.mr.mapreduce_1272377184_4 idxNo:0 { name: "_id_", ns: "random_db_name.tmp.mr.mapreduce_1272377184_4", key: { _id: 1 } } Tue Apr 27 14:06:25 done for 0 records 0.128secs Tue Apr 27 14:06:25 CMD: drop random_db_name.tmp.mr.mapreduce_1272377184_4_inc Tue Apr 27 14:06:25 query random_db_name.$cmd ntoreturn:1 command reslen:157 bytes:141 511ms Tue Apr 27 14:06:26 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:26 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:26 repl: sleep 3sec before next pass Tue Apr 27 14:06:29 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:29 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:29 repl: sleep 3sec before next pass Tue Apr 27 14:06:29 connection accepted from 10.250.118.178:44868 #129687 Tue Apr 27 14:06:29 connection accepted from 10.250.118.178:44869 #129688 Tue Apr 27 14:06:29 connection accepted from 10.211.67.79:50095 #129689 Tue Apr 27 14:06:32 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:32 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:32 repl: sleep 3sec before next pass Tue Apr 27 14:06:35 CMD: drop random_db_name.tmp.mr.mapreduce_1272377195_5 Tue Apr 27 14:06:35 CMD: drop random_db_name.tmp.mr.mapreduce_1272377195_5_inc Tue Apr 27 14:06:35 end connection 10.250.74.240:55243 Tue Apr 27 14:06:35 end connection 10.250.74.240:55249 Tue Apr 27 14:06:35 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:35 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:35 repl: sleep 3sec before next pass Tue Apr 27 14:06:35 end connection 10.250.118.178:44869 Tue Apr 27 14:06:36 connection accepted from 10.250.74.240:55261 #129690 Tue Apr 27 14:06:36 connection accepted from 10.250.74.240:55262 #129691 Tue Apr 27 14:06:37 building new index on { 0: 1 } for random_db_name.tmp.mr.mapreduce_1272377195_5_inc Tue Apr 27 14:06:37 Buildindex random_db_name.tmp.mr.mapreduce_1272377195_5_inc idxNo:0 { ns: "random_db_name.tmp.mr.mapreduce_1272377195_5_inc", key: { 0: 1 }, name: "0_1" } Tue Apr 27 14:06:37 done for 3 records 0.037secs Tue Apr 27 14:06:37 building new index on { _id: 1 } for random_db_name.tmp.mr.mapreduce_1272377195_5 Tue Apr 27 14:06:37 Buildindex random_db_name.tmp.mr.mapreduce_1272377195_5 idxNo:0 { name: "_id_", ns: "random_db_name.tmp.mr.mapreduce_1272377195_5", key: { _id: 1 } } Tue Apr 27 14:06:38 done for 0 records 0.047secs Tue Apr 27 14:06:38 CMD: drop random_db_name.tmp.mr.mapreduce_1272377195_5_inc Tue Apr 27 14:06:38 query random_db_name.$cmd ntoreturn:1 command reslen:157 bytes:141 2854ms Tue Apr 27 14:06:38 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:38 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:38 repl: sleep 3sec before next pass Tue Apr 27 14:06:41 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:41 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:41 repl: sleep 3sec before next pass Tue Apr 27 14:06:42 end connection 10.250.74.240:55262 Tue Apr 27 14:06:43 end connection 10.192.203.32:37320 Tue Apr 27 14:06:44 connection accepted from 10.250.118.178:44874 #129692 Tue Apr 27 14:06:44 connection accepted from 10.250.118.178:44875 #129693 Tue Apr 27 14:06:44 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:44 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:44 repl: sleep 3sec before next pass Tue Apr 27 14:06:47 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:47 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:47 repl: sleep 3sec before next pass Tue Apr 27 14:06:48 end connection 10.211.67.79:50060 Tue Apr 27 14:06:50 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:50 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:50 repl: sleep 3sec before next pass Tue Apr 27 14:06:51 end connection 10.250.118.178:44875 Tue Apr 27 14:06:52 connection accepted from 10.250.74.240:55273 #129694 Tue Apr 27 14:06:52 end connection 10.250.74.240:55261 Tue Apr 27 14:06:52 end connection 10.250.74.240:55273 Tue Apr 27 14:06:52 connection accepted from 10.250.74.240:55274 #129695 Tue Apr 27 14:06:53 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:53 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:53 repl: sleep 3sec before next pass Tue Apr 27 14:06:56 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:56 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:56 repl: sleep 3sec before next pass Tue Apr 27 14:06:57 end connection 10.250.74.240:55274 Tue Apr 27 14:06:59 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:06:59 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:06:59 repl: sleep 3sec before next pass Tue Apr 27 14:07:00 connection accepted from 10.250.118.178:44882 #129696 Tue Apr 27 14:07:00 connection accepted from 10.250.118.178:44883 #129697 Tue Apr 27 14:07:00 CMD: drop random_db_name.tmp.mr.mapreduce_1272377220_6 Tue Apr 27 14:07:00 CMD: drop random_db_name.tmp.mr.mapreduce_1272377220_6_inc Tue Apr 27 14:07:03 repl: from host:production-shard1-002.silentale.net 2500/6219 40% Tue Apr 27 14:07:03 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:07:03 repl: sleep 3sec before next pass Tue Apr 27 14:07:03 end connection 10.250.118.178:44883 decode failed. probably invalid utf-8 string ["c?cilegigi63"@orange.fr] why: TypeError: malformed UTF-8 character sequence at offset 2 Tue Apr 27 14:07:03 mr failed, removing collection Tue Apr 27 14:07:03 CMD: drop random_db_name.tmp.mr.mapreduce_1272377220_6 Tue Apr 27 14:07:03 CMD: drop random_db_name.tmp.mr.mapreduce_1272377220_6_inc Tue Apr 27 14:07:03 query random_db_name.$cmd ntoreturn:1 command reslen:89 bytes:73 2857ms Tue Apr 27 14:07:06 connection accepted from 10.214.251.128:45140 #129698 Tue Apr 27 14:07:06 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:07:06 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:07:06 repl: sleep 3sec before next pass Tue Apr 27 14:07:08 connection accepted from 10.250.118.178:44887 #129699 Tue Apr 27 14:07:08 connection accepted from 10.250.118.178:44888 #129700 Tue Apr 27 14:07:08 end connection 10.250.118.178:44888 Tue Apr 27 14:07:09 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:07:09 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:07:09 repl: sleep 3sec before next pass Tue Apr 27 14:07:12 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:07:12 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:07:12 repl: sleep 3sec before next pass Tue Apr 27 14:07:15 repl: from host:production-shard1-002.silentale.net Tue Apr 27 14:07:15 repl: couldn't connect to server production-shard1-002.silentale.net 10.192.229.187:27017 Tue Apr 27 14:07:15 repl: sleep 3sec before next pass Tue Apr 27 14:07:15 connection accepted from 10.250.118.178:44894 #129701 Tue Apr 27 14:07:15 connection accepted from 10.250.118.178:44895 #129702 Tue Apr 27 14:07:16 end connection 10.214.251.128:44783 Tue Apr 27 14:07:18 CMD: drop random_db_name.tmp.mr.mapreduce_1272377238_7 Tue Apr 27 14:07:18 CMD: drop random_db_name.tmp.mr.mapreduce_1272377238_7_inc Tue Apr 27 14:07:18 Got signal: 11 (Segmentation fault). Tue Apr 27 14:07:18 Backtrace: 0x6a8309 0x3303c322a0 0x7400f9 0x7a02c3 0x79dbbb 0x71388b 0x713b5a 0x714c4f 0x714bdc 0x714ac5 0x54d691 0x54f433 0x649bdc 0x67e441 0x67f347 0x55662f 0x55a6cc 0x5f310a 0x5f8137 0x6a9894 /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x6a8309] /lib64/libc.so.6 [0x3303c322a0] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(js_EmitTree+0x420f) [0x7400f9] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod [0x7a02c3] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(js_CompileTokenStream+0xee) [0x79dbbb] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod [0x71388b] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(JS_CompileUCScriptForPrincipals+0x83) [0x713b5a] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(JS_EvaluateUCScriptForPrincipals+0x71) [0x714c4f] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(JS_EvaluateUCScript+0x5a) [0x714bdc] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(JS_EvaluateScript+0x80) [0x714ac5] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo7SMScope4execERKSsS2_bbbi+0x101) [0x54d691] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo7SMScope12injectNativeEPKcPFNS_7BSONObjERKS3_E+0x253) [0x54f433] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo2mr16MapReduceCommand3runEPKcRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x27c) [0x649bdc] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x821) [0x67e441] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x4d7) [0x67f347] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x3f) [0x55662f] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpE+0x116c) [0x55a6cc] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod [0x5f310a] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERK11sockaddr_in+0x1227) [0x5f8137] /opt/mongodb-linux-x86_64-1.4.1/bin/mongod(_ZN5mongo10connThreadEv+0x244) [0x6a9894] Tue Apr 27 14:07:18 dbexit: Tue Apr 27 14:07:18 shutdown: going to close listening sockets... Tue Apr 27 14:07:18 going to close listening socket: 16 Tue Apr 27 14:07:18 going to close listening socket: 18 Tue Apr 27 14:07:18 shutdown: going to flush oplog... Tue Apr 27 14:07:18 shutdown: going to close sockets... Tue Apr 27 14:07:18 shutdown: waiting for fs preallocator... Tue Apr 27 14:07:18 shutdown: closing all files... Tue Apr 27 14:07:19 msync errno:12 Cannot allocate memory Tue Apr 27 14:07:19 msync errno:12 Cannot allocate memory 117/237 49% 173/237 72% Tue Apr 27 14:07:23 end connection 10.250.118.178:44895 Tue Apr 27 14:07:23 closeAllFiles() finished Tue Apr 27 14:07:23 shutdown: removing fs lock... Tue Apr 27 14:07:23 dbexit: really exiting now ERROR: Client::~Client _context should be NULL: conn Tue Apr 27 14:07:23 connection accepted from 10.250.74.240:55288 #129703 Tue Apr 27 14:07:23 Got signal: 11 (Segmentation fault).