Major - P3 We have a replicaset running on 2 servers (+1 arbiter), but in time to time we start getting the assertion 10334 error without any modifications.
Currently our way to solve the issue is to remove the primary server (switching automatically to the secondary one), and start from scratch the syncing to add again the replicaset.
This works, but also is not at all scalable and not something that is useful.
When this error starts, it's not for a particular row, but any new updated row in any of the collections (we have only one database).
Primary mongo.log
Tue Mar 5 09:56:29 [conn311369] update fbridge.activity_feed query: { _id: ObjectId('511e4130a298d8d31600071e') } update: { user_id: 186564, created_at: new Date(1360937264084), updated_at: new Date(1362495389507), network: "youtube", source: "total_views", external_id: "9223372036854775807", data: { username: "jayz", url: "https://www.youtube.com/channel/UCN-sc1xJr-QQNj_uNIM9wTA", profile_pic: "https://i3.ytimg.com/i/N-sc1xJr-QQNj_uNIM9wTA/1.jpg?v=506e1b60", channel_id: "UCN-sc1xJr-QQNj_uNIM9wTA", user_id: "N-sc1xJr-QQNj_uNIM9wTA", title: "profile stats", importance: 21, total: 346413303 }, activities: [ { total: 345112999, datetime: new Date(1362053676294) }, { total: 345113211, datetime: new Date(1362060046295) }, { total: 345158210, datetime: new Date(1362066608364) }, { total: 345174676, datetime: new Date(1362069756052) }, { total: 345177823, datetime: new Date(1362072928729) }, { total: 345189802, datetime: new Date(1362079298240) }, { total: 345191299, datetime: new Date(1362085671950) }, { total: 345214768, datetime: new Date(1362095423013) }, { total: 345214953, datetime: new Date(1362098570659) }, { total: 345245828, datetime: new Date(1362101745768) }, { total: 345250603, datetime: new Date(1362104925365) }, { total: 345253486, datetime: new Date(1362108080900) }, { total: 345290143, datetime: new Date(1362117651783) }, { total: 345302522, datetime: new Date(1362121018881) }, { total: 345334617, datetime: new Date(1362124196967) }, { total: 345342162, datetime: new Date(1362127742719) }, { total: 345347896, datetime: new Date(1362130922444) }, { total: 345377353, datetime: new Date(1362134105852) }, { total: 345393550, datetime: new Date(1362137284890) }, { total: 345408136, datetime: new Date(1362146824899) }, { total: 345416329, datetime: new Date(1362150003392) }, { total: 345431641, datetime: new Date(1362156546651) }, { total: 345439207, datetime: new Date(1362159727780) }, { total: 345439307, datetime: new Date(1362162907416) }, { total: 345453340, datetime: new Date(1362166087432) }, { total: 345459572, datetime: new Date(1362169266274) }, { total: 345459604, datetime: new Date(1362175627141) }, { total: 345487907, datetime: new Date(1362178805641) }, { total: 345501425, datetime: new Date(1362182169302) }, { total: 345530937, datetime: new Date(1362188525718) }, { total: 345543997, datetime: new Date(1362191704769) }, { total: 345578692, datetime: new Date(1362198071059) }, { total: 345587409, datetime: new Date(1362204426821) }, { total: 345620199, datetime: new Date(1362210969752) }, { total: 345628177, datetime: new Date(1362214149516) }, { total: 345648694, datetime: new Date(1362220508954) }, { total: 345654209, datetime: new Date(1362223688273) }, { total: 345683776, datetime: new Date(1362233231826) }, { total: 345693756, datetime: new Date(1362236412325) }, { total: 345693844, datetime: new Date(1362239768187) }, { total: 345712253, datetime: new Date(1362242950823) }, { total: 345720069, datetime: new Date(1362246131435) }, { total: 345720168, datetime: new Date(1362249314338) }, { total: 345737934, datetime: new Date(1362252491362) }, { total: 345744975, datetime: new Date(1362255669723) }, { total: 345773190, datetime: new Date(1362265208480) }, { total: 345786343, datetime: new Date(1362268565761) }, { total: 345817953, datetime: new Date(1362274929301) }, { total: 345818396, datetime: new Date(1362278110393) }, { total: 345828157, datetime: new Date(1362281287184) }, { total: 345838128, datetime: new Date(1362284471147) }, { total: 345839445, datetime: new Date(1362290823824) }, { total: 345843025, datetime: new Date(1362294018386) }, { total: 345900246, datetime: new Date(1362297369007) }, { total: 345910111, datetime: new Date(1362300978751) }, { total: 345937570, datetime: new Date(1362308172725) }, { total: 345944584, datetime: new Date(1362311771273) }, { total: 345944652, datetime: new Date(1362315369671) }, { total: 345960847, datetime: new Date(1362318974098) }, { total: 345964878, datetime: new Date(1362322576553) }, { total: 345964970, datetime: new Date(1362326173992) }, { total: 345980870, datetime: new Date(1362329773503) }, { total: 345983468, datetime: new Date(1362333373759) }, { total: 345983568, datetime: new Date(1362336979005) }, { total: 346000246, datetime: new Date(1362340576749) }, { total: 346001927, datetime: new Date(1362344186565) }, { total: 346028843, datetime: new Date(1362351365920) }, { total: 346032100, datetime: new Date(1362354983101) }, { total: 346059841, datetime: new Date(1362362188001) }, { total: 346063090, datetime: new Date(1362365778292) }, { total: 346063676, datetime: new Date(1362369387496) }, { total: 346102159, datetime: new Date(1362372983589) }, { total: 346106516, datetime: new Date(1362376581997) }, { total: 346108354, datetime: new Date(1362380172428) }, { total: 346135377, datetime: new Date(1362383766845) }, { total: 346140482, datetime: new Date(1362387378939) }, { total: 346156283, datetime: new Date(1362394586447) }, { total: 346160413, datetime: new Date(1362398169435) }, { total: 346173117, datetime: new Date(1362405386582) }, { total: 346176202, datetime: new Date(1362408980407) }, { total: 346188884, datetime: new Date(1362416189380) }, { total: 346191124, datetime: new Date(1362419790815) }, { total: 346191313, datetime: new Date(1362423392031) }, { total: 346203399, datetime: new Date(1362426994065) }, { total: 346203731, datetime: new Date(1362430607995) }, { total: 346203782, datetime: new Date(1362434196104) }, { total: 346224473, datetime: new Date(1362437772275) }, { total: 346227228, datetime: new Date(1362441391493) }, { total: 346227382, datetime: new Date(1362445002461) }, { total: 346249015, datetime: new Date(1362448602163) }, { total: 346252020, datetime: new Date(1362452203186) }, { total: 346252021, datetime: new Date(1362455782426) }, { total: 346283645, datetime: new Date(1362459399052) }, { total: 346306223, datetime: new Date(1362462992079) }, { total: 346334751, datetime: new Date(1362470187140) }, { total: 346363460, datetime: new Date(1362473786489) }, { total: 346386791, datetime: new Date(1362480993367) }, { total: 346395653, datetime: new Date(1362484590510) }, { total: 346408901, datetime: new Date(1362491790792) }, { total: 346413303, datetime: new Date(1362495389507) } ], _id: ObjectId('511e4130a298d8d31600071e') } idhack:1 keyUpdates:1 exception: Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115 code:10334 locks(micros) w:4955 2ms
Tue Mar 5 09:56:29 [conn311367] Assertion: 10334:Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115
0xb07561 0xacd449 0xacd5cc 0x5721e8 0x796fed 0x874ac4 0x6583e4 0x814697 0x814988 0x861ace 0x824e7f 0x821779 0x842e39 0x8467c1 0x849227 0x7ae1ef 0x7b4654 0x56c688 0xaf57b5 0x31aa607851
/var/lib/mongo/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xb07561]
/var/lib/mongo/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xacd449]
/var/lib/mongo/bin/mongod() [0xacd5cc]
/var/lib/mongo/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x468) [0x5721e8]
/var/lib/mongo/bin/mongod(_ZN5mongo13unindexRecordEPNS_16NamespaceDetailsEPNS_6RecordERKNS_7DiskLocEb+0x11d) [0x796fed]
/var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbbb+0x204) [0x874ac4]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails11cappedAllocEPKci+0x4b4) [0x6583e4]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails6_allocEPKci+0x27) [0x814697]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE+0x38) [0x814988]
/var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci+0x6e) [0x861ace]
/var/lib/mongo/bin/mongod() [0x824e7f]
/var/lib/mongo/bin/mongod(_ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pbb+0x49) [0x821779]
/var/lib/mongo/bin/mongod() [0x842e39]
/var/lib/mongo/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x3081) [0x8467c1]
/var/lib/mongo/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0x849227]
/var/lib/mongo/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4cf) [0x7ae1ef]
/var/lib/mongo/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xbd4) [0x7b4654]
/var/lib/mongo/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x98) [0x56c688]
/var/lib/mongo/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x415) [0xaf57b5]
/lib64/libpthread.so.0() [0x31aa607851]
Tue Mar 5 09:56:29 [conn311367] update fbridge.activity_feed query: { _id: ObjectId('511e3e89a298d8c91300009d') } update: { user_id: 165710, created_at: new Date(1360936585660), updated_at: new Date(1362495389683), network: "twitter", source: "followers", external_id: "221918717", data: { title: "profile stats", importance: 21, total: 4597 }, activities: [ { total: 4452, datetime: new Date(1361010492725) }, { total: 4453, datetime: new Date(1361024889455) }, { total: 4454, datetime: new Date(1361039296622) }, { total: 4457, datetime: new Date(1361046500412) }, { total: 4459, datetime: new Date(1361053693225) }, { total: 4461, datetime: new Date(1361060906789) }, { total: 4462, datetime: new Date(1361075295338) }, { total: 4464, datetime: new Date(1361082486722) }, { total: 4465, datetime: new Date(1361089700348) }, { total: 4466, datetime: new Date(1361096884614) }, { total: 4467, datetime: new Date(1361118504998) }, { total: 4468, datetime: new Date(1361125693487) }, { total: 4469, datetime: new Date(1361140106779) }, { total: 4470, datetime: new Date(1361154504897) }, { total: 4473, datetime: new Date(1361161693609) }, { total: 4476, datetime: new Date(1361168890816) }, { total: 4477, datetime: new Date(1361204897505) }, { total: 4481, datetime: new Date(1361212116607) }, { total: 4482, datetime: new Date(1361219309225) }, { total: 4484, datetime: new Date(1361226511133) }, { total: 4485, datetime: new Date(1361234139730) }, { total: 4484, datetime: new Date(1361241769693) }, { total: 4485, datetime: new Date(1361249377018) }, { total: 4486, datetime: new Date(1361257011427) }, { total: 4487, datetime: new Date(1361279236204) }, { total: 4489, datetime: new Date(1361294516943) }, { total: 4490, datetime: new Date(1361302096782) }, { total: 4492, datetime: new Date(1361324961264) }, { total: 4493, datetime: new Date(1361378134986) }, { total: 4494, datetime: new Date(1361385741808) }, { total: 4495, datetime: new Date(1361400977422) }, { total: 4503, datetime: new Date(1361408424137) }, { total: 4504, datetime: new Date(1361416059541) }, { total: 4507, datetime: new Date(1361423673384) }, { total: 4508, datetime: new Date(1361453712116) }, { total: 4509, datetime: new Date(1361461353632) }, { total: 4510, datetime: new Date(1361468971037) }, { total: 4512, datetime: new Date(1361476593070) }, { total: 4513, datetime: new Date(1361491645504) }, { total: 4517, datetime: new Date(1361499284468) }, { total: 4518, datetime: new Date(1361514514185) }, { total: 4519, datetime: new Date(1361537360835) }, { total: 4520, datetime: new Date(1361544990425) }, { total: 4521, datetime: new Date(1361560068925) }, { total: 4522, datetime: new Date(1361575296421) }, { total: 4523, datetime: new Date(1361582904025) }, { total: 4524, datetime: new Date(1361590546730) }, { total: 4526, datetime: new Date(1361598149470) }, { total: 4527, datetime: new Date(1361605752264) }, { total: 4528, datetime: new Date(1361643709644) }, { total: 4529, datetime: new Date(1361658924566) }, { total: 4528, datetime: new Date(1361666549888) }, { total: 4532, datetime: new Date(1361674191897) }, { total: 4533, datetime: new Date(1361719701036) }, { total: 4532, datetime: new Date(1361734978101) }, { total: 4534, datetime: new Date(1361742635957) }, { total: 4537, datetime: new Date(1361765283703) }, { total: 4538, datetime: new Date(1361772870508) }, { total: 4539, datetime: new Date(1361780486230) }, { total: 4541, datetime: new Date(1361803354597) }, { total: 4546, datetime: new Date(1361818588450) }, { total: 4545, datetime: new Date(1361841298843) }, { total: 4548, datetime: new Date(1361848907518) }, { total: 4547, datetime: new Date(1361864165414) }, { total: 4548, datetime: new Date(1361879375078) }, { total: 4549, datetime: new Date(1361887010507) }, { total: 4553, datetime: new Date(1361894458765) }, { total: 4555, datetime: new Date(1361902055273) }, { total: 4557, datetime: new Date(1361909703566) }, { total: 4558, datetime: new Date(1361940187582) }, { total: 4559, datetime: new Date(1361962845856) }, { total: 4561, datetime: new Date(1361993312405) }, { total: 4562, datetime: new Date(1362000959330) }, { total: 4563, datetime: new Date(1362023800453) }, { total: 4564, datetime: new Date(1362061714245) }, { total: 4565, datetime: new Date(1362069352405) }, { total: 4566, datetime: new Date(1362076955801) }, { total: 4567, datetime: new Date(1362084589182) }, { total: 4568, datetime: new Date(1362099648350) }, { total: 4569, datetime: new Date(1362122504388) }, { total: 4571, datetime: new Date(1362168063232) }, { total: 4572, datetime: new Date(1362183326315) }, { total: 4573, datetime: new Date(1362190922410) }, { total: 4578, datetime: new Date(1362259322264) }, { total: 4579, datetime: new Date(1362266939694) }, { total: 4580, datetime: new Date(1362274536657) }, { total: 4582, datetime: new Date(1362282152885) }, { total: 4585, datetime: new Date(1362289805683) }, { total: 4586, datetime: new Date(1362320476631) }, { total: 4585, datetime: new Date(1362328105123) }, { total: 4586, datetime: new Date(1362335714252) }, { total: 4587, datetime: new Date(1362350961394) }, { total: 4589, datetime: new Date(1362358579732) }, { total: 4592, datetime: new Date(1362366036740) }, { total: 4593, datetime: new Date(1362373627602) }, { total: 4592, datetime: new Date(1362388885446) }, { total: 4593, datetime: new Date(1362434436199) }, { total: 4595, datetime: new Date(1362457286968) }, { total: 4596, datetime: new Date(1362464890616) }, { total: 4597, datetime: new Date(1362495389683) } ], _id: ObjectId('511e3e89a298d8c91300009d') } idhack:1 keyUpdates:1 exception: Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115 code:10334 locks(micros) w:5091 2ms
Tue Mar 5 09:56:29 [conn311369] Assertion: 10334:Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115
0xb07561 0xacd449 0xacd5cc 0x5721e8 0x796fed 0x874ac4 0x6583e4 0x814697 0x814988 0x861ace 0x824e7f 0x821779 0x842e39 0x8467c1 0x849227 0x7ae1ef 0x7b4654 0x56c688 0xaf57b5 0x31aa607851
/var/lib/mongo/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xb07561]
/var/lib/mongo/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xacd449]
/var/lib/mongo/bin/mongod() [0xacd5cc]
/var/lib/mongo/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x468) [0x5721e8]
/var/lib/mongo/bin/mongod(_ZN5mongo13unindexRecordEPNS_16NamespaceDetailsEPNS_6RecordERKNS_7DiskLocEb+0x11d) [0x796fed]
/var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbbb+0x204) [0x874ac4]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails11cappedAllocEPKci+0x4b4) [0x6583e4]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails6_allocEPKci+0x27) [0x814697]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE+0x38) [0x814988]
/var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci+0x6e) [0x861ace]
/var/lib/mongo/bin/mongod() [0x824e7f]
/var/lib/mongo/bin/mongod(_ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pbb+0x49) [0x821779]
/var/lib/mongo/bin/mongod() [0x842e39]
/var/lib/mongo/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x3081) [0x8467c1]
/var/lib/mongo/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0x849227]
/var/lib/mongo/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4cf) [0x7ae1ef]
/var/lib/mongo/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xbd4) [0x7b4654]
/var/lib/mongo/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x98) [0x56c688]
/var/lib/mongo/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x415) [0xaf57b5]
/lib64/libpthread.so.0() [0x31aa607851]
Tue Mar 5 09:56:29 [conn311369] update fbridge.activity_feed query: { _id: ObjectId('51200078a298d8bf160023a3') } update: { user_id: 187354, created_at: new Date(1361051768998), updated_at: new Date(1362495389898), network: "facebook", source: "likes", external_id: "354824454594731", data: { title: "profile stats", importance: 21, total: 1072 }, activities: [ { total: 1064, datetime: new Date(1361051768998) }, { total: 1065, datetime: new Date(1361068506727) }, { total: 1066, datetime: new Date(1361118918627) }, { total: 1067, datetime: new Date(1361135663429) }, { total: 1069, datetime: new Date(1361186081004) }, { total: 1070, datetime: new Date(1361203005482) }, { total: 1069, datetime: new Date(1361236868640) }, { total: 1068, datetime: new Date(1361287280168) }, { total: 1069, datetime: new Date(1361337708743) }, { total: 1070, datetime: new Date(1361405224516) }, { total: 1071, datetime: new Date(1361472560335) }, { total: 1072, datetime: new Date(1361506051164) }, { total: 1073, datetime: new Date(1361556457661) }, { total: 1074, datetime: new Date(1361573765574) }, { total: 1075, datetime: new Date(1361624185245) }, { total: 1076, datetime: new Date(1361674584233) }, { total: 1077, datetime: new Date(1361725036623) }, { total: 1079, datetime: new Date(1361809267815) }, { total: 1081, datetime: new Date(1361842935425) }, { total: 1082, datetime: new Date(1361859684096) }, { total: 1083, datetime: new Date(1361910101728) }, { total: 1084, datetime: new Date(1361994547900) }, { total: 1079, datetime: new Date(1362011722643) }, { total: 1081, datetime: new Date(1362028875509) }, { total: 1082, datetime: new Date(1362062566107) }, { total: 1079, datetime: new Date(1362079283567) }, { total: 1070, datetime: new Date(1362096090636) }, { total: 1072, datetime: new Date(1362113390600) }, { total: 1074, datetime: new Date(1362148086695) }, { total: 1072, datetime: new Date(1362165253631) }, { total: 1071, datetime: new Date(1362251247893) }, { total: 1072, datetime: new Date(1362268582590) }, { total: 1071, datetime: new Date(1362337255954) }, { total: 1068, datetime: new Date(1362354616010) }, { total: 1069, datetime: new Date(1362389705872) }, { total: 1070, datetime: new Date(1362460042510) }, { total: 1072, datetime: new Date(1362495389898) } ], _id: ObjectId('51200078a298d8bf160023a3') } idhack:1 nmoved:1 keyUpdates:0 exception: Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115 code:10334 locks(micros) w:5151 2ms
Tue Mar 5 09:56:30 [conn339077] Assertion: 10334:Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115
0xb07561 0xacd449 0xacd5cc 0x5721e8 0x796fed 0x874ac4 0x6583e4 0x814697 0x814988 0x861ace 0x824e7f 0x821779 0x7ac5b2 0x7acdd9 0x7b4644 0x56c688 0xaf57b5 0x31aa607851 0x31aa2e811d
/var/lib/mongo/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xb07561]
/var/lib/mongo/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xacd449]
/var/lib/mongo/bin/mongod() [0xacd5cc]
/var/lib/mongo/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x468) [0x5721e8]
/var/lib/mongo/bin/mongod(_ZN5mongo13unindexRecordEPNS_16NamespaceDetailsEPNS_6RecordERKNS_7DiskLocEb+0x11d) [0x796fed]
/var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbbb+0x204) [0x874ac4]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails11cappedAllocEPKci+0x4b4) [0x6583e4]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails6_allocEPKci+0x27) [0x814697]
/var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE+0x38) [0x814988]
/var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci+0x6e) [0x861ace]
/var/lib/mongo/bin/mongod() [0x824e7f]
/var/lib/mongo/bin/mongod(_ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pbb+0x49) [0x821779]
/var/lib/mongo/bin/mongod(_ZN5mongo14checkAndInsertEPKcRNS_7BSONObjE+0xc2) [0x7ac5b2]
/var/lib/mongo/bin/mongod(_ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE+0x739) [0x7acdd9]
/var/lib/mongo/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xbc4) [0x7b4644]
/var/lib/mongo/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x98) [0x56c688]
/var/lib/mongo/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x415) [0xaf57b5]
/lib64/libpthread.so.0() [0x31aa607851]
/lib64/libc.so.6(clone+0x6d) [0x31aa2e811d]