We have 6 servers in replica set, accessed using C++ and python drivers.
One of servers sometimes throw in logs assertions like this:
Thu Jan 17 11:00:43 [conn103] Assertion: 10334:Invalid BSONObj size: 1769280243 (0xF3127569) first element: : ?type=100
0xaffd31 0xac5eb9 0xac603c 0x572567 0x66f857 0x84c46b 0x84fcdd 0x85145f 0x851543 0x83dcf3 0x841fc1 0x844a27 0x7ade1f 0x7b4284 0x5703f2 0xaedfc1 0x355a607851 0x355a2e811d
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xaffd31]
/usr/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xac5eb9]
/usr/bin/mongod() [0xac603c]
/usr/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x497) [0x572567]
/usr/bin/mongod(_ZNK5mongo11BSONElement14embeddedObjectEv+0xa7) [0x66f857]
/usr/bin/mongod(_ZNK5mongo3Mod5applyERNS_15BSONBuilderBaseENS_11BSONElementERNS_8ModStateE+0x191b) [0x84c46b]
/usr/bin/mongod(_ZN5mongo11ModSetState17createNewFromModsERKSsRNS_15BSONBuilderBaseERNS_18BSONIteratorSortedERKSt4pairIKSt17_Rb_tree_iteratorIS7_IS1_N5boost10shared_ptrINS_8ModStateEEEEESF_ERKNS_9LexNumCmpE+0x4dd)
[0x84fcdd]
/usr/bin/mongod(_ZN5mongo11ModSetState20createNewObjFromModsERKSsRNS_14BSONObjBuilderERKNS_7BSONObjE+0x7f) [0x85145f]
/usr/bin/mongod(_ZN5mongo11ModSetState17createNewFromModsEv+0x63) [0x851543]
/usr/bin/mongod() [0x83dcf3]
/usr/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x3081) [0x841fc1]
/usr/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0x844a27]
/usr/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4cf) [0x7ade1f]
/usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xbd4) [0x7b4284]
/usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x82) [0x5703f2]
/usr/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x411) [0xaedfc1]
/lib64/libpthread.so.0() [0x355a607851]
/lib64/libc.so.6(clone+0x6d) [0x355a2e811d]
Thu Jan 17 11:00:43 [conn103] Assertion: 10329:Element too large
0xaffd31 0xac5eb9 0x5745c7 0x572fdb 0x65b17f 0x7b3aeb 0x5703f2 0xaedfc1 0x355a607851 0x355a2e811d
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xaffd31]
/usr/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xac5eb9]
/usr/bin/mongod(_ZNK5mongo7BSONObj8toStringERNS_17StringBuilderImplINS_16TrivialAllocatorEEEbbi+0x317) [0x5745c7]
/usr/bin/mongod(_ZNK5mongo11BSONElement8toStringERNS_17StringBuilderImplINS_16TrivialAllocatorEEEbbi+0x69b) [0x572fdb]
/usr/bin/mongod(_ZNK5mongo7OpDebug6reportERKNS_5CurOpE+0xaef) [0x65b17f]
/usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x43b) [0x7b3aeb]
/usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x82) [0x5703f2]
/usr/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x411) [0xaedfc1]
/lib64/libpthread.so.0() [0x355a607851]
/lib64/libc.so.6(clone+0x6d) [0x355a2e811d]
We tracked down to piece of code which causes this, but it consists of 3 queries within single connections:
conn->insert( "collTickets", ticketObj );
conn->getLastError(); // ensure insertion was done
conn->update( "userInfo", BSON( "_id" << userId ), BSON( "$inc" << BSON( "ticketCount" << 1 ) ), false/update mode/ );
conn->update( "ticketAcks", BSON( "_id" << mongo::OID( ackIdStr ) ), BSON( "$set" << BSON( "lastTicket" << ticketOdj[ "_id" ].OID().str() << "accountant" << managerId ) ) );
conn.done();
objects sizes are guaranteed to be much less than 16MB:
truck:PRIMARY> Object.bsonsize( db.userInfo.findOne( { _id:1842412 } ) )
43705
truck:PRIMARY> db.collTickets.find( {userId:1842412} ).forEach( function ( e ) { print(Object.bsonsize(e));} )
255
356
truck:PRIMARY> db.ticketAcks.find( {toUserId:1842412} ).forEach( function ( e ) { print(Object.bsonsize(e));} )
269
6732
6178
6414
6334
6100
truck:PRIMARY> db.version()
2.2.2
What can cause this assertion failures?
How we can avoid them?
- is duplicated by
-
SERVER-8236 Assertion in mongo log: 10334:Invalid BSONObj size: 1769280243 (0xF3127569) first element: : ?type=100
-
- Closed
-