-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: 2.0.3
-
Component/s: MapReduce
-
None
-
Environment:RHEL 5.8
-
Linux
Hello all,
I have been using Mongo in non-sharded mode for quite sometime and have just recently tried my hand at sharding data. To do so I have 3 machines setup as mongod with 1 of them also serving as my mongos and config server.
I have ~212 thousand documents in the sharded collection but each document has what can be a pretty long array.
This dataset was working fine for about a day but after doing an update where I added a new field to most of the documents I am getting the following error:
Thu Mar 29 14:56:22 [conn5166] Assertion: 10334:Invalid BSONObj size: 18303487 (0xFF491701) first element: 0:
{ count: 6294467.0 }The complete log from mongos is here:
Thu Mar 29 14:56:22 [conn5166] Assertion: 10334:Invalid BSONObj size: 18303487 (0xFF491701) first element: 0: { count: 6294467.0 }
0x584432 0x508461 0x9bacbd 0x9bb0c8 0x9b3e87 0x9b43a5 0x9ca9bb 0x97cc14 0x97e20f 0x940e25 0x9441b1 0x8869d7 0x88df49 0xaa37d6 0x637497 0x3d32e0677d 0x3d31ed49ad
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x112) [0x584432]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x471) [0x508461]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr9JSReducer7_reduceERKSt6vectorINS_7BSONObjESaIS3_EERS3_Ri+0xf2d) [0x9bacbd]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr9JSReducer6reduceERKSt6vectorINS_7BSONObjESaIS3_EE+0x88) [0x9bb0c8]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr5State14reduceInMemoryEv+0xe7) [0x9b3e87]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr5State9checkSizeEv+0xf5) [0x9b43a5]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr16MapReduceCommand3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0xa1b) [0x9ca9bb]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6a4) [0x97cc14]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6ff) [0x97e20f]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x35) [0x940e25]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x11e1) [0x9441b1]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod [0x8869d7]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x559) [0x88df49]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x76) [0xaa37d6]
/export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x287) [0x637497]
/lib64/libpthread.so.0 [0x3d32e0677d]
/lib64/libc.so.6(clone+0x6d) [0x3d31ed49ad]
Thu Mar 29 14:56:22 [conn5166] warning: ClientCursor::YieldLock not closed properly
Thu Mar 29 14:56:22 [conn5166] mr failed, removing collection :: caused by :: 10334 Invalid BSONObj size: 18303487 (0xFF491701) first element: 0: { count: 6294467.0 }
Thu Mar 29 14:56:22 [conn5166] CMD: drop lgt8.tmp.mr.bwa_mapping_49
Thu Mar 29 14:56:22 [conn5166] CMD: drop lgt8.tmp.mr.bwa_mapping_49_inc
Thu Mar 29 14:56:22 [conn5166] command lgt8.$cmd command: { mapreduce: "bwa_mapping", map: " function() {
var conds = [];
var thiselm = this;
this.hits.forEach(
function(h) {
var good = t...", reduce: " function(key,values) {
var result = {count:0.0};
values.forEach(function(value) {
result.count += value.count;
...", query: {}, out: "tmp.mrs.bwa_mapping_1333047154_116" } ntoreturn:1 exception: Invalid BSONObj size: 18303487 (0xFF491701) first element: 0: { count: 6294467.0 } code:10334 reslen:187 228442ms
I have tried running repairDatabase(), which ran successfully but does not take care of the problem. I have also run validate() on the collection and validate() reports everything as OK (even when run in verbose mode). I am able to query this database without error however I am unable to perform the map reduce described above (which is counting a subset of the array elements and doing a group by). I am able to perform a map reduce on the elements of the document NOT contained in the array.
I am not sure how much of this information is helpful but I thought I would include it just in case.
Any help on this would be greatly appreciated.
- related to
-
SERVER-6750 Clarify message for error 10334 - Invalid BSONObj error when doing Map/Reduce
-
- Closed
-