-
Type: Bug
-
Resolution: Cannot Reproduce
-
Priority: Critical - P2
-
None
-
Affects Version/s: 1.6.5
-
Component/s: None
-
None
-
Environment:Linux cloud-mongo22 2.6.32-28-server #55-Ubuntu SMP Mon Jan 10 23:57:16 UTC 2011 x86_64 GNU/Linux
2 shards each one with 3 replica members, 3 config servers, mongos running with tomcat API servers.
-
ALL
At least three times on different 1.6 versions we have gotten into this situation. Attached are the database files with the corrupt data.
The general flow is that the LEADERBOARD_ collections get created and dropped based on time and other variables, they last for about 24h.
There is a group() that is run on these tables every 5 minutes.
For example:
query ee895ca8156f4c7fa4104b1b4c9a8c38.$cmd ntoreturn:1 command: { group: { ns: "LEADEBOARD_xxxxxxxxxxx", key:
, cond: null, $reduce: "function(doc,prev)
{ prev.numEntries ++ }", initial:
{ numEntries: 0 }} } reslen:26486 1103ms
to generate some data on the rank.
Here is a sample of magnitude of the problem: 49111 collections with only 318 unique names:
echo "show collections" | mongo localhost:27017/ee895ca8156f4c7fa4104b1b4c9a8c38 | sort -u | wc -l
318
echo "show collections" | mongo localhost:27017/ee895ca8156f4c7fa4104b1b4c9a8c38 | wc -l
49111
show collections will show (partial output)
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l
LEADEBOARD_cityofwonder-10-2_l