-
Type: Bug
-
Resolution: Cannot Reproduce
-
Priority: Critical - P2
-
None
-
Affects Version/s: 2.4.4
-
Component/s: Index Maintenance
-
Environment:Ubuntu 12.04 - On Digital Ocean - Single MongoDB node - Very low load, very small DB,
-
Linux
-
We got the following on mongodb.log:
Thu Dec 26 14:48:51.331 [conn265] update reef.companies query: { _id: ObjectId('521f7f99e4b00b44b9987a59') } update: { $push: { capitals: { legal: { proceeding: { re fid: null, state: "none" } }, id: "ecb646fa-9cb8-4a53-bdf0-1b4821ca4f4e", shareholders: [ { entity: { company: { refid: "52bc3f7fe4b07f4f0a49fc4a", name: "EOG Canada Holdings ULC (CANCELADA)" } }, amount_mo: 2125000000 }, { entity: { company: { refid: "52bc2d36e4b07f4f0a49fc29", name: "EOG Canada Investments ULC" } }, amount_mo: 6375000000 } ], total_mo: 8500000000, since_dd: new Date(1323226800000), inc_mo: 8500000000, is_statute_change: true, is_art299: true } } } idhack:1 keyUpdates:0 ex ception: btree: key+recloc already in index code:10287 locks(micros) w:1905 1ms
Then, right after that operation, the referenced document disappeared from the DB. Running db.companies.find({"_id" : ObjectId("521f7f99e4b00b44b9987a59")}); returns no results.
We tried db.runCommand({validate:"companies"}); and everything looks OK.
We also run a reIndex() on the collection. No errors. The document still gone.
We know the doc was lost due to the above error because that's exactly when it disappeared. And our application codebase does not include a single "remove/delete" operation (we only do "soft" deletes).
Additional info:
- HINT: We DO have textSearchEnabled=true
- It's running on a VM (on Digital Ocean)
- It's a single node installation
- Very very low load (a single user entering data via a UI)
- Only 15 documents on that collection
- The one being updated with $push (which was lost) was 3521 bytes
- Config file:
$ cat /etc/mongodb.conf logappend=true master=true setParameter=textSearchEnabled=true rest=true logpath=/var/log/mongodb/mongodb.log dbpath=/var/lib/mongodb
Right now we restored the lost document from our last backup. However, we are very concerned about losing data in the future. Specially since this is our first project using MongoDB and with very little volume and load we've already lost data.
Thanks