-
Type: Question
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: 3.6.15
-
Component/s: Performance
-
None
-
Fully Compatible
We need to delete a large amount of data (as an example from one of our collections we need to delete approximately 1 billion documents).
Every collection has an index(desc) on a createdAt field so we tried:
db.ourCollection.remove({'createdAt': {$lt: ISODate('2019-07-01')}})
Doing this deletes about 1 million documents every hour, which is extremely slow.
The replica set is continuously being written to.
How can we remove so much data more efficiently and reliably?
v3.6.15
Amazon Linux AMI release 2018.03