Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-44819

Slow remove on large dataset

    • Type: Icon: Question Question
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.6.15
    • Component/s: Performance
    • None
    • Fully Compatible

      We need to delete a large amount of data (as an example from one of our collections we need to delete approximately 1 billion documents).
       
      Every collection has an index(desc) on a createdAt field so we tried:

      db.ourCollection.remove({'createdAt': {$lt: ISODate('2019-07-01')}})

      Doing this deletes about 1 million documents every hour, which is extremely slow.
       
      The replica set is continuously being written to.
       
      How can we remove so much data more efficiently and reliably? 
       
      v3.6.15
      Amazon Linux AMI release 2018.03

            Assignee:
            Unassigned Unassigned
            Reporter:
            axiodis Alexandru Martin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: