We need to backup parts of our DB such that a user can download all his data.

We currently have used the following approach:

 

1. Have a program that generates the required query files for a user to be used with mongodump

2. Then run mongodump with the query file.

 

We now run into an issue where a user wants to create a backup of project where he have uploaded many files.

 

We therefore have a query file with 17MB of individual ids we need to backup from our mongodb.

 

If we run mongodump with this query file we get the following error:

 

Failed: error writing data for collection `cxdb.ProjectFiles.files` to disk: error reading collection: BSONObj size: 21808047 (0x14CC3AF) is invalid. Size must be between 0 and 16793600(16MB) First element: find: "ProjectFiles.files"

 

The query files contains all ids we need to backup:

{{

{"_id":\{"$in":["T93X642SSDFYHwJQY", ...]}

}}

 

In total it contains 1461270 ids we need to backup.

If we delete some entries from this document and get its size smaller, everything works as expected.

 

Unfortunately we cannot create another query which would reduce the size of the query file. 

Is there a better solution for this usecase? Or is it expected that the query files must be less than 16MB?