-
Type: Task
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: 3.2.11
-
Component/s: mongodump
-
1
-
Not Needed
-
v3.4, v3.2
Currently, oplog dumping uses the dumpQueryToIntent (previously dumpQueryToWriter) that checks the command-line query to be non-empty. I believe that when oplog is being dumped, this check should be skipped and the total oplog entry count should not be obtained, because it effectively scans the oplog in full (since there is on index on "ts" field and the query is something along the lines of {ts:{$gt:...}}).
While having the progress bar is certainly helpful, forcing an extra (possibly 100+GB) oplog scan to output it properly seems a bit overkill, since this leads to excessive disk utilization on databases with large historical oplogs (not to mention killing the cache on MMAPv1 engines, straining the database).
Also, although I'm not sure, maybe the "query" instead of "dump.query" should be checked, then it will work for oplog queries automatically.
- duplicates
-
TOOLS-2068 mongodump oplog delay
- Closed