-
Type:
Bug
-
Resolution: Incomplete
-
Priority:
Major - P3
-
None
-
Affects Version/s: 4.0.18
-
Component/s: Internal Client, Performance, Shell
-
None
-
ALL
-
Hello,
I have the following situation:
- I have a GridFS bucket with 810,959 files (.files count) stored in 2,499,160 chunks (.chunks count)
- I want to iterate over all entries in .chunks and print (just for sake of simplicity) the _id of each entry
- for this I use the following command
db.sandboxReports.chunks.find({},{_id:1}).forEach(r=>print(r._id))
- what I expect is to have the screen filled with 2m+ IDs (I know, useless, but again, for sake of simplicity)
- what happens is that after exactly 101 IDs, the script freezes. Waited for hours (I am a curious guy), but it never recovered
- I read the same issue https://jira.mongodb.org/browse/SERVER-35106 (unfortunately closed with CNR) and tried setting the batchSize to 100
- this time it does not freeze forever, but between batches there is significant delays of up to minutes (which makes the solution unusable since there are 2m+ records and there are only 1,440 minutes in a day)
- I also found out that the problem comes up occasionally, but never a working solution is provided (seems hard to reproduce), for eg. see:
https://stackoverflow.com/questions/36407641/node-js-mongo-find-each-stopping-after-first-batch
https://stackoverflow.com/questions/52064752/mongo-cursor-hangs-after-first-fetch
During the time the query executes, I have the following output from currentOp:
{
"host" : "HOST_NAME_HERE:27017",
"desc" : "conn11650",
"connectionId" : 11650,
"client" : "127.0.0.1:48568",
"appName" : "MongoDB Shell",
"clientMetadata" : {
"application" : {
"name" : "MongoDB Shell"
},
"driver" : {
"name" : "MongoDB Internal Client",
"version" : "4.0.18"
},
"os" : {
"type" : "Linux",
"name" : "Ubuntu",
"architecture" : "x86_64",
"version" : "18.04"
}
},
"active" : true,
"currentOpTime" : "2020-05-18T21:37:24.801+0000",
"opid" : 9652856,
"lsid" : {
"id" : UUID("ab8e0656-ddd0-496e-8c1b-6c95dc9e35b3"),
"uid" : BinData(0,"lrHY+QTs83BmclcR9MKKYcSvfFmsJ+jW+bP+OfRjiFM=")
},
"secs_running" : NumberLong(906),
"microsecs_running" : NumberLong(906036759),
"op" : "getmore",
"ns" : "DB_NAME.sandboxReports.chunks",
"command" : {
"getMore" : NumberLong("171272634643"),
"collection" : "sandboxReports.chunks",
"lsid" : {
"id" : UUID("ab8e0656-ddd0-496e-8c1b-6c95dc9e35b3")
},
"$clusterTime" : {
"clusterTime" : Timestamp(1589836937, 5),
"signature" : {
"hash" : BinData(0,"v8OUtyPICaqX5BTz+tZ3u/rAsV8="),
"keyId" : NumberLong("6785026434701197326")
}
},
"$db" : "sandboxReports"
},
"originatingCommand" : {
"find" : "sandboxReports.chunks",
"filter" : { },
"projection" : {
"_id" : 1
},
"lsid" : {
"id" : UUID("ab8e0656-ddd0-496e-8c1b-6c95dc9e35b3")
},
"$clusterTime" : {
"clusterTime" : Timestamp(1589816462, 13),
"signature" : {
"hash" : BinData(0,"ZTJm56Md0AIWuKYHN/EiyyopmLY="),
"keyId" : NumberLong("6785026434701197326")
}
},
"$db" : "sandboxReports"
},
"planSummary" : "COLLSCAN",
"numYields" : 42856,
"locks" : {
"Global" : "r",
"Database" : "r",
"Collection" : "r"
},
"waitingForLock" : false,
"lockStats" : {
"Global" : {
"acquireCount" : {
"r" : NumberLong(42857)
}
},
"Database" : {
"acquireCount" : {
"r" : NumberLong(42857)
}
},
"Collection" : {
"acquireCount" : {
"r" : NumberLong(42857)
}
}
}
},
Server is MongoDB 4.0.18 running on Ubuntu 18.04 setup in replica set mode.
I have tried to run the command on different servers from replica set
If I can help you with other information, please let me know.
Any help is greatly appreciated.
Regards,
Puiu
- related to
-
-
- Closed
-