-
Type:
Improvement
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Fully Compatible
-
Execution Team 2022-04-04, Execution Team 2022-04-18
-
135
As of SERVER-62427, files mapping to namespaces will include the namespace in the backup cursor response.
This function currently has a quadratic runtime.
One approach to improve performance could be to store the catalog entries outside of this class, so we don't have to fetch them for each backup block, and pass them in sorted into this function.
Backup cursor performance as of SERVER-62427 (~4 minutes, 10000 collections):
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:22:13.282+00:00"},"s":"I", "c":"STORAGE", "id":24200, "ctx":"conn1","msg":"Opened backup cursor","attr":{"backupId":{"uuid":{"$uuid":"2d400d44-082e-4da7-92db-0399e3513c28"}},"term":1}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:22:22.686+00:00"},"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn1","msg":"Slow query","attr":{"type":"command","ns":"admin.$cmd.aggregate","appName":"MongoDB Shell","command":{"aggregate":1,"pipeline":[{"$backupCursor":{}}],"cursor":{},"lsid":{"id":{"$uuid":"c4cd6a2e-b41c-489f-88d8-5320700b88ab"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648750902,"i":7}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"cursorid":5957148721134066174,"keysExamined":0,"docsExamined":0,"numYields":0,"nreturned":101,"reslen":14873,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":3}},"Global":{"acquireCount":{"r":2}}},"readConcern":{"level":"local","provenance":"implicitDefault"},"writeConcern":{"w":"majority","wtimeout":0,"provenance":"implicitDefault"},"storage":{},"remote":"127.0.0.1:39852","protocol":"op_msg","durationMillis":9594}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:26:20.941+00:00"},"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn1","msg":"Slow query","attr":{"type":"command","ns":"admin.$cmd","appName":"MongoDB Shell","command":{"getMore":5957148721134066174,"collection":"$cmd.aggregate","lsid":{"id":{"$uuid":"c4cd6a2e-b41c-489f-88d8-5320700b88ab"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648750902,"i":7}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"originatingCommand":{"aggregate":1,"pipeline":[{"$backupCursor":{}}],"cursor":{},"lsid":{"id":{"$uuid":"c4cd6a2e-b41c-489f-88d8-5320700b88ab"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648750902,"i":7}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"cursorid":5957148721134066174,"keysExamined":0,"docsExamined":0,"numYields":0,"nreturned":19943,"reslen":2857705,"locks":{},"readConcern":{"level":"local","provenance":"implicitDefault"},"writeConcern":{"w":"majority","wtimeout":0,"provenance":"implicitDefault"},"remote":"127.0.0.1:39852","protocol":"op_msg","durationMillis":238253}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:26:20.978+00:00"},"s":"I", "c":"STORAGE", "id":24203, "ctx":"conn1","msg":"Closed backup cursor","attr":{"backupId":{"uuid":{"$uuid":"2d400d44-082e-4da7-92db-0399e3513c28"}}}}
Backup cursor performance prior to SERVER-62427 (<1 second, 10000 collections):
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:47:37.900+00:00"},"s":"I", "c":"STORAGE", "id":24200, "ctx":"conn1","msg":"Opened backup cursor","attr":{"backupId":{"uuid":{"$uuid":"f8d60327-892f-4313-ad25-618f3e256b8c"}},"term":1}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:47:37.903+00:00"},"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn1","msg":"Slow query","attr":{"type":"command","ns":"admin.$cmd.aggregate","appName":"MongoDB Shell","command":{"aggregate":1,"pipeline":[{"$backupCursor":{}}],"cursor":{},"lsid":{"id":{"$uuid":"40ebb2e0-2337-442a-af0e-9284de06bc1f"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648752426,"i":31}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"cursorid":3445142455559510098,"keysExamined":0,"docsExamined":0,"numYields":0,"nreturned":101,"reslen":13084,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":3}},"Global":{"acquireCount":{"r":2}}},"readConcern":{"level":"local","provenance":"implicitDefault"},"writeConcern":{"w":"majority","wtimeout":0,"provenance":"implicitDefault"},"storage":{},"remote":"127.0.0.1:41894","protocol":"op_msg","durationMillis":176}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:47:38.006+00:00"},"s":"I", "c":"STORAGE", "id":24203, "ctx":"conn1","msg":"Closed backup cursor","attr":{"backupId":{"uuid":{"$uuid":"f8d60327-892f-4313-ad25-618f3e256b8c"}}}}
- is caused by
-
SERVER-62427 Files mapping to namespaces in a backup cursor will include the namespace in the response
-
- Closed
-