-
Type:
Improvement
-
Resolution: Done
-
Priority:
Unknown
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
Optimize Cursor.to_list.
Context
Cursor.to_list is currently implemented like this:
async def to_list(self) -> list[_DocumentType]:
return [x async for x in self]
And:
def to_list(self) -> list[_DocumentType]:
return [x for x in self]
This is expensive especially in the async case because we need to iterate (and await) on every single document in the cursor.
It should be more efficient to build up the list via each batch, something like this:
async def to_list(self) -> list[_DocumentType]:
res = []
while self.alive:
res.extend(await self._next_batch())
return res
Definition of done
Implement iteration via batches and benchmark the improvement.
Note, if we want to make the batch iteration api public we should open a new ticket.
- causes
-
PYTHON-4610 test_to_list_tailable takes 20 seconds
-
- Closed
-
-
PYTHON-4617 Test Failure - Storage MMAPv1
-
- Closed
-
- related to
-
PYTHON-3079 Async PyMongo Beta
-
- Development Complete
-