With PyMongo using aggregate $out/$merge is as simple as calling the aggregate() method:
coll.aggregate([{'$out': 'output-collection'}])
However in Motor the aggregate method call does not execute the aggregate command. The command is run lazily when the resulting cursor is first iterated. So an application needs to iterate the return in order to run the $out/$merge aggregation:
cursor = motor_coll.aggregate([{'$out': 'output-collection'}]) # Iterate the cursor to run the $out operation. await cursor.to_list(length=None)
Or more succinctly:
await motor_coll.aggregate([{'$out': 'output-collection'}]).to_list(length=None)
Or yet another way:
async for _ in motor_coll.aggregate([{'$out': 'output-collection'}]): pass
This was indirectly reported in https://developer.mongodb.com/community/forums/t/out-from-datalake-to-attlas-cluster-with-pymongo/15177/ :
with motor driver gets even worst because the driver swallows the output and never tells
- is related to
-
PYTHON-4655 Decide whether to make Async Methods that Use Aggregate Sync Instead
- Closed