I'm trying to implement an incremental map reduce using pymongo that looks like this:

https://gist.github.com/4215887

The ideia is that on the first time the map reduce function gets called, it uses a finalize function that saves the current timestamp for each record. The output will be saved on a _stats collection.

On every subsequent execution it first looks for the greatest timestamp on the _stats collection and then it uses that to perform a query against the ObjectId timestamp, that guarantees that only the new records will be processed.

The problem is that for some weird reason it only works with a time.sleep(1) on the map_reduce method. It seems like some sort of sync vs async problem. If I remove the sleep, a query using the timestamp will return not only, but all the record of that collection.