Currently the oplog can be used as a means to track changes on the system. You may want to do this for things such as detecting changes to determine cache invalidations.
Naturally, tailing the oplog is not an ideal medium for this as it's format can change at any time and MongoDB provides no guarantees when basing things off the oplog.
Furthermore, the oplog stores all events that occur on the system including internal events like removes and inserts from migrations that I may not want or may want to filter out.
Finally, the oplog is a mongod-wide component which under heavy traffic means I may be tailing much more data than I actually need if my application is focused on just one collection or DB.
Ideally, creating an activity stream that is optionally published out would expose this through an API layer independent of the oplog. This would allow for applications like this to no longer directly rely on the oplog.
If we could also filter on the activity stream similar to how audit filters are applied it would allow for only reading the changes you care about as oppose to filtering on the application side and transmitting a ton of data needlessly.
You could also do things like only allowing people with certain permissions to receive certain changes. I.E. if I don't have permission to a DB, I can't view the changes that have occurred on it. Currently if you want to tail some subset of the oplog, you have to be able to tail the whole oplog which causes a bit of a security permissions problem.
- is depended on by
-
SERVER-2150 Persistent Queries
- Closed
- is duplicated by
-
SERVER-5042 Implement support for reliable change notifications
- Closed
-
SERVER-30252 Write oplog operations to kafka
- Closed
-
SERVER-13755 Support for tailable cursors over non capped collections
- Closed
-
SERVER-28708 Event handler in auditing functionality.
- Closed
- related to
-
SERVER-5042 Implement support for reliable change notifications
- Closed