-
Type: Improvement
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
StorEng - Refinement Pipeline
Pretty soon we'll probably want to be using the handle_general callback to indicate things like "flush_tier complete", or more details about flush_tier or other system state changes. We already have events we can't track in python, like WT_EVENT_COMPACT_CHECK. (Also, the handle_progress is callback is not being set, so there's no way to test it in Python - that could be handled in this ticket if we wish).
We could imagine a shiny new capability that allows us to have arbitrary python functions attached to the callbacks - the tests could have their own specific python to does the "handling". It's possible, but probably more complicated than we need.
For example, we don't have a Python callback for handle_message and handle_error, rather when we receive these callbacks, we put the strings into stdout.txt and stderr.txt respectively, and the Python program can check the contents if it wants. Thus, in our SWIG interface, we have C code that does a reasonable thing. For the handle_general case, we'd probably break things down by the operation type. For tiered storage callbacks, it probably just makes sense to record the results in a file. We'd likely have a tiered storage event like "tiered storage complete" along with a URI that has a "manifest" of the new objects. When we receive one of these, we might create a event_messages.txt file with a line like:
Apr 24 2023 01:42:23 PM GMT-0400: WT_EVENT_FLUSH_TIER_COMPLETE: s3:/bucket12345/prefix_WiredTiger-0000000001.backup
It probably makes sense to do this once we have a "general" callback in place for tiered storage, then we have something real to work with. We could do a similar callback for WT_EVENT_COMPACT_CHECK as part of this work - that could more more sophisticated, with the ability to cancel the compact operation as part of some condition.