We want to be able to exercise upgrade and downgrade scenarios (similar to what gets tested in the MongoDB's multiversion test suite).
An initial idea might be to have the script build the previous release and run some workgen workload while alternating upgrading and downgrading.