As I’ve worked on multiversion BFs I’ve noticed that a lot of these on the infra side is due to binaries not being downloaded or present in the right place (BF-24950). After “setup multiversion” we should “check multiversion” & fail immediately if it’s all expected binaries are not available. Another thing to note is that we download all binaries for "setup multiversion" even in cases where we do not need them all. We've had a lot of bugs in this code so I think this is risky. What if we download all versions and the multiversion test runs against the wrong version – but does not fail because the version was available?

We don't have to do both of these, but we should make a decision based on effort vs. value. The goal of this ticket is to reduce the number of false positive multiversion failiures which costs everyone time.

Consider:

• Implementing "check multiversion" so that we know that all expected downloads have completed successfully
• Downloading only the binaries absolutely necessary for a given task (evergreen changes)