Currently, in our test tagging system, there are various usages of featureFlagXX, requires_fcv_yy, and FeatureFlagUtil.isEnabled. (The current guidance for feature flag test tagging is here).

For example, currently if we have a JS test dependent on a feature flag, we tag it with featureFlagXX.
However, featureFlagXX tags aren't considered by the jstests/parallel/basic{,Plus}.js tests (only requires_fcv_yy would be) and featureFlagXX tags aren't considered by the jstests/core/selinux.js test (only no_selinux would be).

Additionally, for suites like replica_sets_multiversion, if they are being run on a feature flags enabled variant like Shared Library Enterprise RHEL 8.0 (all feature flags), even if the feature flag hasn’t been enabled by default and released yet, the feature flag will be enabled on this variant. So even if the test is tagged with featureFlagXX, since the feature flag is enabled, it will be run in replica_sets_multiversion, so we need to additionally tag it with requires_fcv_yy to prevent it from being run. Edit: As part of SERVER-71467, we determined that we should not be running multiversion tests with feature flags except for explicit multiversion tests. So, tagging tests with featureFlagXX should be sufficient for exclusion from suites like replica_sets_multiversion.

We should investigate if we can improve and simplify the tagging system so that featureFlagXX is both the exhaustive and definitive way to tag tests dependent on in-development or recently-released feature work.

Another solution is that the test exclusion code could consult the IDL to determine the FCV a given feature is enabled for, and exclude it appropriately, rather than needing to have both the feature flag tag and the fcv tag.

Additionally, we should investigate moving away from using FeatureFlagUtil.isEnabled where possible, since it is wasteful to start up a whole cluster only to see that the feature flag is not enabled and then skip the whole test.

As part of this investigation we can create a table which describes the different combinations of test of {in-development feature, test of feature in last-stable or earlier, test of feature in last-continuous but not last-stable} x {binVersion latest cluster, mixed binVersion last-stable + latest cluster, mixed binVersion last-continuous + latest cluster, pseudo-mixed latest cluster with future git tag cluster} for whether a test should or shouldn't run. And then we can map these back into whatever the build variants and implicit _multiversion test suites are or are not doing today in order to audit our testing coverage.