-
Type: Task
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Replication
FeatureFlagUtils is a JS library used in the test runner to check if feature flags are present and/or enabled.
The README recommends using this library for querying the setting for a feature flag, and using it to skip tests in multiversion environments.
The way the library works is by taking a connection to the cluster as input. If the connection is to a mongod node, it uses it directly; otherwise, it makes a new connection to an arbitrary mongod in the cluster. Then it does the following algorithm using that connection:
- Use getParameter to get the FCV of the node
- Use getParameter again to get the featureFlag information
- Compare the minVersion for the featureFlag retrieved from the server to the FCV retrieved from the server to see if the feature flag could be enabled on the version
- If the feature flag can be enabled on the version, and the server says it is enabled, and the user of the library didn't pass the 'ignoreFCV' option, conclude that the feature flag is enabled.
- Otherwise, conclude that it is not-present if we didn't see it in our getParameter output, or conclude that it is present-but-disabled if we did.
There are a few scenarios where it is unclear what it means for a 'feature flag to be enabled' on the cluster, mostly around multiversion testing, where the correctness of this algorithm is questionable:
- If the feature flag exists on mongos, and we are in a suite where we use last-LTS mongos but current version shards, the library will always report the status of the feature flags on the shards, even if it is missing on mongos
- On a mixed-version cluster, the value returned by isEnabled and isPresentAndEnabled depends on the connection passed to the library - you will get different answers if you ask different version nodes as to whether the feature flag exists. This isn't obvious from the API and is especially dangerous in passthrough tests where we may talk more to secondaries, etc, depending on the suite configuration.
It also contains the "isPresentAndEnabled" helper, which it notes is easy to misuse because of the fact that it can silently prevent tests from running. This can occur not only after a feature flag is removed, as the test notes, but whenever the connection that happened to be passed to the library was to an older-version node.
It would be nice if we could
- Write down the possible states of a feature flag in a cluster, including multiversion
- Get rid of isPresentAndEnabled checks (WE have seen some cases where tests are accidentally not running after the feature flag is enabled, like
SERVER-73509). - Provide a simple command in mongos/mongod that can be used to determine the cluster's current state, so the passed in "db" to this library doesn't need to be adjusted carefully as it is today
- related to
-
SERVER-72517 Tests for former featureFlagTimeseriesUpdatesAndDeletes are not running
- Closed
-
SERVER-89943 Feature flag util must retry upon errors in suites with stepdowns and kill primary
- Closed
-
SERVER-91008 The methods of feature_flag_utils.js should be resilient to ShutdownInProgress errors received from server nodes
- Closed
-
SERVER-92442 FeatureFlagUtil JS library functions can fail on network errors
- Closed
-
SERVER-63302 Fix FeatureFlagUtil.isEnabled FCV version check
- Closed
-
SERVER-73655 Add linter rule to require SERVER TODO on js test feature flag checks
- Backlog