Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-80220

JavaScript tests tagged with in-development feature flags are not being run by burn_in_tests

    • Decision Automation Group
    • Fully Compatible
    • ALL
    • DAG 2023-09-18
    • 2

      buildscripts/burn_in_tests.py generate-test-membership-map-file-for-ci generates the association of a particular JavaScript test file and all of the resmoke YAML suite files the test would run under. However the decision about whether a test is suitable to run depends on the state of resmokelib.config.EXCLUDE_WITH_ANY_TAGS which in particular is populated by the names of all of the in-development feature flags unless --runAllFeatureFlagTests is specified.

      The buildscripts/burn_in_tests.py script does not enable the --runAllFeatureFlagTests mode of resmoke and so these tests are currently never run by burn_in_tests. This defeats a large portion of the use case for burn_in_tests because engineers are writing new tests for their projects under an in-development feature flag and means these JavaScript test files won't be stressed to determine if the tests are unreliable.

            Assignee:
            mikhail.shchatko@mongodb.com Mikhail Shchatko
            Reporter:
            max.hirschhorn@mongodb.com Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: