Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-5392

Make Evergreen green again

    • Type: Icon: Epic Epic
    • Resolution: Unresolved
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: Evergreen, Test Failure
    • Java Drivers
    • Hide
      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?
      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • To Do
    • Evergreen test failures
    • 3
    • 1.5
    • 3
    • 100
    • Hide

      Engineer(s): Valentin

      2024-07-23: Sent to Backlog

      What was completed over the last two weeks?

      • Migrating Unified Tests to JUnit 5

      Focus for next two weeks?

      • Sent to backlog to get started on BulkWrite

      2024-07-05: Set target end date to 2024-07-19

      What was completed over the last two weeks?

      • Migrating Unified Tests to JUnit 5
      Show
      Engineer(s): Valentin 2024-07-23: Sent to Backlog What was completed over the last two weeks? Migrating Unified Tests to JUnit 5 Focus for next two weeks? Sent to backlog to get started on BulkWrite 2024-07-05: Set target end date to 2024-07-19 What was completed over the last two weeks? Migrating Unified Tests to JUnit 5

      Summary

      We want CI builds to behave in accordance to the CI server name. Intermittent failures that we have take up team resources:

      1. it's more difficult to understand whether we have an actual issue with the driver both when we release a driver and when we submit PRs;
      2. we have to re-run Evergreen tasks, sometimes more than once, and that takes time.

      Intermittent test failures can be classified as follows (this classification may not be exhaustive and may be expanded):

      1. Not caused by bugs
        1. Caused by external resources.
        2. Caused by the test having known race conditions which are mitigated in practice by artificial delays being part of the test.
      2. Caused by bugs
        1. Caused by the test having bugs, e.g., an unknown race condition.
        2. Caused by the server or driver bugs.

      We should start with addressing failures that are not caused by bugs. Once they are addressed, we can assess the situations and decide what to do next.

      Reports of some (ideally, all) intermittent and non-intermittent test failures can be seen here.

      Motivation

      Who is the affected end user?

      The dbx-java team.

      How does this affect the end user?

      They are annoyed and less efficient than could have been. They also have increased chances of missing a bug.

      How likely is it that this problem or use case will occur?

      Intermittent failures happen all the time.

      If the problem does occur, what are the consequences and how severe are they?

      The consequences are tolerable at the moment, but we can do better.

      Is this issue urgent?

      No.

      Is this ticket required by a downstream team?

      No.

      Is this ticket only for tests?

      Yes.

      Cast of Characters

      Not applicable.

      Channels & Docs

      Not applicable.

            Assignee:
            Unassigned Unassigned
            Reporter:
            valentin.kovalenko@mongodb.com Valentin Kavalenka
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              5 weeks, 4 days