-
Type: Task
-
Resolution: Done
-
Priority: Unknown
-
None
-
Component/s: None
-
None
-
Needed
-
Summary
The utils.sh and activate_venv.sh scripts provided by Drivers Evergreen Tools are deprecated and will eventually be removed. Drivers must replace their use with the new venv-utils.sh, activate-kmstlsvenv.sh, and find-python3.sh scripts instead.
The new venv-utils.sh and activate-kmstlsvenv.sh are hygienic alternatives to utils.sh and activate_venv.sh that do not leak variables into the environment and do not depend on environment variables for their behavior. Both of these old script behaviors has been observed to cause unexpected and often difficult-to-diagnose distro-specific task failures.
Furthermore, the existing built-in logic to detect a viable python binary capable of creating a virtual environment has been observed to fail on several distros both old and new. To facilitate a more consistent and comprehensive method for finding a viable python binary, a new find-python3.sh script has been provided to replace the current ad hoc implementations both in Drivers and in Drivers Evergreen Tools.
Before:
cd drivers-evergreen-tools/.evergreen/csfle . ./activate_venv.sh # Leaks PYTHON set prior to venv. . ./set-temp-creds.sh # Uses leaked non-venv PYTHON.
After:
cd drivers-evergreen-tools/.evergreen/csfle . ./activate-kmstlsvenv.sh # No leaks. . ./set-temp-creds.sh # Uses venv python.
Before:
. drivers-evergreen-tools/.evergreen/utils.sh PYTHON="path/to/python" # Ad hoc, complicated, and inconsistent. venvcreate "$PYTHON" venv # Leaks PYTHON, depends on undocumented OS env var.
After:
. drivers-evergreen-tools/.evergreen/venv-utils.sh . drivers-evergreen-tools/.evergreen/find-python3.sh PYTHON="$(find_python3)" # Uniform python lookup utility. venvcreate "$PYTHON" venv # No leaks or undocumented dependencies.
Motivation
Who is the affected end user?
Drivers that are using the utils.sh or activate_venv.sh scripts provided by Drivers Evergreen Tools.
How does this affect the end user?
Drivers must update their scripts to replace usage of utils.sh or activate_venv.sh as described above. This should only affect Evergreen configurations and scripts; Drivers implementations should not require any modifications.
How likely is it that this problem or use case will occur?
Depends on the Driver's use of the scripts and related environment variables such as PYTHON, PYTHON_BINARY, and OS.
If the problem does occur, what are the consequences and how severe are they?
Time lost to diagnosing unexpected and problematic Python virtual environment creation or activation behavior.
Is this issue urgent?
No.
Is this ticket required by a downstream team?
No.
Is this ticket only for tests?
Yes. Evergreen configuration and scripts only.
- related to
-
DRIVERS-2519 Utilize find-python3.sh in create-serverless.sh
- Closed
- split to
-
MOTOR-1064 Remove use of activate_venv.sh and utils.sh
- Closed
-
CDRIVER-4519 Remove use of activate_venv.sh and utils.sh
- Closed
-
CSHARP-4399 Remove use of activate_venv.sh and utils.sh
- Closed
-
CXX-2612 Remove use of activate_venv.sh and utils.sh
- Closed
-
GODRIVER-2643 Remove use of activate_venv.sh and utils.sh
- Closed
-
JAVA-4806 Remove use of activate_venv.sh and utils.sh
- Closed
-
NODE-4800 Remove use of activate_venv.sh and utils.sh
- Closed
-
PHPC-2164 Refactor run-ocsp-responder.sh to use activate-ocspvenv.sh
- Closed
-
PHPLIB-1038 Remove use of activate_venv.sh and utils.sh
- Closed
-
PYTHON-3507 Remove use of activate_venv.sh and utils.sh
- Closed
-
RUBY-3175 Remove use of activate_venv.sh and utils.sh
- Closed
-
RUST-1537 Remove use of activate_venv.sh and utils.sh
- Closed