-
Type: Task
-
Resolution: Fixed
-
Priority: Blocker - P1
-
Affects Version/s: None
-
Component/s: Testing Infrastructure
-
None
-
Fully Compatible
-
STM 2022-01-24
-
2
Tasks: https://evergreen.mongodb.com/waterfall/mongodb-mongo-master?task_filter=benchmark
Context
As a database contributor
when I run a patch build containing Google Microbenchmarks
Then I expect to see red/green results(or async signal processed BFs), as a way to alert me to performance regressions
AC
- Ensure signal processing is running in google benchmarks
- Check historical values for google benchmarks through the evergreen API of the base commit
- Make the threshold configurable in a yaml file in google benchmarks
- we'll use 20% for this use case, meaning if the task is 20% worse for Latency (since google benchmarks only reports latency) than the baseline commit then the task will be marked failed (red)
- The key in the yaml file should be the key for the metric in the json perf output
- related to
-
SERVER-65385 Disable threshold checks for Google Microbenchmarks on DEBUG/Sanitizer variants
- Closed
-
SERVER-87396 Add new resmoke hook to check result of a benchmark
- Closed
-
SERVER-65986 Disable threshold checks for highly variable benchmarks
- Closed