Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-4366

High variance in GridFsMultiFileDownload benchmark

XMLWordPrintableJSON

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Python Drivers
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      After the changes in PYTHON-4350, the GridFsMultiFileDownload benchmark has high variance. For example the most recent build dropped by 15%: https://spruce.mongodb.com/task/mongo_python_driver_perf_tests_perf_6.0_standalone_e40ff3516821ce79252bef60fce41bf3d93ae110_24_04_11_01_19_51/trend-charts?execution=0&sortBy=STATUS&sortDir=ASC

      This could also be related to HELP-58235.

      For now we should increase the benchmark runtime from 60 seconds back to 3-5 minutes.

      Also see this:

      https://performance-analyzer.server-tig.prod.corp.mongodb.com/perf-analyzer-viz/?comparison_id=bd0d07ee-4c84-4573-8877-e33e100b92d9&selected_tab=data-table&percent_filter=0%7C%7C100&z_filter=0%7C%7C10

        1. Screenshot 2024-04-15 at 11.58.52 AM.png
          Screenshot 2024-04-15 at 11.58.52 AM.png
          138 kB

            Assignee:
            Unassigned Unassigned
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: