Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-4498

GridFsMultiFileDownload benchmark is inconsistent

XMLWordPrintableJSON

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Unknown Unknown
    • None
    • Affects Version/s: None
    • Component/s: Performance Benchmarking
    • None
    • Python Drivers
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?

      Context

      Ever since PYTHON-4350 we have been seeing that some of the benchmarks are widely inconsistent. For example, GridFsMultiFileDownload:

      Before PYTHON-4350, GridFsMultiFileDownload ran in 12 seconds:

       [2024/04/08 16:36:50.157] test/performance/perf_test.py::TestGridFsMultiFileDownload::runTest Completed TestGridFsMultiFileDownload 2194.621 MB/s, MEDIAN=0.119s, total time=12.557s

      After PYTHON-4350, GridFsMultiFileDownload runs in 30.705 seconds:

       [2024/04/09 19:27:45.464] test/performance/perf_test.py::TestGridFsMultiFileDownload::runTest Completed TestGridFsMultiFileDownload 2092.982 MB/s, MEDIAN=0.125s, total time=30.705s, iterations=239

      It may seem counter intuitive but I think we can fix this by reducing the benchmark runtime back to 10-12 seconds by lowering MIN_ITERATION_TIME from 30 to 10. One explanation is that the mongod is suffering from a problem similar to HELP-58235.

        1. Screenshot 2024-06-12 at 4.58.39 PM.png
          Screenshot 2024-06-12 at 4.58.39 PM.png
          72 kB

            Assignee:
            Unassigned Unassigned
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: