Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-2779

Standardize performance benchmark reporting metrics

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • None
    • Needed

      The Benchmarking spec says:

      In addition to timing data, all micro-benchmark tasks will be measured in terms of "megabytes/second" (MB/s) of documents processed, with higher scores being better. (In this document, "megabyte" refers to the SI decimal unit, i.e. 1,000,000 bytes.) This makes cross-benchmark comparisons easier.

      However, each driver's benchmark suite reports different measurements right now. Python reports "bytes_per_second", Java reports ops_per_second, C says "ops_per_second" but actually reports bytes/sec, etc..

      We should also standardize the "Test" names that are reported so that we can more easier compare results across drivers. For example, the "GridFS download" test is called "GridFS Download" in Java, "TestGridFsDownload" in Python and C, "gridFsDownload" in Node.

      Related to DRIVERS-2666.

            Assignee:
            Unassigned Unassigned
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: