Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-88156

Investigate query regressions caused by new tcmalloc

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • ALL
    • 105

       

      The query team will need to help to investigate the follow regressions caused by new tcmalloc:

      1. see if they are real regressions, comparing to 7.0 and 4.4
      2. see anything we can do to improve the performance or mitigate the regression
      3. if we can't mitigate the regression by 8.0, are we ok with accepting the regerssions.

      See linked BF for more details

       

      when to look team high value variant task test measurement args compare_value base_value compare_stable_mean base_stable_mean compare_stable_std base_stable_std percent_change percent_change_stable_mean_pre_tcmalloc percent_change_stable_mean post_tcmalloc_percent_difference
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.CoveredBlockingSort ops_per_sec {'thread_level': 8} 2303.819261 2558.949341 2284.815807 2558.724812 62.28993283 50.93584901 -9.970110593 8.174161705 -10.70490281 -18.87906452
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.CoveredBlockingSort ops_per_sec {'thread_level': 4} 1197.170227 1520.293392 1327.293364 1477.355154 98.7207345 88.27517031 -21.2540005 0.4255418707 -10.15746207 -10.58300394
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndex ops_per_sec {'thread_level': 4} 321.0089165 407.5318724 313.7054018 403.4309061 15.00419225 18.30759946 -21.23096664 17.72710222 -22.24061244 -39.96771466
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndex ops_per_sec {'thread_level': 4} 175.4321608 214.1474478 182.3838088 230.9510648 8.677265709 9.173532759 -18.07879917 8.76413404 -21.029241 -29.79337504
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndex ops_per_sec {'thread_level': 2} 94.1735679 113.7684276 97.4075431 118.8629683 4.417804311 5.051015727 -17.22346009 7.333480867 -18.05055477 -25.38403564
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndex ops_per_sec {'thread_level': 2} 186.9318716 215.6048861 177.4249362 213.1479603 9.162248186 9.860331384 -13.29887046 15.63551331 -16.75973069 -32.395244
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndex ops_per_sec {'thread_level': 8} 321.4553357 387.6041521 317.1011934 378.5299303 12.79904176 24.87429549 -17.06607528 22.55724508 -16.22823769 -38.78548277
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndex ops_per_sec {'thread_level': 8} 182.2259651 197.8001031 190.1255698 212.0954353 9.149668645 10.47084902 -7.873675405 19.09169954 -10.35848106 -29.45018061
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndexAndQuery ops_per_sec {'thread_level': 4} 472.0235877 593.8188681 513.3369294 627.7908276 54.51898786 22.12879464 -20.51051035 5.019179322 -18.23121542 -23.25039474
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndexAndQuery ops_per_sec {'thread_level': 8} 850.8236454 1053.142528 844.9992657 1029.236282 24.6806082 32.00298096 -19.21096883 11.25711461 -17.90036161 -29.15747622
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndexAndQuery ops_per_sec {'thread_level': 8} 489.2203946 573.5481304 522.5201519 617.7730735 17.34770767 27.12095605 -14.70281766 7.332195018 -15.41875581 -22.75095082
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithIndexAndQuery ops_per_sec {'thread_level': 2} 256.5209122 298.1675517 272.4670346 315.8694028 24.78345118 14.03017729 -13.96752909 6.483621249 -13.74060537 -20.22422662
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithoutIndex ops_per_sec {'thread_level': 4} 319.6540753 403.1615705 307.0049363 397.7343969 23.47728593 20.85929911 -20.71315852 24.81852804 -22.81157006 -47.63009809
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithoutIndex ops_per_sec {'thread_level': 4} 173.445472 221.4328134 180.2043116 228.7798043 9.651717845 7.760417382 -21.67128738 10.51253422 -21.23242162 -31.74495584
      before 8.0 query   perf-mongo-perf-standalone.arm.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithoutIndex ops_per_sec {'thread_level': 2} 90.24841806 111.1212095 95.79137074 116.3690731 4.953257884 5.250007388 -18.78380514 7.718877626 -17.6831368 -25.40201443
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithoutIndex ops_per_sec {'thread_level': 2} 186.5463886 216.3163596 175.7588085 209.1513937 10.62350806 13.06663983 -13.76223744 16.92587147 -15.96574835 -32.89161982
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 agg-query-comparison_read_commands Aggregation.DistinctWithoutIndex ops_per_sec {'thread_level': 8} 321.7592514 380.5027587 315.0043478 374.4834062 15.028912 24.24478576 -15.43839197 24.26604975 -15.88296235 -40.1490121
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 aggregation_read_commands Aggregation.ExpressionObject ops_per_sec {'thread_level': 1} 168.3875758 197.431574 168.0688826 191.6232032 5.956206197 5.940455487 -14.71091866 2.802598853 -12.2919982 -15.09459705
      before 8.0 query   perf-3-node-replSet.arm.aws.2023-11 change_streams_latency 1_1c_1024d_secondary_avg_latency ops_per_sec {'thread_level': 24} -0.002987125246 -0.003165260634 -0.00295013946 -0.003399273038 0.000137387152 0.0001778094067 -5.627826843 8.735547551 -13.21263614 -21.94818369
      before 8.0 query   perf-3-node-replSet.arm.aws.2023-11 change_streams_preimage_latency 1_whenAvailablefd_preImage_1c_1024d_avg_latency ops_per_sec {'thread_level': 24} -0.003507143901 -0.004056270175 -0.0035948088 -0.004048648062 0.0001152573391 0.0001354187094 -13.53771447 11.20964963 -11.20964963 -22.41929926
      before 8.0 query   perf-3-node-replSet.arm.aws.2023-11 change_streams_preimage_latency 1_whenAvailablefdbc_preImage_1c_1024d_avg_latency ops_per_sec {'thread_level': 24} -0.003451726134 -0.003943360967 -0.003628172461 -0.004046589201 0.0001017050332 0.0001365820931 -12.46740627 10.33998558 -10.33998558 -20.67997115
      before 8.0 query   perf-3-node-replSet.arm.aws.2023-11 change_streams_preimage_latency 1_whenAvailablefdbc_whenAvailablefd_preImage_1c_1024d_avg_latency ops_per_sec {'thread_level': 24} -0.003408330457 -0.004146594828 -0.003496483986 -0.004259003849 9.17E-05 0.0001677831441 -17.80411161 12.68692347 -17.90371388 -30.59063735
      before 8.0 query   perf-mongo-perf-standalone.intel.aws.2023-11 views-aggregation Aggregation.IdentityView.Lookup.UncorrelatedPrefixJoin.LargeDataset ops_per_sec {'thread_level': 1} 0.08089168917 0.1947295157 0.08037230074 0.1887018932 0.002010184332 0.005784289073 -58.45946164 0.05125582275 -57.40779312 -57.45904894

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            xiaochen.wu@mongodb.com Xiaochen Wu
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved: