Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-84755

Investigate throughput reduction for large number of documents

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization

      The BigCollection benchmark in the mongo-perf runs multiple tests where the total data size of the collection is kept the same, while the number of documents increases and their size decreases by the same factor. Both Scan and Filter queries reveal a cliff in the throughput for the collection with the largest number of documents (1638400). This holds both for classic and SBE engines.

      From the investigation in SERVER-80583 on VM, 1-thread throughput in ops-per-sec

      Document number Document size  Batch
      size
      Classic SBE
      25 16777216 0 2.286 2.152
      400 1048576 0 2.781 2.505
      6400 65536 0 2.788 2.583
      102400 4096 0 2.248 2.145
      1638400 256 0 0.702 0.799
      400 1048576 1 2.937 2.823
      6400 65536 16 2.846 2.825
      102400 4096 256 2.358 2.432
      1638400 256 4096 0.745 0.901

      This seems to be partially due to the WiredTiger, and partially due to the predicate computation ( higher computational cost for larger number of documents). Excerpt from the flame graphs in the attachment for the SBE engine:

      PlanExecutorSBE::getNext :  1.89% vs. 44.73%

      FilterStage::getNext : 1.67% vs. 40.77%

      WiredTigerRecordStoreCursorBase::next : 0.93% vs. 23.84%

      sbe::vm::ByteCode::runPredicate : 0.24% vs 7.46%

        1. Filter.BC.6400.16.sbe.svg
          325 kB
        2. Filter.BC.1638400.4096.sbe.svg
          372 kB

            Assignee:
            backlog-query-optimization [DO NOT USE] Backlog - Query Optimization
            Reporter:
            milena.ivanova@mongodb.com Milena Ivanova
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: