Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-78485

$project after $search adds 20x latency

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • ALL
    • QI 2023-07-24, QI 2023-08-07

       

       

       

       

      Materializing the first batch without limit takes 736ms 
      measure(() => db.listings.aggregate([{ $search: { queryString: { defaultPath: "data.host_type", query: "h*" }} }])) 

       

      Adding a $project stage should not add significant overhead but:

       

      Materializing the first batch with limit takes 13,407ms
      measure(() => db.listings.aggregate([{ $search: { queryString: { defaultPath: "data.host_type", query: "h*" }} }, { $project: { highlight: { "$meta": "searchScore" } }}])) 

       

      $project itself isn't slow though because the same query with a really large limit is very fast

      Materializing the first batch takes 156ms
      measure(() => db.listings.aggregate([{ $search: { queryString: { defaultPath: "data.host_type", query: "h*" }} }, { $project: { highlight: { "$meta": "searchScore" } }}, {$limit: 1000} ])) 

       

       

       

      Use cases:

      • We have reports of users experiencing timeouts when iterating over the entire resultset of a $search for batch processing. They address this by issuing many smaller queries using skip+limit, but this is also slow for large skip values.
      • Developers debugging on commandline often forgo using limits, they shouldn't see 13s latency overhead for 

        1. no_project_stats.js
          11 kB
        2. project_stats.js
          11 kB

            Assignee:
            Unassigned Unassigned
            Reporter:
            evan.darke@mongodb.com Evan Darke
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: