Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-12639

Performance regression with RegEx

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.6.0-rc0
    • Affects Version/s: 2.5.5
    • Component/s: Querying
    • ALL

      I recently tried to upgrade from 2.4.9 to 2.5.5 and I'm experiencing catastrophic regex query times with 2.5.5 so I had to downgrade again. My query simply is sth. like this:
      {
      "folder":/^weblogic/,
      "type":"file"
      }

      2.4.9 gives me this query plan:
      "cursor": "BtreeCursor type_folder multi",
      "nscanned": NumberInt(2583),

      2.5.5 gives this query plan:
      "cursor": "Complex Plan",
      "nscanned": NumberInt(8470527),

      This plan is then running for literally ages
      I wonder why 2.5.5 isn't even considering my "type_folder" index for this simple query.

      Unfortunately I cannot provide test data because I could only reproduce the problem with our live database which data is quite large and non-disclosable. Nevertheless here is the full explain plan for 2.5.5 in case you can discern anything:

      {
      "cursor": "Complex Plan",
      "n": NumberInt(2582),
      "nscannedObjects": NumberInt(0),
      "nscanned": NumberInt(8470527),
      "nscannedObjectsAllPlans": NumberInt(0),
      "nscannedAllPlans": NumberInt(8470527),
      "nYields": NumberInt(66196),
      "nChunkSkips": NumberInt(0),
      "millis": NumberInt(4006),
      "allPlans": [

      { "cursor": "Complex Plan", "n": NumberInt(2582), "nscannedObjects": NumberInt(0), "nscanned": NumberInt(8470527), "nChunkSkips": NumberInt(0) }

      ],
      "server": "edison:27017",
      "stats": {
      "type": "SORT",
      "works": NumberInt(8473112),
      "yields": NumberInt(66196),
      "unyields": NumberInt(66196),
      "invalidates": NumberInt(0),
      "advanced": NumberInt(2582),
      "needTime": NumberInt(8470528),
      "needFetch": NumberInt(0),
      "isEOF": NumberInt(1),
      "forcedFetches": NumberInt(0),
      "children": [
      {
      "type": "FETCH",
      "works": NumberInt(8470528),
      "yields": NumberInt(66196),
      "unyields": NumberInt(66196),
      "invalidates": NumberInt(0),
      "advanced": NumberInt(2582),
      "needTime": NumberInt(8467945),
      "needFetch": NumberInt(0),
      "isEOF": NumberInt(1),
      "alreadyHasObj": NumberInt(0),
      "forcedFetches": NumberInt(0),
      "matchTested": NumberInt(0),
      "children": [
      {
      "type": "AND_HASH",
      "works": NumberInt(8470527),
      "yields": NumberInt(66196),
      "unyields": NumberInt(66196),
      "invalidates": NumberInt(0),
      "advanced": NumberInt(2582),
      "needTime": NumberInt(8467945),
      "needFetch": NumberInt(0),
      "isEOF": NumberInt(1),
      "flaggedButPassed": NumberInt(0),
      "flaggedInProgress": NumberInt(0),
      "mapAfterChild_0": NumberInt(3459),
      "children": [
      {
      "type": "IXSCAN",
      "works": NumberInt(3460),
      "yields": NumberInt(66196),
      "unyields": NumberInt(66196),
      "invalidates": NumberInt(0),
      "advanced": NumberInt(3459),
      "needTime": NumberInt(0),
      "needFetch": NumberInt(0),
      "isEOF": NumberInt(1),
      "keyPattern": "

      { folder: 1 }

      ",
      "bounds":

      { "folder": [ [ "weblogic", "weblogid" ], [ "/^weblogic/", "/^weblogic/" ] ] }

      ,
      "isMultiKey": NumberInt(0),
      "yieldMovedCursor": NumberInt(0),
      "dupsTested": NumberInt(0),
      "dupsDropped": NumberInt(0),
      "seenInvalidated": NumberInt(0),
      "matchTested": NumberInt(0),
      "keysExamined": NumberInt(3460),
      "children": [

      ]
      },
      {
      "type": "IXSCAN",
      "works": NumberInt(8467067),
      "yields": NumberInt(66196),
      "unyields": NumberInt(66196),
      "invalidates": NumberInt(0),
      "advanced": NumberInt(8467067),
      "needTime": NumberInt(0),
      "needFetch": NumberInt(0),
      "isEOF": NumberInt(1),
      "keyPattern": "

      { type: 1 }

      ",
      "bounds":

      { "type": [ [ "file", "file" ] ] }

      ,
      "isMultiKey": NumberInt(0),
      "yieldMovedCursor": NumberInt(0),
      "dupsTested": NumberInt(0),
      "dupsDropped": NumberInt(0),
      "seenInvalidated": NumberInt(0),
      "matchTested": NumberInt(0),
      "keysExamined": NumberInt(8467067),
      "children": [

      ]
      }
      ]
      }
      ]
      }
      ]
      }
      }

            Assignee:
            david.storch@mongodb.com David Storch
            Reporter:
            rgpublic rgpublic
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: