Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36087

Executing $text statements in conjunction with 'count' or 'sort' provokes Out Of Memory

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.6.6
    • Component/s: Querying
    • None
    • ALL
    • Hide

      1) Create a database with 1 collection

      2) Insert 20 million documents in that collection

      3) documents contain:

      • ID field
      • Text field (with a text index) 'text'
      • Date field (with an index) 'start'

      4) Text field has the value 'andreas.estermann@dynatrace.com' for all documents. Date field values are distributed over time.

      5) Issue the following queries in the mongodb shell:

      db.collection.find({"text": "andreas.estermann@dynatrace.com"})
      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count()
      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25)
      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1})
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }})
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count()
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25)
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1})

      6) Find the following results for the queries

      db.collection.find({"text": "andreas.estermann@dynatrace.com"})

      -> executes within milliseconds, result correct, expected

      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count()

      -> executes within about 1 minute, result correct, expected

      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25)

      -> executes within milliseconds, result correct, expected

      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1})

      -> executes within milliseconds, result correct, expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }})

      -> executes within milliseconds, result correct, expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count()

      -> crashes server because of excessive memory consumption, not expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25)

      -> executes within milliseconds, result correct, expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1})

      -> crashes server because of excessive memory consumption, not expected

      Show
      1) Create a database with 1 collection 2) Insert 20 million documents in that collection 3) documents contain: ID field Text field (with a text index) 'text' Date field (with an index) 'start' 4) Text field has the value 'andreas.estermann@dynatrace.com' for all documents. Date field values are distributed over time. 5) Issue the following queries in the mongodb shell: db.collection.find({"text": "andreas.estermann@dynatrace.com"}) db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count() db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25) db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1}) db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}) db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count() db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25) db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1}) 6) Find the following results for the queries db.collection.find({"text": "andreas.estermann@dynatrace.com"}) -> executes within milliseconds, result correct, expected db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count() -> executes within about 1 minute, result correct, expected db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25) -> executes within milliseconds, result correct, expected db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1}) -> executes within milliseconds, result correct, expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}) -> executes within milliseconds, result correct, expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count() -> crashes server because of excessive memory consumption, not expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25) -> executes within milliseconds, result correct, expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1}) -> crashes server because of excessive memory consumption, not expected

      Executing a full text search and then executing additional functions on the result set seems to result in MongoDB using all the server memory which ultimately leads to a crash.

      The machine used to reproduce the issue only has 4 GB of memory. However, the same behavior can be observed on our production machines that have a lot more memory. It just takes longer (up to 40 minutes) until the memory is used up and the server crashes.

        1. mongod.log
          71 kB
          Andreas Estermann
        2. mongod.log
          541 kB
          Andreas Estermann
        3. metrics.interim
          42 kB
          Andreas Estermann
        4. metrics.interim
          46 kB
          Andreas Estermann
        5. metrics.2018-08-13T14-39-37Z-00000
          405 kB
          Andreas Estermann
        6. metrics.2018-08-13T13-11-05Z-00000
          583 kB
          Andreas Estermann
        7. metrics.2018-08-13T09-48-27Z-00000
          252 kB
          Andreas Estermann
        8. diagnostics.tar.gz
          44.82 MB
          Andreas Estermann

            Assignee:
            david.storch@mongodb.com David Storch
            Reporter:
            estermann@qumram.com Andreas Estermann
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: