Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-1383

Poor mongoexport performance when using a query

    • Type: Icon: New Feature New Feature
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.7.1
    • Affects Version/s: 1.5.3
    • Component/s: Performance
    • None
    • Environment:

      mongoexport is potentially very slow when passing in a query argument.

      Below is an example of a shell find() that takes 30ms for 571 result documents, but many minutes when trying to export these documents using mongoexport.

      If it is known for sure that a collection won't change during an export, can mongoexport be told to be more efficient and less strict when creating the output documents? Possibly it's a simple matter of allowing {$snapshot: false} to be specificed, or maybe there is another reason for the large change in efficiency.

      Either way, providing a simple query which uses an index such as the example given, is currently an unfeasible way to export collection subsets, and one must rely on other manual means to achieve this.

        1. Simple query in shell
          > db.articles.find( {"authors.name": "cannon cp"}

          ).explain()
          {
          "cursor" : "BtreeCursor authors.name_1",
          "nscanned" : 571,
          "nscannedObjects" : 571,
          "n" : 571,
          "millis" : 30,
          "indexBounds" : [
          [

          { "authors.name" : "cannon cp" }

          ,

          { "authors.name" : "cannon cp" }

          ]
          ]
          }

        1. Same query in mongoexport takes multiple minutes with high nscanned
          > ./mongoexport -vv -h localhost -c articles -d pubmed -q ' {"authors.name": "cannon cp"}

          ' -o articles.json
          connected to: localhost
          exported 571 records

        1. Server output
          Thu Jul 8 14:58:51 query pubmed.articles reslen:174620 nscanned:1387320
          Unknown macro: { query}

          nreturned:101 39276ms
          Thu Jul 8 15:03:23 getmore pubmed.articles cid:7259446948513230975 getMore:

          Unknown macro: { query}

          bytes:716901 nreturned:470 271669ms
          Thu Jul 8 15:03:24 end connection 127.0.0.1:54779

            Assignee:
            eliot Eliot Horowitz (Inactive)
            Reporter:
            crudson Doug Hudson
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: