Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-3157

Enforce maxTimeMS for every query sent to the server

    • Type: Icon: Story Story
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.19.0
    • Affects Version/s: 1.16.0-beta.7, 1.15.4
    • Component/s: CRUD, Query, Schema
    • None
    • 3
    • Needed
    • Hide

      The querybar now contains a new box to specify maxTimeMS (see attachment)

      Show
      The querybar now contains a new box to specify maxTimeMS (see attachment)
    • Iteration Antelope, Iteration Bison

      Currently query filters that can be entered in the "Documents" panel don't use the maxtimems parameter. It is possible with non selective filters not supported by indexes to cause to run very heavy queries (COLLSCAN) on the MongoDB server causing possibly performance issues for the other clients. The parameter maxTimeMS is not set in the count and in the find commands sent.

      For example I loaded 5.7M tweets on a test MongoDB instances and I tried to use the filter

      {source: "a"}

      which causes a full collection scan over 5.7M documents:

      2018-09-28T10:42:42.588+0200 I NETWORK  [conn6] received client metadata from 127.0.0.1:51337 conn6: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.6.6" }, os: { type: "Darwin", name: "Mac OS X", architecture: "x86_64", version: "17.7.0" } }
      2018-09-28T10:44:52.172+0200 I COMMAND  [conn5] command test.tweets command: count { count: "tweets", query: { source: "a" }, skip: 0, lsid: { id: UUID("884572c0-9842-4d2f-ada2-72250fc64b9a") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 numYields:47574 reslen:44 locks:{ Global: { acquireCount: { r: 95150 } }, Database: { acquireCount: { r: 47575 } }, Collection: { acquireCount: { r: 47575 } } } protocol:op_query 137667ms
      2018-09-28T10:46:53.077+0200 I COMMAND  [conn1] command test.tweets appName: "MongoDB Compass Beta" command: find { find: "tweets", filter: { source: "a" }, limit: 20, returnKey: false, showRecordId: false, lsid: { id: UUID("884572c0-9842-4d2f-ada2-72250fc64b9a") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 cursorExhausted:1 numYields:47243 nreturned:0 reslen:99 locks:{ Global: { acquireCount: { r: 94488 } }, Database: { acquireCount: { r: 47244 } }, Collection: { acquireCount: { r: 47244 } } } protocol:op_query 120877ms
      

      The query is limited to 20 documents but has no maxTimeMS set.

      If you use the schema tab and you run the same filter, maxTimeMS is used, but I noticed that the same queries are then run without the timeout set:

      2018-09-28T13:33:44.380+0200 I COMMAND  [conn10] command test.tweets command: count { count: "tweets", query: { source: "a" }, maxTimeMS: 10000, lsid: { id: UUID("bbda3184-e246-44bd-a5dd-43650f767757") }, $db: "test" } planSummary: COLLSCAN numYields:5903 reslen:182 locks:{ Global: { acquireCount: { r: 11808 } }, Database: { acquireCount: { r: 5904 } }, Collection: { acquireCount: { r: 5904 } } } protocol:op_query 10019ms
      2018-09-28T13:33:44.390+0200 I COMMAND  [conn8] command test.tweets command: count { count: "tweets", query: { source: "a" }, maxTimeMS: 10000, lsid: { id: UUID("e927734d-2de4-4d15-b958-5ed758860118") }, $db: "test" } planSummary: COLLSCAN numYields:5916 reslen:182 locks:{ Global: { acquireCount: { r: 11834 } }, Database: { acquireCount: { r: 5917 } }, Collection: { acquireCount: { r: 5917 } } } protocol:op_query 10017ms
      
      2018-09-28T13:36:03.315+0200 I COMMAND  [conn11] command test.tweets command: count { count: "tweets", query: { source: "a" }, skip: 0, lsid: { id: UUID("f153a56e-d826-4256-bc41-3402c718bf42") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 numYields:48309 reslen:44 locks:{ Global: { acquireCount: { r: 96620 } }, Database: { acquireCount: { r: 48310 } }, Collection: { acquireCount: { r: 48310 } } } protocol:op_query 148950ms
      2018-09-28T13:38:24.293+0200 I COMMAND  [conn10] command test.tweets command: find { find: "tweets", filter: { source: "a" }, limit: 20, returnKey: false, showRecordId: false, lsid: { id: UUID("f153a56e-d826-4256-bc41-3402c718bf42") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 cursorExhausted:1 numYields:47984 nreturned:0 reslen:99 locks:{ Global: { acquireCount: { r: 95970 } }, Database: { acquireCount: { r: 47985 } }, Collection: { acquireCount: { r: 47985 } } } protocol:op_query 140951ms
      

      I tested this with Compass 1.16.0 beta 7.

        1. image-2019-05-16-09-52-37-178.png
          57 kB
          Massimiliano Marcon

            Assignee:
            durran.jordan@mongodb.com Durran Jordan
            Reporter:
            emilio.scalise@mongodb.com Emilio Scalise
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: