Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8927

Add a new administrative command to provide statistics about the memory cache of all documents in a collection

    • Type: Icon: New Feature New Feature
    • Resolution: Won't Fix
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: Diagnostics
    • None

      SERVER-7593 adds the (experimental) administrative command pagesInRAM.

      The command is helpful but can be misleading if one is interested in how many documents of a collection are in memory (cached).

      Example of pagesInRAM()

      Given collection with "nrecords" : 155219, "deletedCount" : 80617.
      Run a query to page-in all documents (they need to fit in RAM):

      db.coll.find({doesnexist:1})
      

      The output of db.coll.getPagesInRAM() is the following:

        0     size [===================================                                   ]   50.00%
        1     size [=========                                                             ]   12.50%
        2     size [====================                                                  ]   28.13%
        3     size [=====                                                                 ]    6.25%
        4     size [=====                                                                 ]    6.25%
        5     size [=                                                                     ]    0.78%
        6     size [=                                                                     ]    0.04%
        7     size [=                                                                     ]    0.54%
        8     size [=                                                                     ]    0.03%
        9     size [===========================                                           ]   37.15%
       10     size [======================================================================]  100.00%
       11     size [======================================================================]  100.00%
       12     size [======================================================================]  100.00%
       13     size [================================                                      ]   44.33%
      

      Still, all documents are in memory although the output might be interpreted differently.
      Deleted documents need not be in RAM in order to cache all documents that are alive.

      documentsInRAM()

      I propose to add a subcommand documentsInRAM, that doesn't scan extents, but uses the id index to scan through all non-deleted documents and check if they are in memory. Furthermore, the command can output an estimate of memory fragmentation caused by the page-structure of virtual memory.

      Example of documentsInRAM()

      > db.coll.documentsInRAM()
      {
              "inMem" : 1,
              "nscanned" : 155219,
              "netBytesInRAM" : 95626152,
              "pageBytesInRAM" : 98111488,
              "memFragmentation" : 0.02533175319897296,
              "ok" : 1
      }
      

      All documents are fully in memory ("inMem" : 1.0). The memory fragmentation is only about 2.5% since the overhead caused by virtual memory paging (pageBytesInRAM - netBytesInRAM) is low in this case.

            Assignee:
            Unassigned Unassigned
            Reporter:
            bwaldvogel Benedikt Waldvogel
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: