Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-19795

mongod memory consumption higher than WT cache size (at least on Windows 2008 R2)

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: WiredTiger
    • Environment:
      W 2008 R2
    • Windows

      Although mongod.exe process memory is well capped ,as required by the --wiredTigerCacheSizeGB, the OS (Windows 2008 R2) keeps huge part of os memory cache active on files.

      See screen shot of RMAP tool :

      • green : no problem because of use of --wiredTigerEngineConfigString direct_io=[data] , no data file part is cached by the os, no problem.
      • red : the os does keep in ram large parts (see column "Active File"), this is ok , but the issue here is that the sum of ram used because of mongod ( "mongod privateByte" + "os active file" ) exceed --wiredTigerCacheSizeGB , in fact at then end it would have the whole size of the database in memory. And this would lead to OOM we cannot prevent since we cannot cap "OS active file" locked by the OS as asked by WT.

      the "--wiredTigerEngineConfigString direct_io=[data]" is a workaround to this problem , but then it lead to too much slowness for queries and in fact is not applicable for our volumes of process and data.

      Could you make --wiredTigerCacheSizeGB take in acount the whole ram used because of mongo (os+process) , please ?

      Mongo64_2008+\mongod.exe" --port 4444 --dbpath D:\Homeware\XOne_services\PreBETA\MongoData\CurvesFR --directoryperdb --journal --nohttpinterface --wiredTigerCacheSizeGB 1 --wiredTigerDirectoryForIndexes --replSet MongoServiceCacheCurves --oplogSize 1024 --storageEngine wiredTiger --auth --keyFile x.keyfile

        1. 3.1.8 direct IO off.png
          36 kB
          Nick Judson
        2. direct_IO_ON.png
          35 kB
          Nick Judson
        3. HighMemoryUsage.png
          53 kB
          Eitan Klein
        4. mongo_318.png
          107 kB
          Nick Judson
        5. MongoNoRelease_1.png
          152 kB
          Nick Judson
        6. MongoPartialMemRelease.png
          157 kB
          Nick Judson
        7. MongoShutdownCacheRelease.png
          351 kB
          Nick Judson
        8. OS Caching.png
          52 kB
          Nick Judson
        9. serverstatus.json
          24 kB
          Marc Girollet
        10. ShutDownMongoD.png
          58 kB
          Nick Judson
        11. sqlserver.png
          110 kB
          Nick Judson
        12. WT_RAMMAP.png
          569 kB
          Marc Girollet
        13. WT_WORKLOAD.png
          41 kB
          Marc Girollet

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            marc.girollet@sgcib.com Marc Girollet
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: