Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-19795

mongod memory consumption higher than WT cache size (at least on Windows 2008 R2)

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: WiredTiger
    • Environment:
      W 2008 R2
    • Windows

      Although mongod.exe process memory is well capped ,as required by the --wiredTigerCacheSizeGB, the OS (Windows 2008 R2) keeps huge part of os memory cache active on files.

      See screen shot of RMAP tool :

      • green : no problem because of use of --wiredTigerEngineConfigString direct_io=[data] , no data file part is cached by the os, no problem.
      • red : the os does keep in ram large parts (see column "Active File"), this is ok , but the issue here is that the sum of ram used because of mongod ( "mongod privateByte" + "os active file" ) exceed --wiredTigerCacheSizeGB , in fact at then end it would have the whole size of the database in memory. And this would lead to OOM we cannot prevent since we cannot cap "OS active file" locked by the OS as asked by WT.

      the "--wiredTigerEngineConfigString direct_io=[data]" is a workaround to this problem , but then it lead to too much slowness for queries and in fact is not applicable for our volumes of process and data.

      Could you make --wiredTigerCacheSizeGB take in acount the whole ram used because of mongo (os+process) , please ?

      Mongo64_2008+\mongod.exe" --port 4444 --dbpath D:\Homeware\XOne_services\PreBETA\MongoData\CurvesFR --directoryperdb --journal --nohttpinterface --wiredTigerCacheSizeGB 1 --wiredTigerDirectoryForIndexes --replSet MongoServiceCacheCurves --oplogSize 1024 --storageEngine wiredTiger --auth --keyFile x.keyfile

        1. WT_RAMMAP.png
          WT_RAMMAP.png
          569 kB
        2. WT_WORKLOAD.png
          WT_WORKLOAD.png
          41 kB
        3. serverstatus.json
          24 kB
        4. HighMemoryUsage.png
          HighMemoryUsage.png
          53 kB
        5. ShutDownMongoD.png
          ShutDownMongoD.png
          58 kB
        6. sqlserver.png
          sqlserver.png
          110 kB
        7. mongo_318.png
          mongo_318.png
          107 kB
        8. OS Caching.png
          OS Caching.png
          52 kB
        9. MongoNoRelease_1.png
          MongoNoRelease_1.png
          152 kB
        10. MongoPartialMemRelease.png
          MongoPartialMemRelease.png
          157 kB
        11. MongoShutdownCacheRelease.png
          MongoShutdownCacheRelease.png
          351 kB
        12. direct_IO_ON.png
          direct_IO_ON.png
          35 kB
        13. 3.1.8 direct IO off.png
          3.1.8 direct IO off.png
          36 kB

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            marc.girollet@sgcib.com Marc Girollet
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: