-
Type: Bug
-
Resolution: Won't Fix
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: MMAPv1, Performance, Stability, Storage, Usability
-
Environment:Windows, especially with journaling and heavy insert load
-
Storage Execution
-
Windows
The Windows operating system does not do a good job of maintaining an optimal working set (pages assigned to physical memory) for mongod.exe. From observation, it appears that memory mapped files are given too high a priority and may be allowed to consume so much memory that system responsiveness drops and out-of-memory failures become possible. Linux behaves differently, but on Windows we need to take a more active role in managing our footprint.
On newer versions of Windows (Windows Server 2003 and Windows Server 2003 R2, Vista, Windows 7, Windows Server 2008 and Windows Server 2008 R2) we can use a Windows API (SetProcessWorkingSetSizeEx()) to cap our memory consumption at a level that we determine will work well on the hardware we find ourselves running on. In older versions of Windows (Windows XP) we can use EmptyWorkingSet() or SetProcessWorkingSetSize(handle, -1, -1) to free the entire working set when we approach our predetermined limit. Because of the way Windows first moves pages onto a standby list, these pages can be brought back into the working set quickly ("soft" page faults), so freeing the entire working set is not as bad as you might expect. Also, Windows does a poor enough job of keeping the right pages in the working set that throwing it all away and building up a new working set may be better anyway.
We could also look at tracking the pages that we absolutely know we want in memory and empty the working set even on newer versions of Windows and then fault all of the pages we want back into memory on a worker thread (by reading one byte from each page we want). Some performance testing will be required to see if this is a workable strategy or not.
- is duplicated by
-
SERVER-4300 "Assertion: 10000:out of memory BufBuilder" then crashed on Windows 7 64bits
- Closed
-
SERVER-4478 Mongod should allow a setting to periodically flush memory on windows
- Closed
-
SERVER-4301 Why does Mongo become slower when RAM is about used up
- Closed
- related to
-
SERVER-3911 memory leak with journaling in windows
- Closed
-
SERVER-538 SetProcessWorkingSetSizeEx for win32?
- Closed