Microsoft KB article 2549369 explains how the FILE_FLAG_RANDOM_ACCESS hint used during the CreateFile() call disables read-aheads and prevents automatic unmapping of views after pages are read.

It also states:

Operating system performance may degrade when one or more processes access multiple large files using the CreateFile() API and the FILE_FLAG_RANDOM_ACCESS flag. The performance degradation is due to the system cache consuming available memory

Also referring to Cache and Memory Manager Potential Performance Issues, it states:

This issue is described in detail in KB article 2549369. FILE_FLAG_RANDOM_ACCESS flag is a hint for Cache Manager to keep mapped views of the file in memory as long as possible (until Memory Manager doesn't signal low memory condition). At the same time, this flag instructs Cache Manager to disable prefetching of file data.

This situation has been mitigated to some extent by working set trimming improvements in Windows Server 2012 and Windows Server 2012 R2, but the issue itself needs to be primarily addressed by the application vendor by not using FILE_FLAG_RANDOM_ACCESS.

I am currently working with a customer that has 775GB database size with 128GB memory. WiredTiger cache is set to default of 76GB. Over time the Windows file cache was using 86GB while the total process private memory usage was 32GB. Total amount of free memory was 674MB or 0.52%. If that is not a low memory condition I don't know what is!

As a workaround I have suggested that the OS file system cache be configured to only consume 40% of available memory.

Therefore I am raising this to determine if we should investigate the use of the FILE_FLAG_RANDOM_ACCESS or start instructing customers that experience excessive FS cache usage to configure the maximum limit.