Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-14198

Std::set<pointer> and Windows Heap Allocation Reuse produces non-deterministic results

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.6.4, 2.7.2
    • Affects Version/s: 2.7.1
    • Component/s: Storage
    • None
    • Fully Compatible
    • Server 2.7.2

      std::set<pointer> based types are unreliable if the memory addresses are not unique, but could be reused during the lifetime of the process. This also applies to other types such as like std::map.

      In the dump below from Visual Studio 2013 of the mmfiles std::set, we see element 6 contains a paradox.

      -		thingsToFlushWrapper	{_vector={ size=7 } }	mongo::OwnedPointerVector<mongo::MongoFile::Flushable>
      -		_vector	{ size=7 }	std::vector<mongo::MongoFile::Flushable *,std::allocator<mongo::MongoFile::Flushable *> >
      		[size]	7	__int64
      		[capacity]	9	__int64
      +		[0]	0x000000d48e6802c0 {_theFile=0x000000d48a5a5500 {_view_write=0x00000057d6000000 _view_private=0x00000057da000000 ...} ...}	mongo::MongoFile::Flushable *
      +		[1]	0x000000d48e680470 {_theFile=0x000000d48a5a65e0 {fd=0xdddddddddddddddd maphandle=0xdddddddddddddddd ...} ...}	mongo::MongoFile::Flushable *
      +		[2]	0x000000d48e680a10 {_theFile=0x000000d48a5a6ca0 {_view_write=0x0000004014000000 _view_private=0x0000004018000000 ...} ...}	mongo::MongoFile::Flushable *
      +		[3]	0x000000d48e681700 {_theFile=0x000000d48a642288 {_view_write=0x0000000000000000 _view_private=0x0000000000000000 ...} ...}	mongo::MongoFile::Flushable *
      +		[4]	0x000000d48e680740 {_theFile=0x000000d48a643d88 {_view_write=0x000000401c000000 _view_private=0x000000401d000000 ...} ...}	mongo::MongoFile::Flushable *
      +		[5]	0x000000d48e686650 {_theFile=0x000000d48aa07ce8 {_view_write=0x00000057d4000000 _view_private=0x00000057d5000000 ...} ...}	mongo::MongoFile::Flushable *
      -		[6]	0x000000d48e684a30 {_theFile=0x000000d48e2f2a38 {_view_write=0x00000057de000000 _view_private=0x00000057df000000 ...} ...}	mongo::MongoFile::Flushable *
      -		[mongo::WindowsFlushable]	{_theFile=0x000000d48e2f2a38 {_view_write=0x00000057de000000 _view_private=0x00000057df000000 _willNeedRemap=...} ...}	mongo::WindowsFlushable
      +		mongo::MongoFile::Flushable	{...}	mongo::MongoFile::Flushable
      -		_theFile	0x000000d48e2f2a38 {_view_write=0x00000057de000000 _view_private=0x00000057df000000 _willNeedRemap=true ...}	mongo::MemoryMappedFile *
      +		[mongo::DurableMappedFile]	{_view_write=0x00000057de000000 _view_private=0x00000057df000000 _willNeedRemap=true ...}	mongo::DurableMappedFile
      +		mongo::MongoFile	{_filename="D:/data/db/foob.ns" }	mongo::MongoFile
      		fd	0x0000000000000370	void *
      		maphandle	0x00000000000002c0	void *
      -		views	{ size=2 }	std::vector<void *,std::allocator<void *> >
      		[size]	2	__int64
      		[capacity]	2	__int64
      		[0]	0x00000057de000000	void *
      		[1]	0x00000057df000000	void *
      +		[Raw View]	0x000000d48e2f2a78 {...}	std::vector<void *,std::allocator<void *> > *
      		len	16777216	unsigned __int64
      		_uniqueId	0	const unsigned __int64
      +		_flushMutex	{...}	boost::mutex
      		_view	0x00000057ca000000	void *
      		_fd	0x0000000000000270	void *
      +		_filename	"D:/data/db/foob.ns"	std::basic_string<char,std::char_traits<char>,std::allocator<char> >
      +		_flushMutex	{...}	boost::mutex &
      +		__vfptr	0x00007ff68bced080 {mongod.exe!const mongo::WindowsFlushable::`vftable'} {0x00007ff689dc421d {mongod.exe!mongo::WindowsFlushable::`vector deleting destructor'(unsigned int)}, ...}	void * *
      

      The inner class of type mongo::MemoryMappedFile contains a fd of 0x370 while the outer class of mongo::WindowsFlushable contains a fd of 0x270.

      We assume the following invariant will hold:
      mongo::WindowsFlushable::_fd == mongo::MemoryMappedFile::fd

      This is logical since mongo::WindowsFlushable::_fd is simply a copy of mongo::MemoryMappedFile::fd.

      But this invariant will fail to hold under the following scenario.
      1. mongo::WindowsFlushable::_fd <= mongo::MemoryMappedFile::fd
      2. mongo::MemoryMappedFile is deleted
      3. a new mongo::MemoryMappedFile is allocated at the same exact location
      4. mongo::MemoryMappedFile::fd now is set to a new value
      5. Crash since
      mmap_win.cpp contains the following check

                      if ( MongoFile::getAllFiles().count(_theFile) == 0 ) {
                          // this was deleted while we were unlocked
                          return;
                      }
      

      which will incorrectly pass because it believes the file is still allocated.

      In this example, I can validate that this is not just dirty memory because
      1. the item in question exists in mmfiles set
      2. the handle in this case is valid via WinDBG

      0:002> !handle 370 f
      Handle 370
        Type         	File
        Attributes   	0
        GrantedAccess	0x12019f:
               ReadControl,Synch
               Read/List,Write/Add,Append/SubDir/CreatePipe,ReadEA,WriteEA,ReadAttr,WriteAttr
        HandleCount  	2
        PointerCount 	65536
        No Object Specific Information available
      

            Assignee:
            mark.benvenuto@mongodb.com Mark Benvenuto
            Reporter:
            mark.benvenuto@mongodb.com Mark Benvenuto
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: