Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5670

externally preallocated data files may be ignored when allocating extents

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.1.2
    • Affects Version/s: None
    • Component/s: Storage
    • None
    • ALL

      In the following scenario externally preallocated data files may be skipped:

      1) data files (test.0,1,2,...) are allocated manually (for example with dd).
      2) Mongod is started and database 'test' is created to use these data files.
      3) Mongod is restarted, and the 'test' database is opened. Because the database already exists at the time it is opened, openAllFiles() is called which increments the database's 'n' to the total number of existing data files minus one.
      4) A collection in the test database grows in size, with growing extent sizes, until the followupSize() call to determine a new collection size returns Extent::maxSize().
      5) Database::suitableFile() is called with Extent::maxSize(). No data file under the Database's 'n' limit can fit an extent of this size because the specified max extent size is equal to the max file size and some room is needed for the file header.
      6) Database::suitableFile() fails to identify an existing file (under the 'n' limit) that can fit an extent of Extent::maxSize(), so it allocates a new file with addAFile() and returns that new file.
      7) Subsequent extent allocations also request a file for an extent of size Extent::maxSize() and skip the unused files as well.

      I did this to demonstrate the issue:

      I manually preallocated some data files

      for i in `seq 0 29`; do dd if=/dev/zero of=test.$i bs=8192 count=262016 ; done
      

      I started mongod and began running this script

      db.createCollection( 'c', { autoIndexId:false } );
      big = new Array( 1000000 ).toString();
      while( 1 ) {
          db.c.insert( { b:big } );
          db.getLastError();
      }
      

      I let the script run for a little while, then stopped and restarted mongod and ran the script again. Most of the data files that were unused by the prior mongod session remained unused as new data files were allocated.

            Assignee:
            milkie@mongodb.com Eric Milkie
            Reporter:
            aaron Aaron Staple
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: