Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-19562

Restore error: insertion error with out-of-memory condition

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.4
    • Component/s: Admin, WiredTiger
    • None
    • Environment:
      Gentoo, Docker 1.7.0
    • Linux

      Failed to mongorestore database. MongoDB is running in docker container with memory constraint 4220MB. WiredTiger storage cacheSizeGB is set to 4. Tried to restore backup with 120GB in GridFS and failed:

      2015-07-23T23:48:35.968+0300    [##############..........]  anydb.fs.chunks  69.2 GB/117.9 GB  (58.7%)
      2015-07-23T23:48:36.395+0300    Failed: restore error: anydb.fs.chunks: error restoring from fs.chunks.bson: insertion error: EOF
      
      2015-07-23T20:47:30.965+0000 I WRITE    [conn4] insert anydb.fs.chunks ninserted:211 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 1 } }, Collection: { acquireCount: { w: 1 } } } 245ms
      2015-07-23T20:47:31.381+0000 I WRITE    [conn4] insert anydb.fs.chunks ninserted:207 keyUpdates:0 writeConflicts:0 numYields:0 locks:{ Global: { acquireCount: { r: 1, w: 1 } }, Database: { acquireCount: { w: 1 } }, Collection: { acquireCount: { w: 1 } } } 268ms
      2015-07-23T20:48:02.582+0000 E STORAGE  [conn4] WiredTiger (12) [1437684466:622716][1:0x7f9014b34700], session.commit_transaction: journal/WiredTigerLog.0000000707 write error: failed to write 217600 bytes at offset 64507904: Cannot allocate memory
      2015-07-23T20:48:02.658+0000 I -        [conn4] Invariant failure: s->commit_transaction(s, NULL) resulted in status UnknownError 12: Cannot allocate memory at src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp 305
      2015-07-23T20:48:35.250+0000 E STORAGE  WiredTiger (12) [1437684512:848464][1:0x7f9017d3c700], file:anydb/collection/9--8126075289585441344.wt, session.checkpoint: anydb/collection/9--8126075289585441344.wt write error: failed to write 196608 bytes at offset 73760747520: Cannot allocate memory
      2015-07-23T20:48:35.250+0000 E STORAGE  WiredTiger (12) [1437684515:250525][1:0x7f9017d3c700], checkpoint-server: checkpoint server error: Cannot allocate memory
      2015-07-23T20:48:35.250+0000 E STORAGE  WiredTiger (-31804) [1437684515:250560][1:0x7f9017d3c700], checkpoint-server: the process must exit and restart: WT_PANIC: WiredTiger library panic
      2015-07-23T20:48:35.250+0000 I -        Fatal Assertion 28558
      2015-07-23T20:48:35.682+0000 I CONTROL  
       0xf605f9 0xf09361 0xeece61 0xd893ea 0x1395d89 0x1395f45 0x13963e4 0x132b363 0x7f901bd3d182 0x7f901ae3defd
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"B605F9"},{"b":"400000","o":"B09361"},{"b":"400000","o":"AECE61"},{"b":"400000","o":"9893EA"},{"b":"400000","o":"F95D89"},{"b":"400000","o":"F95F45"},{"b":"400000","o":"F963E4"},{"b":"400000","o":"F2B363"},{"b":"7F901BD35000","o":"8182"},{"b":"7F901AD43000","o":"FAEFD"}],"processInfo":{ "mongodbVersion" : "3.0.4", "gitVersion" : "0481c958daeb2969800511e7475dc66986fa9ed5", "uname" : { "sysname" : "Linux", "release" : "3.17.7-gentoo", "version" : "#2 SMP Sun Dec 28 23:21:45 EET 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFFA0ED4000", "elfType" : 3 }, { "b" : "7F901BD35000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3 }, { "b" : "7F901BB2D000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3 }, { "b" : "7F901B929000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3 }, { "b" : "7F901B625000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F901B31F000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3 }, { "b" : "7F901B109000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F901AD43000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3 }, { "b" : "7F901BF53000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf605f9]
       mongod(_ZN5mongo10logContextEPKc+0xE1) [0xf09361]
       mongod(_ZN5mongo13fassertFailedEi+0x61) [0xeece61]
       mongod(+0x9893EA) [0xd893ea]
       mongod(__wt_eventv+0x489) [0x1395d89]
       mongod(__wt_err+0x95) [0x1395f45]
       mongod(__wt_panic+0x24) [0x13963e4]
       mongod(+0xF2B363) [0x132b363]
       libpthread.so.0(+0x8182) [0x7f901bd3d182]
       libc.so.6(clone+0x6D) [0x7f901ae3defd]
      -----  END BACKTRACE  -----
      2015-07-23T20:48:35.682+0000 I -        
      
      ***aborting after fassert() failure
      

        1. mongod.conf
          1 kB
        2. Dockerfile
          0.5 kB

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            lisio Andrey Kostin
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: