2015-01-28T00:00:05.194+0000 I NETWORK [initandlisten] connection accepted from 172.20.43.15:53210 #91 (7 connections now open) 2015-01-28T00:00:05.200+0000 I NETWORK [conn91] end connection 172.20.43.15:53210 (6 connections now open) 2015-01-28T06:36:41.241+0000 I NETWORK [initandlisten] connection accepted from 127.0.0.1:37033 #92 (7 connections now open) 2015-01-28T06:41:59.477+0000 I COMMAND [conn92] CMD: drop clm.customer 2015-01-28T06:41:59.845+0000 E CONTROL [journal] 13601 Couldn't remap private view: errno:12 Cannot allocate memory 2015-01-28T06:41:59.845+0000 I CONTROL [journal] aborting 2015-01-28T06:41:59.845+0000 I CONTROL [journal] mem info: vsize: 212426 resident: 4839 mapped: 106477 2015-01-28T06:41:59.881+0000 F - [journal] Got signal: 6 (Aborted). 0xf41389 0xf40cf2 0xf41076 0x3a1e8329a0 0x3a1e832925 0x3a1e834105 0xeef560 0xd185c6 0xd074e4 0xf8e994 0x3a1ec079d1 0x3a1e8e8b5d ----- BEGIN BACKTRACE ----- 0xf41389 0xf40cf2 0xf41076 0x3a1e8329a0 0x3a1e832925 0x3a1e834105 0xeef560 0xd185c6 0xd074e4 0xf8e994 0x3a1ec079d1 0x3a1e8e8b5d ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"B41389"},{"b":"400000","o":"B40CF2"},{"b":"400000","o":"B41076"},{"b":"3A1E800000","o":"329A0"},{"b":"3A1E800000","o":"32925"},{"b":"3A1E800000","o":"34105"},{"b":"400000","o":"AEF560"},{"b":"400000","o":"9185C6"},{"b":"400000","o":"9074E4"},{"b":"400000","o":"B8E994"},{"b":"3A1EC00000","o":"79D1"},{"b":"3A1E800000","o":"E8B5D"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc1", "gitVersion" : "c8171e7f969519af8b87a43425ae291ee69a0191", "uname" : { "sysname" : "Linux", "release" : "2.6.32-431.17.1.el6.x86_64", "version" : "#1 SMP Fri Apr 11 17:27:00 EDT 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFF828FF000", "elfType" : 3 }, { "path" : "/lib64/libpthread.so.0", "elfType" : 3 }, { "path" : "/lib64/librt.so.1", "elfType" : 3 }, { "path" : "/lib64/libdl.so.2", "elfType" : 3 }, { "b" : "7FAE4665E000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3 }, { "path" : "/lib64/libm.so.6", "elfType" : 3 }, { "path" : "/lib64/libgcc_s.so.1", "elfType" : 3 }, { "path" : "/lib64/libc.so.6", "elfType" : 3 }, { "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf41389] mongod(+0xB40CF2) [0xf40cf2] mongod(+0xB41076) [0xf41076] libc.so.6(+0x329A0) [0x3a1e8329a0] libc.so.6(gsignal+0x35) [0x3a1e832925] libc.so.6(abort+0x175) [0x3a1e834105] mongod(_ZN5mongo16MemoryMappedFile16remapPrivateViewEPv+0x140) [0xeef560] mongod(_ZN5mongo17DurableMappedFile19remapThePrivateViewEv+0x26) [0xd185c6] mongod(+0x9074E4) [0xd074e4] mongod(+0xB8E994) [0xf8e994] libpthread.so.0(+0x79D1) [0x3a1ec079d1] libc.so.6(clone+0x6D) [0x3a1e8e8b5d] ----- END BACKTRACE ----- 2015-01-28T06:44:02.634+0000 I CONTROL ***** SERVER RESTARTED ***** 2015-01-28T06:44:02.643+0000 I CONTROL [initandlisten] MongoDB starting : pid=14101 port=27017 dbpath=/u01/data 64-bit host=bapdmong-14.nova.dev 2015-01-28T06:44:02.643+0000 I CONTROL [initandlisten] 2015-01-28T06:44:02.643+0000 I CONTROL [initandlisten] ** WARNING: /proc/sys/vm/overcommit_memory is 2 2015-01-28T06:44:02.643+0000 I CONTROL [initandlisten] ** Journaling works best with it set to 0 or 1 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] db version v2.8.0-rc1 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] git version: c8171e7f969519af8b87a43425ae291ee69a0191 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] allocator: tcmalloc 2015-01-28T06:44:02.644+0000 I CONTROL [initandlisten] options: { processManagement: { fork: true }, replication: { replSet: "repl0" }, storage: { dbPath: "/u01/data", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/u01/logs/log.rep01" } } 2015-01-28T06:44:02.653+0000 I JOURNAL [initandlisten] journal dir=/u01/data/journal 2015-01-28T06:44:02.653+0000 I JOURNAL [initandlisten] recover begin 2015-01-28T06:44:02.653+0000 I JOURNAL [initandlisten] recover lsn: 2192769159 2015-01-28T06:44:02.654+0000 I JOURNAL [initandlisten] recover /u01/data/journal/j._0 2015-01-28T06:44:02.658+0000 I JOURNAL [initandlisten] recover skipping application of section seq:0 < lsn:2192769159 2015-01-28T06:44:02.658+0000 I JOURNAL [initandlisten] recover skipping application of section seq:411593419 < lsn:2192769159 2015-01-28T06:44:02.658+0000 I JOURNAL [initandlisten] recover skipping application of section seq:411771446 < lsn:2192769159 2015-01-28T06:44:02.662+0000 I JOURNAL [initandlisten] recover cleaning up 2015-01-28T06:44:02.662+0000 I JOURNAL [initandlisten] removeJournalFiles 2015-01-28T06:44:02.662+0000 I JOURNAL [initandlisten] recover done 2015-01-28T06:44:02.756+0000 E CONTROL [initandlisten] mmap private failed with out of memory. (64 bit build) 2015-01-28T06:44:02.756+0000 I - [initandlisten] Assertion: 13636:file /u01/data/clm.5 open/create failed in createPrivateMap (look in log for more information) 2015-01-28T06:44:02.780+0000 I CONTROL [initandlisten] 0xf41389 0xee8d61 0xece71f 0xece7cc 0xd19e47 0xd19fca 0xd02f13 0xd33289 0xd2bd76 0xd2e2ca 0x90235c 0x7da9e1 0x7df304 0x3a1e81ed1d 0x7d8259 ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"B41389"},{"b":"400000","o":"AE8D61"},{"b":"400000","o":"ACE71F"},{"b":"400000","o":"ACE7CC"},{"b":"400000","o":"919E47"},{"b":"400000","o":"919FCA"},{"b":"400000","o":"902F13"},{"b":"400000","o":"933289"},{"b":"400000","o":"92BD76"},{"b":"400000","o":"92E2CA"},{"b":"400000","o":"50235C"},{"b":"400000","o":"3DA9E1"},{"b":"400000","o":"3DF304"},{"b":"3A1E800000","o":"1ED1D"},{"b":"400000","o":"3D8259"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc1", "gitVersion" : "c8171e7f969519af8b87a43425ae291ee69a0191", "uname" : { "sysname" : "Linux", "release" : "2.6.32-431.17.1.el6.x86_64", "version" : "#1 SMP Fri Apr 11 17:27:00 EDT 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFFF12FF000", "elfType" : 3 }, { "path" : "/lib64/libpthread.so.0", "elfType" : 3 }, { "path" : "/lib64/librt.so.1", "elfType" : 3 }, { "path" : "/lib64/libdl.so.2", "elfType" : 3 }, { "b" : "7F9A38353000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3 }, { "path" : "/lib64/libm.so.6", "elfType" : 3 }, { "path" : "/lib64/libgcc_s.so.1", "elfType" : 3 }, { "path" : "/lib64/libc.so.6", "elfType" : 3 }, { "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf41389] mongod(_ZN5mongo10logContextEPKc+0xE1) [0xee8d61] mongod(_ZN5mongo11msgassertedEiPKc+0xAF) [0xece71f] mongod(+0xACE7CC) [0xece7cc] mongod(_ZN5mongo17DurableMappedFile13finishOpeningEv+0x377) [0xd19e47] mongod(_ZN5mongo17DurableMappedFile4openERKSsb+0x10A) [0xd19fca] mongod(_ZN5mongo8DataFile12openExistingEPKc+0x83) [0xd02f13] mongod(_ZN5mongo19MmapV1ExtentManager4initEPNS_16OperationContextE+0x119) [0xd33289] mongod(_ZN5mongo26MMAPV1DatabaseCatalogEntryC1EPNS_16OperationContextERKNS_10StringDataES5_bb+0x1F6) [0xd2bd76] mongod(_ZN5mongo12MMAPV1Engine23getDatabaseCatalogEntryEPNS_16OperationContextERKNS_10StringDataE+0x1DA) [0xd2e2ca] mongod(_ZN5mongo14DatabaseHolder6openDbEPNS_16OperationContextERKNS_10StringDataEPb+0x10C) [0x90235c] mongod(_ZN5mongo13initAndListenEi+0xAE1) [0x7da9e1] mongod(main+0x134) [0x7df304] libc.so.6(__libc_start_main+0xFD) [0x3a1e81ed1d] mongod(+0x3D8259) [0x7d8259] ----- END BACKTRACE ----- 2015-01-28T06:44:02.789+0000 I STORAGE [initandlisten] warning database /u01/data clm could not be opened 2015-01-28T06:44:02.789+0000 I STORAGE [initandlisten] DBException 13636: file /u01/data/clm.5 open/create failed in createPrivateMap (look in log for more information) 2015-01-28T06:44:02.789+0000 I STORAGE [initandlisten] exception in initAndListen: 13636 file /u01/data/clm.5 open/create failed in createPrivateMap (look in log for more information), terminating 2015-01-28T06:44:02.789+0000 W REPL [initandlisten] ReplicationCoordinatorImpl::shutdown() called before startReplication() finished. Shutting down without cleaning up the replication system 2015-01-28T06:44:02.789+0000 I COMMAND [initandlisten] now exiting 2015-01-28T06:44:02.789+0000 I NETWORK [initandlisten] shutdown: going to close listening sockets... 2015-01-28T06:44:02.789+0000 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27017.sock 2015-01-28T06:44:02.789+0000 I NETWORK [initandlisten] shutdown: going to flush diaglog... 2015-01-28T06:44:02.789+0000 I NETWORK [initandlisten] shutdown: going to close sockets... 2015-01-28T06:44:02.790+0000 I STORAGE [initandlisten] shutdown: waiting for fs preallocator... 2015-01-28T06:44:02.790+0000 I STORAGE [initandlisten] shutdown: final commit... 2015-01-28T06:44:02.791+0000 I STORAGE [initandlisten] shutdown: closing all files... 2015-01-28T06:44:02.792+0000 I STORAGE [initandlisten] closeAllFiles() finished 2015-01-28T06:44:02.792+0000 I JOURNAL [initandlisten] journalCleanup... 2015-01-28T06:44:02.792+0000 I JOURNAL [initandlisten] removeJournalFiles 2015-01-28T06:44:02.793+0000 I STORAGE [initandlisten] shutdown: removing fs lock... 2015-01-28T06:44:02.793+0000 I COMMAND [initandlisten] dbexit: rc: 100 2015-01-28T06:48:36.420+0000 I CONTROL ***** SERVER RESTARTED ***** 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] MongoDB starting : pid=15090 port=27017 dbpath=/u01/data 64-bit host=bapdmong-14.nova.dev 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] ** WARNING: /proc/sys/vm/overcommit_memory is 2 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] ** Journaling works best with it set to 0 or 1 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] db version v2.8.0-rc1 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] git version: c8171e7f969519af8b87a43425ae291ee69a0191 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] allocator: tcmalloc 2015-01-28T06:48:36.430+0000 I CONTROL [initandlisten] options: { processManagement: { fork: true }, replication: { replSet: "repl0" }, storage: { dbPath: "/u01/data", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/u01/logs/log.rep01" } } 2015-01-28T06:48:36.435+0000 I JOURNAL [initandlisten] journal dir=/u01/data/journal 2015-01-28T06:48:36.435+0000 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed 2015-01-28T06:48:36.534+0000 E CONTROL [initandlisten] mmap private failed with out of memory. (64 bit build)