***** SERVER RESTARTED ***** Wed Sep 19 07:45:19 BackgroundJob starting: DataFileSync Wed Sep 19 07:45:19 shardKeyTest passed Wed Sep 19 07:45:19 versionCmpTest passed Wed Sep 19 07:45:19 versionArrayTest passed Wed Sep 19 07:45:19 isInRangeTest passed Wed Sep 19 07:45:19 shardObjTest passed Wed Sep 19 07:45:19 [initandlisten] MongoDB starting : pid=19989 port=27017 dbpath=/var/lib/mongodb 64-bit host=REDACTED Wed Sep 19 07:45:19 [initandlisten] db version v2.2.0, pdfile version 4.5 Wed Sep 19 07:45:19 [initandlisten] git version: f5e83eae9cfbec7fb7a071321928f00d1b0c5207 Wed Sep 19 07:45:19 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49 Wed Sep 19 07:45:19 [initandlisten] options: { auth: "true", bind_ip: "127.0.0.1", config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", fork: "true", logappend: "true", logpath: "/var/log/mongodb/mongodb.log", nohttpinterface: "true", verbose: true } Wed Sep 19 07:45:19 [initandlisten] flushing directory /var/lib/mongodb Wed Sep 19 07:45:19 [initandlisten] journal dir=/var/lib/mongodb/journal Wed Sep 19 07:45:19 [initandlisten] recover : no journal files present, no recovery needed Wed Sep 19 07:45:19 [initandlisten] flushing directory /var/lib/mongodb/journal Wed Sep 19 07:45:19 [initandlisten] flushing directory /var/lib/mongodb/journal Wed Sep 19 07:45:19 [initandlisten] opening db: kpsommar3 Wed Sep 19 07:45:19 [initandlisten] query kpsommar3.system.namespaces query: { options.temp: { $in: [ true, 1 ] } } ntoreturn:0 keyUpdates:0 nreturned:0 reslen:20 18ms Wed Sep 19 07:45:19 [initandlisten] opening db: kpsommar Wed Sep 19 07:45:19 [initandlisten] query kpsommar.system.namespaces query: { options.temp: { $in: [ true, 1 ] } } ntoreturn:0 keyUpdates:0 nreturned:0 reslen:20 0ms Wed Sep 19 07:45:19 [initandlisten] opening db: test Wed Sep 19 07:45:20 [initandlisten] query test.system.namespaces query: { options.temp: { $in: [ true, 1 ] } } ntoreturn:0 keyUpdates:0 nreturned:0 reslen:20 11ms Wed Sep 19 07:45:20 [initandlisten] opening db: kpsommar2 Wed Sep 19 07:45:20 [initandlisten] query kpsommar2.system.namespaces query: { options.temp: { $in: [ true, 1 ] } } ntoreturn:0 keyUpdates:0 nreturned:0 reslen:20 0ms Wed Sep 19 07:45:20 [initandlisten] opening db: kpsommar4 Wed Sep 19 07:45:20 [initandlisten] query kpsommar4.system.namespaces query: { options.temp: { $in: [ true, 1 ] } } ntoreturn:0 keyUpdates:0 nreturned:0 reslen:20 4ms Wed Sep 19 07:45:20 [initandlisten] opening db: local Wed Sep 19 07:45:20 [initandlisten] enter repairDatabases (to check pdfile version #) Wed Sep 19 07:45:20 [initandlisten] kpsommar3 Wed Sep 19 07:45:20 [initandlisten] kpsommar Wed Sep 19 07:45:20 [initandlisten] test Wed Sep 19 07:45:20 [initandlisten] kpsommar2 Wed Sep 19 07:45:20 [initandlisten] kpsommar4 Wed Sep 19 07:45:20 [initandlisten] done repairDatabases Wed Sep 19 07:45:20 BackgroundJob starting: snapshot Wed Sep 19 07:45:20 BackgroundJob starting: ClientCursorMonitor Wed Sep 19 07:45:20 [initandlisten] opening db: admin Wed Sep 19 07:45:20 [initandlisten] fd limit hard:8192 soft:1024 max conn: 819 Wed Sep 19 07:45:20 [initandlisten] waiting for connections on port 27017 Wed Sep 19 07:45:20 BackgroundJob starting: PeriodicTask::Runner Wed Sep 19 07:45:20 BackgroundJob starting: TTLMonitor Wed Sep 19 07:45:35 [initandlisten] connection accepted from 127.0.0.1:36461 #1 (1 connection now open) Wed Sep 19 07:45:35 [conn1] note: no users configured in admin.system.users, allowing localhost access Wed Sep 19 07:45:35 [conn1] run command admin.$cmd { whatsmyuri: 1 } Wed Sep 19 07:45:35 [conn1] command admin.$cmd command: { whatsmyuri: 1 } ntoreturn:1 keyUpdates:0 reslen:62 0ms Wed Sep 19 07:45:35 [conn1] run command admin.$cmd { replSetGetStatus: 1.0, forShell: 1.0 } Wed Sep 19 07:45:35 [conn1] command admin.$cmd command: { replSetGetStatus: 1.0, forShell: 1.0 } ntoreturn:1 keyUpdates:0 reslen:76 0ms Wed Sep 19 07:45:36 [conn1] opening db: kpsommar Wed Sep 19 07:45:36 [conn1] query kpsommar.microlog cursorid:4326036852609868158 ntoreturn:0 keyUpdates:0 locks(micros) W:69288 r:267 nreturned:101 reslen:24015 0ms Wed Sep 19 07:45:36 [conn1] run command admin.$cmd { replSetGetStatus: 1.0, forShell: 1.0 } Wed Sep 19 07:45:36 [conn1] command admin.$cmd command: { replSetGetStatus: 1.0, forShell: 1.0 } ntoreturn:1 keyUpdates:0 reslen:76 0ms Wed Sep 19 07:45:37 [conn1] end connection 127.0.0.1:36461 (0 connections now open) Wed Sep 19 07:45:38 [initandlisten] connection accepted from 127.0.0.1:36462 #2 (1 connection now open) Wed Sep 19 07:45:38 [conn2] run command admin.$cmd { whatsmyuri: 1 } Wed Sep 19 07:45:38 [conn2] command admin.$cmd command: { whatsmyuri: 1 } ntoreturn:1 keyUpdates:0 reslen:62 0ms Wed Sep 19 07:45:38 [conn2] run command admin.$cmd { replSetGetStatus: 1.0, forShell: 1.0 } Wed Sep 19 07:45:38 [conn2] command admin.$cmd command: { replSetGetStatus: 1.0, forShell: 1.0 } ntoreturn:1 keyUpdates:0 reslen:76 0ms Wed Sep 19 07:45:39 [conn2] opening db: kpsommar2 Wed Sep 19 07:45:39 [conn2] ERROR: mmap private failed with out of memory. (64 bit build) Wed Sep 19 07:45:39 [conn2] Assertion: 13636:file /var/lib/mongodb/kpsommar2.1 open/create failed in createPrivateMap (look in log for more information) 0xade6e1 0x8036eb 0x80386c 0xbe2ec3 0xbe3e60 0x738be3 0x88be35 0x88c44f 0x88c65a 0x88cf41 0xaf335b 0xaf3623 0xaf3d3f 0x6b3be7 0xb5ba7d 0xb5d052 0x56fa52 0x5dbf11 0x7fcab42c6a3f 0x7fcab36a771d /usr/sbin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xade6e1] /usr/sbin/mongod(_ZN5mongo11msgassertedEiPKc+0x9b) [0x8036eb] /usr/sbin/mongod() [0x80386c] /usr/sbin/mongod(_ZN5mongo8MongoMMF13finishOpeningEv+0x1e3) [0xbe2ec3] /usr/sbin/mongod(_ZN5mongo8MongoMMF4openESsb+0x70) [0xbe3e60] /usr/sbin/mongod(_ZN5mongo13MongoDataFile12openExistingEPKc+0x93) [0x738be3] /usr/sbin/mongod(_ZN5mongo8Database16openExistingFileEi+0x595) [0x88be35] /usr/sbin/mongod(_ZN5mongo8Database12openAllFilesEv+0x1f) [0x88c44f] /usr/sbin/mongod(_ZN5mongo8DatabaseC1EPKcRbRKSs+0x1da) [0x88c65a] /usr/sbin/mongod(_ZN5mongo14DatabaseHolder11getOrCreateERKSsS2_Rb+0x441) [0x88cf41] /usr/sbin/mongod(_ZN5mongo6Client7Context11_finishInitEb+0x6b) [0xaf335b] /usr/sbin/mongod(_ZN5mongo6Client7ContextC1ERKSsSsbb+0x93) [0xaf3623] /usr/sbin/mongod(_ZN5mongo6Client11ReadContextC1ERKSsSsb+0x24f) [0xaf3d3f] /usr/sbin/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x1567) [0x6b3be7] /usr/sbin/mongod() [0xb5ba7d] /usr/sbin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x3a2) [0xb5d052] /usr/sbin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x82) [0x56fa52] /usr/sbin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x411) [0x5dbf11] /lib64/libpthread.so.0(+0x6a3f) [0x7fcab42c6a3f] /lib64/libc.so.6(clone+0x6d) [0x7fcab36a771d] Wed Sep 19 07:45:39 [conn2] warning database /var/lib/mongodb kpsommar2 could not be opened Wed Sep 19 07:45:39 [conn2] DBException 13636: file /var/lib/mongodb/kpsommar2.1 open/create failed in createPrivateMap (look in log for more information) Wed Sep 19 07:45:39 [conn2] assertion 13636 file /var/lib/mongodb/kpsommar2.1 open/create failed in createPrivateMap (look in log for more information) ns:kpsommar2.microlog query:{} Wed Sep 19 07:45:39 [conn2] problem detected during query over kpsommar2.microlog : { $err: "file /var/lib/mongodb/kpsommar2.1 open/create failed in createPrivateMap (look in log for more information)", code: 13636 } Wed Sep 19 07:45:39 [conn2] query kpsommar2.microlog ntoreturn:0 keyUpdates:0 exception: file /var/lib/mongodb/kpsommar2.1 open/create failed in createPrivateMap (look in log for more information) code:13636 locks(micros) W:15659 r:2 reslen:153 0ms Wed Sep 19 07:45:39 [conn2] run command admin.$cmd { replSetGetStatus: 1.0, forShell: 1.0 } Wed Sep 19 07:45:39 [conn2] command admin.$cmd command: { replSetGetStatus: 1.0, forShell: 1.0 } ntoreturn:1 keyUpdates:0 reslen:76 0ms Wed Sep 19 07:45:42 [conn2] end connection 127.0.0.1:36462 (0 connections now open)