2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] MongoDB starting : pid=8654 port=27002 dbpath=/tmp/mongodb/2 64-bit host=boris.ghen.be 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] db version v3.4.1 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] git version: 5e103c4f5583e2566a45d740225dc250baacfbd7 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2j 26 Sep 2016 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] allocator: tcmalloc 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] modules: none 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] build environment: 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] distarch: x86_64 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] target_arch: x86_64 2017-02-08T23:16:12.212+0100 I CONTROL [initandlisten] options: { net: { ipv6: true, port: 27002 }, replication: { replSet: "mytest" }, storage: { dbPath: "/tmp/mongodb/2" } } 2017-02-08T23:16:12.250+0100 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=1462M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted. 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2017-02-08T23:16:12.555+0100 I CONTROL [initandlisten] 2017-02-08T23:16:12.643+0100 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/tmp/mongodb/2/diagnostic.data' 2017-02-08T23:16:12.725+0100 I REPL [initandlisten] Did not find local voted for document at startup; NoMatchingDocument: Did not find replica set lastVote document in local.replset.election 2017-02-08T23:16:12.725+0100 I REPL [initandlisten] Did not find local replica set configuration document at startup; NoMatchingDocument: Did not find replica set configuration document in local.system.replset 2017-02-08T23:16:12.726+0100 I NETWORK [thread1] waiting for connections on port 27002 2017-02-08T23:16:25.205+0100 I NETWORK [thread1] connection accepted from [fd1b:229d:8fec::8]:34436 #1 (1 connection now open) 2017-02-08T23:16:25.206+0100 I - [conn1] end connection [fd1b:229d:8fec::8]:34436 (1 connection now open) 2017-02-08T23:16:25.206+0100 I NETWORK [thread1] connection accepted from [fd1b:229d:8fec::8]:34438 #2 (1 connection now open) 2017-02-08T23:16:25.206+0100 I NETWORK [conn2] received client metadata from [fd1b:229d:8fec::8]:34438 conn2: { driver: { name: "NetworkInterfaceASIO-Replication", version: "3.4.1" }, os: { type: "Linux", name: "Arch", architecture: "x86_64", version: "rolling" } } 2017-02-08T23:16:25.207+0100 I ASIO [NetworkInterfaceASIO-Replication-0] Connecting to boris.ghen.be:27001 2017-02-08T23:16:25.207+0100 I ASIO [NetworkInterfaceASIO-Replication-0] Successfully connected to boris.ghen.be:27001 2017-02-08T23:16:25.328+0100 I REPL [replExecDBWorker-0] Starting replication storage threads 2017-02-08T23:16:25.329+0100 I REPL [ReplicationExecutor] New replica set config in use: { _id: "mytest", version: 2, protocolVersion: 1, members: [ { _id: 0, host: "boris.ghen.be:27001", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "boris.ghen.be:27002", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: 2000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('589b98b41942d443613dbe8d') } } 2017-02-08T23:16:25.329+0100 I REPL [ReplicationExecutor] This node is boris.ghen.be:27002 in the config 2017-02-08T23:16:25.329+0100 I REPL [ReplicationExecutor] transition to STARTUP2 2017-02-08T23:16:25.329+0100 I REPL [ReplicationExecutor] Member boris.ghen.be:27001 is now in state PRIMARY 2017-02-08T23:16:25.501+0100 I REPL [initial sync-0] DataReplicator::_anyActiveHandles_inlock returned false as nothing is active. 2017-02-08T23:16:25.585+0100 I REPL [initial sync-0] sync source candidate: boris.ghen.be:27001 2017-02-08T23:16:25.586+0100 I STORAGE [replication-0] dropAllDatabasesExceptLocal 1 2017-02-08T23:16:25.586+0100 I REPL [replication-0] ****** 2017-02-08T23:16:25.586+0100 I REPL [replication-0] creating replication oplog of size: 990MB... 2017-02-08T23:16:25.626+0100 I STORAGE [replication-0] Starting WiredTigerRecordStoreThread local.oplog.rs 2017-02-08T23:16:25.627+0100 I STORAGE [replication-0] The size storer reports that the oplog contains 0 records totaling to 0 bytes 2017-02-08T23:16:25.627+0100 I STORAGE [replication-0] Scanning the oplog to determine where to place markers for truncation 2017-02-08T23:16:25.850+0100 I REPL [replication-0] ****** 2017-02-08T23:16:25.850+0100 I ASIO [NetworkInterfaceASIO-RS-0] Connecting to boris.ghen.be:27001 2017-02-08T23:16:25.851+0100 I ASIO [NetworkInterfaceASIO-RS-0] Successfully connected to boris.ghen.be:27001 2017-02-08T23:16:25.851+0100 I ASIO [NetworkInterfaceASIO-RS-0] Connecting to boris.ghen.be:27001 2017-02-08T23:16:25.852+0100 I ASIO [NetworkInterfaceASIO-RS-0] Successfully connected to boris.ghen.be:27001 2017-02-08T23:16:25.853+0100 I REPL [replication-0] CollectionCloner::start called, on ns:admin.system.version 2017-02-08T23:16:25.917+0100 I INDEX [InitialSyncInserters-admin.system.version0] build index on: admin.system.version properties: { v: 2, key: { version: 1 }, name: "incompatible_with_version_32", ns: "admin.system.version" } 2017-02-08T23:16:25.917+0100 I INDEX [InitialSyncInserters-admin.system.version0] building index using bulk method; build may temporarily use up to 500 megabytes of RAM 2017-02-08T23:16:25.958+0100 I INDEX [InitialSyncInserters-admin.system.version0] build index on: admin.system.version properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "admin.system.version" } 2017-02-08T23:16:25.958+0100 I INDEX [InitialSyncInserters-admin.system.version0] building index using bulk method; build may temporarily use up to 500 megabytes of RAM 2017-02-08T23:16:25.958+0100 I COMMAND [InitialSyncInserters-admin.system.version0] setting featureCompatibilityVersion to 3.4 2017-02-08T23:16:25.991+0100 I REPL [repl writer worker 15] data clone finished, status: OK 2017-02-08T23:16:25.991+0100 I REPL [replication-0] No need to apply operations. (currently at { : Timestamp 1486592185000|1 }) 2017-02-08T23:16:25.991+0100 I REPL [initial sync-0] Initial sync attempt finishing up. 2017-02-08T23:16:25.992+0100 I REPL [initial sync-0] Initial Sync Statistics: { failedInitialSyncAttempts: 0, maxFailedInitialSyncAttempts: 10, initialSyncStart: new Date(1486592185329), initialSyncEnd: new Date(1486592185992), initialSyncElapsedMillis: 663, initialSyncAttempts: [ { durationMillis: 140, status: "OK", syncSource: "boris.ghen.be:27001" } ], fetchedMissingDocs: 0, appliedOps: 0, initialSyncOplogStart: Timestamp 1486592185000|1, initialSyncOplogEnd: Timestamp 1486592185000|1, databases: { databasesCloned: 1, admin: { collections: 1, clonedCollections: 1, start: new Date(1486592185853), end: new Date(1486592185991), elapsedMillis: 138, admin.system.version: { documentsToCopy: 1, documentsCopied: 1, indexes: 2, fetchedBatches: 1, start: new Date(1486592185853), end: new Date(1486592185991), elapsedMillis: 138 } } } } 2017-02-08T23:16:25.992+0100 I REPL [replication-0] Finished fetching oplog during initial sync: CallbackCanceled: Callback canceled. Last fetched optime and hash: { ts: Timestamp 1486592185000|1, t: 1 }[-6259125569176361901] 2017-02-08T23:16:25.992+0100 I REPL [initial sync-0] DataReplicator::_anyActiveHandles_inlock returned false as nothing is active. 2017-02-08T23:16:25.992+0100 I REPL [initial sync-0] DataReplicator::_anyActiveHandles_inlock returned false as nothing is active. 2017-02-08T23:16:26.007+0100 I REPL [initial sync-0] initial sync done; took 0s. 2017-02-08T23:16:26.016+0100 I REPL [initial sync-0] Starting replication fetcher thread 2017-02-08T23:16:26.016+0100 I REPL [initial sync-0] Starting replication applier thread 2017-02-08T23:16:26.016+0100 I REPL [initial sync-0] Starting replication reporter thread 2017-02-08T23:16:26.017+0100 I REPL [rsSync] transition to RECOVERING 2017-02-08T23:16:26.017+0100 I REPL [rsBackgroundSync] could not find member to sync from 2017-02-08T23:16:26.017+0100 I REPL [rsSync] transition to SECONDARY 2017-02-08T23:16:36.019+0100 I REPL [ReplicationExecutor] Cannot find self in new replica set configuration; I must be removed; NodeNotFound: No host described in new configuration 3 for replica set mytest maps to this node 2017-02-08T23:16:36.019+0100 I REPL [ReplicationExecutor] New replica set config in use: { _id: "mytest", version: 3, protocolVersion: 1, members: [ { _id: 0, host: "boris.ghen.be:27001", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: 2000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('589b98b41942d443613dbe8d') } } 2017-02-08T23:16:36.019+0100 I REPL [ReplicationExecutor] This node is not a member of the config 2017-02-08T23:16:36.019+0100 I REPL [ReplicationExecutor] transition to REMOVED 2017-02-08T23:16:36.019+0100 I NETWORK [ReplicationExecutor] legacy transport layer closing all connections 2017-02-08T23:16:36.019+0100 I NETWORK [ReplicationExecutor] Skip closing connection for connection # 2 2017-02-08T23:16:41.420+0100 I NETWORK [thread1] connection accepted from [fd1b:229d:8fec::8]:34962 #3 (2 connections now open) 2017-02-08T23:16:41.420+0100 I - [conn3] end connection [fd1b:229d:8fec::8]:34962 (2 connections now open) 2017-02-08T23:16:46.422+0100 I REPL [ReplicationExecutor] New replica set config in use: { _id: "mytest", version: 4, protocolVersion: 1, members: [ { _id: 0, host: "boris.ghen.be:27001", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "boris.ghen.be:27002", arbiterOnly: true, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: 2000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('589b98b41942d443613dbe8d') } } 2017-02-08T23:16:46.422+0100 I REPL [ReplicationExecutor] This node is boris.ghen.be:27002 in the config 2017-02-08T23:16:46.422+0100 I REPL [ReplicationExecutor] transition to ARBITER 2017-02-08T23:16:47.019+0100 I REPL [rsBackgroundSync] sync source candidate: boris.ghen.be:27001 2017-02-08T23:16:47.020+0100 I - [rsSync] Fatal assertion 34361 OplogOutOfOrder: Attempted to apply an oplog entry ({ ts: Timestamp 1486592192000|1, t: 1 }) which is not greater than our last applied OpTime ({ ts: Timestamp 1486592201000|1, t: 1 }). at src/mongo/db/repl/sync_tail.cpp 808 2017-02-08T23:16:47.020+0100 I - [rsSync] ***aborting after fassert() failure 2017-02-08T23:16:47.024+0100 F - [rsSync] Got signal: 6 (Aborted). 0x56437b3ee3ea 0x56437b3ed529 0x56437b3ed9f6 0x7f896025c080 0x7f895fee004f 0x7f895fee147a 0x56437a6790e5 0x56437aec7af0 0x56437aea7cf9 0x7f8960a3e58f 0x7f8960252454 0x7f895ff957df ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"564379F82000","o":"146C3EA","s":"_ZN5mongo15printStackTraceERSo"},{"b":"564379F82000","o":"146B529"},{"b":"564379F82000","o":"146B9F6"},{"b":"7F896024B000","o":"11080"},{"b":"7F895FEAD000","o":"3304F","s":"gsignal"},{"b":"7F895FEAD000","o":"3447A","s":"abort"},{"b":"564379F82000","o":"6F70E5","s":"_ZN5mongo42fassertFailedWithStatusNoTraceWithLocationEiRKNS_6StatusEPKcj"},{"b":"564379F82000","o":"F45AF0","s":"_ZN5mongo4repl8SyncTail16oplogApplicationEPNS0_22ReplicationCoordinatorE"},{"b":"564379F82000","o":"F25CF9","s":"_ZN5mongo4repl10RSDataSync4_runEv"},{"b":"7F8960983000","o":"BB58F"},{"b":"7F896024B000","o":"7454"},{"b":"7F895FEAD000","o":"E87DF","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.4.1", "gitVersion" : "5e103c4f5583e2566a45d740225dc250baacfbd7", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.9.8-1-ck-core2", "version" : "#1 SMP PREEMPT Sat Feb 4 19:07:18 EST 2017", "machine" : "x86_64" }, "somap" : [ { "b" : "564379F82000", "elfType" : 3, "buildId" : "0EDFC080723A2C07416982449BDEE04B7F49A47C" }, { "b" : "7FFDD67D5000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "0DA094797D07E1F73A7EB40890C8D505CD52D7BE" }, { "b" : "7F896365C000", "path" : "/usr/lib/libz.so.1", "elfType" : 3, "buildId" : "E9416977C7BA82FD5C643A5BBD71D707040BD09E" }, { "b" : "7F8963454000", "path" : "/usr/lib/libsnappy.so.1", "elfType" : 3, "buildId" : "E4F75978F93D4568DF94DBB86B37354C69F57F5E" }, { "b" : "7F89631E1000", "path" : "/usr/lib/libpcre.so.1", "elfType" : 3, "buildId" : "C7BC8017E62BA0A20D39E0E21543D6C4691241E3" }, { "b" : "7F8962FD7000", "path" : "/usr/lib/libpcrecpp.so.0", "elfType" : 3, "buildId" : "F1973FF166366F215A973CC306A19157F11FC3FB" }, { "b" : "7F8962C9D000", "path" : "/usr/lib/libwiredtiger-2.9.1.so", "elfType" : 3, "buildId" : "A2AD23863DF162D4B7698DF42C2E0E8D78521036" }, { "b" : "7F8962A1C000", "path" : "/usr/lib/libyaml-cpp.so.0.5", "elfType" : 3, "buildId" : "214F88F71D23F3042072C4123128BD1DDF9B356B" }, { "b" : "7F896279E000", "path" : "/usr/lib/libboost_program_options.so.1.63.0", "elfType" : 3, "buildId" : "B5787CB3481D49E19A067D0F17676084EAF596DB" }, { "b" : "7F8962584000", "path" : "/usr/lib/libboost_filesystem.so.1.63.0", "elfType" : 3, "buildId" : "67C46E5B9CC031B1A35D893E34DAB45E3361CF19" }, { "b" : "7F896235C000", "path" : "/usr/lib/libboost_thread.so.1.63.0", "elfType" : 3, "buildId" : "F8145423003600BC8E9D505B841FF53B3627C26F" }, { "b" : "7F8962158000", "path" : "/usr/lib/libboost_system.so.1.63.0", "elfType" : 3, "buildId" : "D5718CC8E19A351106EAACB658DC23ED0628CFDB" }, { "b" : "7F8961F3F000", "path" : "/usr/lib/libboost_iostreams.so.1.63.0", "elfType" : 3, "buildId" : "B4BEAE9888ABAFE63C050069DAAF47FE07A6C3A7" }, { "b" : "7F8961D38000", "path" : "/usr/lib/libboost_chrono.so.1.63.0", "elfType" : 3, "buildId" : "80C44D3AC0E1F501DA089DAC8B3F497BC5254555" }, { "b" : "7F8961A23000", "path" : "/usr/lib/libboost_regex.so.1.63.0", "elfType" : 3, "buildId" : "A108B940BB80E8F5CBC9B84B042438C864714BF0" }, { "b" : "7F8961806000", "path" : "/usr/lib/libsasl2.so.3", "elfType" : 3, "buildId" : "79D4F55099FE180EE217B9625266ECC3B39F3EFA" }, { "b" : "7F8961595000", "path" : "/usr/lib/libssl.so.1.0.0", "elfType" : 3, "buildId" : "39C6A0FF8859ABE21E0FCCAD73B379216FDA9094" }, { "b" : "7F8961117000", "path" : "/usr/lib/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "A98B4DA51D83CCAE4811973B3DA5235CF5F373B3" }, { "b" : "7F8960F0F000", "path" : "/usr/lib/librt.so.1", "elfType" : 3, "buildId" : "348114FCB35A100A6E346D485EE0BBAFBE691F98" }, { "b" : "7F8960D0B000", "path" : "/usr/lib/libdl.so.2", "elfType" : 3, "buildId" : "17EC69093FC89B16363635EA7F76F46145A08384" }, { "b" : "7F8960983000", "path" : "/usr/lib/libstdc++.so.6", "elfType" : 3, "buildId" : "898BD9EBFA531BC9E963B5310C99826A2F680E64" }, { "b" : "7F896067F000", "path" : "/usr/lib/libm.so.6", "elfType" : 3, "buildId" : "9BF277AF31D8B043B4806962D815937A068FFF4F" }, { "b" : "7F8960468000", "path" : "/usr/lib/libgcc_s.so.1", "elfType" : 3, "buildId" : "4845EA37359F26E6376A3BC6311ED370D7492BDC" }, { "b" : "7F896024B000", "path" : "/usr/lib/libpthread.so.0", "elfType" : 3, "buildId" : "13D24A5495AE34F58E52403F9153263488B62CF4" }, { "b" : "7F895FEAD000", "path" : "/usr/lib/libc.so.6", "elfType" : 3, "buildId" : "8CBA3297F538691EB1875BE62986993C004F3F4D" }, { "b" : "7F895FC39000", "path" : "/usr/lib/libtcmalloc.so.4", "elfType" : 3, "buildId" : "CC7F1829875C1494A87F5F946968E3B202EC3554" }, { "b" : "7F895FA25000", "path" : "/usr/lib/liblz4.so.1", "elfType" : 3, "buildId" : "500F234068990A0F081B80CCC4361259769FDF6E" }, { "b" : "7F895F815000", "path" : "/usr/lib/libbz2.so.1.0", "elfType" : 3, "buildId" : "779B2C2CE5DA3FF35F45FE7BF9517D2B7DA6FDDD" }, { "b" : "7F895DD15000", "path" : "/usr/lib/libicudata.so.58", "elfType" : 3, "buildId" : "81E2101E5546B98156C13769D4939895E171B238" }, { "b" : "7F895D89B000", "path" : "/usr/lib/libicui18n.so.58", "elfType" : 3, "buildId" : "9144415EDD422B10F0E7533C04AF4286AFD01724" }, { "b" : "7F895D4EF000", "path" : "/usr/lib/libicuuc.so.58", "elfType" : 3, "buildId" : "D2177D1B49D8C5F17B1FF2CA622468258FA3F6AA" }, { "b" : "7F895D2D8000", "path" : "/usr/lib/libresolv.so.2", "elfType" : 3, "buildId" : "F010A6ED2580DE89841FC17D540BC28A0420F6A3" }, { "b" : "7F8963873000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "C778954FFB0E2239CD454D40C9DE504E8601D990" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x3A) [0x56437b3ee3ea] mongod(+0x146B529) [0x56437b3ed529] mongod(+0x146B9F6) [0x56437b3ed9f6] libpthread.so.0(+0x11080) [0x7f896025c080] libc.so.6(gsignal+0xCF) [0x7f895fee004f] libc.so.6(abort+0x16A) [0x7f895fee147a] mongod(_ZN5mongo42fassertFailedWithStatusNoTraceWithLocationEiRKNS_6StatusEPKcj+0x0) [0x56437a6790e5] mongod(_ZN5mongo4repl8SyncTail16oplogApplicationEPNS0_22ReplicationCoordinatorE+0x1030) [0x56437aec7af0] mongod(_ZN5mongo4repl10RSDataSync4_runEv+0x239) [0x56437aea7cf9] libstdc++.so.6(+0xBB58F) [0x7f8960a3e58f] libpthread.so.0(+0x7454) [0x7f8960252454] libc.so.6(clone+0x5F) [0x7f895ff957df] ----- END BACKTRACE -----