2019-11-29T17:23:49.231+0300 I CONTROL  [main] ***** SERVER RESTARTED *****
2019-11-29T17:23:49.243+0300 I CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2019-11-29T17:23:49.273+0300 I CONTROL  [initandlisten] MongoDB starting : pid=54040 port=27017 dbpath=/var/lib/mongo 64-bit host=abd01.iaae.com
2019-11-29T17:23:49.273+0300 I CONTROL  [initandlisten] db version v4.0.13
2019-11-29T17:23:49.273+0300 I CONTROL  [initandlisten] git version: bda366f0b0e432ca143bc41da54d8732bd8d03c0
2019-11-29T17:23:49.273+0300 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2019-11-29T17:23:49.274+0300 I CONTROL  [initandlisten] allocator: tcmalloc
2019-11-29T17:23:49.274+0300 I CONTROL  [initandlisten] modules: enterprise
2019-11-29T17:23:49.274+0300 I CONTROL  [initandlisten] build environment:
2019-11-29T17:23:49.274+0300 I CONTROL  [initandlisten]     distmod: rhel70
2019-11-29T17:23:49.274+0300 I CONTROL  [initandlisten]     distarch: x86_64
2019-11-29T17:23:49.274+0300 I CONTROL  [initandlisten]     target_arch: x86_64
2019-11-29T17:23:49.274+0300 I CONTROL  [initandlisten] options: { config: "/var/lib/mongo/automation-mongod.conf", net: { bindIp: "0.0.0.0", port: 27017 }, processManagement: { fork: true, timeZoneInfo: "/usr/share/zoneinfo" }, replication: { enableMajorityReadConcern: false, replSetName: "ABD_DB" }, storage: { dbPath: "/var/lib/mongo", engine: "wiredTiger", journal: { enabled: true }, wiredTiger: { engineConfig: { directoryForIndexes: true } } }, systemLog: { destination: "file", logAppend: true, path: "/var/logs/mongodb/mongod.log" } }
2019-11-29T17:23:49.275+0300 W STORAGE  [initandlisten] Detected unclean shutdown - /var/lib/mongo/mongod.lock is not empty.
2019-11-29T17:23:49.275+0300 W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
2019-11-29T17:23:49.275+0300 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=9204M,cache_overflow=(file_max=0M),session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2019-11-29T17:23:56.264+0300 I STORAGE  [initandlisten] WiredTiger message [1575037436:264044][54040:0x7f393540bbc0], txn-recover: Main recovery loop: starting at 214/77890688 to 224/256
2019-11-29T17:23:56.264+0300 I STORAGE  [initandlisten] WiredTiger message [1575037436:264885][54040:0x7f393540bbc0], txn-recover: Recovering log 214 through 224
2019-11-29T17:23:57.317+0300 I STORAGE  [initandlisten] WiredTiger message [1575037437:317200][54040:0x7f393540bbc0], file:index/27--2264927244963632737.wt, txn-recover: Recovering log 215 through 224
2019-11-29T17:24:00.567+0300 I STORAGE  [initandlisten] WiredTiger message [1575037440:567920][54040:0x7f393540bbc0], file:index/27--2264927244963632737.wt, txn-recover: Recovering log 216 through 224
2019-11-29T17:24:03.876+0300 I STORAGE  [initandlisten] WiredTiger message [1575037443:876820][54040:0x7f393540bbc0], file:sizeStorer.wt, txn-recover: Recovering log 217 through 224
2019-11-29T17:24:07.063+0300 I STORAGE  [initandlisten] WiredTiger message [1575037447:63731][54040:0x7f393540bbc0], file:index/37--2264927244963632737.wt, txn-recover: Recovering log 218 through 224
2019-11-29T17:24:10.335+0300 I STORAGE  [initandlisten] WiredTiger message [1575037450:335047][54040:0x7f393540bbc0], file:index/39--2264927244963632737.wt, txn-recover: Recovering log 219 through 224
2019-11-29T17:24:13.615+0300 I STORAGE  [initandlisten] WiredTiger message [1575037453:615731][54040:0x7f393540bbc0], file:index/39--2264927244963632737.wt, txn-recover: Recovering log 220 through 224
2019-11-29T17:24:16.715+0300 I STORAGE  [initandlisten] WiredTiger message [1575037456:715549][54040:0x7f393540bbc0], file:index/39--2264927244963632737.wt, txn-recover: Recovering log 221 through 224
2019-11-29T17:24:19.522+0300 I STORAGE  [initandlisten] WiredTiger message [1575037459:522673][54040:0x7f393540bbc0], file:index/43--2264927244963632737.wt, txn-recover: Recovering log 222 through 224
2019-11-29T17:24:19.524+0300 I STORAGE  [initandlisten] WiredTiger message [1575037459:524626][54040:0x7f393540bbc0], file:index/43--2264927244963632737.wt, txn-recover: Recovering log 223 through 224
2019-11-29T17:24:19.624+0300 I STORAGE  [initandlisten] WiredTiger message [1575037459:624240][54040:0x7f393540bbc0], file:index/43--2264927244963632737.wt, txn-recover: Recovering log 224 through 224
2019-11-29T17:24:19.704+0300 I STORAGE  [initandlisten] WiredTiger message [1575037459:704648][54040:0x7f393540bbc0], file:index/43--2264927244963632737.wt, txn-recover: Set global recovery timestamp: 0
2019-11-29T17:24:20.811+0300 E STORAGE  [initandlisten] WiredTiger error (5) [1575037460:811484][54040:0x7f393540bbc0], file:index/39--2264927244963632737.wt, WT_SESSION.checkpoint: __posix_sync, 99: /var/lib/mongo/index/39--2264927244963632737.wt: handle-sync: fdatasync: Input/output error Raw: [1575037460:811484][54040:0x7f393540bbc0], file:index/39--2264927244963632737.wt, WT_SESSION.checkpoint: __posix_sync, 99: /var/lib/mongo/index/39--2264927244963632737.wt: handle-sync: fdatasync: Input/output error
2019-11-29T17:24:20.811+0300 E STORAGE  [initandlisten] WiredTiger error (-31804) [1575037460:811606][54040:0x7f393540bbc0], file:index/39--2264927244963632737.wt, WT_SESSION.checkpoint: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic Raw: [1575037460:811606][54040:0x7f393540bbc0], file:index/39--2264927244963632737.wt, WT_SESSION.checkpoint: __wt_panic, 494: the process must exit and restart: WT_PANIC: WiredTiger library panic
2019-11-29T17:24:20.811+0300 F -        [initandlisten] Fatal Assertion 50853 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 409
2019-11-29T17:24:20.811+0300 F -        [initandlisten]***aborting after fassert() failure
2019-11-29T17:24:20.836+0300 F -        [initandlisten] Got signal: 6 (Aborted).
 0x5606bfde7ef1 0x5606bfde7109 0x5606bfde75ed 0x7f3931fed5d0 0x7f3931c47207 0x7f3931c488f8 0x5606be307079 0x5606be739206 0x5606be7a3951 0x5606be294791 0x5606be294bab 0x5606be7764b3 0x5606be8739a4 0x5606be7b6459 0x5606be7b734b 0x5606be79cbda 0x5606be7bce49 0x5606be74f397 0x5606be74bca8 0x5606be70ed2a 0x5606be714efb 0x5606be6f6ed2 0x5606beb73b3a 0x5606be37b448 0x5606be37f67f 0x5606be308c09 0x7f3931c333d5 0x5606be379d2f
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"5606BD8BC000","o":"252BEF1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"5606BD8BC000","o":"252B109"},{"b":"5606BD8BC000","o":"252B5ED"},{"b":"7F3931FDE000","o":"F5D0"},{"b":"7F3931C11000","o":"36207","s":"gsignal"},{"b":"7F3931C11000","o":"378F8","s":"abort"},{"b":"5606BD8BC000","o":"A4B079","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"5606BD8BC000","o":"E7D206"},{"b":"5606BD8BC000","o":"EE7951"},{"b":"5606BD8BC000","o":"9D8791","s":"__wt_err_func"},{"b":"5606BD8BC000","o":"9D8BAB","s":"__wt_panic"},{"b":"5606BD8BC000","o":"EBA4B3"},{"b":"5606BD8BC000","o":"FB79A4"},{"b":"5606BD8BC000","o":"EFA459"},{"b":"5606BD8BC000","o":"EFB34B","s":"__wt_txn_checkpoint"},{"b":"5606BD8BC000","o":"EE0BDA"},{"b":"5606BD8BC000","o":"F00E49","s":"__wt_txn_recover"},{"b":"5606BD8BC000","o":"E93397","s":"__wt_connection_workers"},{"b":"5606BD8BC000","o":"E8FCA8","s":"wiredtiger_open"},{"b":"5606BD8BC000","o":"E52D2A","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_"},{"b":"5606BD8BC000","o":"E58EFB","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb"},{"b":"5606BD8BC000","o":"E3AED2"},{"b":"5606BD8BC000","o":"12B7B3A","s":"_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE"},{"b":"5606BD8BC000","o":"ABF448"},{"b":"5606BD8BC000","o":"AC367F","s":"_ZN5mongo11mongoDbMainEiPPcS1_"},{"b":"5606BD8BC000","o":"A4CC09","s":"main"},{"b":"7F3931C11000","o":"223D5","s":"__libc_start_main"},{"b":"5606BD8BC000","o":"ABDD2F"}],"processInfo":{ "mongodbVersion" : "4.0.13", "gitVersion" : "bda366f0b0e432ca143bc41da54d8732bd8d03c0", "compiledModules" : [ "enterprise" ], "uname" : { "sysname" : "Linux", "release" : "3.10.0-957.12.2.el7.x86_64", "version" : "#1 SMP Tue May 14 21:24:32 UTC 2019", "machine" : "x86_64" }, "somap" : [ { "b" : "5606BD8BC000", "elfType" : 3, "buildId" : "1E4C3CE7F1CD1060AE96009AB424CC5BDB1D9503" }, { "b" : "7FFF0D6E3000", "elfType" : 3, "buildId" : "D149FABD4DF969958863221F946E26C130432B47" }, { "b" : "7F3934FC5000", "path" : "/lib64/libldap-2.4.so.2", "elfType" : 3, "buildId" : "F1FADDDE0D21D5F4E2DCADEDD3B85B6E7AAC9883" }, { "b" : "7F3934DB6000", "path" : "/lib64/liblber-2.4.so.2", "elfType" : 3, "buildId" : "3192C56CD451E18EB9F29CB045432BA9C738DD29" }, { "b" : "7F39348FD000", "path" : "/lib64/libnetsnmpmibs.so.31", "elfType" : 3, "buildId" : "26B6C17A2D031E0B3BD83D24D463E613A8671F91" }, { "b" : "7F39346EE000", "path" : "/lib64/libsensors.so.4", "elfType" : 3, "buildId" : "DA4D8D3702D3F0ED057441AC263709F2ED01157B" }, { "b" : "7F39344EA000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "357693C8F1F49D93010C4E31529C07CDD2BD3D08" }, { "b" : "7F3934282000", "path" : "/lib64/librpm.so.3", "elfType" : 3, "buildId" : "BA9B5DF3BBE4764C0BA2AD6B9BBD5E10512BC082" }, { "b" : "7F3934055000", "path" : "/lib64/librpmio.so.3", "elfType" : 3, "buildId" : "069D6EB6C16272B4A4428CAC4087799F18318AA5" }, { "b" : "7F3933DE7000", "path" : "/lib64/libnetsnmpagent.so.31", "elfType" : 3, "buildId" : "F1EEC61134F8FD1553560FAB87D01014AD18D32B" }, { "b" : "7F3933BDC000", "path" : "/lib64/libwrap.so.0", "elfType" : 3, "buildId" : "8C4AA46577D3AA7EBF8338BDFAECC6586EF29906" }, { "b" : "7F39338DA000", "path" : "/lib64/libnetsnmp.so.31", "elfType" : 3, "buildId" : "D443C7B944B5C1E0E09122F430D48099E22619EF" }, { "b" : "7F3933668000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "AEF5E6F2240B55F90E9DF76CFBB8B9D9F5286583" }, { "b" : "7F3933206000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "3593FA778645A59EA272DBBB59D318C60940E792" }, { "b" : "7F3932FE9000", "path" : "/lib64/libsasl2.so.3", "elfType" : 3, "buildId" : "E2F2017F821DD1B9D307DA1A9B8014F2941AEB7B" }, { "b" : "7F3932D9C000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "BCC30853830CD911E58700591830DF51ABCBD7BA" }, { "b" : "7F3932B33000", "path" : "/lib64/libcurl.so.4", "elfType" : 3, "buildId" : "9114859D3C4BEC47A03CA321EE367DCA799638CD" }, { "b" : "7F3932831000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "5B14BE4D749631673523A61074C10959D50F5455" }, { "b" : "7F3932618000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "C444AE61E7CBB716FD9C18A0B46A7FE8F4FCF3E5" }, { "b" : "7F3932410000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "EFDE2029C9A4A20BE5B8D8AE7E6551FF9B5755D2" }, { "b" : "7F39321FA000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "179F202998E429AA1215907F6D4C5C1BB9C90136" }, { "b" : "7F3931FDE000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "96900CB0FF25B26F2BBDF247DE1408242E4773D8" }, { "b" : "7F3931C11000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "EB9F22A3891E5FD3494DFD9ED199E20AE71BB08D" }, { "b" : "7F393521A000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "A527FE72908703C5972AE384E78D1850D1881EE7" }, { "b" : "7F39319BF000", "path" : "/lib64/libssl3.so", "elfType" : 3, "buildId" : "2E28F6A705F2ECEA8460D4716D5D1C24B5DDA5E4" }, { "b" : "7F3931798000", "path" : "/lib64/libsmime3.so", "elfType" : 3, "buildId" : "8D0B4010959C321022DF9CE239277A9D7B34A76A" }, { "b" : "7F393146B000", "path" : "/lib64/libnss3.so", "elfType" : 3, "buildId" : "F5A64BB37FA3972E545EF459A51310F0AB56FA56" }, { "b" : "7F393123B000", "path" : "/lib64/libnssutil3.so", "elfType" : 3, "buildId" : "E0705772325A52C3372FFFB8BDE5F786E2E200D6" }, { "b" : "7F3931037000", "path" : "/lib64/libplds4.so", "elfType" : 3, "buildId" : "084D2194302908913F68B9DCD27DE46FA5B50522" }, { "b" : "7F3930E32000", "path" : "/lib64/libplc4.so", "elfType" : 3, "buildId" : "799B28AD9A5460D78376E2C11260F2E858B95DE3" }, { "b" : "7F3930BF4000", "path" : "/lib64/libnspr4.so", "elfType" : 3, "buildId" : "DE762A28174110911B273E175D54F222B313CFE0" }, { "b" : "7F3930866000", "path" : "/usr/lib64/perl5/CORE/libperl.so", "elfType" : 3, "buildId" : "E2C3C10A756404CC8888CD6CA8DFAD26064EF3CB" }, { "b" : "7F393064C000", "path" : "/lib64/libnsl.so.1", "elfType" : 3, "buildId" : "B1DC715EF11A8ACF2176AE6D34914328F9DBE9CC" }, { "b" : "7F3930415000", "path" : "/lib64/libcrypt.so.1", "elfType" : 3, "buildId" : "740CAD898E29E1F3B73A323CCEC4A7C88911647F" }, { "b" : "7F3930212000", "path" : "/lib64/libutil.so.1", "elfType" : 3, "buildId" : "BFB22A176189006BBF996A7664CCC6BDB2EB9BD9" }, { "b" : "7F3930002000", "path" : "/lib64/libbz2.so.1", "elfType" : 3, "buildId" : "0C85C0386F0CF41EA39969CF7F58A558D1AD3235" }, { "b" : "7F392FDEC000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "B9D5F73428BD6AD68C96986B57BEA3B7CEDB9745" }, { "b" : "7F392FBD4000", "path" : "/lib64/libelf.so.1", "elfType" : 3, "buildId" : "A22999760AD7CF97A32E5A78C34238D56034F9DD" }, { "b" : "7F392F9AE000", "path" : "/lib64/liblzma.so.5", "elfType" : 3, "buildId" : "3B2C97C1937B73A69C412A96D0810C43DF0C6F54" }, { "b" : "7F392F7A4000", "path" : "/lib64/libpopt.so.0", "elfType" : 3, "buildId" : "7AE00165FBAF6920DD5AED6905820DDBAB589E84" }, { "b" : "7F392F57D000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "D2DD4DA3FDE1477D25BFFF80F3A25FDB541A8179" }, { "b" : "7F392F378000", "path" : "/lib64/libcap.so.2", "elfType" : 3, "buildId" : "D594598CEBC1D6DC4ECE2ABF4C29398FDB00587E" }, { "b" : "7F392F16F000", "path" : "/lib64/libacl.so.1", "elfType" : 3, "buildId" : "7F39882FC0B80BE53790C2EAC307D39F7DE1AD6E" }, { "b" : "7F392EF41000", "path" : "/lib64/liblua-5.1.so", "elfType" : 3, "buildId" : "BDD4B9CFC1D3F31D3A5A430D2F9080E020C5B0BA" }, { "b" : "7F392EB82000", "path" : "/lib64/libdb-5.3.so", "elfType" : 3, "buildId" : "746F78257330663AD584998981E6C4D0BF49C940" }, { "b" : "7F392E959000", "path" : "/lib64/libaudit.so.1", "elfType" : 3, "buildId" : "1B298E71E97B742F93F2F6D8FC7BDDA655CDF5EF" }, { "b" : "7F392E670000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "45BAB0BB455BDFA960FDA22E4124CF17B67CC930" }, { "b" : "7F392E46C000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "B4BE1023D9606A88169DF411BF94AF417D7BA1A0" }, { "b" : "7F392E239000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "A9B3906192687CC45D483AE3C58C8AF745A6726A" }, { "b" : "7F392E029000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "94B3BCB669126166B77CDCE6092679A6AA2004C8" }, { "b" : "7F392DE25000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7F392DBF2000", "path" : "/lib64/libidn.so.11", "elfType" : 3, "buildId" : "2B77BBEFFF65E94F3E0B71A4E89BEB68C4B476C5" }, { "b" : "7F392D9C8000", "path" : "/lib64/libssh2.so.1", "elfType" : 3, "buildId" : "689404B6B895EACFAD65BA16E07E5BF0004F5E0C" }, { "b" : "7F392D7C5000", "path" : "/lib64/libfreebl3.so", "elfType" : 3, "buildId" : "B758881F4B6AF6C28C07A1A57713CBD2144628D4" }, { "b" : "7F392D563000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "9CA3D11F018BEEB719CDB34BE800BF1641350D0A" }, { "b" : "7F392D35E000", "path" : "/lib64/libattr.so.1", "elfType" : 3, "buildId" : "2617ECC6738047E207AE3ADD990BD6A34D11B265" }, { "b" : "7F392D158000", "path" : "/lib64/libcap-ng.so.0", "elfType" : 3, "buildId" : "43578677DF613E9D58128ED4AE0C344FBC1E44C0" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x5606bfde7ef1]
 mongod(+0x252B109) [0x5606bfde7109]
 mongod(+0x252B5ED) [0x5606bfde75ed]
 libpthread.so.0(+0xF5D0) [0x7f3931fed5d0]
 libc.so.6(gsignal+0x37) [0x7f3931c47207]
 libc.so.6(abort+0x148) [0x7f3931c488f8]
 mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x5606be307079]
 mongod(+0xE7D206) [0x5606be739206]
 mongod(+0xEE7951) [0x5606be7a3951]
 mongod(__wt_err_func+0x90) [0x5606be294791]
 mongod(__wt_panic+0x39) [0x5606be294bab]
 mongod(+0xEBA4B3) [0x5606be7764b3]
 mongod(+0xFB79A4) [0x5606be8739a4]
 mongod(+0xEFA459) [0x5606be7b6459]
 mongod(__wt_txn_checkpoint+0x1DB) [0x5606be7b734b]
 mongod(+0xEE0BDA) [0x5606be79cbda]
 mongod(__wt_txn_recover+0x939) [0x5606be7bce49]
 mongod(__wt_connection_workers+0x37) [0x5606be74f397]
 mongod(wiredtiger_open+0x1A88) [0x5606be74bca8]
 mongod(_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_+0xAA) [0x5606be70ed2a]
 mongod(_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbbbb+0x67B) [0x5606be714efb]
 mongod(+0xE3AED2) [0x5606be6f6ed2]
 mongod(_ZN5mongo23initializeStorageEngineEPNS_14ServiceContextENS_22StorageEngineInitFlagsE+0x56A) [0x5606beb73b3a]
 mongod(+0xABF448) [0x5606be37b448]
 mongod(_ZN5mongo11mongoDbMainEiPPcS1_+0xF0F) [0x5606be37f67f]
 mongod(main+0x9) [0x5606be308c09]
 libc.so.6(__libc_start_main+0xF5) [0x7f3931c333d5]
 mongod(+0xABDD2F) [0x5606be379d2f]
-----  END BACKTRACE  -----