Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36520

Segmentation fault when add member from another region to replica set

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.6.2
    • Component/s: Replication
    • None
    • ALL
    • Hide

      Start a new instance and wait about 8 min.

      Show
      Start a new instance and wait about 8 min.

      I have a primary instance at America region that has a database (size about 70GB), and I'm trying to add a new member at Europe region, but I got this error after new instance at Europe start about 8 min.

      2018-08-08T09:37:02.009+0000 I NETWORK  [conn60] end connection 159.42.68.252:34862 (1 connection now open)
      2018-08-08T09:37:13.018+0000 I NETWORK  [listener] connection accepted from 159.42.100.149:35586 #61 (2 connections now open)
      2018-08-08T09:37:13.018+0000 I NETWORK  [conn61] end connection 159.42.100.149:35586 (1 connection now open)
      2018-08-08T09:37:13.125+0000 I NETWORK  [listener] connection accepted from 159.42.100.149:35588 #62 (2 connections now open)
      2018-08-08T09:37:13.238+0000 I NETWORK  [conn62] end connection 159.42.100.149:35588 (1 connection now open)
      2018-08-08T09:37:13.695+0000 I NETWORK  [listener] connection accepted from 159.42.68.252:34864 #63 (2 connections now open)
      2018-08-08T09:37:13.695+0000 I NETWORK  [listener] connection accepted from 159.42.68.252:34866 #64 (3 connections now open)
      2018-08-08T09:37:13.695+0000 I NETWORK  [conn63] end connection 159.42.68.252:34864 (2 connections now open)
      2018-08-08T09:37:13.920+0000 F -        [thread65] Invalid access at address: 0x18
      2018-08-08T09:37:13.936+0000 F -        [thread65] Got signal: 11 (Segmentation fault). 0x2afae07721a1 0x2afae07713b9 0x2afae0771a26 0x2afae294e7e0 0x2afae2948470 0x2afadff43806 0x2afadff48591 0x2afadf5d4bce 0x2afadf5d4ef0 0x2afae004779a 0x2afae0048c48 0x2afadf5bbf12 0x2afae0284d4a 0x2afae028570c 0x2afae0285ae4 0x2afae03079e9 0x2afae0307c31 0x2afadf5bae1d 0x2afae087fba0 0x2afae2946aa1 0x2afae2c45bcd
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"2AFADE575000","o":"21FD1A1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"2AFADE575000","o":"21FC3B9"},{"b":"2AFADE575000","o":"21FCA26"},{"b":"2AFAE293F000","o":"F7E0"},{"b":"2AFAE293F000","o":"9470","s":"pthread_mutex_lock"},{"b":"2AFADE575000","o":"19CE806","s":"_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE"},{"b":"2AFADE575000","o":"19D3591","s":"_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE"},{"b":"2AFADE575000","o":"105FBCE","s":"_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE"},{"b":"2AFADE575000","o":"105FEF0","s":"_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE"},{"b":"2AFADE575000","o":"1AD279A","s":"_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE"},{"b":"2AFADE575000","o":"1AD3C48","s":"_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE"},{"b":"2AFADE575000","o":"1046F12"},{"b":"2AFADE575000","o":"1D0FD4A","s":"_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_"},{"b":"2AFADE575000","o":"1D1070C","s":"_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_"},{"b":"2AFADE575000","o":"1D10AE4","s":"_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m"},{"b":"2AFADE575000","o":"1D929E9","s":"_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code"},{"b":"2AFADE575000","o":"1D92C31","s":"_ZN4asio6detail9scheduler3runERSt10error_code"},{"b":"2AFADE575000","o":"1045E1D"},{"b":"2AFADE575000","o":"230ABA0"},{"b":"2AFAE293F000","o":"7AA1"},{"b":"2AFAE2B5D000","o":"E8BCD","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.6.2", "gitVersion" : "489d177dbd0f0420a8ca04d39fd78d0a2c539420", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "2.6.32-696.13.2.el6.x86_64", "version" : "#1 SMP Thu Oct 5 17:03:21 PDT 2017", "machine" : "x86_64" }, "somap" : [ { "b" : "2AFADE575000", "elfType" : 3, "buildId" : "86F94733F5B93AA0FC80E73922EA2BD1DB68D8C4" }, { "b" : "7FFE87A60000", "elfType" : 3, "buildId" : "73026319DA5E5EBC8422A91D5D045B69F2FD5CF0" }, { "b" : "2ABFEF02C000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "F0BE1166EDCFFB2422B940D601A1BBD89352D80F" }, { "b" : "2ABFE6646000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "2C58F69B34107BCE6E0E65B2642CBCE7B9571ECE" }, { "b" : "2ABFE7CB3000", "path" : "/usr/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "35E3C56DFAC90FF49EB65AB43CE40A45F0256DC6" }, { "b" : "2ABFF2898000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "8CE34B59DECD0B577E05E1D9085F7EDD18AD6F17" }, { "b" : "2ABFF36A0000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "1F7E85410384392BC51FA7324961719A10125F31" }, { "b" : "2ABFF24A5000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "1F4B47E6CD57B2A0E10406329448ED33C095FBAF" }, { "b" : "2AFAE2729000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "D0DDA4AC0E92F42FB6D3A3E7D59FEEBACC81D3F3" }, { "b" : "2ABFF353F000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "9704352BD4F54A3839DA5AE80D9C4633FDC600D1" }, { "b" : "2ABFF3B5D000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "E09BB3F6D737C8219B3086F294257BAC8E28B123" }, { "b" : "2ABFF2E09000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "D176C2235EE14AC4CE5763D19CFDE5F2DA83EBD8" }, { "b" : "2ABFE82F1000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "2CBD7B734972204176F168E3BC983371F08D46B0" }, { "b" : "2ABFE8935000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "68864AF40B36D187E9B61CCAD2741DB1A1E647A9" }, { "b" : "2ABFF081D000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "63E88CB382D4FE28625992FEF61B73586121A3FB" }, { "b" : "2ABFE9221000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "13CCE164ACE084DAB18CB6280557C0535FCBDE80" }, { "b" : "2ABFF3C4D000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "5FA8E5038EC04A774AF72A9BB62DC86E1049C4D6" }, { "b" : "2ABFE9E64000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "3E8E0EB7EDCBF0C22F201C8E56EB2CB7F5017490" }, { "b" : "2ABFEA86F000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "AF374BAFB7F5B139A0B431D3F06D82014AFF3251" }, { "b" : "2ABFF3A73000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "B4576BE308DDCF7BC31F7304E4734C3D846D0236" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x2afae07721a1]
       mongod(+0x21FC3B9) [0x2afae07713b9]
       mongod(+0x21FCA26) [0x2afae0771a26]
       libpthread.so.0(+0xF7E0) [0x2afae294e7e0]
       libpthread.so.0(pthread_mutex_lock+0x0) [0x2afae2948470]
       mongod(_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE+0x46) [0x2afadff43806]
       mongod(_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE+0x41) [0x2afadff48591]
       mongod(_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE+0x10E) [0x2afadf5d4bce]
       mongod(_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE+0x20) [0x2afadf5d4ef0]
       mongod(_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE+0x12A) [0x2afae004779a]
       mongod(_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE+0x28) [0x2afae0048c48]
       mongod(+0x1046F12) [0x2afadf5bbf12]
       mongod(_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_+0x7A) [0x2afae0284d4a]
       mongod(_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_+0x3AC) [0x2afae028570c]
       mongod(_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m+0x164) [0x2afae0285ae4]
       mongod(_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code+0x389) [0x2afae03079e9]
       mongod(_ZN4asio6detail9scheduler3runERSt10error_code+0xD1) [0x2afae0307c31]
       mongod(+0x1045E1D) [0x2afadf5bae1d]
       mongod(+0x230ABA0) [0x2afae087fba0]
       libpthread.so.0(+0x7AA1) [0x2afae2946aa1]
       libc.so.6(clone+0x6D) [0x2afae2c45bcd]
      -----  END BACKTRACE  -----
      

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            jongchana.phuwasate JONGCHANA PHUWASATE
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: