Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-58050

mongod standalone sharded replicaset issue during upgrade 4.4.0 to 4.4.6

    • Type: Icon: Question Question
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None

      Hi,

      on our dev env we run ong 4.4.0 binaries consisting of:

      1 standalone config replica set

      2 standalone shards replicaset 

      1 router

      I've disabled balancer, stopped config and started with 4.4.6 binaries.

      This went fine. Then I proceeded with stopping shard0001 and starting with 4.4.6 binaries.

      At this moment it seems to not be progressing for over 20 minutes.

      log of the mongod:

      {"t":{"$date":"2021-06-24T11:10:30.195+02:00"},"s":"I", "c":"CONTROL", "id":20698, "ctx":"main","msg":"***** SERVER RESTARTED *****"}
      {"t":{"$date":"2021-06-24T11:10:30.199+02:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
      {"t":{"$date":"2021-06-24T11:10:30.200+02:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
      {"t":{"$date":"2021-06-24T11:10:30.201+02:00"},"s":"I", "c":"NETWORK", "id":4648602, "ctx":"main","msg":"Implicit TCP FastOpen in use."}
      {"t":{"$date":"2021-06-24T11:10:30.300+02:00"},"s":"I", "c":"STORAGE", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":43037,"port":28018,"dbPath":"/ctr/trademarkrepository/shard/data","architecture":"64-bit","host":"deng-pl111"}}
      {"t":{"$date":"2021-06-24T11:10:30.300+02:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.6","gitVersion":"72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7","openSSLVersion":"OpenSSL 1.0.1e-fips 11 Feb 2013","modules":[],"allocator":"tcmalloc","environment":{"distmod":"rhel62","distarch":"x86_64","target_arch":"x86_64"}}}}
      {"t":{"$date":"2021-06-24T11:10:30.300+02:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Oracle Linux Server release 6.9","version":"Kernel 4.1.12-61.1.28.el6uek.x86_64"}}}
      {"t":{"$date":"2021-06-24T11:10:30.300+02:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/ctr/trademarkrepository/shard/data/automation-mongod.conf","net":{"bindIp":"0.0.0.0","port":28018},"processManagement":{"fork":true},"replication":{"replSetName":"shard0001"},"security":{"keyFile":"/ctr/trademarkrepository/keyfile_mongo_ctr_dvlp"},"setParameter":{"transactionLifetimeLimitSeconds":"600"},"sharding":{"clusterRole":"shardsvr"},"storage":{"dbPath":"/ctr/trademarkrepository/shard/data","engine":"wiredTiger","wiredTiger":{"engineConfig":{"cacheSizeGB":256.0,"directoryForIndexes":true}}},"systemLog":{"destination":"file","logAppend":true,"logRotate":"reopen","path":"/ctr/trademarkrepository/shard/log/mongod.log"}}}}
      {"t":{"$date":"2021-06-24T11:10:30.301+02:00"},"s":"I", "c":"STORAGE", "id":22297, "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
      {"t":{"$date":"2021-06-24T11:10:30.301+02:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=262144M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
      {"t":{"$date":"2021-06-24T11:10:30.897+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1624525830:897381][43037:0x7fe1d6387b40], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 395 through 396"}}
      {"t":{"$date":"2021-06-24T11:10:30.921+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1624525830:921098][43037:0x7fe1d6387b40], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 396 through 396"}}
      {"t":{"$date":"2021-06-24T11:10:31.139+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1624525831:139469][43037:0x7fe1d6387b40], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Main recovery loop: starting at 395/62380416 to 396/256"}}
      {"t":{"$date":"2021-06-24T11:10:31.140+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1624525831:140297][43037:0x7fe1d6387b40], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 395 through 396"}}
      {"t":{"$date":"2021-06-24T11:10:31.168+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1624525831:168326][43037:0x7fe1d6387b40], txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 396 through 396"}}
      {"t":{"$date":"2021-06-24T11:10:31.227+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1624525831:227706][43037:0x7fe1d6387b40], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global recovery timestamp: (1624525800, 1)"}}
      {"t":{"$date":"2021-06-24T11:10:31.227+02:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1624525831:227776][43037:0x7fe1d6387b40], txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global oldest timestamp: (0, 0)"}}

       

       

            Assignee:
            eric.sedor@mongodb.com Eric Sedor
            Reporter:
            kinwai.cheung@clarivate.com Kin Wai Cheung
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: