Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17543

WT secondary fall to “recovery” when MMAP secondary keep going under insert only workload

    • Type: Icon: Bug Bug
    • Resolution: Incomplete
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Replication, Storage
    • ALL

      Version - RC11

      Environment:
      3 members replica set with the setting below:

      • Primary WT (compression enabled 1GB cache)
      • Secondary WT no compression (journal and block)
      • Secondary MMAP

      Machine – Dedicated 6 core machine for WT and 4 core for MMAP

       Used hammer.mongo to do insert only workload , 6 threads
       OS: Windows

      MongoDB shell version: 3.0.0-rc11
      connecting to: 127.0.0.1:5002/test
      EitanRs3a:PRIMARY> rs.status()
      {
              "set" : "EitanRs3a",
              "date" : ISODate("2015-03-11T15:23:10.223Z"),
              "myState" : 1,
              "members" : [
                      {
                              "_id" : 0,
                              "name" : "eitan5:5002",
                              "health" : 1,
                              "state" : 1,
                              "stateStr" : "PRIMARY",
                              "uptime" : 10657,
                              "optime" : Timestamp(1426087390, 2870),
                              "optimeDate" : ISODate("2015-03-11T15:23:10Z"),
                              "electionTime" : Timestamp(1426076744, 2),
                              "electionDate" : ISODate("2015-03-11T12:25:44Z"),
                              "configVersion" : 3,
                              "self" : true
                      },
                      {
                              "_id" : 1,
                              "name" : "eitan1:5002",
                              "health" : 1,
                              "state" : 2,
                              "stateStr" : "SECONDARY",
                              "uptime" : 10572,
                              "optime" : Timestamp(1426087297, 6834),
                              "optimeDate" : ISODate("2015-03-11T15:21:37Z"),
                              "lastHeartbeat" : ISODate("2015-03-11T15:23:09.143Z"),
                              "lastHeartbeatRecv" : ISODate("2015-03-11T15:23:10.112Z"
      ),
                              "pingMs" : 0,
                              "syncingTo" : "eitan5:5002",
                              "configVersion" : 3
                      },
                      {
                              "_id" : 2,
                              "name" : "eitan6:5001",
                              "health" : 1,
                              "state" : 3,
                              "stateStr" : "RECOVERING",
                              "uptime" : 10555,
                              "optime" : Timestamp(1426079421, 2354),
                              "optimeDate" : ISODate("2015-03-11T13:10:21Z"),
                              "lastHeartbeat" : ISODate("2015-03-11T15:23:08.363Z"),
                              "lastHeartbeatRecv" : ISODate("2015-03-11T15:23:08.628Z"
      ),
                              "pingMs" : 1,
                              "configVersion" : 3
                      }
              ],
              "ok" : 1
      }
      

      I will work with michael.grundy@10gen.com to understand the why happen before the replica set fall beyond

        1. MMAPSecondery.txt
          273 kB
          Eitan Klein
        2. rs3primary.txt
          4.64 MB
          Eitan Klein
        3. WTSecondery.txt
          195 kB
          Eitan Klein

            Assignee:
            eitan.klein Eitan Klein
            Reporter:
            eitan.klein Eitan Klein
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: