Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17222

WT is doing busy wait while being in recovery

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.0.0-rc9, 3.1.0
    • Affects Version/s: 3.0.0-rc8
    • Component/s: Storage
    • Fully Compatible
    • ALL
    • Hide

      2 replica set WT enabled
      MongoD run w/ the command line below

      start %MongoD%\mongod --replSet EitanRs3 --port 5001 --dbpath F:\data\db1 --wiredTigerCacheSizeGB 1 --storageEngine wiredTiger --logpath d:\data\rs3a.txt

      %MongoD%\mongod --replSet EitanRs3 --port 5002 --dbpath d:\data\db1 --wiredTigerCacheSizeGB 1 --storageEngine wiredTiger --logpath d:\data\rs3.txt

      insert only workload on windows

      Show
      2 replica set WT enabled MongoD run w/ the command line below start %MongoD%\mongod --replSet EitanRs3 --port 5001 --dbpath F:\data\db1 --wiredTigerCacheSizeGB 1 --storageEngine wiredTiger --logpath d:\data\rs3a.txt %MongoD%\mongod --replSet EitanRs3 --port 5002 --dbpath d:\data\db1 --wiredTigerCacheSizeGB 1 --storageEngine wiredTiger --logpath d:\data\rs3.txt insert only workload on windows

      When replica member get into recovering mode, WT is doing busy wait for checking the conditions below every 10 msec and every 1000 msec

      __log_close_server

      /* Wait until the next event. */
      WT_ERR(__wt_cond_wait(session,
      conn->log_close_cond, 10000));

      static void *
      __evict_server(void *arg)
      {

      /* Don't rely on signals: check periodically. */
      WT_ERR(__wt_cond_wait(session, cache->evict_cond, 100000));

      2015-02-07T02:41:24.780-0800 W REPL [rsBackgroundSync] we are too stale to use eitan5:5002 as a sync source

      This is a regression from previous cycles (first time I hit this issue on RC8). And it’s reproducible after ~7 hours

      EitanRs3:PRIMARY> rs.status()
      {
              "set" : "EitanRs3",
              "date" : ISODate("2015-02-07T22:37:05.762Z"),
              "myState" : 1,
              "members" : [
                      {
                              "_id" : 0,
                              "name" : "eitan5:5002",
                              "health" : 1,
                              "state" : 1,
                              "stateStr" : "PRIMARY",
                              "uptime" : 70819,
                              "optime" : Timestamp(1423329892, 681),
                              "optimeDate" : ISODate("2015-02-07T17:24:52Z"),
                              "electionTime" : Timestamp(1423277828, 2),
                              "electionDate" : ISODate("2015-02-07T02:57:08Z"),
                              "configVersion" : 2,
                              "self" : true
                      },
                      {
                              "_id" : 1,
                              "name" : "eitan5:5001",
                              "health" : 1,
                              "state" : 3,
                              "stateStr" : "RECOVERING",
                              "uptime" : 70786,
                              "optime" : Timestamp(1423304149, 756),
                              "optimeDate" : ISODate("2015-02-07T10:15:49Z"),
                              "lastHeartbeat" : ISODate("2015-02-07T22:37:04.951Z"),
                              "lastHeartbeatRecv" : ISODate("2015-02-07T22:37:04.951Z"
      ),
                              "pingMs" : 0,
                              "configVersion" : 2
                      }
              ],
              "ok" : 1
      }
      

        1. rs3.txt
          9.13 MB
          Eitan Klein
        2. rs3a.txt
          708 kB
          Eitan Klein

            Assignee:
            sue.loverso@mongodb.com Susan LoVerso
            Reporter:
            eitan.klein Eitan Klein
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: