When replica member get into recovering mode, WT is doing busy wait for checking the conditions below every 10 msec and every 1000 msec
__log_close_server
/* Wait until the next event. */
WT_ERR(__wt_cond_wait(session,
conn->log_close_cond, 10000));
static void *
__evict_server(void *arg)
{
/* Don't rely on signals: check periodically. */
WT_ERR(__wt_cond_wait(session, cache->evict_cond, 100000));
2015-02-07T02:41:24.780-0800 W REPL [rsBackgroundSync] we are too stale to use eitan5:5002 as a sync source
This is a regression from previous cycles (first time I hit this issue on RC8). And it’s reproducible after ~7 hours
EitanRs3:PRIMARY> rs.status() { "set" : "EitanRs3", "date" : ISODate("2015-02-07T22:37:05.762Z"), "myState" : 1, "members" : [ { "_id" : 0, "name" : "eitan5:5002", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 70819, "optime" : Timestamp(1423329892, 681), "optimeDate" : ISODate("2015-02-07T17:24:52Z"), "electionTime" : Timestamp(1423277828, 2), "electionDate" : ISODate("2015-02-07T02:57:08Z"), "configVersion" : 2, "self" : true }, { "_id" : 1, "name" : "eitan5:5001", "health" : 1, "state" : 3, "stateStr" : "RECOVERING", "uptime" : 70786, "optime" : Timestamp(1423304149, 756), "optimeDate" : ISODate("2015-02-07T10:15:49Z"), "lastHeartbeat" : ISODate("2015-02-07T22:37:04.951Z"), "lastHeartbeatRecv" : ISODate("2015-02-07T22:37:04.951Z" ), "pingMs" : 0, "configVersion" : 2 } ], "ok" : 1 }