Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-231

Failed to retrieve any result when using SlaveOK with all slaves are down

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.5
    • Affects Version/s: 2.3
    • Component/s: Cluster Management
    • None
    • Environment:

      Java driver 2.3.

      We have 3 mongo servers. Each has 48G with 24 CPUs. The servers are running with replica set.

      [joseph.wang@lpsdb1.la2 ~]$ /usr/local/mongodb-linux-x86_64-1.6.3/bin/mongo localhost:4110
      MongoDB shell version: 1.6.3
      connecting to: localhost:4110/test
      > rs.status()
      {
      "set" : "prod",
      "date" : "Sat Dec 11 2010 13:04:28 GMT-0800 (PST)",
      "myState" : 1,
      "members" : [

      { "_id" : 0, "name" : "lpsdb1.la2.estalea.net:4110", "health" : 1, "state" : 1, "self" : true }

      ,

      { "_id" : 1, "name" : "mongo-prod-mem2.lps.la2.estalea.net:4110", "health" : 1, "state" : 2, "uptime" : 11095, "lastHeartbeat" : "Sat Dec 11 2010 13:04:27 GMT-0800 (PST)" }

      ,

      { "_id" : 2, "name" : "mongo-prod-mem3.lps.la2.estalea.net:4110", "health" : 1, "state" : 2, "uptime" : 11178, "lastHeartbeat" : "Sat Dec 11 2010 13:04:26 GMT-0800 (PST)" }

      ],
      "ok" : 1
      }

      The connection pool was connecting to all three servers. We manually brought down two slaves to see if we can get query result (as part of fault tolerance testing).
      There were no update/writes. Just query. When 1 slave was down, we'd no problem getting query result. When 2 slaves were down, we got no query result.
      When a consumer query comes in, we fork out multiple threads, each responsible for fetch data from specific collection.

      MongoConnection.java shows our singleton connection pool code.
      BaseTableQueryEngine.java shows our query to one of our collections.

      As you can see, we set slaveOk at the query level.

      if (db != null) {
      DBCollection coll = db.getCollection(currentCollection);
      DBCursor cur = null;

      cur = coll.find(dbQuery).addOption(
      Bytes.QUERYOPTION_SLAVEOK);

      DBObject dbObject = db.getLastError();
      if (dbObject != null && dbObject.get("err") != null)

      { log.warn("BaseTableQueryEngine: Encounter error for query " + dbQuery.toString()); setSuccess(false); }

      if (enable_debug)

      { log.debug("BaseTableQueryEngine: Run query " + dbQuery.toString()); log.debug("BaseTableQueryEngine: Found " + cur.count() + " in " + (System.currentTimeMillis() - fStart)); }

      while (cur.hasNext()) {
      BasicDBObject dbo = (BasicDBObject) cur.next();
      BaseTableRow row = new BaseTableRow(dbo);
      if (row.isValid())

      { tuples.add(row.getTuple()); }

      long time = (timeout - (System.currentTimeMillis() - fStart));
      if (time < 0)

      { break; }

      }

      if (enable_debug)

      { log.debug("BaseTableQueryEngine: tuples " + tuples.size() + " in time " + (System.currentTimeMillis() - fStart)); }

      }

        1. BaseTableQueryEngine.java
          4 kB
          Joseph Wang
        2. MongoConnnection.java
          3 kB
          Joseph Wang

            Assignee:
            antoine Antoine Girbal
            Reporter:
            joseph.wang Joseph Wang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: