Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-34498

mongo balancer often won't start back up

    • Type: Icon: Improvement Improvement
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Sharding
    • None

      sh.status() output

        active mongoses:
          "3.6.2" : 30
        balancer:
          Currently enabled:  yes
          Currently running:  no
          Failed balancer rounds in last 5 attempts:  0
          Migration Results for the last 24 hours:
              No recent migrations
      

      No locks

      mongos> db.locks.find({'_id': 'balancer'}).pretty()
      mongos> 
      

      This collection is sharded so it ought to be doing something. It's done nothing in over 4 days and I have multiple collections out of balance that it should be working on.

      mongos> db.chunks.aggregate([ { $match: {ns: "some.collection"}}, { $group:{ _id: "$shard", cnt:

      { $sum: 1}

      } }, {$sort: {"_id": 1}} ]);

      { "_id" : "rs0", "cnt" : 647 } { "_id" : "rs1", "cnt" : 648 } { "_id" : "rs2", "cnt" : 648 } { "_id" : "rs3", "cnt" : 648 } { "_id" : "rs4", "cnt" : 648 } { "_id" : "rs5", "cnt" : 648 } { "_id" : "rs6", "cnt" : 649 } { "_id" : "rs7", "cnt" : 648 } { "_id" : "rs8", "cnt" : 642 } { "_id" : "rs9", "cnt" : 591 }
      
      

      mongos> printjson(sh.isBalancerRunning()); printjson(sh.isBalancerRunning()); new ISODate(); printjson(sh.startBalancer()); new ISODate(); db.version()
      false
      false
      assert.soon failed, msg:Waited too long for lock balancer to change to state undefined
      doassert@src/mongo/shell/assert.js:15:14
      assert.soon@src/mongo/shell/assert.js:199:13
      sh.waitForDLock@src/mongo/shell/utils_sh.js:194:1
      sh.waitForBalancer@src/mongo/shell/utils_sh.js:287:9
      sh.startBalancer@src/mongo/shell/utils_sh.js:164:5
      @(shell):1:96

      2018-04-16T10:19:51.289-0700 E QUERY [thread1] Error: assert.soon failed, msg:Waited too long for lock balancer to change to state undefined :
      doassert@src/mongo/shell/assert.js:15:14
      assert.soon@src/mongo/shell/assert.js:199:13
      sh.waitForDLock@src/mongo/shell/utils_sh.js:194:1
      sh.waitForBalancer@src/mongo/shell/utils_sh.js:287:9
      sh.startBalancer@src/mongo/shell/utils_sh.js:164:5
      @(shell):1:96

      mongos> db.version()
      3.6.2

      
      

            Assignee:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Reporter:
            mkruse@adobe.com Matthew Kruse
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: