Original Summary

MongoDB 3.6 Balancer much slower than 3.4

Original Description

I have a test for balancer between 3.6 and 3.4, and I found balancing on 3.6 is much slower than 3.4.

I insert 100 million docs to 3.6 and 3.4, the single doc size is 2 kbytes. 

Collection initial as below:

db.runCommand({shardCollection: "ycsb.test", key: {_id: "hashed"}, numInitialChunks: 6500}) 

Test Results:

1. 3.4.15 with MMAPv1 engine: 
   1. from 1 shard to 2 shards: use 41866 seconds, after balance, 3250 chunks on shard.
   2. from 2 sahrds to 4 shards: use 30630 seconds, after balance 1625 chunks on 1 shard.
2. 3.6.5 with MMAPv1 engine:
   1. from 1 shard to 2 shards: use 90200 seconds, after balance, 3250 chunks on shard.
   2. from 2 sahrds to 4 shards: use 44679 seconds, after balance 1625 chunks on 1 shard.
3. 3.4.15 with wiredTiger engine:
   1. from 1 shard to 2 shards: use 35635 seconds, after balance, 3250 chunks on shard.
   2. from 2 sahrds to 4 shards: use 10740 seconds, after balance 1625 chunks on 1 shard.
4. 3.6.5 with wiredTiger engine:
   1. from 1 shard to 2 shards: use 49762 seconds, after balance, 3250 chunks on shard.
   2. from 2 sahrds to 4 shards: use 18961 seconds, after balance 1625 chunks on 1 shard.

MongoDB configuration for MMAPv1 engine:

security:
  authorization: disabled
sharding:
  clusterRole: shardsvr
replication:
  replSetName: rs1
systemLog:
  logAppend: true
  destination: file
  path: /home/adun/3.4/log/mongod.log

processManagement:
  fork: true
  pidFilePath: /home/adun/3.4/log/mongod.pid

net:
  port: 27017
  bindIp: 127.0.0.1,192.168.10.31
  maxIncomingConnections: 65536

storage:
  dbPath: /home/adun/3.4/data
  directoryPerDB: true
  engine: mmapv1

MongoDB configuration for wiredTiger engine:

security:
  authorization: disabled
sharding:
  clusterRole: shardsvr
replication:
  replSetName: rs1
systemLog:
  logAppend: true
  destination: file
  path: /home/adun/3.4/log/mongod.log

processManagement:
  fork: true
  pidFilePath: /home/adun/3.4/log/mongod.pid

net:
  port: 27017
  bindIp: 127.0.0.1,192.168.10.31
  maxIncomingConnections: 65536

storage:
  dbPath: /home/adun/3.4/data
  directoryPerDB: true
  engine: wiredTiger

  wiredTiger:
    engineConfig:
      cacheSizeGB: 32
      directoryForIndexes: true

Other Questions:

1. If we set numInitialChunks to a small value, such as 100, mongodb will create/split chunk by itself, but even if we insert the same data/same records number, 3.6 will create chunks abont 10% more than 3.4. (both MMAPv1 and wiredTiger)