Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-47222

Mongos high cpu usage on getShardIdsForRange while dealing shard key range query

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.2.5, 4.0.17
    • Component/s: Sharding
    • None
    • ALL
    • Sharding 2020-04-20

      Hi,

      We have encountered an issue that after upgrading mongo from 3.4 to 4.0(through 3.6), the mongos's cpu usage has raised for multiple times. After some investigation we found the mainly cpu cost is doing the ShardId string compare from the getShardIdsForRange() call while inserting ShardId into the result set.  The case was that user's query range for the shard key was [MinKey, MaxKey](e.g., doing a range query on a hashed shard key), and the collection's routing table(chunk map) was very large(about 100k chunks). So there will be tens of thousands of inserting into the ShardId result set, costing a lot of cpus. And I notice that in 3.4, there was a ChunkRangeMap that maintains each shard's chunk ranges which can optimize this procedure. I noticed that SERVER-33929 has removed this ChunkRangeMap due to some reason but bringing this performance degradation.  To resolve this issue, I think we can have a fast path for getShardIdsForRange() when the range(all shard key fields) is [MinKey, MaxKey], we only need to return all ShardIds through getAllShardIds(). Looking forward for your feedback. Thanks!

            Assignee:
            blake.oler@mongodb.com Blake Oler
            Reporter:
            mingyan.zc@gmail.com Cen Zheng
            Votes:
            1 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: