Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22116

Memory limit of MigrationSourceManager is too big

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.4.23
    • Affects Version/s: 3.0.8, 3.2.0
    • Component/s: Sharding
    • None
    • Sharding
    • Fully Compatible
    • ALL

      MigrationSourceManger is caching the document id of modification during moving chunk.

      But I think the threshold of memory usage is too big.
      According to the source code, current limit is 5TB. Move chunk command is checking whether the memory usage is greater than 500*1024*1024, but MigrationSourceManager::mbUsed() already return MB not Bytes. So eventually MoveChunkCommand will be aborted when memory usage is greater than 5TB.

      https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/move_chunk_command.cpp#L397
      if (shardingState->migrationSourceManager()->mbUsed() > (500 * 1024 * 1024)) {
      // This is too much memory for us to use so we're going to abort the migration
      ...

      https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_source_manager.cpp#L557
      long long MigrationSourceManager::mbUsed() const

      { stdx::lock_guard<stdx::mutex> lk(_mutex); return _memoryUsed / (1024 * 1024); }

            Assignee:
            backlog-server-sharding [DO NOT USE] Backlog - Sharding Team
            Reporter:
            sunguck.lee@gmail.com 아나 하리
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: