Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-910

Allow MongoClient to automatically transition from replica set to sharded endpoint without restarts

    • To Do
    • Allow MongoClient to survive cluster topology changes from sharded cluster to replica set
    • Needed
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CDRIVER-4683 Execution Blocked
      CXX-2714 Execution Blocked
      CSHARP-4717 Execution Blocked
      GODRIVER-2905 Execution Blocked
      JAVA-5071 Execution Blocked
      NODE-5453 Execution Blocked
      MOTOR-1153 Execution Blocked
      PYTHON-3836 Execution Blocked
      PHPLIB-1195 Execution Blocked
      RUBY-3297 Execution Blocked
      RUST-1702 Execution Blocked
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CDRIVER-4683 Execution Blocked CXX-2714 Execution Blocked CSHARP-4717 Execution Blocked GODRIVER-2905 Execution Blocked JAVA-5071 Execution Blocked NODE-5453 Execution Blocked MOTOR-1153 Execution Blocked PYTHON-3836 Execution Blocked PHPLIB-1195 Execution Blocked RUBY-3297 Execution Blocked RUST-1702 Execution Blocked

      Summary

      Definitions:

      • router endpoint - server endpoint that routes requests to shards (i.e. mongos)
      • replica set endpoint - server endpoint that emulates a replica set or standalone (i.e. mongod)

      Starting in 8.0, sharded clusters will be the only supported topology in MongoDB. To allow customers currently using a MongoDB replica set to migrate to MongoDB 8.0 without needing to reconfigure their applications, the server team will build a "replica set endpoint" that makes a single-shard cluster appear to be a replica set (PM-2965). However, there are limitations to the replica set endpoint:

      • It can only be used with single-shard clusters, so customers have to switch to the router endpoint if they want to add more shards (e.g. to scale horizontally).
        • Note that the there have been discussions about supporting multi-shard clusters with the replica set endpoint, but it's currently not in the scope of PM-2965. In any case, horizontal scalability will likely still be limited for customers using the replica set endpoint.
      • It will likely be removed with a future version of MongoDB (9.0 or later), so probably can't be used indefinitely.
        • It's not decided when the replica set endpoint feature will be removed.
          As a result, customers may eventually be motivated or required to stop using the replica set endpoint.

      To switch to the router endpoint, customers using existing drivers will have to update their connection string and restart their applications. We want to offer customers a better migration experience that doesn't require reconfiguring and restarting their application to switch to the router endpoint.

      Updated from the original description:

      It might be useful to allow a MongoClient to survive cluster topology changes from a replica set to a sharded cluster (or vice versa). For example, client is connected to mongoses A and B and an admin restarts A and B as a replica set, the client could rediscover A and B as replica set members.

      Currently the SDAM spec does not allow this. When a MongoClient is connected to a replica set, it will remove servers that are discovered to be mongos nodes. When a MongoClient is connected to a sharded cluster (a set of mongos nodes), it will remove servers that are not mongos nodes.

      I can imagine this addition to the SDAM spec would allow it:

      If all nodes are removed from the Topology, clients MUST reset the TopologyType to Unknown and rediscover the original seed addresses.

      Originally requested in: PYTHON-2131

      Motivation

      Who is the affected end user?

      Customers who upgrade existing replica sets to MongoDB 8.0 or who chose use the new replica set endpoint for a new MongoDB 8.0 cluster (see PM-2965).

      How does this affect the end user?

      Customers may delay migrating off the replica set endpoint, reducing their motivation to horizontally scale their clusters or preventing them from migrating to a future MongoDB version that doesn't have the replica set endpoint feature.

      How likely is it that this problem or use case will occur?

      All customers using the MongoDB 8.0 replica set endpoint will eventually need to switch to the router endpoint.

      If the problem does occur, what are the consequences and how severe are they?

      Customers who want or need to switch to the router endpoint must change their connection string and restart their applications to re-initialize the MongoClient.

      Is this issue urgent?

      This feature will be useful when MongoDB 8.0 is released. It will become even more useful when a future MongoDB version is released that removes the replica set endpoint feature.

      Is this ticket required by a downstream team?

      No.

      Is this ticket only for tests?

      No.

      Acceptance Criteria

      To allow migrating from the replica set endpoint to the router endpoint, drivers must be able to:

      1. Discover the router endpoint(s) somehow.
      2. Give customers some way to configure whether or not they want their applications to automatically switch to the router endpoint(s), if available.
      3. Update the current topology from replica set or standalone to sharded cluster.

      Open questions:

      1. Should we try to make this work for pre-8.0 MongoDB versions?
      2. Should we try to make this work for self-hosted databases or Atlas only?
      3. Are there other topology transitions that may become important in the future?
        1. Future customers may switch from sharded to load-balanced clusters. Should we try to support no-restart transitions from sharded to load-balanced? (suggested by tyler.brock@mongodb.com)

            Assignee:
            shane.harvey@mongodb.com Shane Harvey
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Steve Silvester Steve Silvester
            Esha Bhargava Esha Bhargava
            Rishabh Bisht Rishabh Bisht
            Votes:
            2 Vote for this issue
            Watchers:
            35 Start watching this issue

              Created:
              Updated: