Summary

Add spec test which checks that the order of read preference tag matching is obeyed.

Motivation

SERVER-51835 represents a bug in the server's implementation of the replica set monitor from MongoDB 4.4 (PM-1519). Had a driver spec test existed then the driver spec tests which were integrated into the server's automated testing would have flagged it. Adding a spec test would confirm existing drivers following the documented behavior and ensure any new drivers don't make the same mistake as the server.

If the tag set list contains multiple documents, MongoDB tries each document in succession until a match is found. Once a match is found, that tag set is used to find all eligible matching members, and the remaining tag sets are ignored. If no members match any of the tag sets, the read operation returns with an error.

https://www.mongodb.com/docs/manual/core/read-preference-tags/#order-of-tag-matching

From what I found in the existing driver spec tests the only tests I found using multiple tag sets are either (a) for testing maxStalenessSeconds is obeyed or (b) uses a "doesntexist" tag which don't match any of the servers anyway.

$ jq '{file: input_filename, root: .} | select((.root.read_preference.tag_sets | length) > 1 and (.root.suitable_servers | length) != (.root.topology_description.servers | length) and .root.read_preference.maxStalenessSeconds == null) | .file' -- $(git grep -l tag_sets -- '*.json')
"source/server-selection/tests/server_selection/ReplicaSetNoPrimary/read/Secondary_multi_tags.json"
"source/server-selection/tests/server_selection/ReplicaSetNoPrimary/read/Secondary_multi_tags2.json"

Who is the affected end user?

Authors of drivers behavior on Drivers and Sharding team (mongos is driver-like).

How does this affect the end user?

Specification was misunderstood on Sharding team.

How likely is it that this problem or use case will occur?

This problem did happen (twice independently even, in both the scanning replica set monitor and in the new streamable replica set monitor). Read preference tags are commonly used in Atlas for ANALYTICS nodes and other resource separation strategies.

If the problem does occur, what are the consequences and how severe are they?

If the read preference tag matching isn't obeyed then applications will see queries they intended to target a constrained set of hosts going more commonly spread across all hosts and not only going to other hosts when the most preferred set of hosts are unavailable.

Is this issue urgent?

Not urgent.

Is this ticket required by a downstream team?

Not required by a downstream team.

Is this ticket only for tests?

Yes only for tests.