Starting in MongoDB 8.0, explicit or implicit creation of system.buckets.-prefixed collections without providing time-series options is rejected with an error:

test> db.createCollection("system.buckets.xyzzy")
MongoServerError[IllegalOperation]: Creation of a timeseries bucket collection without timeseries options is not allowed
test> db.system.buckets.xyzzy.insertOne({})
MongoServerError: Creation of a timeseries bucket collection without timeseries options is not allowed
test> db.other.renameCollection("system.buckets.xyzzy")
MongoServerError[IllegalOperation]: Cannot rename non timeseries buckets collection 'test.other' to a timeseries buckets namespace 'test.system.buckets.xyzzy'.

This change was initially expected to have no impact, as system.buckets.-prefixed collections without time-series options were not expected to exist in realistic scenarios. However, during the deployment of MongoDB 8.0 in Atlas, it was discovered that some clusters have those invalid bucket collections (see HELP-67589, HELP-69056), due to issues such as SERVER-87678.

In order to avoid blocking upgrades, tentatively since MongoDB 8.0.5, clusters will be able to upgrade to FCV 8.0 if their database contains invalid bucket collections (see SERVER-99290, CA-282). However, creation of new system.buckets.-prefixed collections without time-series options still need to be disallowed.

As system.buckets.-prefixed collections without time-series options are not expected to be used by queries, we do not expect this change to have an impact on those. However, tools which may expect to be able to do a 1:1 clone of any arbitrary collection should be able to deal gracefully with the scenario that attempting to clone an existing system.buckets.-prefixed collections without time-series option will fail due to the restriction on creation of new collections. Similarly, attempts to rename collections from regular namespaces to system.buckets.-prefixed namespaces and vice versa will fail.

Description of Linked Ticket

Prior to MongoDB 5.0 it wasn't possible for an external client to create a system.buckets collection, even if such a client was authenticated as the __system user. This is because collections which are prefixed with system. are protected at a level outside the access control system where unrecognized system collections cannot be created (full stop). With the introduction of timeseries collections in MongoDB 5.0, this constraint was relaxed and it became possible for external clients to create system.buckets collections directly. The intended use case was likely for mongorestore for performance reasons to directly write the bucket documents. However despite the only valid use case for system.buckets collection to be for a timeseries collection, this isn't actually enforced by the server at all.

This gap in input validation leaves open a risk where Server engineers make poor assumptions about NamespaceString::isTimeseriesBucketsCollection() implying the collection is actually timeseries without consulting the local mdb catalog. Or where mongosync wants to rely on checking whether the namespace has a system.buckets. prefix and therefore requires special DDL handling to deal with the associated view namespace. We should instead move the server semantics into one where the assumptions engineers are making turn out also to be true.

rs:PRIMARY> db.dropDatabase()
rs:PRIMARY> db.system.buckets.a.runCommand("create")
{
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1690936687, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1690936687, 1)
}