ISSUE DESCRIPTION AND IMPACT
In MongoDB 4.4 and earlier, TTL indexes with an expireAfterSeconds value of NaN (not-a-number) were functional as indexes for queries but did not expire any documents. When such a config is in place, the following error is logged by MongoDB 4.2 and 4.4 primary replica set nodes:
2022-08-05T21:00:00.000+0000 E INDEX [TTLMonitor] Error processing ttl index: { v: 2, key: { created: 1 }, name: "created_1", ns: "dbname.collname", expireAfterSeconds: nan.0, background: true } -- DurationOverflow: Cannot negate the minimum duration
In MongoDB 5.0.0-5.0.13 and 6.0.0-6.0.1, as part of SERVER-56676, TTL indexes began treating NaN as 0, and the listIndexes command necessary for initial syncs and mongodump backups began returning 0 instead of NaN.
Because of this change in unspecified behavior, when a TTL index with this improper configuration exists, the following can trigger the sudden expiration of TTL-indexed documents in a collection:
- On MongoDB 4.4, when:
- upgrading to MongoDB 5.0.0-5.0.13
- initially syncing from a 5.0.0-5.0.13 or 6.0.0-6.0.1 node
- On MongoDB 4.2, when initially syncing from a 5.0.0-5.0.13 or 6.0.0-6.0.1 node
- On MongoDB 5.0.0-5.0.13 or 6.0.0-6.0.1,
- When restoring from a mongodump of a 4.2 or 4.4 collection that has a TTL configured with expireAfterSeconds: NaN
- When initially syncing from a version 4.2 or 4.4 node that has a TTL configured with expireAfterSeconds: NaN
Now that this issue is addressed, the following case may still present concern:
- On MongoDB 4.2 and 4.4, when initially syncing from a 5.0.14+ or 6.0.2+ node that still has a TTL index with an invalid config, a newly synced node will end up with a system-dependent TTL configuration of expireAfterSeconds:<int32_max>. Use the collMod command to set a reasonable value, or remove the TTL index.
WORKAROUNDS AND REMEDIATION
In general, avoid this issue by avoiding expireAfterSeconds: NaN as a configuration and correct this config anywhere it exists.
The following script for the mongosh shell reports any TTL indexes with an expireAfterSeconds: NaN configuration:
Note: Do not use the legacy mongo shell for this operation.
function getNaNIndexes() { const nan_idx = []; const dbs = db.adminCommand({ listDatabases: 1 }).databases; dbs.forEach((d) => { const listCollCursor = db .getSiblingDB(d.name) .runCommand({ listCollections: 1 }).cursor; const collDetails = { db: listCollCursor.ns.split(".$cmd")[0], colls: listCollCursor.firstBatch.map((c) => c.name), }; collDetails.colls.forEach((c) => db .getSiblingDB(collDetails.db) .getCollection(c) .getIndexes() .forEach((entry) => { if (Object.is(entry.expireAfterSeconds, NaN)) { nan_idx.push({ ns: `${collDetails.db}.${c}`, index: entry }); } }) ); }); return nan_idx; }; getNaNIndexes();
Once identified, correct any TTL indexes with the expireAfterSeconds: NaN configuration and establish an unambiguous, valid configuration with a specified behavior. The collMod command allows you to modify the expireAfterSeconds value for an existing index.
MongoDB intends to help protect against this behavior change by:
- In this ticket,
SERVER-68477, MongoDB versions 5.0.14+ and 6.0.2+ now render badly configured TTL indexes ineffective, rather than applying a meaning of expireAfterSeconds: 0 to these indexes. - Releasing
SERVER-68522in MongoDB 5.0.11. With this fix, MongoDB 5.0.11+ will refuse to start if a TTL index with expireAfterSeconds: NaN exists, to ensure that the normal upgrade path from 4.4 is protected from this unexpected change in behavior. SeeSERVER-68522for additional details.
Original description
Currently listIndexes, and subsequently initial sync, do not properly handle NaN values for expireAfterSeconds. This can result in unexpected TTL behavior, especially when upgrading from MongoDB 4.4 to MongoDB 5.0 or when migrating earlier index definitions to MongoDB 5.0 or 6.0.
- is related to
-
SERVER-68359 Prevent TTLMonitor from processing index if expireAfterSeconds is NaN
- Closed
-
SERVER-68621 Investigate potentially unintended additions to the stableAPI
- Closed
- related to
-
SERVER-68522 Prevent 5.0 binary from starting in FCV 4.4 with misconfigured TTL index
- Closed
-
SERVER-69091 Handle additional out-of-bounds expireAfterSeconds values for TTL indexes
- Closed
-
SERVER-69402 Update FCV version for ttl_index_options.js
- Closed
-
SERVER-69783 use integral type for TTL index test case in ttl_index_options.js
- Closed
-
SERVER-76421 [5.0] ttl_expire_nan_upgrade.js checks indexes on node before it becomes secondary
- Closed
-
SERVER-68487 add BSONElement method to check for NaN field values
- Closed