-
Type: Improvement
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Execution
-
Fully Compatible
-
Execution Team 2024-10-14, Execution Team 2024-10-28
CollectionTruncateMarkersWithPartialExpiration have a confusing contract where some records are accounted for, but aren't expireable when constructed.
In a stable state (when truncate markers track pre-existing and new records in a collection), the 'partial marker' ('lastHighestRecordId' and 'lastHighestWallTime') is
- Always updated to reflect the highest seen record
- Strictly increasing
- Thus, the 'partial marker' is valid/non-null even if the highest seen record has since been truncated, and the collection is empty.
However, the CollectionTruncateMarkersWithPartialExpiration constructor can be passed
- A non-empty queue of markers,
- Non-zero 'leftoverRecordsCount' and 'leftoverRecordsBytes'
- With an uninitialized 'partial marker'
Resulting in a contradictory state where the CollectionTruncateMarkersWithPartialExpiration
- Track records via the non-empty 'markers' queue & know there are records and bytes not accounted for by the markers.
- Can't expire the leftover records because '_lastHighestRecordId' and '_lastHighestWallTime' were never initialized.
Eventually, if a new record is inserted, the 'partial marker' is updated, and the system can support partial marker expiration.
There are only 2 current subclasses of CollectionTruncateMarkersWithPartialExpiration:
- ChangeCollectionTruncateMarkers: Serverless only, thus doesn't run in production.
- PreImagesTruncateMarkersPerNsUUID: Tracks pre-images generated from a specific user collection.
Today both subclasses ensure the partial marker is updated post-construction
Thus, the scenario described shouldn't impact production correctness. However, improving the contract could make CollectionTruncateMarkersWithPartialExpiration easier to reason about, test, and debug.
- is depended on by
-
SERVER-95714 Construct PreImagesTruncateMarkersPerNsUUID with highest record metrics
- In Code Review
- is related to
-
SERVER-95290 Add unittests for PreImageTruncateMarkersPerNsUUID API
- Closed