Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-79260

Avoid relying on fast count for change_collection_server_stats.js and multitenant_pre_image_expired_document_remover.js

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 7.2.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • Execution EMEA Team 2023-08-07, Execution EMEA Team 2023-09-04
    • 0

      change_collection_server_stats.js and multitenant_pre_image_expired_document_remover.js both rely on count() to test the number of pre-images/ change collection entries.

      count() uses fast-count; fast-count is not guaranteed to be correct for unreplicated, untimestamped collections which rely on WT truncate. WT truncate doesn't keep track of the amount of documents truncated (with the benefit of extremely fast document removal). For this reason, these collections rely on CollectionTruncateMarkers to estimate how much data / the amount of documents exist with a given range of the collections. These initial estimated numbers are used by truncate to update the fast-count.

      CollectionTruncateMarkers for change-collections and pre-images collections are lazily initialized at startup, and are estimates. For this reason, the tests which rely on fast-count should opt for counting the number of change-collection entries and pre-images by fetching the collection and counting the number of entries explicitly (an example with pre-images).

       

       

       

       

      Old description:
      In multitenant_pre_image_expired_document_remover.js, if the preimage removal job is set to run every 1 second, the fast count() of images in a tenant's collection doesn't always match the number of pre-images returned from a find().

      At some level, we expect initial fast count estimates to be off around startup because the markers are initialised as estimates. However, we should investigate whether or not we would expect the counts to match within the time an assert.soon completes and explore the limitations of the estimation process.

            Assignee:
            miquel.munoz@mongodb.com Miquel Munoz (Inactive)
            Reporter:
            haley.connelly@mongodb.com Haley Connelly
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: