Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-10995

Investigate problems with tiered storage local_retention retaining files.

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • StorEng - Refinement Pipeline

      In working on WT-10988, after a flush_tier in a python test, we'd like to make sure a locally retained file is removed.  We want that so we can be sure that the version of the file in the cloud was not overwritten (what we are fixing in the test).

      However, even with local_retention set at 0 or 1 (minute), and waiting for longer than a minute, the local file does not get removed.  Add trace statements in the code reveal that the loop is running every minute, but apparently the handle for the local file remains open, which prevents the local file removal - and the removal is requeued.  So this may be a failure in sweep or some other code that is keeping the file open.  Maybe block handles?

      Even more troubling is that doing a reopen on the connection does not cause the file to be removed. We seem to be stuck in the same loop.  More investigation is needed.

      Also, can we get the check loop to run more than once a minute, at least for testing and benchmarks?  This may be a common case where we push something to the cloud, want local retention to expire quickly so we can check retrieving from the cloud.  I can see this "removal wait loop" might become somewhat common for tests and benchmarks.

      Note: FIXME added in test added for WT-10988.

            Assignee:
            donald.anderson@mongodb.com Donald Anderson
            Reporter:
            donald.anderson@mongodb.com Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: