Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-5544

Fix cache stuck during salvage

    • 8
    • Storage Engines 2020-02-24, Storage Engines 2020-03-09, Storage Engines 2020-03-23, Storage - Tora 2020-05-04

      KB Edit: This CONFIG has the "checkpoints=wiredtiger" signature characteristic of WT-5521.

       


      This is a cache stuck full with clean page case with 0% dirty page, captured by format stress test job on margay-aws. The cache dump 

      http://build.wiredtiger.com:8080/job/wiredtiger-test-format-stress/73287/

      There is only 1 internal clean page (0 MB) for table file:WiredTigerLAS.wt, and 1 internal dirty page (0 MB) for table file:WiredTiger.wt. And the total found cache size is 0MB vs tracked inuse 534MB. See below for the detailed cache dump:

      0x7f6f5fd8a690:=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
      0x7f6f5fd8a690:transaction state dump
      0x7f6f5fd8a690:current ID: 5
      0x7f6f5fd8a690:last running ID: 5
      0x7f6f5fd8a690:metadata_pinned ID: 5
      0x7f6f5fd8a690:oldest ID: 5
      0x7f6f5fd8a690:durable timestamp: (0, 202)
      0x7f6f5fd8a690:oldest timestamp: (0, 202)
      0x7f6f5fd8a690:pinned timestamp: (0, 202)
      0x7f6f5fd8a690:stable timestamp: (0, 0)
      0x7f6f5fd8a690:has_durable_timestamp: yes
      0x7f6f5fd8a690:has_oldest_timestamp: yes
      0x7f6f5fd8a690:has_pinned_timestamp: yes
      0x7f6f5fd8a690:has_stable_timestamp: no
      0x7f6f5fd8a690:oldest_is_pinned: yes
      0x7f6f5fd8a690:stable_is_pinned: no
      0x7f6f5fd8a690:checkpoint running: yes
      0x7f6f5fd8a690:checkpoint generation: 2
      0x7f6f5fd8a690:checkpoint pinned ID: 0
      0x7f6f5fd8a690:checkpoint txn ID: 0
      0x7f6f5fd8a690:oldest named snapshot ID: 0
      0x7f6f5fd8a690:session count: 13
      0x7f6f5fd8a690:Transaction state of active sessions:
      0x7f6f5fd8a690:=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
      0x7f6f5fd8a690:cache dump
      0x7f6f5fd8a690:cache full: yes
      0x7f6f5fd8a690:cache clean check: yes (153.678%)
      0x7f6f5fd8a690:cache dirty check: no (0.000%)
      0x7f6f5fd8a690:file:wt.wt(<live>) eviction disabled at open:
      0x7f6f5fd8a690:  Opened exclusively. Cannot walk tree, skipping.
      0x7f6f5fd8a690:file:WiredTigerLAS.wt(<live>) eviction disabled at open:
      0x7f6f5fd8a690:internal: 1 pages, 0MB, 1/0 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      0x7f6f5fd8a690:leaf: 0 pages
      0x7f6f5fd8a690:file:WiredTiger.wt(<live>):
      0x7f6f5fd8a690:internal: 1 pages, 0MB, 0/1 clean/dirty pages, 0/0 clean/dirty MB, 0MB max page, 0MB max dirty page
      0x7f6f5fd8a690:leaf: 0 pages
      0x7f6f5fd8a690:cache dump: total found: 0MB vs tracked inuse 534MB
      0x7f6f5fd8a690:total dirty bytes: 0MB 

      The configuration:

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      alter=0
      assert_commit_timestamp=1
      assert_read_timestamp=0
      auto_throttle=1
      backups=1
      bitcnt=6
      bloom=1
      bloom_bit_count=62
      bloom_hash_count=19
      bloom_oldest=0
      cache=348
      cache_minimum=20
      checkpoints=wiredtiger
      checkpoint_log_size=56
      checkpoint_wait=89
      checksum=uncompressed
      chunk_size=1
      compaction=0
      compression=none
      data_extend=0
      data_source=table
      delete_pct=17
      dictionary=0
      direct_io=0
      encryption=none
      evict_max=2
      file_type=row-store
      firstfit=0
      huffman_key=0
      huffman_value=0
      independent_thread_rng=0
      in_memory=0
      insert_pct=5
      internal_key_truncation=1
      internal_page_max=9
      isolation=snapshot
      key_gap=18
      key_max=256
      key_min=256
      leaf_page_max=9
      leak_memory=0
      logging=0
      logging_archive=0
      logging_compression=none
      logging_file_max=3021
      logging_prealloc=0
      lsm_worker_threads=3
      major_timeout=0
      memory_page_max=6
      merge_max=12
      mmap=1
      modify_pct=67
      ops=0
      prefix_compression=1
      prefix_compression_min=3
      prepare=0
      quiet=1
      random_cursor=0
      read_pct=6
      rebalance=1
      repeat_data_pct=3
      reverse=0
      rows=1000000
      runs=1
      salvage=1
      split_pct=59
      statistics=0
      statistics_server=0
      threads=29
      timer=4
      timing_stress_aggressive_sweep=0
      timing_stress_checkpoint=1
      timing_stress_lookaside_sweep=0
      timing_stress_split_1=0
      timing_stress_split_2=0
      timing_stress_split_3=0
      timing_stress_split_4=0
      timing_stress_split_5=0
      timing_stress_split_6=1
      timing_stress_split_7=0
      timing_stress_split_8=0
      transaction_timestamps=1
      transaction-frequency=100
      truncate=1
      value_max=914
      value_min=256
      verify=1
      wiredtiger_config=
      write_pct=5
      ############################################
      

            Assignee:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Reporter:
            luke.chen@mongodb.com Luke Chen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: