Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12721

Investigate large schema lock contention during checkpoint and session sweep

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Schema Management
    • Storage Engines
    • 8
    • 나비 (nabi) - 2024-04-16, Nick - 2024-04-30, Megabat - 2024-05-14, 2024-05-28 - FOLLOW ON SPRINT, 2024-06-11 - Dinosaurs go rawr, 2024-06-25 - Climbing BTrees, 2024-07-09 - Nyan Cat

      There has been a vast amount of HELP tickets that deal with large contention around the schemaLock affecting checkpoint. HELP-52527, HELP-53809, HELP-57159 are all examples of tickets where we see this behaviour. Common symptoms of the problem all show sweep server being active and at the same time of the checkpoint. For example:

      Note: There are missing FTDC data within the help tickets because to capture statistics, it will need to fetch the schemaLock too.

      There has been a long history that deals with schemaLock contention with checkpoint and sweep server. This is because the start of checkpoint requires both the SCHEMA_LOCK  and table lock to fetch the dhandle list. The sweep server acts on a regular interval basis and also requires the table lock. It is postulated that the sweep server constantly requires the table lock and contends with checkpoint making the large impact of schema lock.

      This ticket aims to investigate into the potential reason why we have contention on the schema lock from the help tickets.

       

        1. calls-prepare-evict.png
          calls-prepare-evict.png
          96 kB
        2. ep_app_thread_wait.png
          ep_app_thread_wait.png
          548 kB
        3. ep_full_ftdc.png
          ep_full_ftdc.png
          546 kB
        4. metrics.2024-04-05T00-25-22Z-00000
          3.27 MB
        5. schemaLock.png
          schemaLock.png
          153 kB
        6. screenshot-1.png
          screenshot-1.png
          308 kB
        7. Screenshot 2024-04-05 at 3.41.44 PM.png
          Screenshot 2024-04-05 at 3.41.44 PM.png
          100 kB
        8. Screenshot 2024-04-09 at 10.38.20 AM.png
          Screenshot 2024-04-09 at 10.38.20 AM.png
          118 kB

            Assignee:
            Unassigned Unassigned
            Reporter:
            jie.chen@mongodb.com Jie Chen
            Sean Watt
            Votes:
            1 Vote for this issue
            Watchers:
            19 Start watching this issue

              Created:
              Updated: