Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-11678

Investigate issues generated by background compaction in MDB tests

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Checkpoints, Compaction
    • None
    • 5
    • c(3x10^8)-StorEng - 2023-11-14, 2023-11-28 - Anthill Tiger, 2023-12-12 - Heisenbug

      The following MDB patch build enables the background compaction by default and hit the following issue in the test src/mongo/db/modules/enterprise/jstests/fcbis/in_progress_index_builds_are_restarted_on_fcbis_node.js:

      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF986F4F04","b":"FFFF984AD000","o":"247F04","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"44"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF986F07E0","b":"FFFF984AD000","o":"2437E0","s":"abruptQuit","s+":"8C"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF98BAC668","b":"FFFF98BAC000","o":"668","s":"__kernel_rt_sigreturn","s+":"0"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF96D25530","b":"FFFF96CF3000","o":"32530","s":"gsignal","s+":"B0"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF96D26D74","b":"FFFF96CF3000","o":"33D74","s":"abort","s+":"154"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF8106D448","b":"FFFF80EAD000","o":"1C0448","s":"__wt_abort","s+":"28"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF8103D80C","b":"FFFF80EAD000","o":"19080C","s":"__hs_delete_reinsert_from_pos.constprop.0","s+":"7AC"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF8103DB2C","b":"FFFF80EAD000","o":"190B2C","s":"__hs_insert_record","s+":"26C"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF8103EBD4","b":"FFFF80EAD000","o":"191BD4","s":"__wt_hs_insert_updates","s+":"C44"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF81096B48","b":"FFFF80EAD000","o":"1E9B48","s":"__rec_write_wrapup","s+":"108"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF81098288","b":"FFFF80EAD000","o":"1EB288","s":"__wt_reconcile","s+":"1A8"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF80FABC38","b":"FFFF80EAD000","o":"FEC38","s":"__wt_sync_file","s+":"4B8"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF810ED6D0","b":"FFFF80EAD000","o":"2406D0","s":"__checkpoint_tree","s+":"BC"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF810EE29C","b":"FFFF80EAD000","o":"24129C","s":"__txn_checkpoint_wrapper","s+":"6BC"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF810EF5DC","b":"FFFF80EAD000","o":"2425DC","s":"__wt_txn_checkpoint","s+":"14C"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF810BCF84","b":"FFFF80EAD000","o":"20FF84","s":"__wt_session_compact","s+":"D24"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF80FD756C","b":"FFFF80EAD000","o":"12A56C","s":"__compact_server","s+":"44C"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF96E8022C","b":"FFFF96E79000","o":"722C","s":"start_thread","s+":"AC"}}
      [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I  CONTROL  31445   [thread66] "Frame","attr":{"frame":{"a":"FFFF96DC7A1C","b":"FFFF96CF3000","o":"D4A1C","s":"thread_start","s+":"C"}}
      

      A first analysis of the artifacts is required. Please note that this patch build was executed while PM-3091 was still in progress and the logic for the background compaction was unfinished.

      Update: this ticket became a general investigation around issues that appear when background compaction is enabled by default in a MDB patch build. The current issues that are being tracked are:

      • OOO ts errors (WT-12032)
      • Timeout errors - Those are ok are compaction may prevent collections from being exclusively accessed. Background compaction will be enabled by default in the right tests, see SERVER-80123 for that work.
      • Investigate RTS and background compaction (WT-11939)
      • Errors when the salvage API is used (WT-11984)
      • Errors when the import is used (WT-11985)
      • Errors when backup cursors are used (SERVER-83338 & SERVER-83477)
      • Errors related to encryption - (SERVER-83699)

            Assignee:
            etienne.petrel@mongodb.com Etienne Petrel
            Reporter:
            etienne.petrel@mongodb.com Etienne Petrel
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: