-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Checkpoints, Compaction
-
None
-
5
-
c(3x10^8)-StorEng - 2023-11-14, 2023-11-28 - Anthill Tiger, 2023-12-12 - Heisenbug
The following MDB patch build enables the background compaction by default and hit the following issue in the test src/mongo/db/modules/enterprise/jstests/fcbis/in_progress_index_builds_are_restarted_on_fcbis_node.js:
[js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF986F4F04","b":"FFFF984AD000","o":"247F04","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"44"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF986F07E0","b":"FFFF984AD000","o":"2437E0","s":"abruptQuit","s+":"8C"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF98BAC668","b":"FFFF98BAC000","o":"668","s":"__kernel_rt_sigreturn","s+":"0"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.893+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF96D25530","b":"FFFF96CF3000","o":"32530","s":"gsignal","s+":"B0"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF96D26D74","b":"FFFF96CF3000","o":"33D74","s":"abort","s+":"154"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF8106D448","b":"FFFF80EAD000","o":"1C0448","s":"__wt_abort","s+":"28"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF8103D80C","b":"FFFF80EAD000","o":"19080C","s":"__hs_delete_reinsert_from_pos.constprop.0","s+":"7AC"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF8103DB2C","b":"FFFF80EAD000","o":"190B2C","s":"__hs_insert_record","s+":"26C"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF8103EBD4","b":"FFFF80EAD000","o":"191BD4","s":"__wt_hs_insert_updates","s+":"C44"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.894+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF81096B48","b":"FFFF80EAD000","o":"1E9B48","s":"__rec_write_wrapup","s+":"108"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF81098288","b":"FFFF80EAD000","o":"1EB288","s":"__wt_reconcile","s+":"1A8"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF80FABC38","b":"FFFF80EAD000","o":"FEC38","s":"__wt_sync_file","s+":"4B8"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF810ED6D0","b":"FFFF80EAD000","o":"2406D0","s":"__checkpoint_tree","s+":"BC"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF810EE29C","b":"FFFF80EAD000","o":"24129C","s":"__txn_checkpoint_wrapper","s+":"6BC"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF810EF5DC","b":"FFFF80EAD000","o":"2425DC","s":"__wt_txn_checkpoint","s+":"14C"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF810BCF84","b":"FFFF80EAD000","o":"20FF84","s":"__wt_session_compact","s+":"D24"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF80FD756C","b":"FFFF80EAD000","o":"12A56C","s":"__compact_server","s+":"44C"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF96E8022C","b":"FFFF96E79000","o":"722C","s":"start_thread","s+":"AC"}} [js_test:in_progress_index_builds_are_restarted_on_fcbis_node] d20043| 2023-08-21T08:35:52.895+00:00 I CONTROL 31445 [thread66] "Frame","attr":{"frame":{"a":"FFFF96DC7A1C","b":"FFFF96CF3000","o":"D4A1C","s":"thread_start","s+":"C"}}
A first analysis of the artifacts is required. Please note that this patch build was executed while PM-3091 was still in progress and the logic for the background compaction was unfinished.
Update: this ticket became a general investigation around issues that appear when background compaction is enabled by default in a MDB patch build. The current issues that are being tracked are:
- OOO ts errors (
WT-12032) - Timeout errors - Those are ok are compaction may prevent collections from being exclusively accessed. Background compaction will be enabled by default in the right tests, see
SERVER-80123for that work. - Investigate RTS and background compaction (
WT-11939) - Errors when the salvage API is used (
WT-11984) - Errors when the import is used (WT-11985)
- Errors when backup cursors are used (
SERVER-83338&SERVER-83477) - Errors related to encryption - (
SERVER-83699)
- is related to
-
SERVER-83338 Update backup cursor tests not resilient to checkpoints
- Closed
-
SERVER-83699 Investigate possible race between the EncryptionKeyManager and background compaction
- Closed
-
WT-11939 Investigate if background compaction can run in parallel with RTS
- Closed
- related to
-
WT-11985 Create a test that shows checkpoints can cause import to fail
- Open
-
SERVER-80123 Enable background compaction by default in appropriate test suites
- Closed
-
WT-11984 Investigate salvage and background compaction
- Closed