Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12540

Testy fails due to OOM killer

    • Storage Engines
    • 5
    • 2024-07-09 - Nyan Cat

      Error signature:

      process of this unit has been killed by the OOM killer.
      Feb 24 00:12:09  systemd[1]: testy-run@sample.service: Killing process 325476 (eviction-ser 2) with signal SIGKILL.
      Feb 24 00:12:11  systemd[1]: testy-run@sample.service: Failed with result 'oom-kill'.
      Feb 24 00:12:11  systemd[1]: testy-run@sample.service: Consumed 1h 54min 5.983s CPU time.
      

      Logs since the start:

      Feb 23 21:46:48  systemd[1]: Started "testy-run: A 24/7 WiredTiger workload service".
      Feb 23 22:55:43  bash[325444]: Warning: Unable to identify WiredTiger build. Please consider either:
      Feb 23 22:55:43  bash[325444]: - Setting 'WT_BUILDDIR' environment variable to specify the build directory.
      Feb 23 22:55:43  bash[325444]: - Calling workgen from the root of the build directory.
      Feb 23 22:55:43  bash[325444]: [1708728943:455021][325444:0x7fcffd70e640], sweep-server: [WT_VERB_DEFAULT][WARNING]: Session 18 (@: 0x0x560e1e37ccd0 name: connection-open-session) did not run a sweep for 60 minutes.
      Feb 23 22:55:43  bash[325444]: [1708728943:562460][325444:0x7fcffd70e640], sweep-server: [WT_VERB_DEFAULT][WARNING]: Session 56 (@: 0x0x560e1e38ad40 name: connection-open-session) did not run a sweep for 60 minutes.
      Feb 24 00:07:40  bash[325444]: [1708733229:154635][325444:0x7fd000714640], file:table_9f24A.wt, eviction-server: [WT_VERB_EVICT][WARNING]: Eviction took more than 1 minute (68560560us). Building disk image took 145523us. History store wrapup took 68261291us.
      Feb 24 00:09:16  bash[325444]: [1708733309:637175][325444:0x7fcfff712640], file:table_LhEyA.wt, eviction-server: [WT_VERB_EVICT][WARNING]: Eviction took more than 1 minute (80519403us). Building disk image took 484371us. History store wrapup took 80034914us.
      Feb 24 00:10:37  bash[325444]: [1708733309:637183][325444:0x7fd001716640], file:table_LhEyA_mirror.wt, eviction-server: [WT_VERB_EVICT][WARNING]: Eviction took more than 1 minute (80519409us). Building disk image took 484317us. History store wrapup took 80034949us.
      Feb 24 00:11:27  bash[325444]: [1708733435:492301][325444:0x7fcffef11640], file:table_laoSn_mirror.wt, eviction-server: [WT_VERB_EVICT][WARNING]: Eviction took more than 1 minute (125856045us). Building disk image took 294154us. History store wrapup took 125561764us.
      Feb 24 00:11:27  bash[325444]: [1708733435:392743][325444:0x7fd000714640], file:table_PiBws_mirror.wt, eviction-server: [WT_VERB_EVICT][WARNING]: Eviction took more than 1 minute (174259710us). Building disk image took 1461016us. History store wrapup took 170931450us.
      

      The workload comes from this commit.

        1. leaks.txt
          1.71 MB

            Assignee:
            ruby.chen@mongodb.com Ruby Chen
            Reporter:
            etienne.petrel@mongodb.com Etienne Petrel
            Etienne Petrel
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: