Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22115

Primary Server Crashes due to Memory Leak

    • Type: Icon: Bug Bug
    • Resolution: Incomplete
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.8
    • Component/s: WiredTiger
    • ALL
    • Hide

      Server details:
      RAM: 14GB
      Data Size: 47.5GB (storage size ~15GB)
      cacheSizeGB: 7
      4 Cores
      3.0.8
      CentOS Linux release 7.2.1511 (Core): Azure, Linux version 3.10.0-229.11.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP

      Replica set: Primary, Secondary and Arbiter.
      Engine: WiredTiger

      Show
      Server details: RAM: 14GB Data Size: 47.5GB (storage size ~15GB) cacheSizeGB: 7 4 Cores 3.0.8 CentOS Linux release 7.2.1511 (Core): Azure, Linux version 3.10.0-229.11.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Replica set: Primary, Secondary and Arbiter. Engine: WiredTiger

      • A replica-set in Azure that was deployed few months ago.
      • This setup was started to crash since Fri, Jan 8 (log example attached).
      • No upgrade/change to the system or application was done in the last week.
      • The crash happens only when instance is primary, after few hours of operation. You can see that during these hours memory usage increases, while all other parameters like connections remain constant (see the attached files).
      • The replica set was 3.0.6, and was upgraded it today to 3.0.8. We also removed it from the replica set and recreated it. Yet the problem continues to happen on this machine.
      • This one of few replica sets in Azure (where this is not recreated), yet it is the most active one.

        1. all_graphs.png
          227 kB
          Moshe Kaplan [X]
        2. Crash_Memory.png
          44 kB
          Moshe Kaplan [X]
        3. iostat.log
          2.31 MB
          Moshe Kaplan [X]
        4. log_crash.txt
          5 kB
          Moshe Kaplan [X]

            Assignee:
            ramon.fernandez@mongodb.com Ramon Fernandez Marina
            Reporter:
            MosheKaplan Moshe Kaplan [X]
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: