[447960.770455] mongod invoked oom-killer: gfp_mask=0x26000c0, order=2, oom_score_adj=0 [447960.770458] mongod cpuset=/ mems_allowed=0 [447960.770465] CPU: 1 PID: 29909 Comm: mongod Not tainted 4.4.0-59-generic #80-Ubuntu [447960.770469] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [447960.770471] 0000000000000286 000000008cf10e98 ffff8801e659faf0 ffffffff813f7583 [447960.770475] ffff8801e659fcc8 ffff880024f9d400 ffff8801e659fb60 ffffffff8120ad5e [447960.770478] ffffffff81cd2dc7 0000000000000000 ffffffff81e67760 0000000000000206 [447960.770481] Call Trace: [447960.770488] [] dump_stack+0x63/0x90 [447960.770493] [] dump_header+0x5a/0x1c5 [447960.770497] [] oom_kill_process+0x202/0x3c0 [447960.770500] [] out_of_memory+0x219/0x460 [447960.770504] [] __alloc_pages_slowpath.constprop.88+0x8fd/0xa70 [447960.770507] [] __alloc_pages_nodemask+0x286/0x2a0 [447960.770510] [] alloc_kmem_pages_node+0x4b/0xc0 [447960.770514] [] copy_process+0x1be/0x1b70 [447960.770519] [] ? aa_sk_perm+0x70/0x210 [447960.770524] [] ? do_tcp_getsockopt.isra.36+0x11b/0x4e0 [447960.770527] [] ? aa_sock_perm+0x4b/0xe0 [447960.770531] [] ? move_addr_to_user+0xb4/0xd0 [447960.770533] [] _do_fork+0x80/0x360 [447960.770537] [] ? do_prlimit+0xdc/0x210 [447960.770539] [] ? tcp_getsockopt+0x2c/0x30 [447960.770542] [] SyS_clone+0x19/0x20 [447960.770546] [] entry_SYSCALL_64_fastpath+0x16/0x71 [447960.770553] Mem-Info: [447960.770558] active_anon:1746565 inactive_anon:324240 isolated_anon:0 active_file:846342 inactive_file:751599 isolated_file:0 unevictable:914 dirty:2630 writeback:48 unstable:0 slab_reclaimable:129203 slab_unreclaimable:6931 mapped:5151 shmem:1096 pagetables:5042 bounce:0 free:24927 free_pcp:30 free_cma:0 [447960.770562] Node 0 DMA free:15904kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [447960.770569] lowmem_reserve[]: 0 3745 15022 15022 15022 [447960.770589] Node 0 DMA32 free:56712kB min:3896kB low:4868kB high:5844kB active_anon:1534208kB inactive_anon:514992kB active_file:845884kB inactive_file:687520kB unevictable:340kB isolated(anon):0kB isolated(file):0kB present:3915776kB managed:3835152kB mlocked:340kB dirty:2092kB writeback:4kB mapped:10720kB shmem:2736kB slab_reclaimable:176680kB slab_unreclaimable:7712kB kernel_stack:2032kB pagetables:5216kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [447960.770595] lowmem_reserve[]: 0 0 11277 11277 11277 [447960.770599] Node 0 Normal free:27092kB min:11728kB low:14660kB high:17592kB active_anon:5452052kB inactive_anon:781968kB active_file:2539484kB inactive_file:2318876kB unevictable:3316kB isolated(anon):0kB isolated(file):0kB present:11927552kB managed:11547980kB mlocked:3316kB dirty:8428kB writeback:188kB mapped:9884kB shmem:1648kB slab_reclaimable:340132kB slab_unreclaimable:20012kB kernel_stack:6064kB pagetables:14952kB unstable:0kB bounce:0kB free_pcp:240kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [447960.770605] lowmem_reserve[]: 0 0 0 0 0 [447960.770609] Node 0 DMA: 0*4kB 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15904kB [447960.770621] Node 0 DMA32: 2016*4kB (UME) 5850*8kB (UME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB (H) 0*4096kB = 56912kB [447960.770630] Node 0 Normal: 6226*4kB (UME) 38*8kB (UM) 0*16kB 1*32kB (H) 2*64kB (H) 1*128kB (H) 0*256kB 1*512kB (H) 1*1024kB (H) 0*2048kB 0*4096kB = 27032kB [447960.770648] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [447960.770650] 1600516 total pagecache pages [447960.770652] 815 pages in swap cache [447960.770653] Swap cache stats: add 10132296, delete 10131481, find 3831558/6676543 [447960.770655] Free swap = 15331372kB [447960.770656] Total swap = 15399036kB [447960.770657] 3964829 pages RAM [447960.770658] 0 pages HighMem/MovableOnly [447960.770660] 115070 pages reserved [447960.770662] 0 pages cma reserved [447960.770675] 0 pages hwpoisoned [447960.770676] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name [447960.770684] [ 416] 0 416 13209 1483 30 3 806 0 systemd-journal [447960.770690] [ 454] 0 454 25742 392 17 3 32 0 lvmetad [447960.770693] [ 490] 0 490 10774 485 23 3 323 -1000 systemd-udevd [447960.770697] [ 960] 0 960 4030 427 11 3 163 0 dhclient [447960.770699] [ 1078] 0 1078 6932 459 18 3 46 0 cron [447960.770702] [ 1081] 107 1081 29297 244 57 4 219 -900 dbus-daemon [447960.770705] [ 1085] 0 1085 1306 406 9 3 25 0 iscsid [447960.770708] [ 1086] 0 1086 1431 881 9 3 0 -17 iscsid [447960.770710] [ 1088] 0 1088 87055 2 28 5 432 0 snapd [447960.770712] [ 1097] 0 1097 132677 207 25 3 261 0 lxcfs [447960.770715] [ 1099] 0 1099 6511 308 18 3 63 0 atd [447960.770718] [ 1101] 104 1101 81586 463 59 3 403 0 rsyslogd [447960.770722] [ 1103] 0 1103 5096 349 14 3 38 0 systemd-logind [447960.770731] [ 1106] 0 1106 1100 284 8 3 36 0 acpid [447960.770734] [ 1130] 0 1130 85627 484 68 3 229 0 accounts-daemon [447960.770737] [ 1136] 0 1136 16380 511 37 3 154 -1000 sshd [447960.770740] [ 1160] 0 1160 3344 268 11 3 25 0 mdadm [447960.770742] [ 1261] 0 1261 227227 627 76 4 213 0 nscd [447960.770745] [ 1263] 0 1263 4868 402 15 3 37 0 irqbalance [447960.770748] [ 1279] 112 1279 24552 544 18 3 108 0 ntpd [447960.770750] [ 1306] 0 1306 67211 278 32 3 85 0 polkitd [447960.770753] [ 1322] 0 1322 1560 41 8 3 26 0 agetty [447960.770755] [ 1325] 0 1325 1606 87 8 3 28 0 agetty [447960.770758] [29909] 114 29909 2369065 2067390 4419 12 9867 0 mongod [447960.770760] Out of memory: Kill process 29909 (mongod) score 270 or sacrifice child [447960.777191] Killed process 29909 (mongod) total-vm:9476260kB, anon-rss:8259476kB, file-rss:11700kB