The compile and compile_all Evergreen tasks has been failing intermittently due to one of the ld processes being killed by OOM killer since Feb 2017. As a workaround—similar to what was done in SERVER-29442 and SERVER-30516—we should reduce the value for the --jobs option specified to SCons when our s390x builders are running the compile and compile_all task to avoid further Evergreen failures. Given that an individual linker process may use >3.5GB of memory of 16GB total, we should prevent SCons from running more than 3 of them as the same time by specifying -j3 rather than -j$(grep -c ^processor /proc/cpuinfo) or -j$(( $(grep -c ^processor /proc/cpuinfo) / 2 )) in each build variant's ${compile_flags} expansion.
[2018/02/16 20:07:33.596] mci-exec 9539 3.3 22.5 3896412 3660772 ? D 19:52 0:31 /opt/mongodbtoolchain/v2/lib/gcc/s390x-mongodb-linux/5.4.0/../../../../s390x-mongodb-linux/bin/ld @/tmp/ccoOHhDJ [2018/02/16 20:07:33.596] mci-exec 9837 3.4 22.6 3896592 3685484 ? D 19:52 0:31 /opt/mongodbtoolchain/v2/lib/gcc/s390x-mongodb-linux/5.4.0/../../../../s390x-mongodb-linux/bin/ld @/tmp/cc8EneGh [2018/02/16 20:07:33.597] mci-exec 13693 5.0 22.2 3883392 3625496 ? D 19:57 0:30 /opt/mongodbtoolchain/v2/lib/gcc/s390x-mongodb-linux/5.4.0/../../../../s390x-mongodb-linux/bin/ld @/tmp/ccLraHsk [2018/02/16 20:07:33.597] mci-exec 14395 5.5 22.3 3885152 3640884 ? D 19:58 0:30 /opt/mongodbtoolchain/v2/lib/gcc/s390x-mongodb-linux/5.4.0/../../../../s390x-mongodb-linux/bin/ld @/tmp/ccekp0ZZ
- is related to
-
SERVER-29442 Reduce concurrency for Solaris builds to avoid OOM failures
- Closed
-
SERVER-30516 Reduce concurrency for Windows compile_all builds to avoid OOM failures
- Closed