Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7839

Add evergreen task to run python tests with ASAN enabled

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Storage Engines
    • StorEng - Defined Pipeline

      The WiredTiger python tests can be run with ASAN:

      • Build WT with ASAN enabled
      • Use the --asan option to run.py when running tests

      To improve the coverage of our automated tests, we should add an evergreen task that runs the python suite with ASAN.

      One tricky detail is that ASAN currently reports various memory leaks that appear to come from Python itself (or possibly our Python SWIG library?).  We will need to either eliminate these errors or recognize and ignore them.

      For reference, here is an example of a few of those memory leaks currently reported by ASAN:

      Direct leak of 1024 byte(s) in 1 object(s) allocated from:
    #0 0x7fb30a707f83 in malloc /data/mci/f6a3cdb8925fada697dec19875e037b4/toolchain-builder/tmp/build-llvm.sh-cpH/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:146:3
    #1 0x7fb309a31c35 in _PyMem_RawMalloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:99:12
    #2 0x7fb309a31c35 in PyMem_RawMalloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:572:12
    #3 0x7fb309a31c35 in _PyObject_Malloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:1645:11
    #4 0x7fb309a31c35 in PyMem_Malloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:605:12
    #5 0x7fb309a31c35 in set_table_resize /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/setobject.c:260:20


Direct leak of 128 byte(s) in 4 object(s) allocated from:
    #0 0x7fb30a707f83 in malloc /data/mci/f6a3cdb8925fada697dec19875e037b4/toolchain-builder/tmp/build-llvm.sh-cpH/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:146:3
    #1 0x7fb309a11537 in _PyMem_RawMalloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:99:12
    #2 0x7fb309a11537 in PyMem_RawMalloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:572:12
    #3 0x7fb309a11537 in PyThread_allocate_lock.localalias.2 /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Python/thread_pthread.h:380:21


Indirect leak of 199200 byte(s) in 209 object(s) allocated from:
    #0 0x7fb30a707f83 in malloc /data/mci/f6a3cdb8925fada697dec19875e037b4/toolchain-builder/tmp/build-llvm.sh-cpH/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:146:3
    #1 0x7fb309a22cf0 in _PyMem_RawMalloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:99:12
    #2 0x7fb309a22cf0 in PyMem_RawMalloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:572:12
    #3 0x7fb309a22cf0 in _PyObject_Malloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:1645:11
    #4 0x7fb309a22cf0 in PyObject_Malloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/obmalloc.c:685:12
    #5 0x7fb309a22cf0 in _PyObject_GC_Alloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Modules/gcmodule.c:2225:26
    #6 0x7fb309a22cf0 in _PyObject_GC_Malloc /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Modules/gcmodule.c:2252:12
    #7 0x7fb309a22cf0 in PyType_GenericAlloc.localalias.10 /data/mci/78f810c8dfd4519e8edccff6788f14f0/toolchain-builder/tmp/build-python.sh-pQV/build-Python-3.9.2/../src/Python-3.9.2/Objects/typeobject.c:1047:15

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            keith.smith@mongodb.com Keith Smith
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: