Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-2190

In-place C extensions cannot be loaded on macOS with Python 2.7

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None

      I'm seeing a the following error when attempting to use the in-place c extensions with Python 2.7 on macOS:

      >>> from bson import _cbson
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      ImportError: dlopen(bson/_cbson.so, 2): no suitable image found.  Did find:
      	file system relative paths not allowed in hardened programs
      

      Full log on a fresh clone:

      $ git clone git@github.com:mongodb/mongo-python-driver.git
      Cloning into 'mongo-python-driver'...
      remote: Enumerating objects: 97, done.
      remote: Counting objects: 100% (97/97), done.
      remote: Compressing objects: 100% (63/63), done.
      remote: Total 30933 (delta 54), reused 57 (delta 34), pack-reused 30836
      Receiving objects: 100% (30933/30933), 14.86 MiB | 9.66 MiB/s, done.
      Resolving deltas: 100% (23585/23585), done.
      $ cd mongo-python-driver
      $ /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python setup.py build_ext -i
      running build_ext
      building 'bson._cbson' extension
      creating build
      creating build/temp.macosx-10.15-x86_64-2.7
      creating build/temp.macosx-10.15-x86_64-2.7/bson
      cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -pipe -Ibson -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c bson/_cbsonmodule.c -o build/temp.macosx-10.15-x86_64-2.7/bson/_cbsonmodule.o
      cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -pipe -Ibson -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c bson/time64.c -o build/temp.macosx-10.15-x86_64-2.7/bson/time64.o
      cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -pipe -Ibson -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c bson/buffer.c -o build/temp.macosx-10.15-x86_64-2.7/bson/buffer.o
      cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -pipe -Ibson -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c bson/encoding_helpers.c -o build/temp.macosx-10.15-x86_64-2.7/bson/encoding_helpers.o
      cc -bundle -undefined dynamic_lookup -Wl,-F. -L/AppleInternal/BuildRoot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.Internal.sdk/usr/local/libressl/lib build/temp.macosx-10.15-x86_64-2.7/bson/_cbsonmodule.o build/temp.macosx-10.15-x86_64-2.7/bson/time64.o build/temp.macosx-10.15-x86_64-2.7/bson/buffer.o build/temp.macosx-10.15-x86_64-2.7/bson/encoding_helpers.o -o /Users/shane/mongo-python-driver/bson/_cbson.so
      ld: warning: directory not found for option '-L/AppleInternal/BuildRoot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.Internal.sdk/usr/local/libressl/lib'
      building 'pymongo._cmessage' extension
      creating build/temp.macosx-10.15-x86_64-2.7/pymongo
      cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -pipe -Ibson -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c pymongo/_cmessagemodule.c -o build/temp.macosx-10.15-x86_64-2.7/pymongo/_cmessagemodule.o
      cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -iwithsysroot /usr/local/libressl/include -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -pipe -Ibson -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c bson/buffer.c -o build/temp.macosx-10.15-x86_64-2.7/bson/buffer.o
      cc -bundle -undefined dynamic_lookup -Wl,-F. -L/AppleInternal/BuildRoot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.Internal.sdk/usr/local/libressl/lib build/temp.macosx-10.15-x86_64-2.7/pymongo/_cmessagemodule.o build/temp.macosx-10.15-x86_64-2.7/bson/buffer.o -o /Users/shane/mongo-python-driver/pymongo/_cmessage.so
      ld: warning: directory not found for option '-L/AppleInternal/BuildRoot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.Internal.sdk/usr/local/libressl/lib'
      $ /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python
      
      WARNING: Python 2.7 is not recommended.
      This version is included in macOS for compatibility with legacy software.
      Future versions of macOS will not include Python 2.7.
      Instead, it is recommended that you transition to using 'python3' from within Terminal.
      
      Python 2.7.16 (default, Feb 29 2020, 01:55:37)
      [GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.29.20) (-macos10.15-objc- on darwin
      Type "help", "copyright", "credits" or "license" for more information.
      >>> import bson
      >>> bson.has_c()
      False
      >>> from bson import _cbson
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      ImportError: dlopen(bson/_cbson.so, 2): no suitable image found.  Did find:
      	file system relative paths not allowed in hardened programs
      >>> from pymongo import _cmessage
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      ImportError: dlopen(pymongo/_cmessage.so, 2): no suitable image found.  Did find:
      	file system relative paths not allowed in hardened programs
      

      However the same python2.7 can successfully use the c extensions when installing from a wheel:

      $ /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python -m pip install --user --upgrade ./mongo-python-driver
      DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
      Processing ./mongo-python-driver
      Building wheels for collected packages: pymongo
        Building wheel for pymongo (setup.py) ... done
        Created wheel for pymongo: filename=pymongo-3.11.0.dev1-cp27-cp27m-macosx_10_15_x86_64.whl size=364018 sha256=c0279586e1adc1791edc9e1cc96f825da960aa46f6667359c7b56919baba5985
        Stored in directory: /Users/shane/Library/Caches/pip/wheels/48/ab/df/80ccba4a1321c605240b466f9e4d8122e4272b156b470930fa
      Successfully built pymongo
      Installing collected packages: pymongo
        Attempting uninstall: pymongo
          Found existing installation: pymongo 3.10.1
          Uninstalling pymongo-3.10.1:
            Successfully uninstalled pymongo-3.10.1
      Successfully installed pymongo-3.11.0.dev1
      $ /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python
      WARNING: Python 2.7 is not recommended.
      This version is included in macOS for compatibility with legacy software.
      Future versions of macOS will not include Python 2.7.
      Instead, it is recommended that you transition to using 'python3' from within Terminal.
      
      Python 2.7.16 (default, Feb 29 2020, 01:55:37)
      [GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.29.20) (-macos10.15-objc- on darwin
      Type "help", "copyright", "credits" or "license" for more information.
      >>> import bson
      >>> bson.has_c()
      True
      >>> from bson import _cbson
      >>>
      

      Using a 2.7 virtualenv also fixes the issue. The error only seems to occur when using the system python directly with in-place c extensions.

            Assignee:
            Unassigned Unassigned
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: