I’m afraid that none of these answers are acceptable to me. Because these answers didn’t fix the cause. Some of them are lucky, or misunderstand, or even wrong. So I’m going to provide my own solution and explain in details for the problem.
Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.
The root cause is quite clear itself. Your Python was trying to open (by dlopen) an unversioned OpenSSL shared library named libcrypto. Apple doesn’t allow anyone to use it for the security reason since Catalina. So the solution is straightforward. Just using versioning OpenSSL instead.
I write a python script named openlib.py to reproduce the problem.
import sys
from ctypes.util import find_library
from ctypes import CDLL
name = sys.argv[1]
path = find_library(name)
print(f"path: {path}")
lib = CDLL(path)
I use system Python for demo here
$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.15.7
BuildVersion: 19H15
$ /usr/bin/python3 --version
Python 3.8.2
$ ls -al /usr/lib/ | grep 'libcrypto\|libssl'
.rwxr-xr-x 1.1M root 22 Sep 8:29 libcrypto.0.9.7.dylib
.rwxr-xr-x 1.4M root 22 Sep 8:29 libcrypto.0.9.8.dylib
.rwxr-xr-x 1.5M root 22 Sep 8:29 libcrypto.35.dylib
.rwxr-xr-x 1.5M root 22 Sep 8:29 libcrypto.41.dylib
.rwxr-xr-x 1.5M root 22 Sep 8:29 libcrypto.42.dylib
.rwxr-xr-x 1.5M root 22 Sep 8:29 libcrypto.44.dylib
.rwxr-xr-x 32k root 22 Sep 8:29 libcrypto.dylib
.rwxr-xr-x 212k root 22 Sep 8:29 libssl.0.9.7.dylib
.rwxr-xr-x 335k root 22 Sep 8:30 libssl.0.9.8.dylib
.rwxr-xr-x 330k root 22 Sep 8:28 libssl.35.dylib
.rwxr-xr-x 313k root 22 Sep 8:29 libssl.43.dylib
.rwxr-xr-x 300k root 22 Sep 8:29 libssl.44.dylib
.rwxr-xr-x 294k root 22 Sep 8:29 libssl.46.dylib
.rwxr-xr-x 32k root 22 Sep 8:29 libssl.dylib
$ /usr/bin/python3 openlib.py libcrypto
path: /usr/lib/libcrypto.dylib
Abort trap: 6
$ /usr/bin/python3 openlib.py libcrypto.35
path: /usr/lib/libcrypto.35.dylib
$ /usr/bin/python3 openlib.py libcrypto.44
path: /usr/lib/libcrypto.44.dylib
As you can see. Python has crashed with argument libcrypto as following diagnostic report. Look similar, right? Bingo!
Process: Python [97291]
Path: /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python
Identifier: Python
Version: 3.8.2 (3.8.2)
Build Info: python3-73040006000000~117
Code Type: X86-64 (Native)
Parent Process: bash [84388]
Responsible: iTerm2 [7705]
User ID: 501
Date/Time: 2020-12-26 00:28:00.281 +0800
OS Version: Mac OS X 10.15.7 (19H15)
Report Version: 12
Anonymous UUID: 1C43F3DB-1783-4B94-B663-7F7E8D331B56
Time Awake Since Boot: 53000 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Application Specific Information:
/usr/lib/libcrypto.dylib
abort() called
Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fff69bba33a __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff69c76e60 pthread_kill + 430
2 libsystem_c.dylib 0x00007fff69b41808 abort + 120
3 libcrypto.dylib 0x00007fff6766b7e4 __report_load + 415
Based on the path of your Python /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python. Apparently, Your Python is installed by Homebrew. I believe Homebrew Python 3 has been linked with keg-only OpenSSL. So It must be some packages that using unversioned OpenSSL. Looking at file ctypes/macholib/dyld.py 2. It searches for any libraries in following directories in the specified order:
DEFAULT_LIBRARY_FALLBACK = [
os.path.expanduser("~/lib"),
"/usr/local/lib",
"/lib",
"/usr/lib",
]
$ /usr/bin/python3
Python 3.8.2 (default, Nov 4 2020, 21:23:28)
[Clang 12.0.0 (clang-1200.0.32.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library
>>> find_library('libcrypto')
'/usr/lib/libcrypto.dylib'
>>>
So the easy fix is linking versioning OpenSSL to the library path in your home directory.
$ pwd
/Users/gasolwu
$ ln -s /usr/lib/libcrypto.44.dylib $HOME/lib/libcrypto.dylib
$ $ ls ~/lib
libcrypto.dylib
After that. Test opening OpenSSL by running the provided script openlib.py. It returns library path successfully without crash.
$ /usr/bin/python3 openlib.py libcrypto
path: /Users/gasolwu/lib/libcrypto.dylib
I use Homebrew Python 3 too. So I fixed it a couple of days ago and have sent a pull request. If you have already upgraded to the latest version of Python and If PR has been merge and the bottle has built and published. Simply run the command with brew reinstall [email protected] will be the easiest way to fix your problem.
Don’t break your system by disabling SIP and using sudo to overwrite your system OpenSSL.
Don’t install duplicate libraries that waste your disk space. There is no need to install another OpenSSL in any location.
Don't use environment variable DYLD_LIBRARY_PATH as follows. You have to declare every time if you don't add this line to your configuration of the shell. If you do, It will affect every program on your machine.
export DYLD_LIBRARY_PATH=/usr/local/opt/openssl/lib:$DYLD_LIBRARY_PATH
Finally. If you fixed the problem by updating python dependencies. You might be lucky. Some packages have fixed the problem by looking for versioning OpenSSL. But many are not.
