Pip error installing cryptography on Big Sur

Question

Using pip version 20.2.4, Python 3.8.2 and Big Sur 11.0.1.

When I run pip install cryptography, I get this error:

  Building wheels for collected packages: cffi
    Building wheel for cffi (setup.py): started
    Building wheel for cffi (setup.py): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /Users/xxx/projects/xxx/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/g8/nj6ghjhgj11j84rsjmqqb00000gn/T/pip-install-aufpkz3c/cffi/setup.py'"'"'; __file__='"'"'/private/var/folders/g8/nj6ghjhgj11j84rsjmqqb00000gn/T/pip-install-aufpkz3c/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/g8/nj6ghjhgj11j84rsjmqqb00000gn/T/pip-wheel-asotxqfl
         cwd: /private/var/folders/g8/nj6ghjhgj11j84rsjmqqb00000gn/T/pip-install-aufpkz3c/cffi/
    Complete output (42 lines):
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.14.6-x86_64-3.8
    creating build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/backend_ctypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/error.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/setuptools_ext.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/cffi_opcode.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/vengine_gen.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/pkgconfig.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/model.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/ffiplatform.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/api.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/vengine_cpy.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/commontypes.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/lock.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/recompiler.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/cparser.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/verifier.py -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/_cffi_include.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/parse_c_type.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/_embedding.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    copying cffi/_cffi_errors.h -> build/lib.macosx-10.14.6-x86_64-3.8/cffi
    running build_ext
    building '_cffi_backend' extension
    creating build/temp.macosx-10.14.6-x86_64-3.8
    creating build/temp.macosx-10.14.6-x86_64-3.8/c
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/local/Cellar/libffi/3.3/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/ffi -I/Users/xxxx/projects/xxxx/venv/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c c/_cffi_backend.c -o build/temp.macosx-10.14.6-x86_64-3.8/c/_cffi_backend.o
    c/_cffi_backend.c:5854:2: error: Apple Arm64 ABI requires ffi_prep_cif_var
    #error Apple Arm64 ABI requires ffi_prep_cif_var
     ^
    c/_cffi_backend.c:6304:9: warning: 'ffi_prep_closure' is deprecated [-Wdeprecated-declarations]
        if (ffi_prep_closure(closure, &cif_descr->cif,
            ^
    /usr/local/Cellar/libffi/3.3/include/ffi.h:341:18: note: 'ffi_prep_closure' has been explicitly marked deprecated here
      __attribute__((deprecated))
                     ^
    1 warning and 1 error generated.
    error: command 'clang' failed with exit status 1
    ----------------------------------------
    ERROR: Failed building wheel for cffi

I didn't ever encounter this before I upgraded MacOS. Any idea what I can try?

Impartial

There were fixes in cffi since the latest release, which look related. Can you try pip install hg+https://foss.heptapod.net/pypy/cffi? – Armistice

Interesting. I get ERROR: Error [Errno 2] No such file or directory: 'hg' while executing command hg clone --noupdate -q https://foss.heptapod.net/pypy/cffi /private/var/folders/g8/nj61c9fhhbjdvbhhbmqqb00000gn/T/pip-req-build-e3dcg2vf ERROR: Cannot find command 'hg' - do you have 'hg' installed and in your PATH? – Impartial

I am able to install cffi from the latest commit as suggested (and importing the module works), but then when I try to install cryptography or argon2-cffi, pip apparently tries to reinstall cffi which of course fails. This is in a fresh virtual environment. Any ideas why? – Vasodilator

It claims to be trying to install cffi "Using cached cffi-1.14.3.tar.gz", even if I use the --no-cache-dir flag... – Vasodilator

I guess it's probably because installing cffi this way doesn't associate a version number with it, so the cryptography package thinks the dependency is unmet? pip freeze shows: cffi @ hg+https://foss.heptapod.net/pypy/cffi/@3440 instead of cffi==1.14.3. – Vasodilator

Actually, it seems this is being caused by bugs in pip on Big Sur: github.com/pypa/pip/issues/9138 github.com/nolze/msoffcrypto-tool/issues/54 – Vasodilator

You can try this solution, I've described here: https://mcmap.net/q/434000/-installing-cryptography-on-an-apple-silicon-m1-mac – Chatty

Answer 1

First install openssl

brew install openssl

Then export compiler options:

export CRYPTOGRAPHY_SUPPRESS_LINK_FLAGS="1" 
export LDFLAGS="-L/usr/local/opt/[email protected]/lib"
export CPPFLAGS="-I/usr/local/opt/[email protected]/include"

and now you can use pip install

pip3 install cryptography==2.8

Inerrant

On Big Sur 11.1 brew install openssl gives you "already installed" but I did the exports and pip install worked. Thanks for the tip. – Straddle

@Straddle Yes but exporting the compiler options does the trick! – Soutache

On Big Sur, the export lines were insufficient. Even though openssl was already installed, I reinstalled it to see if it would help. It still failed for older (1.11.5) versions of cffi – Tremolant

Answer 2

env LDFLAGS="-L$(brew --prefix [email protected])/lib" CFLAGS="-I$(brew --prefix [email protected])/include" pip install cryptography

Kingmaker

This should be the top answer! It still is correct on M1 Arm Mac with Big Sur 11.6 – Cistercian

On my system, brew install openssl gave me openssl@3. Needed to run brew install [email protected], then this worked. – Saline

This worked for me for M1 Arm Mac with Big Sur 11.6. I am using Python 2.7 pip 19.3 and cryptography 3.1.1 – Axle

This worked on M1 os pip but it still cant install cryptography inside virtualenv python 3.8.9. The version cryptography installed on os pip is 38.0.1. – Epidiascope

Answer 3

The answer from @serge-g above worked well on M1 Mac / ARM64 with Homebrew and Big Sur 11.6.

I had to make the following changes:

brew install openssl

export CRYPTOGRAPHY_SUPPRESS_LINK_FLAGS="1" 
export LDFLAGS="-L/opt/homebrew/opt/[email protected]/lib"
export CPPFLAGS="-I/opt/homebrew/opt/[email protected]/include"

At that point installing cryptography worked for me via pip3 install cryptography==2.8

FYI, in my case cryptography was a dependency of the Azure CLI Python package (pip3 install azure-cli). Thanks to @serge-g for pointing me in the right direction!

Cistercian

The answer from @Simeon Aleksov is the best answer because it will work regardless of where homebrew is installed! – Cistercian

Answer 4

According to this issue on cryptography's GitHub, as of November 2020, simply upgrading pip to the latest version allows this to work correctly. I was running into this error myself on pip 19 and upgrading to pip 21.0.1 fixed the issue for me.

Adventurer

Answer 5

You should try poetry shell and pip install cryptography

Inessa