Error 'openssl/crypto.h' file not found when installing PycURL on Mac using pip

Question

When installing Python3 packages on macOS X 10.15 Catalina using pip install -r requirements.txt I am getting this error:

× Running setup.py install for pycurl did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-8r44qz4e/pycurl_09ab3d56141443439eb3f41fa62f7b0f/setup.py:771: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        if LooseVersion(distutils.__version__) > LooseVersion("1.0.1"):
      /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-8r44qz4e/pycurl_09ab3d56141443439eb3f41fa62f7b0f/setup.py:773: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        if LooseVersion(distutils.__version__) < LooseVersion("1.0.3"):
      Using curl-config (libcurl 7.84.0)
      running install
      /usr/local/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.15-x86_64-cpython-39
      creating build/lib.macosx-10.15-x86_64-cpython-39/curl
      copying python/curl/__init__.py -> build/lib.macosx-10.15-x86_64-cpython-39/curl
      running build_ext
      building 'pycurl' extension
      creating build/temp.macosx-10.15-x86_64-cpython-39
      creating build/temp.macosx-10.15-x86_64-cpython-39/src
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DPYCURL_VERSION=\"7.43.0\" -DHAVE_CURL_OPENSSL=1 -DHAVE_CURL_SSL=1 -I/usr/local/Cellar/curl/7.84.0/include -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/docstrings.c -o build/temp.macosx-10.15-x86_64-cpython-39/src/docstrings.o
      In file included from src/docstrings.c:4:
      src/pycurl.h:170:13: fatal error: 'openssl/crypto.h' file not found
      #   include <openssl/crypto.h>
                  ^~~~~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

So It fails on installing PycURL because it cannot find the openssl include directory.

I have first added cryptography==37.0.4 to the requirements.txt file but then I got:

src/pycurl.h:178:13: fatal error: 'openssl/ssl.h' file not found

Then I have tried installing latest openssl with:

brew update
brew install openssl
brew link --force openssl
ln -s /usr/local/opt/openssl/lib/libcrypto.3.dylib /usr/local/lib/
ln -s /usr/local/opt/openssl/lib/libssl.3.dylib /usr/local/lib/
ln -s /usr/local/Cellar/openssl/3.0.4/bin/openssl /usr/local/bin/openssl

But it didn't help.

I also tried:

export PYCURL_SSL_LIBRARY=openssl
export LDFLAGS="-L/usr/local/opt/openssl@3/lib"
export CPPFLAGS="-I/usr/local/opt/openssl@3/include"

pip install --no-cache-dir --compile --ignore-installed --install-option="--with-openssl" pycurl

But also with no success. Any help on this?

Cartwheel

Answer 1

What eventually helped was:

1. Remove pycurl from the requirements.txt file
2. Install openssl with: brew install openssl
3. Find openssl installation directory with

$ brew --prefix openssl
/usr/local/opt/openssl@3

$ ls -la /usr/local/opt/openssl@3
lrwxr-xr-x  1 runner  admin  25 Jul  6 09:13 /usr/local/opt/openssl@3 -> ../Cellar/openssl@3/3.0.4

4. Install PycURL specifying inline the above openssl install directories like this:

PYCURL_SSL_LIBRARY=openssl LDFLAGS="-L/usr/local/Cellar/openssl@3/3.0.4/lib" CPPFLAGS="-I/usr/local/Cellar/openssl@3/3.0.4/include" pip3 install --no-cache-dir pycurl

5. Now the PycURL installs without any problem:

Collecting pycurl
  Downloading pycurl-7.45.1.tar.gz (233 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 233.9/233.9 kB 64.5 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: pycurl
  Building wheel for pycurl (setup.py): started
  Building wheel for pycurl (setup.py): finished with status 'done'
  Created wheel for pycurl: filename=pycurl-7.45.1-cp39-cp39-macosx_10_15_x86_64.whl size=146607 sha256=aff9581cbd6ddec739c529df376ad2d4edf49dafdad179a9b68bddeb60a739b3
  Stored in directory: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-ephem-wheel-cache-hjqadgim/wheels/4d/ef/77/ffab9dc4b0016ce4f780b752e0020815e416f0d27a701a816f
Successfully built pycurl
Installing collected packages: pycurl
Successfully installed pycurl-7.45.1

Cartwheel

It helped to install the pycurl, but still not able to import pycurl? – Jablonski

It usually happens when you install pycurl on the system level and then trying using it within a virtual environment (venv) which uses a different Python version. So, for example, your system Python can be 2.x while your venv may use Python 3.x. Always create a venv for your project. Then after installing everything (requirements.txt + pycurl) inside you venv you should test your installation – Cartwheel

You can test your installation in Terminal by executing which python, then executing <your-python-path> -m pip list. If the installation was successful you should see pycurl on the list, then execute <your-python-path. Now, being in Python, try to execute import pycurl. It should work. Then exit Python with exit(). If you are using VS Code make sure you select the right interpreter from your venv. You should see the Python interpreter version at the bottom right in your IDE. Or else do CMD+SHIFT+P then >Python: Select Interpreter. – Cartwheel

I always use VENV for my projects. I followed your steps to test the installation and tried to import the python, getting this error ImportError: pycurl: libcurl link-time version (7.77.0) is older than compile-time version (7.79.1) couldn't resolve this error yet. how can I resolve this? – Jablonski

If you are using Python > 3.7.x then try switching to Python 3.7.x – Cartwheel

I updated my python version to 3.8.10, still pycurl is giving hard time to get installed – Jablonski

I said to try Python 3.7.x – Cartwheel

I cant try Python 3.7.x, its a project dependency. – Jablonski

This worked like magic. Mac OS 13 Ventura, Django 4.1.9, Celery 5.1.3, M2 Apple Silicon Chip – Mountain

Answer 2

Using brew --prefix openssl to dynamically lookup the paths for LDFLAGS, CPPFLAGS, and PKG_CONFIG_PATH:

OPENSSL_DIR=$(brew --prefix openssl) \
LDFLAGS="-L${OPENSSL_DIR}/lib" CPPFLAGS="-I${OPENSSL_DIR}/include" \
PKG_CONFIG_PATH="${OPENSSL_DIR}/lib/pkgconfig" \
pip install pycurl

Millimicron

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Parameter

Answer 3

first install openssl

brew install openssl

then

export PKG_CONFIG_PATH="/usr/local/opt/openssl@3/lib/pkgconfig"
export CPPFLAGS="-I/usr/local/opt/openssl@3/include"
export LDFLAGS="-L/usr/local/opt/openssl@3/lib"

the exception go away

pip3 install pycurl
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Collecting pycurl
  Using cached pycurl-7.45.2.tar.gz (234 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pycurl
  Building wheel for pycurl (setup.py) ... done
  Created wheel for pycurl: filename=pycurl-7.45.2-cp39-cp39-macosx_10_13_x86_64.whl size=144557 sha256=2c56c17f7987b8739333b8078c6d8b61ed5c8e5239ea182992bbe0eb64208970
  Stored in directory: /Users/xlla/Library/Caches/pip/wheels/23/b0/37/d2c3211ee738adfb8ec6b6e10aa00e78ebc4de363f862a12c5
Successfully built pycurl
Installing collected packages: pycurl
Successfully installed pycurl-7.45.2

Careaga