I'm having issues building wheel for psycopg2 thru pip install -r requirements.txt. I'm on ubuntu 20.04 + python 3.8.5 + venv.
This is my requirements.txt:
amqp==2.6.1
anyjson==0.3.3
asgiref==3.2.10
billiard==3.6.3.0
brotlipy==0.7.0
celery==4.4.7
celery-progress==0.0.12
certifi==2020.6.20
cffi==1.14.2
chardet==3.0.4
cryptography==3.1
Django==3.0.3
dj-database-url==0.5.0
django-celery-results==1.2.1
django-cors-headers==3.5.0
django-crispy-forms==1.9.2
django-heroku==0.3.1
django-rest-framework==0.1.0
django-templated-mail==1.1.1
djangorestframework==3.11.1
djoser==2.0.5
fake-useragent==0.1.11
future==0.18.2
gunicorn==20.0.4
httpie==2.2.0
idna==2.10
kombu==4.6.11
lxml==4.5.2
pika==1.1.0
psycopg2==2.8.5
pycparser==2.20
Pygments==2.7.0
pyOpenSSL==19.1.0
PySocks==1.7.1
python-dateutil==2.8.1
python-decouple==3.3
pytz==2020.1
requests==2.24.0
six==1.15.0
SQLAlchemy==1.3.19
sqlparse==0.3.1
urllib3==1.25.10
vine==1.3.0
whitenoise==5.2.0
This is the output when I pip install -r requirements.txt:
[...]
Collecting urllib3==1.25.10
Using cached urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
Collecting vine==1.3.0
Using cached vine-1.3.0-py2.py3-none-any.whl (14 kB)
Collecting whitenoise==5.2.0
Using cached whitenoise-5.2.0-py2.py3-none-any.whl (19 kB)
Requirement already satisfied: setuptools>=3.0 in ./venv/lib/python3.8/site-packages (from gunicorn==20.0.4->-r requirements.txt (line 24)) (44.0.0)
Building wheels for collected packages: psycopg2
Building wheel for psycopg2 (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/pierre/Workspace/campground_scavanger/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1xr9yjk0/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-1xr9yjk0/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-b8g9assp
cwd: /tmp/pip-install-1xr9yjk0/psycopg2/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for psycopg2
Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: vine, amqp, anyjson, asgiref, billiard, pycparser, cffi, brotlipy, kombu, pytz, celery, celery-progress, certifi, chardet, six, cryptography, sqlparse, Django, dj-database-url, django-celery-results, django-cors-headers, django-crispy-forms, whitenoise, psycopg2, django-heroku, djangorestframework, django-rest-framework, django-templated-mail, djoser, fake-useragent, future, gunicorn, idna, urllib3, requests, Pygments, httpie, lxml, pika, pyOpenSSL, PySocks, python-dateutil, python-decouple, SQLAlchemy
Running setup.py install for psycopg2 ... error
ERROR: Command errored out with exit status 1:
command: /home/pierre/Workspace/campground_scavanger/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1xr9yjk0/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-1xr9yjk0/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-8dtfz_uf/install-record.txt --single-version-externally-managed --compile --install-headers /home/pierre/Workspace/campground_scavanger/venv/include/site/python3.8/psycopg2
cwd: /tmp/pip-install-1xr9yjk0/psycopg2/
Complete output (40 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/__init__.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.8/psycopg2
running build_ext
building 'psycopg2._psycopg' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/psycopg
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.5 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=120005 -DHAVE_LO64=1 -I/home/pierre/Workspace/campground_scavanger/venv/include -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from psycopg/psycopgmodule.c:28:
./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
36 | #include <libpq-fe.h>
| ^~~~~~~~~~~~
compilation terminated.
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /home/pierre/Workspace/campground_scavanger/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1xr9yjk0/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-1xr9yjk0/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-8dtfz_uf/install-record.txt --single-version-externally-managed --compile --install-headers /home/pierre/Workspace/campground_scavanger/venv/include/site/python3.8/psycopg2 Check the logs for full command output.
Googling the first error error: invalid command 'bdist_wheel' led me to run pip install wheel in my venv. Successfully installed wheel-0.36.2. Not sure if this is related whatsoever with my psycopg2 issue hereafter but I'm showing all the steps I'm doing.
I then reran pip install -r requirements.txt and now only the psycopg2 issue shows up:
[...]
Requirement already satisfied: vine==1.3.0 in ./venv/lib/python3.8/site-packages (from -r requirements.txt (line 43)) (1.3.0)
Requirement already satisfied: whitenoise==5.2.0 in ./venv/lib/python3.8/site-packages (from -r requirements.txt (line 44)) (5.2.0)
Requirement already satisfied: setuptools>=3.0 in ./venv/lib/python3.8/site-packages (from gunicorn==20.0.4->-r requirements.txt (line 24)) (44.0.0)
Building wheels for collected packages: psycopg2
Building wheel for psycopg2 (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/pierre/Workspace/campground_scavanger/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-zxwqo979/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-zxwqo979/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-fknnvsn3
cwd: /tmp/pip-install-zxwqo979/psycopg2/
Complete output (40 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/__init__.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.8/psycopg2
running build_ext
building 'psycopg2._psycopg' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/psycopg
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.5 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=120005 -DHAVE_LO64=1 -I/home/pierre/Workspace/campground_scavanger/venv/include -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from psycopg/psycopgmodule.c:28:
./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
36 | #include <libpq-fe.h>
| ^~~~~~~~~~~~
compilation terminated.
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for psycopg2
Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: psycopg2, django-heroku, djangorestframework, django-rest-framework, django-templated-mail, djoser, fake-useragent, future, gunicorn, urllib3, idna, requests, Pygments, httpie, lxml, pika, pyOpenSSL, PySocks, python-dateutil, python-decouple, SQLAlchemy
Running setup.py install for psycopg2 ... error
ERROR: Command errored out with exit status 1:
command: /home/pierre/Workspace/campground_scavanger/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-zxwqo979/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-zxwqo979/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-r4aij71q/install-record.txt --single-version-externally-managed --compile --install-headers /home/pierre/Workspace/campground_scavanger/venv/include/site/python3.8/psycopg2
cwd: /tmp/pip-install-zxwqo979/psycopg2/
Complete output (40 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/__init__.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.8/psycopg2
running build_ext
building 'psycopg2._psycopg' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/psycopg
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.5 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=120005 -DHAVE_LO64=1 -I/home/pierre/Workspace/campground_scavanger/venv/include -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from psycopg/psycopgmodule.c:28:
./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
36 | #include <libpq-fe.h>
| ^~~~~~~~~~~~
compilation terminated.
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /home/pierre/Workspace/campground_scavanger/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-zxwqo979/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-zxwqo979/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-r4aij71q/install-record.txt --single-version-externally-managed --compile --install-headers /home/pierre/Workspace/campground_scavanger/venv/include/site/python3.8/psycopg2 Check the logs for full command output.
I replaced psycopg2==2.8.5 by psycopg2-binary==2.8.5 as per some other stackoverflow threads but the same issue remains during the pip install -r requirements.txt .
If I install manually psycopg2-binary in my venv it works fine:
(venv) pierre@thinkpad:~/Workspace/campground_scavanger$ pip install psycopg2-binary
Collecting psycopg2-binary
Using cached psycopg2_binary-2.8.6-cp38-cp38-manylinux1_x86_64.whl (3.0 MB)
Installing collected packages: psycopg2-binary
Successfully installed psycopg2-binary-2.8.6
But then again if I comment out psycopg2==2.8.5 (or psycopg2-binary==2.8.5 and rerun pip install -r requirements.txt, I keep getting the same error message.
I'm not quite sure why there is a Building wheels for collected packages: psycopg2 even if psycopg2 is commented out in my requirements.txt. Dependency with another package?