Cannot install psycopg2 Ubuntu

Question

Trying to get a server ready for a django project and I'm running into some issues with setup for postgres.

I'm following this guide: https://jee-appy.blogspot.com/2017/01/deply-django-with-nginx.html

And I'm at step 5:

5. Now, we need to configure postgreSQL so that it can communicate with our Django application. For this, install psycopg2 database adapter. But this adapter have some package dependencies, so first install them.

run:

(django_env) $ sudo apt-get install libpq-dev python3-dev

then...

(django_env) $ pip install psycopg2

I do that, and this point in the instructions, I get a "compilation failed" error.

I tried the solutions suggested in this stack exchange question: Trouble with psycopg2 in virtualenv python3 for use with Django and this one: Cannot install psycopg2 on virtualenv

I am very much a linux and django noob, so if this is a duplicate issue, please have mercy on me and leave a comment explaining why you are marking it as a duplicate as you do so.

Thanks for your time!

Here's the output and install command that caused it:

(django_env1) user:/home/projects/sample_project$ sudo pip install psycopg2
Downloading/unpacking psycopg2
  Downloading psycopg2-2.7.3.2.tar.gz (425kB): 425kB downloaded
  Running setup.py (path:/tmp/pip_build_root/psycopg2/setup.py) egg_info for package psycopg2

Installing collected packages: psycopg2
  Running setup.py install for psycopg2
    building 'psycopg2._psycopg' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.7.3.2 (dt dec pq3 ext lo64)" -DPG_VERSION_NUM=90320 -DHAVE_LO64=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.3/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement
    In file included from psycopg/psycopgmodule.c:27:0:
    ./psycopg/psycopg.h:34:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/psycopg2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-_0zc1L-record/install-record.txt --single-version-externally-managed --compile:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/psycopg2

copying lib/_ipaddress.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/tz.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/_json.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/errorcodes.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/psycopg1.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/_range.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/extensions.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/sql.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/pool.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/extras.py -> build/lib.linux-x86_64-2.7/psycopg2

creating build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_bug_gc.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_psycopg2_dbapi20.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_types_extras.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_async_keyword.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_sql.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_ipaddress.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/dbapi20.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_fast_executemany.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_copy.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_with.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_connection.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_module.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_replication.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_extras_dictcursor.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_dates.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_bugX000.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/testconfig.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_errcodes.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_transaction.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_async.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/testutils.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_cursor.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_quote.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/dbapi20_tpc.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_notify.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_lobject.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_cancel.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_green.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_types_basic.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

running build_ext

building 'psycopg2._psycopg' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/psycopg

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.7.3.2 (dt dec pq3 ext lo64)" -DPG_VERSION_NUM=90320 -DHAVE_LO64=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.3/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement

In file included from psycopg/psycopgmodule.c:27:0:

./psycopg/psycopg.h:34:20: fatal error: Python.h: No such file or directory

 #include <Python.h>

                    ^

compilation terminated.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/psycopg2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-_0zc1L-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/psycopg2

Derril

You installed Python3 headers, but Python in your virt. environment is 2.7 (/usr/include/python2.7). – Clerical

Hmmm.. okay, I only barely understand package management tools for linux conceptually, so I'm struggling with this (obviously). – Derril

I thought I had installed python 3 on this virtual environment. How can I set it to point to python 3? – Derril

Fair enough :) I guess next step is to google: "how to set python version when creation new virtual envrionment in virtaulenv" ? – Derril

Answer 1

You've installed the python-dev libraries for Python 3. Your psycopg2 install is using python 2.7

If on Ubuntu 20.04 or above

sudo apt-get install build-essential

Then run the other steps for either Python 2 or 3:

If you want to use Python 3:

Make sure you've installed python3-pip

sudo apt-get install python3-pip

Then:

pip3 install psycopg2

If that still fails, try installing the development headers for postgresql:

sudo apt install python3-dev libpq-dev
pip3 install psycopg2

If on Python 2

sudo apt update
sudo apt-get install postgresql postgresql-contrib
sudo apt-get install libpq-dev
sudo apt-get install python-dev
sudo apt-get install python-pip
pip2 install psycopg2-binary

Copacetic

Looks like you and Maurice are in agreement. – Derril

Yup, but: pip3 install <module-name> – Copacetic

I am still new to package management and linux in general, how can I pint my env. to the right version of python? – Derril

Updated answer :) Use pip3 – Copacetic

Glad I could help! Best of luck, hope to see you back soon as paid-intern :) – Copacetic

thank you, you helped while setting up pg with ansible – Banzai

@EdwardCasanova glad to hear that! I revisit this answer from time to time and try to keep it updated; cool to hear it still helps people :) – Copacetic

Answer 2

Note that on many distros, the development headers needed for compiling against libraries are not installed by default. For psycopg2 on Ubuntu you'll need the python3 and postgresql headers:

sudo apt install python3-dev libpq-dev
pip3 install psycopg2

These can be installed in your activated virtual environment.

Expectation

In case you've multiple versions of python (ex. 3.8, 3.7...), you must specify a specific version of python, like: sudo apt-get install python3.7-dev. – Additament

Answer 3

I was testing my application on a ubuntu docker image that only has python2.7.6 installed (as happens in most big orgs)

If you are using python2.x (though you should plan to move to 3.x asap), check the below dependencies:

sudo apt update
sudo apt-get install postgresql postgresql-contrib
sudo apt-get install libpq-dev # this is required as psycopg2 uses pg_config
sudo apt-get install python-dev
sudo apt-get install python-pip

Now install psycopg2 using:

pip2 install psycopg2-binary

Stinking

This one was missing for me sudo apt-get install libpq-dev. After installing this package I was able to pip install psycopg2 – Bigwig

@FabienQuatravaux Thanks. After installing libpq-dev, now it's working. – Belier

Answer 4

I had the same error trying to install it in a virtualenv (with python3)

I solved it by installing a previous version of psycopg2.

pip install psycopg2==2.7.5

Manner

I used ` pip install psycopg2==2.8.3` – By

Answer 5

This solved mine. I am using Python 3.8.2, Ubuntu 20.04 LTS:

sudo apt-get install python3-dev
sudo apt-get install python3-pip
pip install psycopg2

Reba

Answer 6

To install psycopg2 in ubuntu or mate 20 you need first to install:

sudo apt install libpq-dev

and then:

pip3 install psycopg2

Ellis

Answer 7

In my case, I was facing this problem when I ran pip install -r requirements.txt to install all packages for a Django project on an Ubuntu machine, I ran into this error and many other installation errors.

To solve this one, I ran the following commands:

sudo apt install postgresql postgresql-contrib  

sudo apt install libpq-dev

sudo apt install python3-dev

sudo apt install python3-pip

sudo apt install python3-psycopg2

pip3 install psycopg2

pip3 install psycopg2-binary

Plus, also check if the Ubuntu and Python and Psycopg versions are compatible together.

Also, @Arghya Bhattacharya answers pip install aiopg, solve the issue when i ran into it the second time.

Embroideress

Answer 8

I had to install this one as well on my Ubuntu 20.04 LTS:

sudo apt-get install build-essential

Postmistress

Answer 9

I faced same issue on my ubuntu 18.04 LTS OS. Also, face some issues in Pillow. In both cases (psycopg2 and Pillow) these command solved my issue.

sudo apt install -y build-essential libssl-dev libffi-dev python3-dev libjpeg-dev libjpeg8-dev

sudo apt install libpq-dev

sudo apt-get install python3-pip

Note: I have installed psycopg2 in python=3.8.5 environment.

Mauve

Answer 10

make sure you are using the correct psycopg version for the python version.

Example for python 3.8.

python 3.8, the supported version is psycopg 2.8.4.

Reference ubuntu 20.04 + python 3.8 , pip install psycopg2==2.7.3.2 error #1106

Kinzer

Answer 11

I solved this issue using another package which itself internally installs psycopg2.

pip install aiopg

Fidler