Error trying to install Postgres for python (psycopg2)

Question

I tried to install psycopg2 to my environment, but I get the following error:

(venv)avlahop@apostolos-laptop:~/development/django/rhombus-dental$ sudo pip install psycopg2
Downloading/unpacking psycopg2,
Downloading psycopg2-2.5.2.tar.gz (685kB): 685kB downloaded
Running setup.py egg_info for package psycopg2

Installing collected packages: psycopg2
  Running setup.py install for psycopg2
    building 'psycopg2._psycopg' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.5.2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x09010D -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.1/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement
    In file included from psycopg/psycopgmodule.c:27:0:
    ./psycopg/psycopg.h:30:20: fatal error: Python.h: Δεν υπάρχει τέτοιο αρχείο ή κατάλογος
     #include <Python.h>
                    ^
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip_build_root/psycopg2/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-SgfQCA-record/install-record.txt --single-version-externally-managed:
running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/psycopg2

copying lib/pool.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/errorcodes.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/_json.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/_range.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/extensions.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/psycopg1.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/tz.py -> build/lib.linux-x86_64-2.7/psycopg2

copying lib/extras.py -> build/lib.linux-x86_64-2.7/psycopg2

creating build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/testconfig.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copyng tests/test_bug_gc.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_dates.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_copy.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_cancel.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_bugX000.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_extras_dictcursor.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_psycopg2_dbapi20.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_types_basic.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_async.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_lobject.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_cursor.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_with.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/__init__.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_types_extras.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/testutils.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_notify.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_green.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_quote.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_connection.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_transaction.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/dbapi20.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/test_module.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

copying tests/dbapi20_tpc.py -> build/lib.linux-x86_64-2.7/psycopg2/tests

running build_ext

building 'psycopg2._psycopg' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/psycopg

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.5.2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x09010D -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.1/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement

In file included from psycopg/psycopgmodule.c:27:0:

./psycopg/psycopg.h:30:20: fatal error: Python.h: No such file or directory
 #include <Python.h>

                ^

compilation terminated.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip_build_root/psycopg2/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-SgfQCA-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_root/psycopg2
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.4.1', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 148, in main
    return command.main(args[1:], options)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 169, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 54: ordinal not in range(128)

I also get errors trying to install gunicorn. I followed this example here but nothing. My virtual environment was created using the --no-site-packagesflag. I use ubuntu 13.10 and greek environment. Maybe it's that?

Liquidize

Do you have the python-dev package installed? If this is a Debian-based distribution (e.g. Ubuntu), try apt-get install python-dev and then retry the psycopg2 installation. It is needed in order to compile Python extensions written in C or C++. – Mosasaur

It's worth noting that if you're using a non-Debian distribution, you'll need to use yum install python-devel. The reason for the difference escapes me except to sow confusion and mayhem. – Olivenite

Good question, should be upvoted – Carrnan

@RichardD How would I install python-devel for Python3 with yum? – Ranged

Answer 1

The python-dev package is required for compilation of Python extensions written in C or C++, like psycopg2. If you're running a Debian-based distribution (e.g. Ubuntu), you can install python-dev by running

sudo apt install python-dev

or

sudo apt install python3-dev

depending on your python version (for example, sudo apt install python3.8-dev).

After that, proceed to install psycopg2 in your virtualenv environment as usual.

Mosasaur

If you're using python 3 this might not (or perhaps definitely will not) work, you'll need to do apt-get install python3-dev instead. – Rowen

I had to do apt-get install python3.5-dev for python 3.5 – Sheelagh

Similarly, I had to do apt-get install python3.6-dev for python 3.6 – Greataunt

same for python3.7-dev 🤔 – Conjoin

same for python3.8-dev! – Naxos

I think there might be a pattern here... 🧐 – Mosasaur

same for python3.9-dev – Jackelynjackeroo

Answer 2

For Ubuntu 14.04, from Docker image python:3.4.3-slim this combination worked for me:

sudo apt-get update
sudo apt-get install -y build-essential
sudo apt-get install -y python3.4-dev
sudo apt-get install -y libpq-dev

pip3 install psycopg2

Note build-essential package. It was crucial in my case.

Porthole

The build essential did it for me as well. Using Ubuntu 16.04 and Python 2.7.12 – Nondisjunction

Same here! build-essential was the missing piece. – Fabrikoid

Ubuntu 16.04, Python 3.5, same for me – build-essential was missing. – Terminate

The build-essential was also the missing piece in my WSL Ubuntu 20.04 LTS. – Mailer

this answer worked for me 2 years ag. today it didn't lol – Planetstruck

I was missing libpq-dev package. Thanks for help! – Workingwoman

Answer 3

I tried all the solution but only works be

pip install psycopg2-binary

after installing lib you can use import psycopg2 in python file and then further db connection config

Hydrous

Answer 4

In my case (ubuntu 12.04 with python 2.7 and python 3.4 installed)

sudo apt-get install python3-dev 

was not enough. What did the trick was

sudo apt-get install python3.4-dev

Dachau

Great! It was my problem too. – Doan

In my case, apt install python3.8-dev saved me – Gheber

Answer 5

On Amazon Linux within a python3 venv I had to do the following:

sudo yum install python3-devel postgresql-devel
pip install psycopg2

Distinctive

Answer 6

Debian:

sudo apt-get install gcc

Lubbi

Why was this downvoted? This solved the issue for me. gcc is not pre-installed by default in ubuntu 19. – Uriniferous

this saved our lives trying to install psycogp2 on a ubuntu EC2 server. ty! – Jacklin

Answer 7

For the latest Python 3.7 I had to install python3.7-dev:

sudo apt-get install -y python3.7-dev

Conjoin

Answer 8

First of all install python-dev

sudo apt-get install python-dev

And to adecuate use try this:

This is to do with the encoding of your terminal not being set to UTF-8. Here is my terminal:

$ echo $LANG
es_UY.UTF-8
$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s = '(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89'
>>> s1 = s.decode('utf-8')
>>> print s1
(｡･ω･｡)ﾉ
>>> 

If I unset I got the same error you got:

$ unset LANG
$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s = '(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89'
>>> s1 = s.decode('utf-8')
>>> print s1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-5: ordinal not in range(128)
>>> 

Lorenlorena

The unicode error is simply visualisation-related. The actual fatal error is not being able to locate the Python.h header file. – Mosasaur

Thanks, I also had to install server dev: sudo apt-get install postgresql-server-dev-9.1 , or 9.3 if using postgres 9.3 – Monsignor

Answer 9

I'm using the Ubuntu distribution of Linux and did the following:

Install the latest version of the python3-dev library so C extensions work for python. The first command updates the package list in Ubuntu so the latest version of python3-dev is installed.

sudo apt-get update

sudo apt-get install python3-dev

Then I had to install the the missing development library for PostgreSQL: the "libpq-dev" package.

sudo apt-get install libpq-dev

After that, I was able to finish installing my dependencies.

Tharp

Answer 10

sudo apt-get install -y build-essential after trying everything with no success this helped me solve my problem.

Madeup

Answer 11

In my case, I tried to install psysopg2 version 2.7.3.2 in a virtual environment with python3.8.

The above error occurred.

As it turned out, python3.8 requires psycopg2 version 2.8 or higher.

Cooperman

Answer 12

Remove the current virtual environment and create another one. Install the dependencies first then the rest you've been using. This works

Stillwell

Please add further details to expand on your answer, such as working code or documentation citations. – Githens