Error: pg_config executable not found i try to pip3 psycopg2 with python3.7

Question

im using aws linux , which the default python is 2.7 I yum installed 3.7 and changed the alias python=python3 now when I do python -V Im getting: Python 3.7.6. i run pip3 install psycopg2

and getting :

pip3 install psycopg2
Defaulting to user installation because normal site-packages is not writeable
Collecting psycopg2
  Using cached psycopg2-2.8.5.tar.gz (380 kB)
    ERROR: Command errored out with exit status 1:
     command: /bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-b3bpay2i/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-b3bpay2i/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-3c18jwyz
         cwd: /tmp/pip-install-b3bpay2i/psycopg2/
    Complete output (23 lines):
    running egg_info
    creating /tmp/pip-pip-egg-info-3c18jwyz/psycopg2.egg-info
    writing /tmp/pip-pip-egg-info-3c18jwyz/psycopg2.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-pip-egg-info-3c18jwyz/psycopg2.egg-info/dependency_links.txt
    writing top-level names to /tmp/pip-pip-egg-info-3c18jwyz/psycopg2.egg-info/top_level.txt
    writing manifest file '/tmp/pip-pip-egg-info-3c18jwyz/psycopg2.egg-info/SOURCES.txt'

    Error: pg_config executable not found.

    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.

    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.

    For further information please check the 'doc/src/install.rst' file (also at
    <https://www.psycopg.org/docs/install.html>).

    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

i try to pip3 the binary with : pip3 install psycopg2-binary and did : pip3 -list and the result is :

Package         Version Location                                          Installer
--------------- ------- ------------------------------------------------- ---------
pip             20.1    /usr/local/lib/python3.7/site-packages            pip
psycopg2-binary 2.8.5   /home/ec2-user/.local/lib/python3.7/site-packages pip
setuptools      38.4.0  /usr/lib/python3.7/site-packages                  pip
wheel           0.34.2  /home/ec2-user/.local/lib/python3.7/site-packages pip

and when i check if file do exist :

ls -l /home/ec2-user/.local/lib/python3.7/site-packages
total 4
drwxrwxr-x 3 ec2-user ec2-user  302 Apr 30 15:06 psycopg2
drwxrwxr-x 2 ec2-user ec2-user  102 Apr 30 15:06 psycopg2_binary-2.8.5.dist-info
drwxrwxr-x 2 ec2-user ec2-user 4096 Apr 30 15:06 psycopg2_binary.libs
drwxrwxr-x 4 ec2-user ec2-user  233 Apr 30 16:03 wheel
drwxrwxr-x 2 ec2-user ec2-user  130 Apr 30 16:03 wheel-0.34.2.dist-info

also installed postgresql10 client using

sudo amazon-linux-extras install postgresql10

why i still getting the error ?

Lipetsk

"pg_config is required to build psycopg2 from source" is why you get the error. the -binary package is an alternative, not an additon, you shouldn't have both installed – Discompose

what is pg_config and how do i install it ? do i need also gcc installed ? – Lipetsk

The binary packages come with their own versions of a few C libraries, among which libpq and libssl, which will be used regardless of other libraries available on the client: upgrading the system libraries will not upgrade the libraries used by psycopg2. Build psycopg2 from source if you want to maintain binary upgradeability. – Organotherapy

And yes, you will need gcc installed – Organotherapy

Answer 1

From the docs:

The psycopg2-binary package is meant for beginners to start playing with Python and PostgreSQL without the need to meet the build requirements.

If you are the maintainer of a published package depending on psycopg2 you shouldn’t use psycopg2-binary as a module dependency. For production use you are advised to use the source distribution..

Since this package is needed in order to connect to Postgre database, you will need libpq. psycopg2 library is built as a wrapper around libpq. It requires some PostgreSQL binaries and headers required for building 3rd-party applications for PostgreSQL. Run:

$ sudo apt install python3-dev libpq-dev

The main goal of the command above is to provide all requirements for building psycopg2. To install the latest package in your system, use pip:

$ pip3 install psycopg2

If you cannot find the pg_config, check your PATH. Chances are it is not in your PATH.

On alpine, the library containing pg_config is postgresql-dev. To install, run:

apk update && \
 apk add postgresql-dev gcc python3-dev musl-dev

# Then you can install pyscopg2 say from your requirements.txt file or manually

Organotherapy

Why is it not advised to use the binary package for production use? – Turaco

@GunarGessner a bit late, but according to the psycopg-vs-psycopg-binary docs there are several reasons, e.g. upgradeability and potential libssl conflicts. – Rathenau