Failing to install psycopg2-binary on new docker container

Question

I have encountered a problem while trying to run my django project on a new Docker container. It is my first time using Docker and I can't seem to find a good way to run a django project on it. Having tried multiple tutorials, I always get the error about psycopg2 not being installed.

requirements.txt:

-i https://pypi.org/simple
asgiref==3.2.7
django-cors-headers==3.3.0
django==3.0.7
djangorestframework==3.11.0
gunicorn==20.0.4
psycopg2-binary==2.8.5
pytz==2020.1
sqlparse==0.3.1

Dockerfile:

# pull official base image
FROM python:3.8.3-alpine

# set work directory
WORKDIR /usr/src/app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install dependencies
RUN pip install --upgrade pip

COPY ./requirements.txt .
RUN pip install -r requirements.txt

# copy project
COPY . .

# set project environment variables
# grab these via Python's os.environ
# these are 100% optional here

ENV PORT=8000
ENV SECRET_KEY_TWITTER = "***"

While running docker-compose build, I get the following error:

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source. Please add the directory

containing pg_config to the $PATH or specify the full executable path with the

option:

python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in 'setup.cfg'.

If you prefer to avoid building psycopg2 from source, please install the PyPI

'psycopg2-binary' package instead.

I will gladly answer any questions that might lead to the solution. Also, maybe someone can recommend me a good tutorial on dockerizing django apps?

Coccus

I think you need to have the PostgreSQL client installed before pip installing the package. EDIT: I actually had the same issue when using Py3.8 and i reverted back to Py3.7 and installed it without any issues. – Sedimentology

Answer 1

On Alpine Linux, you will need to compile all packages, even if a pre-compiled binary wheel is available on PyPI. On standard Linux-based images, you won't (https://pythonspeed.com/articles/alpine-docker-python/ - there are also other articles I've written there that might be helpful, e.g. on security).

So change your base image to python:3.8.3-slim-buster or python:3.8-slim-buster and it should work.

Selfabsorption

It actually solved my problem, even if partially, thanks! Unfortunately now I get the error when running the container, django.db.utils.OperationalError: could not translate host name "db" to address: Name or service not known . Gonna try to solve that one now. – Coccus

If you're using Compose, make sure your database service name is db. – Selfabsorption

I am actually, I created a post about it here: #62730417 – Coccus

Unfortunately I get the same error when using python:3.8-slim-buster. – Petersen

Now try switching to installing psycopg-binary. – Selfabsorption

Answer 2

I made it work. This is the code:

FROM python:3.8.3-slim #Image python:3.9.5-slim also works # Image python:3.9.5-slim-buster also works

RUN apt-get update \
    && apt-get -y install libpq-dev gcc \
    && pip install psycopg2
    

Boadicea

This won't work for psycopg2. But if you change to images 3.9.5-slim or 3.9.5-slim-buster, it will. – Antimalarial

Sergio what are you mean it won't work? It is working now, my django project using this settings. – Layout

I did not make it work psycog2 with version 3.8.3. But I managed to do it with version 3.9.5. – Antimalarial

Doing it exactly this way worked for me. What did not work was installing it through the requirements.txt. – Ballonet

Answer 3

This scripts work on MacBook Air M1

Dockerfile

FROM ubuntu:20.04
RUN apt-get update && apt-get -y install libpq-dev gcc && pip install psycopg2
COPY requirements.txt /cs_account/
RUN pip3 install -r requirements.txt

requirements.txt

psycopg2-binary~=2.8.6

Updated answer from the answer of Zoltán Buzás

Underling

Have you tried running the GOARCH=amd64 docker build .... ? Did you still need to install gcc if you specified GOARCH? I don't have an M1 mac, but I am trying to help another engineer with an M1 Mac. Thanks! Note: You'll need to have rosetta installed docs.docker.com/desktop/mac/apple-silicon – Hipolitohipp

Yes, I still had to install libpq-dev and gcc. – Underling

I like you confidence when you say this Dockerfile works on MacBook Air M1 – Polygnotus

Answer 4

This worked for me. Try slim-buster image.

In your Dockerfile

FROM python:3.8.7-slim-buster

and in your requirements.txt file

psycopg2-binary~= <<version_number>>

Seashore

Answer 5

I added this to the top answer because I was getting other errors like below:

gcc: error trying to exec 'cc1plus': execvp: No such file or directory
error: command 'gcc' failed with exit status 1

and

src/pyodbc.h:56:10: fatal error: sql.h: No such file or directory
#include <sql.h>

This is what I did to fix this, so I am not sure how others were getting that to work, however maybe it was some of the other things I was doing?

My solution that I found from other posts when googling those two errors:

FROM python:3.8.3-slim 

RUN apt-get update \
&& apt-get -y install g++ libpq-dev gcc unixodbc unixodbc-dev

Jaymie

Fixed it for me, g++ seems to make the difference. – Jobina

Answer 6

I've made a custom image with

FROM python:alpine
ADD requirements.txt /
RUN apk update --no-cache \
&& apk add build-base postgresql-dev libpq --no-cache --virtual .build-deps \
&& pip install --no-cache-dir --upgrade pip \
&& pip install --no-cache-dir -r /requirements.txt \
&& apk del .build-deps
RUN apk add postgresql-libs libpq --no-cache

and requirements.txt

django
djangorestframework
psycopg2-binary

Aspiration

Answer 7

To others that may have this problem - (I know OP doesn't have this issue) but it could be that you are attempting to install psycopg2 instead of psycopg2-binary which was my problem.

Heiress

Worth to note that there is a difference between psycopg2 and psycopg2-binary: Link - For production use you are advised to use the source distribution. – Jobina

Answer 8

I added the psycopg2-binary~=2.9.9 to the requirements.txt and it worked for me. None of the above suggestions had fixed the issue.

Skylight