How to solve libpq.so.5 Import error when running postgresql on alpine in docker

Question

Hi I am trying to run postgresql in alpine in docker with SQLAlchemy and flask but anytime I run my app I get this error ImportError: Error loading shared library libpq.so.5: No such file or directory (needed by /usr/local/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-x86_64-linux-gnu.so) I have combed stack overflow for a solution but everyone one them seems to tell me to install psycopg2 which I have done already

 FROM python:3.8.1-alpine3.10 AS build

# ENV PYTHONUNBUFFERED 1

WORKDIR /usr/src/app/restful
COPY requirements.txt /usr/src/app/restful
RUN python -m pip install --upgrade pip
RUN apk update && apk upgrade
RUN apk add libffi-dev
#installing dependencies
# dependencies for libpq postgresql-libs postgresql-dev *remove if not
RUN apk add --no-cache --virtual .build-deps gcc libc-dev py-cryptography libpq postgresql-libs postgresql-dev python3-dev musl-dev make openssl-dev gcc
RUN apk update && apk add --no-cache ca-certificates \
    && update-ca-certificates 2>/dev/null || true
RUN apk add build-base python-dev py-pip jpeg-dev zlib-dev
ENV LIBRARY_PATH=/lib:/usr/lib  
WORKDIR /usr/src/app/restful
COPY requirements.txt  /usr/src/app/restful
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
RUN python -m pip install --upgrade pip
RUN pip install -r requirements.txt
RUN pip install gevent

FROM python:3.8.1-alpine3.10
COPY --from=build /usr/local/lib/python3.8/site-packages/ /usr/local/lib/python3.8/site-packages/
RUN mkdir -p /usr/src/app/restful
RUN set -ex && apk --no-cache add sudo
RUN apk --no-cache --update add libffi libressl
RUN apk update && apk add --no-cache supervisor
RUN pip install psycopg2-binary 

Coldiron

Answer 1

You seem to be using staged builds in your Dockerfile, and your apk add postgresql-libs is in the first stage. That second FROM stage is building off of a plain python:3.8.1-alpine3.10, not on all the stuff you had done above it, and you're only copying /usr/local/lib/python3.8/site-packages/ over to the second stage. You'll either need to find the full paths of all those dependencies (which could be prohibitively tedious), or just install the dependencies on the second stage as well. Therefore, you need to either build your second stage with FROM build (to include all the other apk deps), or you need to add RUN apk add postgresql-libs gcc libc-dev in the second stage.

So depending on what you're going for, you probably need this:

<...>
RUN pip install -r requirements.txt
RUN pip install gevent

FROM build
RUN mkdir -p /usr/src/app/restful
<...>

or this:

<...>
RUN set -ex && apk --no-cache add sudo
RUN apk --no-cache --update add libffi libressl postgresql-libs gcc libc-dev
RUN apk update && apk add --no-cache supervisor
<...>

Muslin

It worked for me just adding RUN apk add --no-cache libpq after the second FROM without reinstalling postgresql-libs gcc libc-dev. I'm using arm32v7/python:3.8-alpine3.12 – Bosom

Being in a similar situation, I'm confused on what's the point of multistage build in those situations (having C libs dependencies). Installing python3-dev, build-essential and libpq-dev adds ~350MB to the image, can't seem to find a way to get away from it. In my case (using slim buster) doesn't really work to just add libpq. – Shinny

Answer 2

RUN apt-get update && apt-get install libpq5 -y after calling your build did the trick for me.

Antipodal

Alpine generally doesn't have apt-get – Rachealrachel

Answer 3

This was fixed for me by changing my psycopg2 import to psycopg2-binary.

Erotogenic

Please note that psycopg2-binary can be handy but it is not recommended for production use as per the installation guide. It reads: "The binary package is a practical choice for development and testing but in production it is advised to use the package built from sources." – Versus