PostgreSQL docker: "could not bind IPv6 socket: Cannot assign requested address"

Question

EDIT 2: After a long time, solved! See answer below.

EDIT: I am sorry to say that the the problems went away "on their own" between yesterday and today, without me having done anything. Great non-deterministic lesson to learn here... Bonus fun: the "could not bind IPv6 socket" error still appears in the error logs, so this was probably not even the problem in the first place.

I have a problem with a previously functioning docker PGSQL image. Until an uninspired rebuild yesterday ( :-D ), I've used this build successfully for the last 5+ months.

My system:

• Ubuntu 17.04 64b
• PGSQL 9.6.4
• Docker version 17.11.0-ce, build 1caf76c
• I am mapping host port 5433 to container port 5432

The problem (snippet from the PGSQL logs):

...

LOG: could not bind IPv6 socket: Cannot assign requested address

HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.

...

This is what ss is telling me (when the container is running, obviously):

    $ docker exec -it db ss -atune
    Netid  State      Recv-Q Send-Q     Local Address:Port       Peer Address:Port 
    udp    ESTAB      0      0              127.0.0.1:45876         127.0.0.1:45876  uid:999 ino:147509 sk:00000001 <->
    tcp    LISTEN     0      128            127.0.0.1:5432                  *:*      uid:999 ino:147500 sk:00000002 <->

I've cleaned all docker containers / images, I've reinstalled docker, nothing helped. Who can possibly be using the 5432 port in the container? For that matter, am I reading this correctly, that PGSQL is complaining about the 5432 port being already used in the docker container?

Even if you have no solution, a basic idea of how to proceed with debugging this would be a great help.

EDIT:

postgres.docker file

FROM postgres:9.6.4

ADD bin/postgres-setup.sh /docker-entrypoint-initdb.d/postgres-setup.sh

RUN chmod 755 /docker-entrypoint-initdb.d/postgres-setup.sh && \
    apt-get update && \
    apt-get install -y --no-install-recommends postgresql-plpython3-9.6 python3-pip postgresql-9.6-pldebugger && \
    pip3 install pyexcel pyexcel-xls pyexcel-xlsx pyexcel-xlsxw

Pettway

What command are you using to start the container? Can you provide the Docker file if you have a custom image. – Rondel

@Rondel I've added the docker file; regarding the command, it's complicated (we use a custom build / deploy tool, would have to dig a bit to see the exact command line equivalent). – Pettway

Answer 1

After a long time, we finally figured out what was the problem -- adding explanation here, in case it helps others.

Since pgsql listens only to localhost by default, when it is running in the docker container, where we have the port mapping configuration, the external API was not able to connect to the pgsql server.

The solution is to allow pgsql to listen to all IP addresses:

1. Connect to db container shell: $ docker exec -ti db bash

2. Change the configuration file /var/lib/postgresql/data/postgresql.conf to allow pgsql to listen to all IPs: listen_addresses = '*'

Pettway

You may need to also add an entry to the IPV4 local connections hosts allow rule in your pg_hba.conf file for your docker containers. I forced my docker network to only assign ip addresses from the 10.5.0.0/16 range and whitelisted the range in my pg_hba.conf file. This allowed my application docker containers to connect to my postgres docker container. – Gravante

My default installation of postgres seems to have the listen_addresses = '*' by default. – Ravenna

don't forget to uncomment the line # listen_addresses = '*'... took me hours to see that the line is still commented out with # – Commines

Thank you for documenting this. Common scenario when rolling with a docker-compose locally but attempting to deal with docker in CI environments where network port mapping isn't always supported – Embitter

Answer 2

Certain edits have to be made to the postgres.conf and pg_hba.conf files in order for Postgres inside the container to listen to connections from the host:

See the Gotchas section at the cityseer/postgis repo.

• Check that your postgresql.conf file has the listen_addresses item uncommented and set to listen to all ports, i.e. listen_addresses = '*';

• Check that your pg_hba.conf file allows the docker container's Postgres to provide local and host access per the following two lines:

  local all all trust

  host all all 0.0.0.0/0 trust

Newbill

Answer 3

Do not configure it as listen_addresses = '*'. It is a bad practice to open access to everyone, it makes brute-force attacks so easy

Dorrie