docker-compose: route traffic through vpn except for connections to other services

Question

Consider this docker-compose configuration:

# docker-compose.yml
version: "3.7"

services:
  app:
    build: ./app
    depends_on:
      - db
      - vpn
    ports:
      - "3001:3000"
  db:
    image: postgres
  vpn:
    build: ./vpn
    cap_add:
      - NET_ADMIN

Description

• The app is accessed from the docker host via http://localhost:3001.
• The app needs to connect to a postgres db, which is the second container.
• Also, the app needs to connect to an api, which is only available though a vpn. This is why a third container, vpn, establishes the required vpn connection.

Goal

The app container should should be able to reach the other services within this docker-compose environment, i.e. the db, and route the rest of its traffic through the vpn container, such that it can access the api behind the vpn tunnel.

What I've tried

• I have tried to set the network_mode of the app:

  services:
    app:
      network_mode: "service:vpn"
  

  This routes all traffic of the app container through the vpn. With this, I can reach the api behind the vpn tunnel from the app container. But this is not compatible with ports: - "3001:3000". Also, the db container cannot be reached from the app anymore: ping: bad address 'db'.

• I also have tried to link the db container from the vpn container, hoping that this would make the db service available to the app.

  services:
    app:
      network_mode: "service:vpn"
    vpn:
      links:
        - db
  

  But still db cannot be found by app.

• If I link the db container from the vpn container but do not establish the vpn connection within the vpn container, the db container can be reached from the app.

• And I've experimented with adding 127.0.0.1 db to the /etc/hosts of the app container, vaguely hoping that I could reach the db port directly. But this also does not work.

Does anyone have a clue how to achieve this?

Renny

Answer 1

I've finally found a solution, but it required three steps:

Step 1: network_mode: service

In order to route all traffic of the app container through the vpn, set network_mode on the app container:

services:
  app:
    network_mode: "service:vpn"

Step 2: DNS servers

In order to resolve both the host names behind the vpn tunnel as well as the local docker services, the vpn container needs to talk to both DNS servers: the DNS server behind the tunnel as well as the docker-compose DNS server.

The docker-compose DNS server is always 127.0.0.11 as far as I understood.

To find out the remote DNS server behind the tunnel, establish the tunnel and then run cat /etc/resolv.conf. This will list the DNS server behind the tunnel in the line commented with "by strongSwan".

In the startup script of the vpn container, add both DNS servers to the resolv.conf of the vpn container:

# vpn-container startup script:
echo "nameserver <remote dns server ip>" >> /etc/resolv.conf
echo "nameserver 127.0.0.11" >> /etc/resolv.conf

To test this, log into the vpn container and try to ping a remote ip and the db container:

docker-compose run vpn /bin/bash
ping db  # should work
ping <some-ip-behind-the-vpn-tunnel>  # should also work

Step 3: Expose the port

With network_mode: "service:vpn" on the app container, the app container cannot expose its ports to the host anymore as far as I understood. Instead, the app container and the vpn container appear as the same machine to the docker host, now. Therefore, one can expose the desired ports on the vpn container instead.

services:
  vpn:
    ports:
      - "3001:3000"

The app (!) is then reachable through http://localhost:3001 on the docker host.

Bringing all together: Final docker-compose.yml

# docker-compose.yml
version: "3.7"

services:
  app:
    build: ./app
    depends_on:
      - db
      - vpn
    network_mode: "service:vpn"
  db:
    image: postgres
  vpn:
    build: ./vpn
    cap_add:
      - NET_ADMIN
    ports:
      - "3001:3000"
    command: >
      bash -c "echo 'nameserver <remote dns server ip>' >> /etc/resolv.conf
      && echo 'nameserver 127.0.0.11' >> /etc/resolv.conf
      && ..."

Renny

This works if you try to access the app from the host machine, what if you need to access it from another local machine ? it is not accessible then :-/ – Randolph

What about app accessing dB? Does that work? – Beadruby

Answer 2

in my case it works.

# docker-compose.yml
version: "3.7"

services:
  app:
    network_mode: service:vpn
    depends_on:
      - db
      - vpn

  db:
    image: postgres
    network_mode: bridge

  vpn:
    network_mode: bridge
    cap_add:
      - NET_ADMIN
    ports:
      - "3001:3000"
    links:
      - db

but in this way, db don't communicate network with vpn. i wanna make whole network communicate with vpn. is there any way to db communicate with vpn? or postgres don't communicate with 'external network'? so i don't need to modify this docker-compose?(sorry for my ignorance but i don't know what postgres do exactly.) sorry for writing my problem here but i think we got similar case so i write here. sorry.

Melbamelborn

Welcome to stackoverflow! Stackoverflow works best if you post your question as a new question. From your new question, you can, of course, easily refer to this similar case by pasting a link. Good luck! – Renny

alright, but i gave you other method and i asked to you about db – Melbamelborn

No an answer. Read How to Ask – Eberhart

This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review – Greaser