Publishing docker swarm mode port only to localhost

Question

I've created docker swarm with a website inside swarm, publishing port 8080 outside. I want to consume that port using Nginx running outside swarm on port 80, which will perform server name resolution and host static files.

Problem is, swarm automatically publishes port 8080 to internet using iptables, and I don't know if is it possible to allow only local nginx instance to use it? Because currently users can access site on both 80 and 8080 ports, and second one is broken (without images).

Tried playing with ufw, but it's not working. Also manually changing iptables would be a nightmare, as I would have to do it on every swarm node after every update. Any solutions?

EDIT: I can't use same network for swarm and nginx outside swarm, because overlay network is incompatible with normal, single-host containers. Theoretically I could put nginx to the swarm, but I prefer to keep it separate, on the same host that contains static files.

Discretionary

Answer 1

No, right now you are not able to bind a published port to an IP (even not to 127.0.0.1) or an interface (like the loopback interface lo). But there are two issues dealing with this problem:

• github.com - moby/moby - Assigning service published ports to IP
• github.com - moby/moby - docker swarm mode: ports on 127.0.0.1 are exposed to 0.0.0.0

So you could subscribe to them and/or participate in the discussion.

Further reading:

• How to bind the published port to specific eth[x] in docker swarm mode

Yowl

Answer 2

Yes, if the containers are in the same network you don't need to publish ports for containers to access each other.

In your case you can publish port 80 from the nginx container and not publish any ports from the website container. Nginx can still reach the website container on port 8080 as long as both containers are in the same Docker network.

Laniary

The problem is that they can't be in the same network. Swarm network type is overlay, and if you try to run normal docker container using "docker run --net swarm_net" you receive error that networks are not compatible. I know that it's working using normal docker containers, but swarm changes everything. Also "docker service create" --publish command doesn't support interfaces, so it's impossible to do --publish 127.0.0.1:8080:8080 – Discretionary

@Valvar, @Elton-Stoneman is correct. You just need to run each container as a service and you can use the --publish option if yu need to but you wouldn't specify the IP address with a service. With a service on a SWARM cluster when you use --publish PORT:PORT it sets it up so that that port is mapped on EVERY node in the cluster since you don't know (unless you force it) which machine in the cluster the service will reside on. So, eliminate the IP address in your execution of the docker service create command and only publish the port mapping: --publish 8080:8080 – Solicitude

@Valar, for example run a test nginx service with the following: docker service create --publish 80:80 --replicas 1 --name test-nginx nginx Wait until docker service ls shows it is running and then try 127.0.0.1 from each machine in the cluster or use their IP address and you should get the Welcome to nginx page from each machine EVEN THOUGH you only have one instance running. – Solicitude

Thanks for suggestions, all you say it's true, and I understand it. But problem with that kind of publishing is that I don't have any control who can visit published port, under what circumstances etc. That's why I wanted to put nginx between outer world and published port. What I can see it's quite impossible, so only solution is to put nginx inside swarm and publish only nginx port. – Discretionary

I'd prefer to put Nginx in the Swarm anyway - otherwise your public facing entry point is on a single host with no failover, while your upstreams have HA. But, if you really want to do this, it becomes an infrastrcture problem - expose the app port on the swarm and use firewall rules so only the Nginx box can access the swarm port. – Laniary

Answer 3

"Temp" solution that I am using is leaning on alpine/socat image.

Idea:

• use additional lightweight container that is running outside of swarm and use some port forwarding tool to (e.g. socat is used here)
• add that container to the same network of the swarm service we want to expose only to localhost
• expose this helper container at localhost:HOST_PORT:INTERNAL_PORT
• use socat of this container to forward trafic to swarm's machine

Command:

docker run --name socat-elasticsearch -p 127.0.0.1:9200:9200 --network elasticsearch --rm -it alpine/socat tcp-listen:9200,reuseaddr,fork tcp:elasticsearch:9200

Flag -it can be removed once it can be confirmed all is working fine for you. Also add -d to run it daemonized.

Daemon command:

docker run --name socat-elasticsearch -d -p 127.0.0.1:9200:9200 --network elasticsearch --rm alpine/socat tcp-listen:9200,reuseaddr,fork tcp:elasticsearch:9200

My use case:

Sometimes I need to access ES directly, so this approach is just fine for me. Would like to see some docker's native solution, though.

P.S. Auto-restart feature of docker could be used if this needs to be up and running after host machine restart.

See restart policy docs here:

https://docs.docker.com/engine/reference/commandline/run/#restart-policies---restart

Toilsome

TCP port forwarding via socat between one standalone container and a swarm service works as a charm for me. Only a docker overlay/attachable network for those two is needed. Sort of a "workaround" for now until github.com/moby/moby/issues/32299 someday... – Roter

Answer 4

I solved this problem without socat, which is an extra hop between userspace and kernel

1. Expose the port

    ports:
      - target: 26379 # note: public (internet facing) access is blocked via iptables, see below.
        published: 26379
        mode: host

2. Use iptables to block the port from the internet:

# allow only traffic from loopback (localhost).
sudo iptables -t mangle -I PREROUTING ! -i lo -p tcp --dport 26379 -j DROP

Explanation: Docker has a PREROUTING rule on the nat table (see here for Linux packet flow). We must insert our rule before Docker's rule, so mangle is our only option.

Garfish