Dynamic DNS Resolution with HAProxy and Docker

Question

I'm trying to setup HAProxy inside a Docker host.

Using HAProxy 1.7 and Docker 1.12

My haproxy.cfg looks like:

# Simple configuration for an HTTP proxy listening on port 81 on all
# interfaces and forwarding requests to a single backend "servers" with a
# single server "server1" listening on 127.0.0.1:8000
global
    daemon
    maxconn 256

resolvers docker
    # nameserver dnsmasq 127.0.0.1:53
    nameserver dns 127.0.0.1:53

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms
    default-server init-addr none

frontend http-in
    bind *:80
    default_backend www_somedomain1_com

    # Define hosts
    acl host_www_somedomain1_com hdr(host) -i somedomain1.com
    acl host_www_somedomain1_com hdr(host) -i www.somedomain1.com
    acl host_www_somedomain2_com hdr(host) -i www.somedomain2.com

    ## figure out which one to use
    use_backend www_somedomain1_com if host_www_somedomain1_com
    use_backend www_somedomain2_com if host_www_somedomain2_com

backend www_somedomain1_com
    # Utilizing the Docker DNS to resolve below host
    # server server1 www-somedomain1-com maxconn 32 check port 80
    server server1 www-somedomain1-com resolvers docker check maxconn 32

backend www_somedomain2_com
    # Utilizing the Docker DNS to resolve below host
    # server server1 www-somedomain2-com maxconn 32 check resolvers docker resolve-prefer ipv4
    server server1 www-somedomain2-com maxconn 32 check port 80

I want to use Docker's embedded DNS system - which, in my understanding, is only enabled when using a user defined network.

So I create a network (using the default bridge driver)

docker network create mynetwork

When I run my two named docker containers, (my-haproxy and www-somedomain1-com) I add them to that network with the --net flag.

Docker run commands:

docker run --name myhaproxy --net mynetwork -p 80:80 -d haproxy
docker run --name www-somedomain1-com --net mynetwork -d nginx

I know the Docker dns is functional because I can resolve from one container to the other when I hop on them in a bash shell. I can't get the right combo/config in HAProxy to enable the dynamic DNS resolution.

HAProxy stats page always shows the downstream backends as brown/resolution issue....

Some things that have helped: - the "default-server init-addr none" helps pass the haproxy config check on startup.

Any guidance is greatly appreciated!

Gruel

Can you post your docker run commands? Any reason you aren't using docker-compose for this environment? – Ferrotype

Could you run a dig/nslookup from within the container? – Nicolanicolai

You could also issue a "show stat resolvers" on the socket to get a dump of DNS resolution statistics, we may see what type of errros are reported – Nicolanicolai

@Ferrotype I've updated my post to contain the run commands I am using. I am not using docker-compose because I didn't know about it! (But looking at it now, it appears just what I'd want and I will matriculate to it in the future). However, while I can see the benefits and conveniences offered by docker-compose I still think my root issue lives in the HAProxy config (right?)... – Gruel

@Nicolanicolai I can resolve "www-somedomain1-com" from within the haproxy container (using ping, as neither dig nor nslookup are available in this container). – Gruel

@Nicolanicolai I'm interested to try the "show stat resolvers" - I'm assuming this means I enable the HAProxy stats socket first, etc... Will try later today. – Gruel

Pretty sure you will need to create link between containers. docker run --name myhaproxy --net mynetwork -p 80:80 -d haproxy --link www-somedomain1-com. Also, check below for my docker-compose file using nginx and haproxy, although I am using them in the reverse order that you ar. – Ferrotype

Answer 1

I think your issue is that you are using 127.0.0.1:53 for your resolver dns, when it needs to be 127.0.0.11:53 for the docker bridge network.

Here is my haproxy setup for dev docker stuff:

global
    quiet

defaults
    log global
    mode http
    option forwardfor
    timeout connect 60s
    timeout client 60s
    timeout server 60s
    default-server init-addr none

resolvers docker_resolver
    nameserver dns 127.0.0.11:53

frontend https-proxy
    bind 0.0.0.0:80
    bind 0.0.0.0:443 ssl crt /usr/local/etc/haproxy/dev_server.pem
    redirect scheme https if !{ ssl_fc }

    acl is_api_server hdr(host) -i mywebsite

    use_backend api_server if is_api_server

backend api_server
    server haproxyapi api-server-dev:80 check inter 10s resolvers docker_resolver resolve-prefer ipv4

Messieurs

Is it necessary to add an explicit dns resolver entry? Won't the /etc/resolv.conf entry inserted by docker be sufficient for HAProxy to resolve to an ip? – Suanne

This is indeed puzzling, but I've just verified that using a resolvers section seems to be required if you want haproxy to periodically check the DNS past the startup phase. – Bilicki

This solution also worked for Docker Swarm using an overlay network. It would also be worth checking out haproxy.com/blog/…. – Defect

@Messieurs Thanks a lot, that's exactly what I was seeking for. Haproxy Powaa. – Waterage