Network calls fail during image build on corporate network

Question

I'm having a problem building Docker images on my corporate network. I'm just getting started with Docker, so I have the following Dockerfile for a hello-world type app:

# DOCKER-VERSION 0.3.4
FROM    centos:6.4
# Enable EPEL for Node.js
RUN     rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# Install Node.js and npm
RUN     yum install -y npm
# Bundle app source
ADD . /src
# Install app dependencies
RUN cd /src; npm install
EXPOSE  8080
CMD ["node", "/src/index.js"]

This works fine when I build it on my laptop at home, on my own wireless network. It pulls down the requisite dependencies and builds the image correctly.

However, when I'm on my corporate network at work, this same docker build fails when trying to pull down the RPM from download.fedoraproject.org, with this error message:

Step 2 : RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm ---> Running in e0c26afe9ed5 curl: (5) Couldn't resolve proxy 'some.proxy.address' error: skipping http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm - transfer failed

On my corporate network, I can access that URL just fine from my laptop. But once Docker is trying to build the container, all of a sudden it can't resolve at all. This behavior is the same for a variety of external resources (apt-get, etc.): They all can resolve just fine on my laptop on the corporate network, but Docker can't resolve them.

I don't have the network know-how to figure out what's going on here. Does anyone know why this strange behaviour would be occurring when building Docker containers?

Spellbinder

take a look at #19211063 it might also solve your issue – Rademacher

same problem occured to me. I had forgotten to run apt-get update. – Forb

These can be symptoms of "Docker does not pickup DNS information from Cisco AnyConnect" forums.docker.com/t/… – Jihad

similar issue at home with ADSL connection. At my office, it works – Agonized

Answer 1

I was able to figure out the issue. On Ubuntu, Docker sets the DNS servers for container to Google's servers at 8.8.8.x. As I understand it, this is a workaround on Ubuntu due to the fact that Ubuntu sets /etc/resolv.conf to be 127.0.0.1.

Those Google servers weren't accessible from behind our firewall, which is why we couldn't resolve any URLs.

The fix is to tell Docker which DNS servers to use. This fix depends on how you installed Docker:

Ubuntu Package

If you have the Ubuntu package installed, edit /etc/default/docker and add the following line:

DOCKER_OPTS="--dns <your_dns_server_1> --dns <your_dns_server_2>"

You can add as many DNS servers as you want to this config. Once you've edited this file you'll want to restart your Docker service:

sudo service docker restart

Binaries

If you've installed Docker via the binaries method (i.e. no package), then you set the DNS servers when you start the Docker daemon:

sudo docker -d -D --dns <your_dns_server_1> --dns <your_dns_server_2> &

Spellbinder

what about using the 'docker build' command with Dockfiles...it doesn't look like it works in those circumstances: docker --dns=209.18.47.61 build . 2>&1 | tee ./output.txt – Honey

yeah this should work with docker build. docker build doesn't have a --dns flag on the command itself, but if you set it on the daemon like this then it will apply when using docker build – Spellbinder

Thank you for this solution. At my side the problem was, that docker apparently needs IPv4 to be available and enabled. WTF? Learn: IPv4 is dead. Software being IPv4 only is seriously broken. – Moyer

I had already done this step and started getting the errors again. Restarting the service fixed it again. – Dunford

I am having this issue with docker 1.7 although I have tried to run it either with --dns option or putting the config in /etc/default/docker – Livy

In my case I needed to set the --dns option when starting the docker daemon, but using the nameservers from the network did not work. Instead forcing it to use google's nameservers worked, i.e. 8.8.8.8 and 8.8.4.4. This was docker 1.8.2 running on Centos7 trying to build a container based on Centos6. – Wilsey

@Honey this does not work for docker build see luk's answer to fix that with /etc/docker/daemon.json – Londonderry

Huh finally!! I added DNS from ubuntu dekstop network settings -> wired and there is DNS . I used it and works ;O – Interpreter

Answer 2

I advise changing the DNS settings of the Docker daemon. You can set the default options for the docker daemon by creating a daemon configuration file at /etc/docker/daemon.json. Set DNS server according to your host machine, e.g. my DNS server is 10.0.0.2:

{"dns": ["10.0.0.2", "8.8.8.8"] }

Then you need just restart docker service:

sudo service docker restart

Step-by-step explanation is available here Fix Docker's networking DNS config

Caftan

Answer 3

The following steps works for me ( for both docker build and docker run command). My linux version is Ubuntu 14.04.

• Identify DNS using following command.

  nm-tool | grep DNS

This result DNS:192.168.1.1 in my case

• Create entry in /etc/default/docker.io. My current entry looks like this

DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 --dns 192.168.1.1"

• Restart docker service

 sudo service docker.io restart 

Diaphoretic

The CentOS equivalent is the /etc/sysconfig/docker file, to which I was able to add the DOCKER_OPTS="--dns 8.8.8.8" line and solve my problem. – Towny

OpenSuSE too has the /etc/sysconfig/docker file. – Jaco

Answer 4

For any Linux distribution working with SystemD (Ubuntu 16, RHEL 7...), the path will be displayed with the following command:

$ systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2016-06-29 08:10:33 PDT; 2min 34s ago
     Docs: https://docs.docker.com
 Main PID: 1169 (dockerd)
    Tasks: 19
   Memory: 85.0M
      CPU: 1.779s
   CGroup: /system.slice/docker.service
           ├─1169 /usr/bin/dockerd --dns 172.18.20.11 --dns 172.20.100.15 --dns 8.8.8.8 --dns 8.8.4.4 -H fd://
           └─1232 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --met

The path would be /lib/systemd/system/docker.service. Add the DOCKER_OPTS values, which can have any of the --dns, in the line where the daemon is started.

cat /lib/systemd/system/docker.service | grep dns 
ExecStart=/usr/bin/dockerd --dns 172.18.20.11 --dns 172.20.100.15 --dns 8.8.8.8 --dns 8.8.4.4  -H fd://

Carolyncarolyne

Thank you! After editing docker.service I had to stop the docker service with sudo service docker stop and then systemctl daemon-reload and finally sudo service docker start – Enjambement

You are better off adding a unit file in /etc/systemd/system/docker.service.d with the modified configuration (which systemd helpfully merges/overrides the system config) rather than changing the system version. The latter will be lost on upgrades. – Manipur

Answer 5

Docker (at least >=1.13, probably earlier) on Mac and Windows allow you configure the DNS in Preferences -> Daemon -> Advanced:

The following config sets two corporate DNS servers (use your own values here) with fallback to Google public DNS servers.

Jihad

Thanks Jason--if this doesn't work, see my later answer for the newer way to set DNS at the UI. – Bluing

Answer 6

Specify your DNS to the Docker daemon.

First of all get your DNS address

$ nmcli dev show | grep 'IP4.DNS'
IP4.DNS[1]:                             10.0.0.2

Test if the problem is really with the DNS by launching a docker container forcing this new DNS

$ docker run --dns 10.0.0.2 <image_name> <command_name>

If this solves the problem, you can apply this fix for all the docker daemons in the following way

Edit or create a file /etc/docker/daemon.json

Add the following line to this file

{
"dns": ["10.0.0.2", "8.8.8.8"]
}

Restart docker

$ sudo service docker restart

A very nice guide for doing ALL this process can be found here.

https://development.robinwinslow.uk/2016/06/23/fix-docker-networking-dns/

Catfall

This is a borderline link-only answer. You should expand your answer to include as much information here, and use the link only for reference. – Milesmilesian

if you installed by snap daemon.json will be at /var/snap/docker/<id>/config/daemon.json – Maleki

Answer 7

Solution without restarting Docker service

It is possible to modify the DNS settings for a single Docker image without affecting other docker build calls (and without restarting the Docker service) by overriding the resolv.conf at build time:

FROM ubuntu:18.04

RUN echo "nameserver 123.123.123.123" > /etc/resolv.conf && apt update

Replace the IP 123.123.123.123 with the one which is used within your corporate network (use nmcli dev show | grep 'IP4.DNS' to get the currently used DNS server).

Downsides:

• This does not affect any other line from the Dockerfile. Hence, you have to prefix every line with the fix, if it depends on DNS resolution

Mecham

No longer works, as /etc/resolv.conf was made read-only by Docker – Tripp

Thanks for your comment, do you know in which version this change did happen? – Mecham

Answer 8

On my Ubuntu 16.04 machine, sometimes, Google's DNS do not work for building Docker images.

cat /etc/docker/daemon.json
{"dns": [""8.8.8.8"] }

I have to manually find out my Service Providers DNS using the following command

nmcli device show <interfacename> | grep IP4.DNS

125.22.47.102

and add it to my daemon.json as show below

cat /etc/docker/daemon.json 

{"dns": ["125.22.47.102","8.8.8.8"] }

 restart docker

sudo service docker restart

(PS nm-tool is deprecated from Ubuntu 15.04)

Brae

Answer 9

Updated info September 2021

Inspired by Jason's answer; setting DNS server in the JSON didn't work for me in the current version, but there's now another place to set it:

When you turn on the toggle, the 8.8.8.8 is already there, so I just left it and it works well enough for me in my dev environment. I didn't research it but if wanted, there may be a way to add a list, perhaps separated by commas/semicolons/spaces etc.

Bluing

My Docker Desktop on Windows (v4.12.0) doesn't have that DNS option. It only has the subnet. – Inkerman