Docker container can only access internet with --net=host

Question

Just installed docker 1.10.1 today using their installation guide. However, none of my containers can access the internet unless I used --net=host in the docker run command. I have tried various workarounds from these posts:

1. http://odino.org/cannot-connect-to-the-internet-from-your-docker-containers/
2. My docker container has no internet
3. I can't get Docker containers to access the internet?
4. Docker container cannot access internet

Nothing has worked so far save for adding --net=host to the run command, but I can't build images from a Dockerfile because I can't use --net=host with the build command.

I ran docker network inspect bridge to checkout the settings for the docker network bridge and noticed that it uses (almost) the same subnet and gateway as my work VPN. Could that be causing an issue? That could also explain why when I connect to my work VPN some of the sites do not load.

This is the result from docker network inspect bridge:

[
    {
        "Name": "bridge",
        "Id": "6d603ebd1c437d0d1f02be8406cf362f7f36d33168e42b9883891bae99834fa9",
        "Scope": "local",
        "Driver": "bridge",
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        }
    }
]

This is ifconfig:

docker0   Link encap:Ethernet  HWaddr 02:42:9a:29:4a:c2  
          inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:9aff:fe29:4ac2/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:55 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2080 (2.0 KB)  TX bytes:8498 (8.4 KB)

enx00e09f0004bd Link encap:Ethernet  HWaddr 00:e0:9f:00:04:bd  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:70948 errors:0 dropped:1 overruns:0 frame:0
          TX packets:14839 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:14270948 (14.2 MB)  TX bytes:3460691 (3.4 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:3407 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3407 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:326405 (326.4 KB)  TX bytes:326405 (326.4 KB)

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:172.17.62.55  P-t-P:172.17.62.55  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1406  Metric:1
          RX packets:18 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:1773 (1.7 KB)  TX bytes:1466 (1.4 KB)

wlp6s0    Link encap:Ethernet  HWaddr cc:3d:82:1a:1e:1d  
          inet addr:10.250.9.73  Bcast:10.250.9.255  Mask:255.255.254.0
          inet6 addr: fe80::ce3d:82ff:fe1a:1e1d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4381 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4398 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2246805 (2.2 MB)  TX bytes:835572 (835.5 KB)

Spicer

Answer 1

I can't build images from a Dockerfile because I can't use --net=host with the build command

That is the job of the docker daemon to be able to access the internet when building.

You can help it by passing build-time arguments like

docker build --build-arg HTTP_PROXY=http://...

That is, if you are behind a proxy.
If you are not, check your DNS settings (that issue is in the context of boot2docker, which might not concern you here, but it still can give some clues as to what to inspect).
Here is another example of DNS issue.

The OP wheeler confirms a dns-related issue in the comments:

I had to disable dnsmasq in NetworkManager, not quite sure why it was affecting docker, but DNS resolution started working inside containers when I disabled dnsmasq.

This is a workaround seen before here:

• Disable dnsmasq by commenting it out the "dns=dnsmasq" line in /etc/NetworkManager/NetworkManager.conf and restarting the network-manager and docker.io services (sudo service network-manager restart && sudo service docker.io restart).
• Alternatively enable the commented out DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4" line in /etc/default/docker.io (and also restart the docker.io service).

The latter workaround of course requires the 8.8.8.8 / 8.8.4.4 servers to be reachable from your network.

---

The OP adds:

This solution worked to some extent until I used my VPN to work from home, and the subnet of the docker bridge was colliding with my VPN subnet.

He recommends "Set the ip of the Docker bridge with Systemd"

/etc/systemd/system/docker.service.d/docker.conf should contain this:

[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --bip=192.168.169.1/24

And:

systemctl stop docker

# We need a program called brctl to, well, control the bridge, which is part of the bridge-utils package.
sudo apt-get install bridge-utils

#Bring down the docker0 interface:
sudo ip link set docker0 down

# And delete the bridge.
sudo brctl delbr docker0

# Finally, start the Docker daemon
systemctl start docker

Nomarch

I'm not behind a proxy, and I've tried setting the proper DNS settings by editing DOCKER_OPTS to both google's and OpenDNS's servers and still I cannot ping from within a container. – Spicer

Figured it out: I had to disable dnsmasq in NetworkManager, not quite sure why it was affecting docker, but DNS resolution started working inside containers when I disabled dnsmasq. – Spicer

@Spicer Great! I have included your workaround (and added a second and some link) in the answer for more visibility. – Nomarch

This solution worked to some extent until I used my VPN to work from home, and the subnet of the docker bridge was colliding with my VPN subnet. This guide helped resolve that: container-solutions.com/… – Spicer

@Spicer Interesting. I have included your comment and the relevant extract of your link in the answer for more visibility. – Nomarch

I edited the answer, changing docker ExecStart=/usr/bin/docker -d -H fd:// --bip=192.168.169.1/24 to ExecStart=/usr/bin/docker daemon -H fd:// --bip=192.168.169.1/24 since -d is no longer a valid flag for the docker binary. – Spicer

Answer 2

I had this issue on Ubuntu 16.04 Here is the fix for this by editing the NetworkManager.conf

sudo nano /etc/NetworkManager/NetworkManager.conf

comment out dns=dnsmasq then restart the NetworkManager

sudo service network-manager restart

DONE!

Jessamyn

Answer 3

On RHEL, CentOS, or Fedora, edit the <interface>.network file in /usr/lib/systemd/network/ on your Docker host (ex: /usr/lib/systemd/network/80-container-host0.network) and add the following block within the [Network] section.

[Network]
...
IPForward=true

in my case I added IPForward=ipv4 to /etc/systemd/network/XX-eth0.network and it solved the problem

Notice that

sysctl -w net.ipv4.ip_forward=1

didn't solve this problem for me, only changing network interface on host has result

https://docs.docker.com/engine/install/linux-postinstall/#ip-forwarding-problems

Destinydestitute

Answer 4

For me, the issue was a QEMU installation that was messing with the bridge to the host, which might be related to this issue.

To address it, I removed QEMU and libvirt:

sudo apt-get purge qemu qemu-block-extra qemu-slof qemu-system qemu-system-arm qemu-system-common qemu-system-mips qemu-system-misc qemu-system-ppc qemu-system-sparc qemu-system-x86 qemu-user qemu-user-binfmt qemu-utils libvirt-bin libvirt-daemon libvirt0:amd64.

Brade