Using:
Linux 4.13.0-26-generic #29~16.04.2-Ubuntu SMP Tue Jan 9 22:00:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
I've been noticing my system hesitating while working, and finally tracked it down to something wonky going on with Docker veth devices. It seems like once/minute the following occurs:
- NetworkManager complains udev can't find a device
- It deletes/recreates the device, then complains about no ifupdown configuration found
- It complains about driver 'veth' does not support carrier detection
The following lines are for one example interface:
12:17:26 NetworkManager[1256]: <warn> device (vetha2ac803): failed to find device 60 'vetha2ac803' with udev
12:17:26 NetworkManager[1256]: <info> manager: (vetha2ac803): new Veth device (/org/freedesktop/NetworkManager/Devices/78)
12:17:26 NetworkManager[1256]: <info> devices added (path: /sys/devices/virtual/net/vetha2ac803, iface: vetha2ac803)
12:17:26 NetworkManager[1256]: <info> device added (path: /sys/devices/virtual/net/vetha2ac803, iface: vetha2ac803): no ifupdown configuration found.
12:17:26 kernel: eth0: renamed from vetha2ac803
12:17:26 NetworkManager[1256]: <info> devices removed (path: /sys/devices/virtual/net/vetha2ac803, iface: vetha2ac803)
12:17:26 NetworkManager[1256]: <info> device (vetha2ac803): driver 'veth' does not support carrier detection.
12:17:31 kernel: vetha2ac803: renamed from eth0
Full syslog for an episode:
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <warn> [1520785440.6743] device (vethb7b0026): failed to find device 90 'vethb7b0026' with udev
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.6754] manager: (vethb7b0026): new Veth device (/org/freedesktop/NetworkManager/Devices/123)
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <warn> [1520785440.6757] device (vethcc7ba9a): failed to find device 91 'vethcc7ba9a' with udev
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.6763] manager: (vethcc7ba9a): new Veth device (/org/freedesktop/NetworkManager/Devices/124)
Mar 11 12:24:00 mcrowe-XPS-15-9560 kernel: [ 496.472909] docker0: port 5(vethcc7ba9a) entered blocking state
Mar 11 12:24:00 mcrowe-XPS-15-9560 kernel: [ 496.472910] docker0: port 5(vethcc7ba9a) entered disabled state
Mar 11 12:24:00 mcrowe-XPS-15-9560 kernel: [ 496.472954] device vethcc7ba9a entered promiscuous mode
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.6828] devices added (path: /sys/devices/virtual/net/vethcc7ba9a, iface: vethcc7ba9a)
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.6828] device added (path: /sys/devices/virtual/net/vethcc7ba9a, iface: vethcc7ba9a): no ifupdown configuration found.
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.6830] devices added (path: /sys/devices/virtual/net/vethb7b0026, iface: vethb7b0026)
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.6830] device added (path: /sys/devices/virtual/net/vethb7b0026, iface: vethb7b0026): no ifupdown configuration found.
Mar 11 12:24:00 mcrowe-XPS-15-9560 dockerd[1770]: time="2018-03-11T12:24:00-04:00" level=info msg="shim docker-containerd-shim started" address="/containerd-shim/moby/787be6d744aac449168f9ac3d3016dfacb5cc0329c2d1106b49cf82a653e9593/shim.sock" debug=false module="containerd/tasks" pid=17214
Mar 11 12:24:00 mcrowe-XPS-15-9560 kernel: [ 496.668367] eth0: renamed from vethb7b0026
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.8953] devices removed (path: /sys/devices/virtual/net/vethb7b0026, iface: vethb7b0026)
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.8954] device (vethb7b0026): driver 'veth' does not support carrier detection.
Mar 11 12:24:00 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785440.8959] device (vethcc7ba9a): link connected
Mar 11 12:24:00 mcrowe-XPS-15-9560 kernel: [ 496.692872] docker0: port 5(vethcc7ba9a) entered blocking state
Mar 11 12:24:00 mcrowe-XPS-15-9560 kernel: [ 496.692877] docker0: port 5(vethcc7ba9a) entered forwarding state
Mar 11 12:24:00 mcrowe-XPS-15-9560 libvirtd[1426]: Failed to open file '/sys/class/net/vethb7b0026/operstate': No such file or directory
Mar 11 12:24:00 mcrowe-XPS-15-9560 libvirtd[1426]: unable to read: /sys/class/net/vethb7b0026/operstate: No such file or directory
Mar 11 12:24:01 mcrowe-XPS-15-9560 dockerd[1770]: time="2018-03-11T12:24:01-04:00" level=info msg="shim reaped" id=787be6d744aac449168f9ac3d3016dfacb5cc0329c2d1106b49cf82a653e9593 module="containerd/tasks"
Mar 11 12:24:01 mcrowe-XPS-15-9560 dockerd[1770]: time="2018-03-11T12:24:01.412572018-04:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Mar 11 12:24:01 mcrowe-XPS-15-9560 kernel: [ 497.241175] docker0: port 5(vethcc7ba9a) entered disabled state
Mar 11 12:24:01 mcrowe-XPS-15-9560 kernel: [ 497.241448] vethb7b0026: renamed from eth0
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5112] device (vethcc7ba9a): link disconnected
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <warn> [1520785441.5113] device (vethb7b0026): failed to find device 90 'vethb7b0026' with udev
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5137] manager: (vethb7b0026): new Veth device (/org/freedesktop/NetworkManager/Devices/125)
Mar 11 12:24:01 mcrowe-XPS-15-9560 kernel: [ 497.316333] docker0: port 5(vethcc7ba9a) entered disabled state
Mar 11 12:24:01 mcrowe-XPS-15-9560 kernel: [ 497.322213] device vethcc7ba9a left promiscuous mode
Mar 11 12:24:01 mcrowe-XPS-15-9560 kernel: [ 497.322224] docker0: port 5(vethcc7ba9a) entered disabled state
Mar 11 12:24:01 mcrowe-XPS-15-9560 libvirtd[1426]: Failed to open file '/sys/class/net/vethb7b0026/operstate': No such file or directory
Mar 11 12:24:01 mcrowe-XPS-15-9560 libvirtd[1426]: unable to read: /sys/class/net/vethb7b0026/operstate: No such file or directory
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5479] devices added (path: /sys/devices/virtual/net/vethb7b0026, iface: vethb7b0026)
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5480] device added (path: /sys/devices/virtual/net/vethb7b0026, iface: vethb7b0026): no ifupdown configuration found.
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5488] devices removed (path: /sys/devices/virtual/net/vethb7b0026, iface: vethb7b0026)
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5489] device (vethb7b0026): driver 'veth' does not support carrier detection.
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5495] device (vethcc7ba9a): driver 'veth' does not support carrier detection.
Mar 11 12:24:01 mcrowe-XPS-15-9560 NetworkManager[1256]: <info> [1520785441.5563] devices removed (path: /sys/devices/virtual/net/vethcc7ba9a, iface: vethcc7ba9a)
My docker info shows:
Containers: 15
Running: 7
Paused: 0
Stopped: 8
Images: 37
Server Version: 17.12.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9b55aab90508bd389d7654c4baf173a981477d55
runc version: 9f9c96235cc97674e935002fc3d78361b696a69e
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.13.0-26-generic
Operating System: Linux Mint 18.3
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 23.39GiB
Name: mcrowe-XPS-15-9560
ID: YHBS:L34M:PS4S:7OTF:XVTR:RYNA:IHK7:6XUB:NYDY:DJKU:M7BI:EPRU
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
I've tried having NetworkManager ignore veth interfaces, such as adding the following to NetworkManager.conf:
[keyfile]
unmanaged-devices=interface-name:veth*
Full system info:
System: Host: mcrowe-XPS-15-9560 Kernel: 4.13.0-26-generic x86_64 (64 bit gcc: 5.4.0)
Desktop: MATE 1.18.0 (Gtk 3.18.9) Distro: Linux Mint 18.3 Sylvia
Machine: System: Dell (portable) product: XPS 15 9560
Mobo: Dell model: 05FFDN v: A00 Bios: Dell v: 1.7.0 date: 12/15/2017
CPU: Quad core Intel Core i7-7700HQ (-HT-MCP-) cache: 6144 KB
flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 22464
clock speeds: max: 3800 MHz 1: 2800 MHz 2: 2800 MHz 3: 2800 MHz 4: 2800 MHz 5: 2800 MHz 6: 2800 MHz
7: 2800 MHz 8: 2800 MHz
Graphics: Card-1: Intel Device 591b bus-ID: 00:02.0
Card-2: NVIDIA Device 1c8d bus-ID: 01:00.0
Display Server: X.Org 1.18.4 drivers: intel (unloaded: fbdev,vesa) FAILED: nouveau
Resolution: [email protected]
GLX Renderer: Mesa DRI Intel HD Graphics 630 (Kaby Lake GT2)
GLX Version: 3.0 Mesa 17.2.8 Direct Rendering: Yes
Audio: Card Intel Device a171 driver: snd_hda_intel bus-ID: 00:1f.3 Sound: ALSA v: k4.13.0-26-generic
Network: Card-1: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter driver: ath10k_pci bus-ID: 02:00.0
IF: wlp2s0 state: up mac: <filter>
Card-2: Atheros usb-ID: 001-003
IF: null-if-id state: N/A speed: N/A duplex: N/A mac: N/A
Drives: HDD Total Size: NA (-) ID-1: /dev/nvme0n1 model: N/A size: 512.1GB
Partition: ID-1: / size: 445G used: 182G (44%) fs: ext4 dev: /dev/dm-0
ID-2: /boot size: 473M used: 131M (30%) fs: ext2 dev: /dev/nvme0n1p2
ID-3: swap-1 size: 25.61GB used: 0.00GB (0%) fs: swap dev: /dev/dm-1
RAID: No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors: System Temperatures: cpu: 40.0C mobo: N/A
Fan Speeds (in rpm): cpu: N/A
Info: Processes: 348 Uptime: 24 min Memory: 5150.1/23956.4MB Init: systemd runlevel: 5 Gcc sys: 5.4.0
Client: Shell (bash 4.3.481) inxi: 2.2.35