Enabling live restore on docker isn't keeping the containers alive

Question

I read the Enable Live Restore, but when I tried it.

ubuntu@ip-10-0-0-230:~$ cat /etc/docker/daemon.json 
{
  "live-restore": true
}

I started an nginx container in detached mode.

sudo docker run -d  nginx
c73a20d1bb620e2180bc1fad7d10acb402c89fed9846f06471d6ef5860f76fb5 


$sudo docker ps

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              
 
c73a20d1bb62        nginx               "nginx -g 'daemon of…"   5 seconds ago       Up 4 
seconds

Then I stopped the dockerd

sudo systemctl stop snap.docker.dockerd.service

and I checked that there was no container running

ps aux | grep nginx

After that, I restarted the docker service and still, there wasn't any container.

Any Idea? How this "enable live restore" works?

Decadence

Answer 1

From the documentation, after modifying the daemon.json (adding "live-restore": true) you need to :

Restart the Docker daemon. On Linux, you can avoid a restart (and avoid any downtime for your containers) by reloading the Docker daemon. If you use systemd, then use the command systemctl reload docker. Otherwise, send a SIGHUP signal to the dockerd process.

You can also do this but it's not recommended :

If you prefer, you can start the dockerd process manually with the --live-restore flag. This approach is not recommended because it does not set up the environment that systemd or another process manager would use when starting the Docker process. This can cause unexpected behavior.

It seems that you had not done this step. You said that you've made the modification to the daemon.json and directly started a container and then stopped the dockerd.

In order to make the Live Restore functionality work follow all steps in the right order :

1. Modify the daemon.json by adding "live-restore": true

2. Reload the Docker daemon with the command :

   sudo systemctl reload docker
   

Then try the functionality with your example (firing up a container and making the daemon unavailable).

I've tested and it works if you follow the steps in order :

Tested with Docker version 19.03.2, build 6a30dfc and Ubuntu 19.10 (Eoan Ermine)

You've installed Docker via snap : snap.docker.dockerd.service

Unfortunately, it's not recommended since snap model is not fully compatible with Docker. Furthermore, docker-snap is no longer maintained by Docker, Inc. Users encounters some issues when they installed Docker via snap see 1 2

You should delete the snap Docker installation to avoid any potential overlapping installation issues via this command :

sudo snap remove docker --purge

Then install Docker with the official way and after that try the Live Restore functionality by following the above steps.

Also be careful when restarting the daemon the documentation says that :

Live restore upon restart

The live restore option only works to restore containers if the daemon options, such as bridge IP addresses and graph driver, did not change. If any of these daemon-level configuration options have changed, the live restore may not work and you may need to manually stop the containers.

Also about downtime :

Impact of live restore on running containers

If the daemon is down for a long time, running containers may fill up the FIFO log the daemon normally reads. A full log blocks containers from logging more data. The default buffer size is 64K. If the buffers fill, you must restart the Docker daemon to flush them.

On Linux, you can modify the kernel’s buffer size by changing /proc/sys/fs/pipe-max-size.

Kaveri

Thanks but please check my second last line. P.S I have read all the documentation – Decadence

@Decadence I added an example (following the steps in order) and it works. Maybe we don't have the same environment. I've installed Docker from the official website. I noticed that you have snap.docker.dockerd.service maybe try with the official install ? Because docker-snap is no longer maintained by Docker, Inc. – Kaveri

Okay Yann, I will try the official installation. Thanks. – Decadence

@Decadence You should delete your snap Docker installation before installing Docker through the official way, to avoid any potential overlapping installation issues. (You can check my answer for more info) – Kaveri