Docker prune stuck in "a prune operation is already running"

Question

I run command docker system prune yesterday, it took some time and then my SSH session was disconnected from different reason.

Unfortunately I am getting now:

Error response from daemon: a prune operation is already running.

Obviously there is a lock and prune command is not running anymore.

Does anybody know how remove the lock without stopping and removing all containers?

EDIT: Created an issue in repo: https://github.com/moby/moby/issues/36447

Dhahran

Sounds more like a bug which should be reported to project owners IMVHO. – Mancuso

Answer 1

Restarting docker worked for me.

Shavon

Answer 2

This issue seems to occur when a container is not-responding to docker.

Here is how I've fixed it:

1. First, find the non responding containers with: sudo docker inspect %CONTAINER ID%
2. If a container does not respond, the inspect command will not return anything.
3. Once the %CONTAINER ID% not responding has been identified, find its corresponding pid with: ps -aux | grep %CONTAINER ID%
4. There should be a line looking like:

root 14931 0.0 0.0 7648 428 ? Sl Sep13 0:26 docker-containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/3b0d4cba3f63a71fda99c76f3f777a156056e559fb034da4ed59c0aa340e5669 -address /var/run/docker/containerd/docker-containerd.sock -containerd-binary /usr/bin/docker-containerd -runtime-root /var/run/docker/runtime-runc

5. Then, kill this service with kill -9 %PID%

---

Tip 1: There can be one or many containers not responding

Tip 2: In order to avoid down-time, you can scale-up the service corresponding to the container that does not respond with a docker service scale ....

(my answer complements dparkar's.)

Room

Remove the non responding containers with docker rm -f %CONTAINER ID% worked fine for me without to kill the process. – Prescribe

Answer 3

In my case, it was not stuck, just taking a very long time to complete. It deleted 3500 images once finished!

Softener

I had this same issue - you can run docker system info in a separate terminal session to check if the number of containers/images is going down. – Jareb

Answer 4

Working solution from the github issue :

doublemcz commented on Mar 14

I can confirm that prune stuck because of a non-responding container. When I kill the container first by kill -9 PROCESS_ID where the process Id I get from ps aux | grep docker-containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/CONTAINER_ID

The problem is that you need to know that there is a container that does not respond on docker :-/ The container works (i.e. node.js works fine) but just docker is not able to even inspect it.

Btw this container should not even be there because we run docker service update... with the :latest image. Docker created another container and this was not killed. So there were two running containers with two different versions.

Senary

Answer 5

In my case these steps helped (I have hundreds of containers, so I was not able to identify the bad ones).
I'm using minikube.

1. Ensure that all your stuff running in deployments/replicasets.
2. systemctl restart docker (this command runs forever for me)
3. Now it is possible to identify the dangling processes via ps auxf. All other containers are stopped.
4. Kill them via kill -9 <PID>
5. Now rerun systemctl restart docker. It should now working correctly.
6. Kubernetes will bring back all pods correctly. Just give it some time (it will restart pods multiple times)

Glennglenna

Answer 6

Easy way to fix this issue Question:

Answer: service docker restart

Done:

Youngling

I think you should include the command lines as text like this: service docker restart. – Earthward

@AdrienPavao The answer was the command line "service docker restart",thanks – Youngling

Answer 7

Restart the docker service. It is worked for me.

sudo systemctl restart docker.service

Hypomania

Answer 8

This thread is old, but maybe this information is still of use to some. I had some 'corrupt' images that I could not remove with either of the proposed methods, so I had to do a:

docker rmi -f $(docker images -aq)

To delete all non active images After that restarting docker and prune started working again

Interknit