Enable gpu support by default on docker containers

Question

I'm using a platform (Cytomine) on Ubuntu 18.04 to run some deep learning containerized applications (this platform handles the Docker images and containers automatically, so I only need to create the image and provide its download URL to the platform). So far it's working good but now I need to enable GPU support to run the model efficiently. Thus, I did some local tests with nvidia-docker to manually run the model container with GPU support, it was really easy to have it working because I just had to add one option to the run command:

docker run --gpus all

However, because I cannot add this option to the code on the Cytomine platform I need to find a way of adding/enabling that option by default to all the containers run by docker.

I tried adding this option to the files /etc/docker/daemon.json and /etc/docker/key.json and then restarted docker sudo systemctl restart docker. However, it didn't work.

Also, I found how to create docker config files (docker config); however, this seems to work only with Docker Swarm and I'm not going to use a Swarm for this project.

Thus, I'm looking for a straightforward solution that can be deployed properly. Is there any way to enable this option (--gpus all) by default when running any Docker container? (like somehow including it on the Dockerfile?)

Thanks!

Orran

Answer 1

Here's the right config to set in /etc/docker/daemon.json :

{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    },
    "default-runtime": "nvidia"
}

Don't forget to sudo service docker restart

Yttria

Answer 2

@secavfr answer sort of worked for me.

I set the default runtime to nvidia in the /etc/docker/daemon.json but when running the nvidia-smi I got an error saying:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "nvidia-smi": executable file not found in $PATH: unknown.

I found out though that you can then use an env variable to make GPUs visible to the runtime this way: docker run -it --rm -e NVIDIA_VISIBLE_DEVICES=all ubuntu nvidia-smi

As you can see, you don’t need to add --gpus all at all but the env var is required. For my use case environment vars were allowed.

Maddie