Using GPU with containers and Container Optimized OS in Google Cloud VM

Question

I would like to run a custom Docker image with GPU on Google Compute Engine. I have built and pushed the image to the Google Container Registry. It seems logical to use Container-Optimized OS for a host machine in Google Cloud Engine since I don't need any extra soft on the host machine except Docker, Nvidia GPU drivers and nvidia-container-runtime.

I managed to install nvidia-drivers with this solution.

But I can't run my Docker image with GPU (using --gpu all option) without nvidia-container runtime. This step is specified in official Docker documentation.

Is there a way to install nvidia-container-runtime on Container-Optimized OS in Google Cloud VM?

Tyree

Answer 1

You don't have to set --gpu all, because this is the default option for nvidia-container-runtime. The assumption, that you don't need anything else is wrong, because it requires libnvidia-container.

To precisely answer the question: No, because libnvidia-container needs to be installed on the OS and nvidia-container-runtime needs to be installed within the K8s container. The one exposes an interface - and the other connects it. And so the one is useless without the other.

Forthwith

Thanks for the explanation! I mentioned that I am interested in Google COmpute Enginer instance, not k8s cluster. – Tyree

Does it mean that it's impossible to run containers with GPU on Google Compute instance? – Tyree

If you wish to access the GPU from within Docker, you need to install both components... and you cannot run the run-time directly on GCE. It's GPU <-> kernel module (driver) <-> libnvidia-container <-> nvidia-container-runtime. It's complicated, but this is based upon several abstraction layers. And I don't think that there would be much difference, in between Container Engine & Compute Engine. – Forthwith

And that --gpu switch is merely to use different GPU with different Docker containers. – Forthwith

my question was not about --gpu switch – Tyree