Is GPU pass-through possible with docker for Windows?

Question

I am trying to run an application inside a docker container in Windows 10.

But I am not able to get the GPU working inside docker.

I read that it needs "GPU Pass-through."

How should I get around this?

Chlamydate

Answer 1

Update (December 2020) You can now do GPU pass-through on Windows, if you use WSL 2 as the backend for Docker: WSL 2 GPU Support is Here - that is a slightly neater method than running Docker inside WSL.

Original answer:

GPU access from within a Docker container currently isn't supported on Windows.

You need nvidia-docker, but that is currently only supported on Linux platforms. GPU passthrough with Hyper-v would require Discrete Device Assignment (DDA), which is currently only in Windows Server, and (at least in 2015) there was no plan to change that state of affairs. Hence, NVIDIA are not porting nvidia-docker to Windows at the moment.

A bit more info here: https://devblogs.nvidia.com/nvidia-docker-gpu-server-application-deployment-made-easy/

Update (October 2019): nvidia-docker is deprecated, as Docker 19.03 has native support for NVIDIA GPUs. Instead install nvidia-container-runtime, and use the docker run --gpus all flag. You can also run Windows Containers with GPU acceleration on a Windows host, using Docker 19.03, but not a Linux container.

Update (August 2020): It looks like you can now do GPU pass-through when running Docker inside the Windows Subsystem for Linux (WSL 2).

This link goes through installation, setup and running a TensorFlow Jupyter notebook inside Docker in Ubuntu in WSL 2, with GPU support: https://ubuntu.com/blog/getting-started-with-cuda-on-ubuntu-on-wsl-2

Note - I haven't done this myself yet.

Endarch

nvidia-docker is not the only option for GPU pass-through on UNIX-like OS's (or at least where devices are files) it should be possible to forward GPUs with the --device flag. An example is the ROCm container. – Identify

There is some update about using GPU in container with Windows Docker - techcommunity.microsoft.com/t5/Containers/… but only for DirectX API. – Constitutionality

I just asked similar question and was suggested to use docker. My goal is slightly different - to use GPU acceleration under VirtualBox with Linux on Windows machine. From this answer I assume I cannot follow the suggestion, but looks like two questions are related – Likely

Thanks for your answert. Could you briefly explain how to install nvidia-container-runtime? – Babar

There is a good set of instructions here: github.com/NVIDIA/nvidia-docker. Still Linux only though. Make sure you have an up-to-date version of Docker installed (>19.03) – Endarch

@4Oh4, thanks for your answer. So, currently the only way to pass through a GPU would be to install a separate Linux OS as dual boot? – Babar

If you have an ATI, AMD, or Intel GPU/APU you're out of luck then? – Funiculus

@4Oh4, hi sir, i have similar issue - no support for win10? – Latoyia

is there anyone who can help me out with similar question? #63600936 – Latoyia

@Latoyia Updated answer to reflect new route: running Docker in Ubuntu in WSL2, with GPU pass through – Endarch

@4Oh4, oh thx , but it's tensorflow.. i have pytorch model trained and it needs gpu from inside ubuntu container – Latoyia

@erjan you can replace the TensorFlow container with a different container, that's just an example. For GPU support, your container needs the CUDA Toolkit installed in it as well. This one maybe: ngc.nvidia.com/catalog/containers/nvidia:pytorch – Endarch

Answer 2

2023 updated answer

TLDR: If you're using the latest versions of Windows 11, NVIDIA drivers, and Docker Desktop, this should work pretty much out of the box. Try with this command: docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody

---

I was successful with training models on GPU in TensorFlow 2 using this method.

1. Use Windows 11 if possible. The feature was added in build 20149, which was originally an Insider build of Windows 10, but later changed to Windows 11 as it was revealed. It's not clear to me if the feature was back-ported to Windows 10, report back if you try it.
2. Ensure that you are using the latest drivers for your GPU. (It used to be necessary to download the NVIDIA CUDA WSL driver, but the link is dead now -- documentation seems to suggest that it is a part of the standard driver now.)
3. Install Docker Desktop
   • It will guide you through enabling WSL2 if you haven't already.
   • If you already have it installed, update it to the latest version and enable Settings - General - Use the WSL2 backed engine.
   • To be able to use the docker CLI from inside WSL2 (not just from PowerShell/cmd), enable the integration in Settings - Resources - WSL INTEGRATION.
4. Test using the command docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody

You need to pass --gpus=all to docker run to enable the container to access GPU. (If you use VSCode Remote Containers, add "runArgs": ["--gpus=all"], to devcontainer.json.)

You may come across mentions of --runtime=nvidia in descriptions of images meant for nvidia-docker (like the official TensorFlow images). Simply replace --runtime=nvidia by --gpus=all in the provided commands.

Adley

This works, but less work is to just dual boot PoP OS / other Linux for ML purposes + you achieve a few % better performance – Rattoon

@Ron The only significant performance difference is for disk volume I/O, which can be mostly mitigated by mounting volumes from the WSL filesystem -- either enable CLI integration from Docker Desktop settings and run the docker command from WSL, or use the `\\wsl$` filesystem. – To

Is my understanding right that you must become a member of Windows Insider Program to enable this? If so, do you know what's the plan to get this access without joining Insider Program? It's not possible for me to join this program from various reasons that's why I'm asking. Otherwise good to hear there is such progress. – Nichani

@Nichani You need to have Windows 10 build 20149 or higher installed, which you can get by either: 1. enrolling a Microsoft account in the Insider program and updating your existing install 2. installing fresh from the Insider ISO images: learn.microsoft.com/en-us/windows-insider/isos 3. waiting a few months for the update to hit the main release channel – To

Answer 3

Now that docker on Windows 10 can access WSL2 (as of Windows 10 version 2004) it has cleared the way for GPU support of Linux docker containers on Windows 10.

According to this official blog, MS "will start previewing GPU compute support for WSL in Windows 10 Insider builds within the next few months": https://devblogs.microsoft.com/commandline/the-windows-subsystem-for-linux-build-2020-summary/#wsl-gpu

I expect Docker GPU support to follow shortly after.

Update:

GPU pass-through in Windows is now possible under very specific circumstances, including:

• that the container must be Windows as well
• process level isolation only, no hyper-v
• it only works with DirectX based applications
• for Machine Learning, this means only Microsoft ML will work.

Refer to: https://learn.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/gpu-acceleration

Update 2:

GPU pass through from Linux docker on Windows host is now possible on the latest Windows Insider build, refer to:

https://ubuntu.com/blog/getting-started-with-cuda-on-ubuntu-on-wsl-2

This will flow through to mainstream Windows probably in the next major update.

Update 3:

Confirmed that Windows build version 2021 will include GPU pass-through for WSL. See details of the announcement here: https://blogs.windows.com/windowsdeveloper/2021/05/25/the-windows-developers-guide-to-microsoft-build-2021/

Pluviometer

so it's possible to run docker container with flask app that needs nvidia drivers but only under specific conditions? – Latoyia

I think it will if the nvidia drivers are the standard graphics ones, I don't think CUDA is working yet. – Pluviometer

could u pls take a look at this similar question? #63600936 – Latoyia

Answer 4

Provide a sample !

ffmpeg -hide_banner -codecs | grep 264
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_cuvid ) (encoders: h264_nvenc h264_v4l2m2m nvenc nvenc_h264 )

This would show that the ffmpeg verion can handle: h264_nvenc nvenc nvenc_h264

ffmpeg -i video.mp4 -vf scale=iw/2:ih/2 -vcodec h264_nvenc "video_2.mp4"

On my WSL2/Insider I get a message that I don't have the correct nvidia driver installed. Because the drive is installed down-below in the windows OS.

Queries with Microsoft and NVIDIA turned out the hardware acceleration is not passed through and they have no near future plans to do so.

Windows docker on a widnows os, could be fine. Will check.

Subtropical

Answer 5

I think docker run --gpus all can resolve, you will need install nvidea drivers to your linux docker and cuda sdk. Other way I think is give all permissions using parameter --privileged: docker run --privileged and other parameters you need, you will nedd to think about because with privileged de docker can access your real machine.

Malamud

Answer 6

Note that Hardware Acceleration of video decoding-encoding is not supported in the WSL2/windows-Insider implementation.

Thus, any GPU functions can be used with little limitations on WSL2 but video encoding-decoding cannot be done by the hardware.

Attempt to ask ffmpeg to use the hardware acceleration leads to error messages. (skvideo, which is an ffmpeg wrapper, crashes)

Not sure about gStreamer yet. But for sure it cannot use the Hardware Acceleration as it is not "funneled" through the interface.

As far as I know - there are no current plans to add ths support.

Best, Mickey

Subtropical

Not only is this answer not helpful, it's also incorrect, as the accepted answer states that it can be done with GPU passthrough. – Sachsen

See my second post above. – Subtropical