Getting "NotImplementedError: Could not run 'torchvision::nms' with arguments from CUDA backend" despite having all necessary libraries and imports

Question

The full error:

NotImplementedError: Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'torchvision::nms' is only available for these backends: [CPU, QuantizedCPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradMPS, AutogradXPU, AutogradHPU, AutogradLazy, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].

I get this when attempting to train a YOLOv8 model on a Windows 11 machine, everything works for the first epoch then this occurs.

---

I also get this error immediately after the first epoch ends but I don't think it is relevant.

Error executing job with overrides: ['task=detect', 'mode=train', 'model=yolov8n.pt', 'data=custom.yaml', 'epochs=300', 'imgsz=160', 'workers=8', 'batch=4']

I was trying to train a YOLOv8 image detection model utilizing CUDA GPU.

Appealing

Answer 1

This error occurs when you have Torch and Torchaudio for CUDA but not Torchvision for CUDA installed.

Uninstall Torch and Torchvision, I used pip:

pip uninstall torch torchvision

Then go here to install the proper versions of both using the nice interface. I got the following command:

pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116

Appealing

Answer 2

I had this issue and had no idea how to fix it. I then installed the PyTorch CUDA with pip instead of pip3, and that fixed it.

Priebe

This doesn't sound like a reliable method. – Rollway

This fixed it for me too, no idea why, didn't even know there was a difference.. My only guess is maybe pip3 is using the system's python interpreter and pip is getting redirected to the venv's interpreter, which are different python versions. – Politics

Answer 3

This issue arises because you have installed a PyTorch version that is not compatible with the CUDA version (Compute Platform). You can identify your CUDA version using the nvidia-smi command.

For example, if your CUDA version is 12, but you have installed a PyTorch version that is compatible with CUDA 12.1. To resolve this, uninstall PyTorch and reinstall it to match your CUDA version (computer platform). Find the relevant PyTorch version

If your CUDA version is 12, and you cannot find a PyTorch version compatible with that, install a PyTorch version compatible with a CUDA version less than 12, such as 11.8. No issues will arise in this case.

Note that you cannot install a PyTorch version with a higher CUDA version than your current platform.

If you have installed the CPU version of PyTorch, this problem should not occur.

Vanderbilt

Answer 4

If you get here after updating the new drivers (CUDA 12.4), you have to install a PyTorch that is still in a previous version

pip uninstall torch torchvision torchaudio

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu124

Moore

Answer 5

I don't know what causes this error but I know installing the proper version of torchvision and cuda didn't fix it. The way I solved is by uninstalling all of my packages and then reinstalling. Works just fine now. In hindsight, it may have been better to just update all my packages, but problem fixed none the less.

Boyes

Answer 6

I solved the problem following Pamudu suggestion. First I checked my CUDA version by running nvidia-smi on my Windows terminal. Then I went to the PyTorch web site and found the right command to install the version that matches my CUDA version. In my case the nvidia-smi reported version 12.2, the command below did the trick.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

As you probably has already installed torch so must uninstall everything first.

pip uninstall torch torchvision torchaudio

Mcclean