nvidia-smi Failed to initialize NVML: GPU access blocked by the operating system

Question

when asking for

nvidia-smi

it gives this error:

Failed to initialize NVML: GPU access blocked by the operating system

other information:

$ nvcc --verion

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2015 NVIDIA Corporation
Built on Mon_Feb_16_22:59:02_CST_2015
Cuda compilation tools, release 7.0, V7.0.27

and also:

 $ lspci | grep -i nvidia

 01:00.0 VGA compatible controller: NVIDIA Corporation GF108M [GeForce GT 425M] (rev a1)
 01:00.1 Audio device: NVIDIA Corporation GF108 High Definition Audio Controller (rev a1)

Having searched a lot in the internet I couldn't find a way to solve this problem. when I use ipython notebook and want to run Caffe framework it gives this error:

Check failed: error == cudaSuccess (38 vs. 0)  no CUDA-capable device is detected

Oliverolivera

You have not installed CUDA (in particular, the GPU driver) correctly. I suggest following the directions. – Regnant

@RobertCrovella I followed all of that directions but in all of steps I received the message that says: * is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. e.g. for sudo apt-get install cuda or sudo apt-get install cuda-driver and sudo apt-get install linux-headers-$(uname -r) – Oliverolivera

Answer 1

I noticed that after CUDA installation restarting Ubuntu works, and now I see the GPU details output by nvidia-smi

Oliverolivera

Answer 2

If you believe that both CUDA and graphics driver are installed correctly, but you still cannot make your GPU to be detected, the problem might be in that you are using mobile Nvidia graphics on Optimus-enabled laptop on Linux.

You could either:

• change your application to properly detect GPUs behind Optimus. See documentation here
• or to run your application via Bumblebee (and primus)

Holliehollifield

Answer 3

WSL user here. Running nvidia-smi on either Windows and WSL failed. Reinstalling the Nvidia for WSL driver, on the Windows side, fixed the problem. The problem was created when installing CUDA Toolkit and CUDNN broke the Nvidia for WSL driver.

Lame

does this still work after today's flight? I've started getting the OP's error today after updating to build 21354 – Hennessey

Same here, just installed build 21354 and and it's stopped working! – Blithesome

Answer 4

If you are using WSL2, you have to install the WSL-Ubuntu distribution of CUDA Toolkit, and reboot your entire PC.

https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local

Triplicate

Answer 5

I had the same problem. It was happened because of installing a nvidia toolkit (I am not sure). According to this website (which has useful ideas) I found that cuda driver version in the cuda installer and host was incompatible. (host : 367.57 , installer: 375.26 , At first I could not check the installer version because all the versions was 367.57, but when I reinstall cuda by run file, I found it)

So, I uninstalled cuda and nvidia completely and install cuda again by this help. At first in the installation process I got some errors which I found, nvidia has not completely gone. After uninstalling completely, I installed cuda and now I can run "sudo nvidia-smi" without problem.

Quarterback

Answer 6

I got the error failed to initialize NVML: Driver/Library version mismatch. And nvidia-smi failed to print any info. I tried to find if there were other versions of nvidia driver installed in my ubuntu. But I just found nvidia-driver-390. In the end, reboot helped me solve the problem.

Pumpkinseed

dmesg will help in this case. You will be able to see the detailed error printed by nvidia-smi – Babylonian