Google Colab Error : Failed to get convolution algorithm.This is probably because cuDNN failed to initialize

Question

UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.

 [[{{node conv2d_1/convolution}} = Conv2D[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](zero_padding2d_1/Pad, conv2d_1/kernel/read)]]
 [[{{node metrics/acc/Mean/_255}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_563_metrics/acc/Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

This error I am getting on Google colab. Colab's tensorflow version is 1.12.0.

I couldn't find any solution. Cause All of them were of local system.

Lillis

Can you share a self-contained notebook that reproduces the error you observe? – Cretan

Did you and how did you fix this problem? – Flummox

I've received the same error. It seems the error only shows when you've assign a GPU. I don't get the error when I train a model using CPU. – Err

Answer 1

For future passers-by, this looks like a service bug that is being addressed by the Colab folks now.

Details are on the GitHub issue.

Update: This issue has been fixed service-side. Details: https://github.com/googlecolab/colabtools/issues/384#issuecomment-451001849

Cretan

it is not resolved. getting this nasty error. – Horned

Answer 2

Just install tensorflow and dont install CUDA and cuDNN

This was my first code(Which had the error occurring)

!pip uninstall tensorflow

!pip install tensorflow-gpu==1.15

!apt install --allow-change-held-packages libcudnn7=7.4.1.5-1+cuda10.0

By removing the last line my error got resolved

!pip uninstall tensorflow

!pip install tensorflow-gpu==1.15

Hope it helps!

Ovate