Could not load dynamic library 'libnvinfer.so.7'

Question

I know that this question has been asked a lot, but none of the suggestions seem to work, probably since my setup is somewhat different:

Ubuntu          22.04
python          3.10.8
tensorflow      2.11.0
cudatoolkit     11.2.2
cudnn           8.1.0.77
nvidia-tensorrt 8.4.3.1
nvidia-pyindex  1.0.9

Having created a conda environment 'tf', in the directory home/dan/anaconda3/envs/tf/lib/python3.10/site-packages/tensorrt I have

libnvinfer_builder_resource.so.8.4.3
libnvinfer_plugin.so.8
libnvinfer.so.8
libnvonnxparser.so.8
libnvparsers.so.8
tensorrt.so

When running python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))" I get

tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7';
dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory;
LD_LIBRARY_PATH: :/home/dan/anaconda3/envs/tf/lib

tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7';
dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory;
LD_LIBRARY_PATH: :/home/dan/anaconda3/envs/tf/lib

tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

I'm guessing I should downgrade nvidia-tensorrt, but nothing I've tried seems to work, any advice would be much appreciated.

Becker

Your LD_LIBRARY_PATH is incorrect, it should point to the actual folder containing .so files, as tensorrt is not a python library, so the whole pythone.10/site-packages/tensorrt is completely unnecessary. – Caulescent

@Dr.Snoopy I'm not sure I understand correctly - my LD_LIBRARY_PATH is /home/dan/anaconda3/envs/tf/lib, as printed in the error message, and is exactly where all the .so files are. home/dan/anaconda3/envs/tf/lib/python3.10/site-packages/tensorrt is merely the location where tensorrt is installed. – Becker

You said and showed in your questions that the .so files are in /home/dan/anaconda3/envs/tf/lib/python3.10/site-packages/tensorrt, also it seems that your tensorflow version expects libnvinfer version 7, not version 8 that you have installed. – Caulescent

yes, as that is where pip install nvidia-tensorrt put them - but all the other .so files are under /home/dan/anaconda3/envs/tf/lib. Should I move those under tensorrt/ into `lib/' ? – Becker

I also think so, which is why I'm wondering how to downgrade to version 7? – Becker

No, do not modify the location of any .so files, tensorrt is not a python package, so anaconda is just emulating one. About downgrading, no idea, anaconda might have earlier versions available. – Caulescent

Ahh I see now that using python > 3.8 makes it impossible for pip to find version 7 of tensorrt - the solution seems to be the following: github.com/tensorflow/tensorflow/issues/… Although I am still in the process of trying it myself – Becker

Answer 1

Solution: follow the steps listed here https://github.com/tensorflow/tensorflow/issues/57679#issuecomment-1249197802.

Add the following to ~/.bashrc (for the conda envs as described in my scenario):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${HOME}/anaconda3/lib/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${HOME}/anaconda3/lib/python3.8/site-packages/tensorrt/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${HOME}/anaconda3/envs/tf/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${HOME}/anaconda3/envs/tf/lib/python3.8/site-packages/tensorrt/

Becker

Answer 2

For me the setting a symbolic link from libnvinfer version 7 to 8 worked:

# the following path will be different for you - depending on your install method
$ cd env/lib/python3.10/site-packages/tensorrt

# create symbolic links
$ ln -s libnvinfer_plugin.so.8 libnvinfer_plugin.so.7
$ ln -s libnvinfer.so.8 libnvinfer.so.7

# add tensorrt to library path
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/env/lib/python3.10/site-packages/tensorrt/

Leatherette

ln -s linvinfer.so.8 libnvinfer.so.7 should be ln -s libnvinfer.so.8 libnvinfer.so.7 missing a letter b. – Now

How can you know that it's safe to do that? – Sticky

Thanks for your question, @RezaRoboubi! I didn't claim it is safe but it solved the issue in a different way and I felt it was okay to share it. I probably should add a warning stating that everyone performs that change at their own risk. Please let me and everyone else know if you have evidence that this may indeed be a very bad idea. – Leatherette

Thanks for your reply @thalhamm. No, I have no evidence. – Sticky

It works for me. In my case the path is /usr/local/lib/python3.8/dist-packages/tensorrt_libs. – Sybyl

Answer 3

This has a solution already, but maybe I can add more depth to the problem and a description for folks (like me) who had to install that stuff from scratch.

Most likely (judging from these questions) you will come here because you install kohya_ss.

In my case the machine does not have some of the required packages (tensorflow and tensorrt) and installing these brings versions of certain components that are newer than what kohya_ss expects.

When you see this error, you are probably missing tensorflow and tensorrt (or you have it already, but in a different version)

# install tensor-runtime 
sudo apt install python3-pip -y
pip install tensorrt tensorflow

Now try the following command to see if above install fixed the problem already:

python3 -c 'import tensorflow as tf; print(tf.__version__)' 

If not (if the above command still complains about these missing files), understand that the above install command will give you an installation of tensorflow and tensorrt inside a hidden folder named ~/.local/... This will contain versions of the file you need, but with the wrong version numbers/filenames.

The following command will find this other (wrong) version and its paths:

find . -name libnvinfer.so* -print

This will give you an output in the form

.local/lib/python3.10/site-packages/tensorrt/libnvinfer.so.8

Note the part before the actual filename and cd there.

cd  ~/.local/lib/python3.10/site-packages/tensorrt/

Type the following command. This will create a file with the same content but under a different name (the name that is missing).

ln -s libnvinfer.so.8 libnvinfer.so.7

Do a similar find command with libnvinfer_plugin.so.* file

find . -name libnvinfer_plugin.so.* -print

With the result do the same steps (this may be slightly different for you but same steps as above, i.e. find file with new name, go to that folder, create link with missing name):

cd ~/.local/lib/python3.10/site-packages/nvidia/cuda_runtime/lib/
ln -s libcudart.so.12 libcudart.so.11.0

Add the two paths to the LD_LIBRARY_PATH like this (only the paths which you used in the cd commands, not the file names):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/python3.10/site-packages/tensorrt/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/python3.10/site-packages/nvidia/cuda_runtime/lib/

For good measure also add the /bin path that the installer complained about:

export $PATH=$PATH:/home/sdgui/.local/bin

Verify that python3 can now use tensorflow without the error:

python3 -c 'import tensorflow as tf; print(tf.__version__)' 

Hopefully a message, but no more warning about missing files.

You can then add the two export commands to your .profile or .bashrc file. as pointed out in the answer from the original poster.

Also, if you had tensorrt already installed globally (via sudo), your find command will need to be:

sudo find / -name libnvinfer.so* -print

The steps will be similar, just with global paths and requiring sudo to do it.

Enjoy

Pustule

Answer 4

I changed version and the problem was solved:

pip install --upgrade tensorflow==2.8

Note: When I use v2.10, I get the same error message. v2.8 is stable now.

Sandi