How to set the path that a .so library will search for other .so libraries?

Question

I have a libA.so that depends on libB.so, which is located at ../libB/ (from libA.c). I'm trying to compile things in such a way that I don't have to set any environment variables. I have:

    cc -std=c99 -c -fPIC -I../libB/ -Wall libA.c
    cc -std=c99 -shared libA.o -L../libB -lB -o libA.so

This compiles fine. When I run a program that loads libA with dlopen I get:

dyld: Library not loaded: libB.so
  Referenced from: libA/libA.so
  Reason: image not found
Trace/BPT trap: 5

so libA is not finding libB at runtime. I found this solution to change the runtime path on Mac OS X:

install_name_tool -change libB.so @loader_path/../libB.so libA.so

but I'd like to find a solution that would work on both OS X and Linux. Again, I'm trying to make the end-user do as little as possible so I don't want them to have to set environment variables, and I have to use cc (which for me is Apple LLVM version 4.2 (clang-425.0.27) (based on LLVM 3.2svn), and I'd like for it to work on Linux too, so presumably cc=gcc there).

EDIT My problem may be more complicated than I realized. I'm making this dynamic library in C, but trying to use it from within python. I can use libB.so (which has no dependencies) from within python no problem, and when I load libA.so from within python it finds it (see error above), it's just that at that point libA.so realizes it doesn't know where to find libB.so. If I understand your answers correctly below, the solutions depend on setting the linker path when you compile the executable, which in my case is in python.

Is there no way to tell libA.so where to look for libB.so when I compile it? I can do it afterward with install_name_tool on OSX, but is there not a way with the compiler that would work on both OSX and linux?

Threeply

Answer 1

The bottom line is that your final executable must know where your library resides. You can accomplish that 2 ways (1) exporting the LD_LIBRARY_PATH that includes the path to your library, or (2) using rpath so your executable knows where to find your library. Exporting the LD_LIBRARY_PATH generally looks something like this:

LD_LIBRARY_PATH=/path/to/your/lib:${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH

I prefer to use rpath. To use rpath compile your library as normal (example below for my extended test function library libetf.so)

gcc -fPIC -Wall -W -Werror -Wno-unused -c -o lib_etf.o lib_etf.c
gcc -o libetf.so.1.0 lib_etf.o -shared -Wl,-soname,libetf.so.1

Then to compile an executable making use of this library, you compile to object, then link the object with rpath given as a linker option. You would provide a path for both libA.so and libB.so in your case. Building my testso executable:

gcc -O2 -Wall -W -Wno-unused -c -o testetf.o testetf.c
gcc -o testso testetf.o -L/home/david/dev/src-c/lib/etf -Wl,-rpath=/home/david/dev/src-c/lib/etf -letf

Use ldd to confirm that the executable has correctly located your library:

$ ldd testso
        linux-vdso.so.1 (0x00007fffd79fe000)
        libetf.so.1 => /home/david/dev/src-c/lib/etf/libetf.so.1 (0x00007f4d1ef23000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f4d1eb75000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f4d1f126000)

Note: libetf.so.1 points to /home/david/dev/src-c/lib/etf/libetf.so.1.

Pricket

Answer 2

Although you're not building an executable yourself, the approach is almost the same except you'll set the rpath in libA.so rather than in an executable binary. When setting the rpath, use the special $ORIGIN string so that libB.so's location will always be relative to libA.so.

ld: Using -rpath,$ORIGIN inside a shared library (recursive)

For example:

cc -std=c99 -c -fPIC -I../libB/ -Wall libA.c
cc -std=c99 -shared libA.o -L../libB -lB -o libA.so -Wl,-rpath,\$ORIGIN/../libB

Note that $ORIGIN isn't an environment variable, it's interpreted directly by the runtime loader so is escaped when passed to the linker as shown above.

As an aside, if you prefer to follow a similar approach to what you're doing on OS X, you can change the rpath in the .so file after it's been compiled using the chrpath command - see:

Can I change 'rpath' in an already compiled binary?

[Edited to Add]

Well this is fun! Between reading various posts on -rpath and install_name and playing with various options I think I've found the combo that works. The main trick seems to be setting an install_name on libB.so along with a @loader_path on libA:

cc -shared -o libA.so libA.o -L../libB -lB -Wl,-rpath,@loader_path
cc -shared -o libB.so libB.o -install_name @loader_path/../libB/libB.so

Now libB.so is always located in ../libB/ relative to wherever libA.so is.

Masqat

Thanks so much for your help. I tried your suggestion but unfortunately it didn't work--from the first link, it seems like the way to get it to work is to add the -rpath-link option, but my compiler on OS X does not support it. The chrpath in the second link would also be great, but when I checked on my linux workstation (red hat), it wasn't installed. I was hoping to find a solution that made it so the end user wouldn't have to install any dependencies. Any other ideas? – Threeply

Looks like the $ORIGIN special string isn't supported on OS X... So you should stick with your plan when building on OS X. Use the trick outlined above when building on Linux, or install "chrpath" with: sudo yum install chrpath. Basically if you build it the corresponding way on each platform, your end-users won't have to do anything but download and run. – Masqat

Thanks, your above solution does work for me on OS X, but of course, not on Linux. I thought there must be a simple way to do this on both operating systems, but maybe the answer is that there isn't? – Threeply

No @DanT, I don't believe there's an identical way to do this on both systems. Although conceptually it's pretty much the same, the devil's in the details. This would be a good candidate for a Makefile to wrap up these differences, so that the end result on either build platform would be a simple "make" command. – Masqat

Your second link was exactly what I needed. – Harewood

Answer 3

Use the -rpath linker option.

-rpath dir

Add a directory to the runtime library search path. This is used when linking an ELF executable with shared objects. All -rpath arguments are concatenated and passed to the runtime linker, which uses them to locate shared objects at runtime. The -rpath option is also used when locating shared objects which are needed by shared objects explicitly included in the link; see the description of the -rpath-link option. If -rpath is not used when linking an ELF executable, the contents of the environment variable LD_RUN_PATH will be used if it is defined. The -rpath option may also be used on SunOS. By default, on SunOS, the linker will form a runtime search patch out of all the -L options it is given. If a -rpath option is used, the runtime search path will be formed exclusively using the -rpath options, ignoring the -L options. This can be useful when using gcc, which adds many -L options which may be on NFS mounted filesystems. For compatibility with other ELF linkers, if the -R option is followed by a directory name, rather than a file name, it is treated as the -rpath option.

Jaworski

Thanks for your help. When I man cc, I don't see any options for rpath. Unfortunately I have to use cc. Any ideas? – Threeply

When you say "cc" you need to be more specific. "cc" is system dependent. On Linux it is often just a link to "gcc". In any case, "rpath" is a linker option not a compiler option. So read it in: man ld – Jaworski

In case it's not clear. With gcc (and probably your cc) options can be passed to the linker with the -Wl option. – Jaworski

Thanks, I need all the help and explicitness I can get :) cc for me is Apple LLVM version 4.2 (clang-425.0.27) (based on LLVM 3.2svn). I tried cc -shared libA.o -Wl,-rpath ../libB -L../libB -lB -o libA.so. It makes fine, but when I try to load the library in another program I get the same error as above. – Threeply

Use rpath during compile: gcc -Wall -Wextra -L/path/to/your/lib -Wl,-rpath=/path/to/your/lib -o progname somefile.c The -Wl,... options says pass the following options to the linker (until the next space is encountered -- so no spaces in the library options string). – Pricket

Thanks so much. I think I wasn't specific enough in the question (I edited it above). I tried adding rpath at compile (but didn't do the step for the executable you suggest in your answer, see edit above), but when I run I get an error: dyld: Library not loaded: libA.so Reason: no suitable image found. Did find: libA.so: mach-o, but wrong filetype Trace/BPT trap: 5 – Threeply

I don't think your situation is more complicated than you originally stated. rpath applies equally to shared libraries and to executables. So it doesn't matter that you are running python (the loader doesn't care whether it is python or anything else). Have you retried with David's suggestion? That is, the gcc command line with -Wl that you posted is not correct. You cannot have a space after "-rpath". Did you fix that and retry? Also, I'm not sure whether or how relative paths are handled. You may want to try with an absolute path first. And if that works then experiment with relative paths. – Jaworski

Sorry for the late response--I didn't see there had been activity on the post. I did try David's suggestion, though I had to change the '=' after rpath to a comma for it to make. Thanks for the suggestion of trying first with an absolute path, but neither an absolute nor relative path worked :/ To be explicit: cc -std=c99 -shared libA.o -Wextra -Wl,-rpath,/abs/path/to/dir -L ../libB -lB -o libA.so – Threeply

Also, I would upvote all your great suggestions, but I don't seem to have enough reputation! – Threeply

Er, the space after -L above is also a typo--it was -L../libB – Threeply