Is it OK to use DYLD_LIBRARY_PATH on Mac OS X? And, what's the dynamic library search algorithm with it?

Question

I read some articles discouraging of the use of DYLD_LIBRARY_PATH, as the the path of dynamic library should be fixed using -install_name, @rpath, and @loader_path.

In terms of making a program that runs both on Linux and Mac OS X, DYLD_LIBRARY_PATH of Mac OS X does exactly what LD_LIBRARY_PATH of Linux. And, we can share (almost) the same make file that doesn't have the -install_name and @rpath.

• Is this OK to use DYLD_LIBRARY_PATH on Mac OS X?
• What's the dynamic library search algorithm with Mac OS X when the binary can't find the dynamic library? current directory -> DYLD_LIBRARY_PATH directories ... ?

Planography

My recommendation would be to avoid the use of both DYLD_LIBRARY_PATH and LD_LIBRARY_PATH alike. Using them at times is convenient but in my experience they are usually used incorrectly or as band-aids which tends to end up breaking things. I'm not experienced enough with the Mac way of searching for and specifying dynamically linked libraries to fully answer your question. Man gcc and man dyld should help you with the correct methods to use on the Mac platform. Make -f will help if you end up having to write 2 Makefiles, one for each platform. – Appellee

Answer 1

As you've noted, DYLD_LIBRARY_PATH behaves like LD_LIBRARY_PATH on other *nix. However, there is another environment variable you should look at called DYLD_FALLBACK_LIBRARY_PATH.

In general, these are (both on osx and linux) suggested only for development use as they can cause symbol lookup errors when you override with a library that does not have the same symbol table. A good example of this is when you attempt to override the default install of VecLib (e.g. blas lapack) with a custom install. This will cause symbol not found errors in applications linked to the system VecLib if DYLD_LIBRARY_PATH is set and the reverse (symbol lookup errors in custom applications) if it is not. This is due to the system blas/lapack not being a full implementation of the ATLAS libs.

DYLD_FALLBACK_LIBRARY_PATH will not produce these problems.

When installing libraries to a non-standard location, DYLD_FALLBACK_LIBRARY_PATH is much more sane. This will look for symbols in libraries provided in the default paths and if the symbol is not found there, fall back to the specified path.

The benefit is that this process will not cause symbol lookup errors in applications compiled against the default libraries.

In general, when libraries are installed to non-standard locations absolute paths should be specified which negates the ambiguity of the dynamic lookup.

Rivalee

Another benefit - if you set DYLD_LIBRARY_PATH, X11 will refuse to run. But if you set DYLD_FALLBACK_LIBRARY_PATH, you can have your custom libraries and X11 at the same time. Great answer! – Smitt

Something seems fishy here: According to the docs, the default library search path is used if DYLD_FALLBACK_LIBRARY_PATH is not set. Setting it will cause the default paths to be ignored, causing lots of problems. (At least this is what I find happening with Mountain Lion.) If you modify DYLD_FALLBACK_LIBRARY_PATH, then it seems you must be careful to add the defaults manually. (See man dlopen.) Am I missing something? – Electrotechnology

The Homebrew folks don't appreciate DYLD_FALLBACK_LIBRARY_PATH: github.com/mxcl/homebrew/issues/13463 – Readytowear

DYLD_FALLBACK_LIBRARY_PATH seems to be unusable in El Capitan cause to security reasons. Any ideas? – Irrepealable

when installing cuda we are told to set export DYLD_LIBRARY_PATH=/Developer/NVIDIA/CUDA-7.0/lib:$DYLD_LIBRARY_PATH – Sarita

Answer 2

DYLD_LIBRARY_PATH does not behave like LD_LIBRARY_PATH. The OS X dlopen documentation (https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man3/dlopen.3.html) specifies that when providing an absolute path, it will first look in locations specified by DYLD_LIBRARY_PATH:

When path contains a slash but is not a framework path (i.e. a full path or a partial path to a dylib), dlopen() searches the following until it finds a compatible Mach-O file: $DYLD_LIBRARY_PATH (with leaf name from path ), then the supplied path (using current working directory for relative paths), then $DYLD_FALLBACK_LIBRARY_PATH (with leaf name from path ).

In other words, if you set DYLD_LIBRARY_PATH to /Hello, the following two dlopen calls:

dlopen("/Hello/libfoo.so", RTLD_NOW);
dlopen("/World/libfoo.so", RTLD_NOW);

will both resolve to /Hello/libfoo.so. This is quite counter-intuitive, and represents a security vulnerability. Software using dlopen has no way to guarantee it is loading the correct libraries (perhaps override DYLD_LIBRARY_PATH in its own environment?)

Insulation

A lot of times I have seen $DYLIB_LIBRARY_PATH empty, and it seems like dyld is looking for the library in my cwd. Is this a consequence of the sentence "then the supplied path (using current working directory for relative paths)", in the case that the relative path is null? – Disciplinarian

Answer 3

For documentation on the dynamic link editor's environment variables and how they affect the search for dynamic libraries, man dyld.

DYLD_LIBRARY_PATH

This is a colon separated list of directories that contain libraries. The dynamic linker searches these directories before it searches the default locations for libraries. It allows you to test new versions of existing libraries.

For each library that a program uses, the dynamic linker looks for it in each directory in DYLD_LIBRARY_PATH in turn. If it still can't find the library, it then searches DYLD_FALLBACK_FRAMEWORK_PATH and DYLD_FALLBACK_LIBRARY_PATH in turn.

Use the -L option to otool(1). to discover the frameworks and shared libraries that the executable is linked against.

DYLD_FALLBACK_LIBRARY_PATH

This is a colon separated list of directories that contain libraries. It is used as the default location for libraries not found in their install path. By default, it is set to $(HOME)/lib:/usr/local/lib:/lib:/usr/lib.

DYLD_VERSIONED_LIBRARY_PATH

This is a colon separated list of directories that contain potential override libraries. The dynamic linker searches these directories for dynamic libraries. For each library found dyld looks at its LC_ID_DYLIB and gets the current_version and install name. Dyld then looks for the library at the install name path. Whichever has the larger current_version value will be used in the process whenever a dylib with that install name is required. This is similar to DYLD_LIBRARY_PATH except instead of always overriding, it only overrides is the supplied library is newer.

Generable

Note that otool -L tells you the libraries the executable is linked with not necessarily the ones that will actually be loaded when it is run. You can see this by setting DYLD_LIBRARY_PATH and running otool (the paths displayed for the libraries do not take DYLD_LIBRARY_PATH into account). – Crawfish