Does LD_LIBRARY_PATH really cause inconsistencies?

Question

The blog article "LD_LIBRARY_PATH – or: How to get yourself into trouble!" by the DTU Computing Center states:

3. Inconsistency: This is the most common problem. LD_LIBRARY_PATH forces an application to load a shared library it wasn’t linked against, and that is quite likely not compatible with the original version. This can either be very obvious, i.e. the application crashes, or it can lead to wrong results, if the picked up library not quite does what the original version would have done. Especially the latter is sometimes hard to debug.

Is this really true? LD_LIBRARY_PATH allows us to modify the search path for dynamic libraries, but does it really suppress the soname lookup that ensures binary compatibility?

(Because, by my interpretation, the Program Library HOWTO doesn't say any such thing.)

Or is the author unaware of the concept of maintaining a consistent library versioning scheme, and therefore assuming that one is not in use for the library in question?

Mingmingche

Where are you getting "does it really suppress the soname lookup" from? – Sunless

@trojanfoe: Binary compatibility ("is this .so the same library against which the program was originally linked?") is established by examining the soname. – Mingmingche

OK, but why the assertion that using $LD_LIBRARY_PATH avoids such binary compatibility tests? The statement "is quite likely not compatible with the original version" is referring to functional incompatibility and not binary incompatibility (AFAIK). – Sunless

@trojanfoe: I quote: This can either be very obvious, i.e. the application crashes I think it's clear that the blog author is asserting that the binary compat checks are overridden. Even if not, functional compatibility sits at a level below binary compatibility; you expect to be able to obtain minor functional differences by swapping out libraries with the same soname, e.g. to get a bug fix patch. – Mingmingche

Well the use of the word "compatible" is clearly ambiguous. In my experience the use of $LD_LIBRARY_PATH doesn't behave any differently when loading shared libraries than those obtained from /etc/ld.so.conf.d and the ldconfig cache. – Sunless

@trojanfoe: That's also my experience; hence the question. :) – Mingmingche

It seems that the author wanted to highlight rpath. – Penneypenni

Answer 1

I think the LD_LIBRARY should only be used for testing and not for a final installation, for it allows to use a specified library before the standard library location are used. But The linux documentation project says this about LD_LIBRARY_PATH and puts it more clear than I can.

3.3.1. LD_LIBRARY_PATH

You can temporarily substitute a different library for this particular execution. In Linux, the environment variable LD_LIBRARY_PATH is a colon-separated set of directories where libraries should be searched for first, before the standard set of directories; this is useful when debugging a new library or using a nonstandard library for special purposes. The environment variable LD_PRELOAD lists shared libraries with functions that override the standard set, just as /etc/ld.so.preload does. These are implemented by the loader /lib/ld-linux.so. I should note that, while LD_LIBRARY_PATH works on many Unix-like systems, it doesn't work on all; for example, this functionality is available on HP-UX but as the environment variable SHLIB_PATH, and on AIX this functionality is through the variable LIBPATH (with the same syntax, a colon-separated list).

LD_LIBRARY_PATH is handy for development and testing, but shouldn't be modified by an installation process for normal use by normal users; see ``Why LD_LIBRARY_PATH is Bad'' at http://www.visi.com/~barr/ldpath.html for an explanation of why. But it's still useful for development or testing, and for working around problems that can't be worked around otherwise. If you don't want to set the LD_LIBRARY_PATH environment variable, on Linux you can even invoke the program loader directly and pass it arguments. For example, the following will use the given PATH instead of the content of the environment variable LD_LIBRARY_PATH, and run the given executable:

/lib/ld-linux.so.2 --library-path PATH EXECUTABLE

Just executing ld-linux.so without arguments will give you more help on using this, but again, don't use this for normal use - these are all intended for debugging.

taken at august 13th 2013 from: http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

The link inside the document is old, found a the intended article here: http://xahlee.info/UnixResource_dir/_/ldpath.html

edit

You can override a library to which a program is linked to during building/installing because of the order in which ld.so will lookup a library to load at runtime. A library found at in a location specified inside de environmental variable LD_LIBRARY_PATH will be loaded instead of a library specified the default path ( /lib and the /usr/lib)

from man 8 ld.so

   ld.so loads the shared libraries needed by a program, prepares the pro‐
   gram to run, and then runs it.  Unless  explicitly  specified  via  the
   -static  option to ld during compilation, all Linux programs are incom‐
   plete and require further linking at run time.

   The necessary shared libraries needed by the program are  searched  for
   in the following order

   o      Using      the      environment     variable     LD_LIBRARY_PATH
          (LD_AOUT_LIBRARY_PATH for a.out programs).  Except if  the  exe‐
          cutable is a setuid/setgid binary, in which case it is ignored.

   o      From  the  cache file /etc/ld.so.cache which contains a compiled
          list of candidate libraries previously found  in  the  augmented
          library  path.  Libraries  installed  in  hardware  capabilities
          directories (see below) are prefered to other libraries.

   o      In the default path /lib, and then /usr/lib.

Natty

@LightnessRacesinOrbit I think the author is right. Since one can "override" the default library. I once obtained a machine vision camera that that was controlled by a close source library, which used the opensource xerces XML library. My repository would provide me with a version of xerces that was much newer than the one the camera library was linked against. Who knows how well the camera library will operate if at runtime the I get my system xerces library. Not to mention that I if it is not installed I need to export the LD_LIBRARY_PATH as well to run my apps wich is cumbersome. – Natty

The author claims that you can "override" the "default" library to the extent that binary compatibility is ignored. Your own answer and my experience, plus all documentation I've read, suggest the opposite. So which is it?! – Mingmingche

Who knows how well the camera library will operate if at runtime the I get my system xerces library The soname parameter knows. It's called binary compatibility, and it's achieved by maintaining a consistent version policy with the libraries you publish. Perhaps the blog author, then, is assuming that the library publisher isn't using a version policy at all, and that the application in question was built on that very system against that one version of the pre-installed library. Seems rather short-sighted to me. – Mingmingche

Please read my comments. I know that LD_LIBRARY_PATH changes the lookup path — that's its job. That's what it means. This question is about whether LD_LIBRARY_PATH also changes the way libraries within those paths are looked up. The blog author seems to think that, when LD_LIBRARY_PATH is set, the lookup will also ignore binary compatibility driven by a soname version policy, but I've seen nothing to back that up. – Mingmingche