Conflict between dynamic linking priority in OSX?

Question

There is a dynamic-linking-conflict between different libjpeg dynamic libraries on OSX. First there is a standard native libJPEG.dylib (in /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/). But if you are using MacPorts, you can also have a port-related libjpeg.dylib in (in /opt/local/lib). The latter may for example have been installed as a dependency for some other port.

This creates a problem when you link against your system libJPEG (which is preferred). Then if /opt/local/lib is in DYLD_LIBRARY_PATH, that path will be prioritised when searching for a dynamic lib, resulting in a runtime error when loading symbols:

dyld: Symbol not found: __cg_jpeg_resync_to_restart
 Referenced from:
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
 Expected in: /opt/local/lib/libJPEG.dylib
in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
Trace/BPT trap: 5

So I have two questions (likely related):

1. What is a good way of solving the actual problem (removing /opt/local/lib from DYLD_LIBRARY_PATH obviously solves it but creates problems for other dependencies)?

2. What other paths are searched for dynamic libs (I.e. Where is the "/System/Library" path specified) and why does DYLD_LIBRARY_PATH rank higher priority-wise?

Lichenin

Answer 1

You should not set library paths using DYLD_LIBRARY_PATH. As you've discovered, that tends to explode. Executables and libraries should have their library requirements built into them at link time. Use otool -L to find out what the file is looking for:

$ otool -L /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO:
    /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO (compatibility version 1.0.0, current version 1.0.0)
    ...
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)

For an example of one of my homebrew-built programs:

$ otool -L /usr/local/bin/gifcolor
/usr/local/bin/gifcolor:
    /usr/local/Cellar/giflib/4.1.6/lib/libgif.4.dylib (compatibility version 6.0.0, current version 6.6.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)

Note that it references /usr/local. If you've built it in such a way that it references the wrong library, I recommend rebuilding and pointing it to the correctly library.

If that's impossible, it is possible to edit what path is used using install_name_tool, but there are cases where this doesn't work, such as if the new path is longer than the old path and you didn't link it with -header_pad_max_install_names. Rebuilding with the correct path is preferred.

Note that there are a few "special" paths available that allow libraries to be found relative to their loader. See @executable_path/ and its kin in the dyld(1) man page.

Horst

Thank you for the reply. otooland install_name_tool are useful indeed. The reason for why I am not using the full path for all libraries when linking is because the binary will be distributed to systems without brew or ports. I would like to use system libraries as much as possible and the remaining libraries will be packaged with the app. But during development the directory structure looks different than that of the installed app, so I link non-system libraries with relative paths (as in the version to be distributed), but have /opt/local/ in DYLD until I'm ready for release/packaging. – Lichenin

But it seems that the right way to go is to try to avoid DYLD_LIBRARY_PATH altogether. – Lichenin

I also develop system executables for Mac that require a specific tree structure. The solution I use is two-fold: I install into a tree that looks like my final tree using INSTALL_PATH, DSTROOT, and DEPLOYMENT_LOCATION in Xcode. For anything relying on relative paths (@executable_path), that's enough. For things that require absolute paths, I have a symlink on my dev box from my "real" install path to my Xcode DSTROOT. – Horst

Answer 2

I experienced similar problem while using OpenCV in MacOS El Capitan. Solved the problem using the solution in the link

Solution is to delete some dlylibs in the /usr/local/lib directory and create symbolic links to related files /System/Library/Frameworks/ImageIO.framework/Resources/

cd /usr/local/lib
rm libgif.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libGIF.dylib libGIF.dylib
rm libjpeg.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libJPEG.dylib libJPEG.dylib
rm libtiff.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libTIFF.dylib libTIFF.dylib
rm libpng.dylib
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libPng.dylib libPng.dylib

Lewellen

LIFE SAVING!!!! Thanks a lot. BTW, in my system, there was no libgif.dylib to remove. – Joe

Thanks! libgif was on my system. – Mussel

This is dangerous. Actual macOS applications that link to that location expecting the Apple provided dylibs will break. – Questionable

Link in answer is dead (excuse pun) – Bubalo

@Questionable Very dangerous - but effective - do you have a solution, especially e.g. during a *nix upgrade? – Bubalo

@Lewellen - This works - and has worked for me before - but it comes back to bite one the next time one does a major system upgrade.... – Bubalo

Answer 3

If using Qt Creator, you have to uncheck the Add build library search path to DYLD_LIBRARY_PATH and DYLD_FRAMEWORK_PATH option from the Run section in the Projects tab:

Sidesman

Thanks for this reply, that helped me a lot! I guess that this checkbox was introduced in the latest version of QtCreator. – Jambeau

@Steakfly, Funny enough, it helped me too after posting this. – Sidesman

Answer 4

I had a similar error when trying to run Apache Celix on macOS Sierra If you use Homebrew to install libjpeg, libtiff, libpng which may confuse the linker to use macOS imageIO library. Simple fix is unlink those libs:

brew unlink libpng
brew unlink libtiff
brew unlink libjpeg

Re-link those libs whenever we need to:

brew link libpng
brew link libtiff
brew link libjpeg

Crist

I had this problem while working with Qt creator. The above is a good solution that's easily reversible. If you have problem with libgif in addition you also need brew unlink giflib. – Spectrograph

This is much better/safer than manually setting symlinks as the other answer suggests. – Bjork

Answer 5

I had a similar error, and i solved putting the following variable in my bash_profile:

export DYLD_LIBRARY_PATH=/usr/lib/:$DYLD_LIBRARY_PATH

Brownout

Answer 6

I followed the instructions mdemirst suggested and it fixed my issue. I am using OS X Sierra.

I created a gist just in case someone else runs into the same issue.

Gist to fix Spidermonkey errors

Shearwater