PKG_CHECK_MODULES considered harmful?

Question

Various developers discourage the usage of the PKG_CHECK_MODULES (for example, in this answer) but there is no clear, comprehensive explanation of their reasons as far as I've looked for. So, I ask:

• Why would PKG_CHECK_MODULES be harmful?
• What are the alternatives?

I, for one, used it for the first time today. I found it invaluably useful, specially for dealing with pretty intricate library sets, such as GTK+, where I have all these dependencies:

-I/usr/lib/i386-linux-gnu/gtk-2.0/include -I/usr/include/atk-1.0
-I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 
-I/usr/include/gio-unix-2.0/ -I/usr/include/glib-2.0 
-I/usr/lib/i386-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 
-I/usr/include/freetype2 -I/usr/include/libpng12

-lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 
-lcairo -lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lgmodule-2.0
-lgthread-2.0 -lrt -lglib-2.0 

Possessive

Although the reasons are sound behind the alternative to pkg-config that William Pursell usually proposes, the reality is that there are entire platforms such as GTK that work the 'wrong' way and fixing it would require all of those libraries to change the directory they install themselves to. This would cause massive breakage of the build systems of existing applications. Since I don't think the 'wrong' way actually causes any harm, it's not worth changing. – Conventionalize

Also, pkg-config allows you to keep incompatible versions of libraries (such as GTK 2 and GTK 3) installed in parallel. Although I'm sure William Pursell has thought about this and will be happy to explain how to do it his way ;-) – Conventionalize

@Conventionalize No, I am strictly a non-gui person and have never dealt directly with gtk. But I believe it should be entirely possible to do things like "LDFLAGS=-L$( pkg-config --libs-only-L gtk+-2.0 ) CPPFLAGS=$( pkg-config --cflags gtk+-2.0 ) LIBS=$( pkg-config --libs-only-l gtk+-2.0 )", and those options can be placed in a config.site. To be clear, I have no objections to pkg-config, but I dislike PKG_CHECK_MODULES for the reasons outlined in my answer. – Voltage

@As to installing incompatible versions of libraries...that's what pkgsrc is for! – Voltage

@WilliamPursell pkgsrc as in NetBSD? – Matazzoni

@Matazzoni pkgsrc originated in NetBSD, but works on many platforms. – Voltage

@Conventionalize Comin' at you live from a full decade hence! Seems it was less the location than the tools, that was the problem. Funny thing is, in the end it wasn't that seismic to wean even big messes like Gtk off the autotools/pkg-config crack pipe. It just took motivation and a clear direction forward. Gtk4 and even Gtk3 now build "right" with Meson. (Which is... whatever. It's not CMake, but it's "fine". Rubs me the wrong way for some reason, always has, but it beats autotools. A pair of tweezers, two rusty nails, and a jar of marmalade is a better build configurator than autotools.) – Radionuclide

Answer 1

One significant problem with PKG_CHECK_MODULES is that it causes failures where it should not. If a user installs libfoo in /p/a/t/h and invokes a configure script with LDFLAGS=-L/p/a/t/h, the user is justified in expecting the configury to find libfoo. But, the user also must set PKG_CONFIG_PATH so that the configure script can find foo.pc in order for the configury to succeed, and in my opinion that is broken. It would be possible to invoke AC_CHECK_LIB and then only invoke PKG_CHECK_MODULES if the library is not found through the standard mechanism to avoid that problem. Another issue is that it is entirely possible for PKG_CHECK_MODULES to find a .pc file in which the information is inaccurate, causing the build to fail. In that case, it is necessary to invoke AC_CHECK_LIB after PKG_CHECK_MODULES.

In short, to use PKG_CHECK_MODULES correctly, it is necessary to invoke AC_CHECK_LIBS first, then conditionally invoke PKG_CHECK_MODULES, and then invoke AC_CHECK_LIBS again to validate the information found by PKG_CHECK_MODULES. All of this additional work on the part of the maintainer just to make it easier for users to install their libraries in non-standard location is absurd. The user should set up their tool chain to find libraries through the standard mechanisms.

-- EDIT --

To clarify, I am not suggesting that a package which uses a library which encourages the use of PKG_CHECK_MODULES should avoid using it in their configury. Rather, I am recommending that libraries not encourage its use and stop distributing .pc files. The problem that is trying to be solved by .pc files is better addressed at a higher level. The autotools are not a package management system, and this is a problem that should be addressed by a package management tool.

Voltage

A neat thing about the .pc is that it gives you the CFLAGS the library needs, such as -mms-bitfield. Also the libs required for static compilation, the conflicting modules, and various run-time details, such as location for module installation etc. So your suggestion to rely "on standard mechanism" doesn't seem to cover these cases. It's not only about finding a lib and a symbol. Although I agree that adding more link time checks could be helpful in some cases, that would also slow configure time a bit. Relying on a min. of sys correctness is similar to relying on some cached result – Matazzoni

All of that to say that back to the standard mechanism you propose isn't viable, relying on a correct system with PKG_CONFIG_LIBDIR make it trivial for me to cross-compile for various systems without any pain. – Matazzoni

@Matazzoni You can use pkg-config to populate CFLAGS, CPPFLAGS, LDFLAGS, and LIBS without using PKG_CHECK_MODULES, so you can get all of the benefits of pkg-config without relying on PKG_CHECK_MODULES. (See my comment to the question) – Voltage

@Matazzoni regarding "libs required for static compilation", they are very useful, but am I right that PKG_CHECK_MODULES doesn't support static linking (--static flag for pkg-config)? – Abohm

You say "use pkg-config to populate CFLAGS, CPPFLAGS, LDFLAGS, and LIBS without using PKG_CHECK_MODULES" but your answer says " I am recommending that libraries stop distributing .pc files". Huh? – Kulseth

If a library ships a pc file and recommends using pkg-config, then I suggest that maintainers of a package that depends on that library no use PKG_CHECK_MODULES but instead encourage their users to populate the user flags (via config.site, or some other mechanism) using pkg-config. But overall I would prefer that library maintainers stop relying on pkg-config. – Voltage

sometimes using PKG_CHECK_MODULES is the only sane way to check the version of the package in the configure script. Would you instead advocate grepping for the version in the headers, going through the package history to find a feature you need to test for to ensure the version is not too old? – Rior

@Dima Testing for features is exactly the right thing to do. That's one of the core philosophies of the tool chain. If you need a feature, test for it. If you believe you need version 3.5 of library foo and don't know what feature version 3.5 actually gives you, then you need to be better educated about the libraries you are depending on. – Voltage

this is an ideal world picture, where you know what features to test for. But if I, say, merely try to unvendor Foo of version X in a large project, it might take days to figure out what to test for, why and how... – Rior

it is also not unusual for a release to happen due to a bugfix, and a test for the bug is just too involved to be put into an autoconf macro. – Rior

If you actually need to test the version number, that is an easy configure check to make without relying on PKG_CHECK_MODULES (assuming the library writer makes it easy to query to version number.) But PKG_CHECK_MODULES does not give you what you want, as it cannot be made reliable without a great deal of effort. It is all too easy for PKG_CHECK_MODULES to return a false positive, creating a situation in which the build thinks it's using libfoo v 3.2 when in fact it uses libfoo v1.0. In the ideal world, PKG_CHECK_MODULES works well. I reality, it does not. Feature checks work. – Voltage

There are libraries where to find out anything about it will need a pkg-config call. One that comes to mind is libffi, which does not otherwise tell you where its headers are. – Rior

@Dima calling pkg-config to query the system is not the code smell I am worried about here. (Also note that this answer is 11 years old, and almost completely obsolete. My strong advice at this time is to stop using the autotools and use a modern build system). If you need to call pkg-config, do so. But don't use PKG_CHECK_MODULES. If you do use PKG_CHECK_MODULES (unless it has changed in the last 11 years!), you still need to invoke the appropriate macros to verify that the data in the discovered .pc file is accurate. – Voltage

Also, if you need to invoke pkg-config to find the library on a user's system, I believe that onus is on the user, not the package maintainer. If you choose to install libraries in /p/a/t/h, then you are responsible for telling the toolchain about /p/a/t/h. In the auto-tool world, the correct way to do that is probably to invoke pkg-config in your CONFIG_SITE. That is not the responsibility of the package maintainer. – Voltage

@WilliamPursell - maybe 11 years ago something like cmake and basel looked fresh and crispy and better than autotools, but in 2023 still autotools in many cases provide the cleanest set of macros for the job. Also, I don't think you're right regarding "onus is on the user, not the package maintainer". E.g. on Gentoo libffi's include path can't be found without pkg-config, full stop. That's where it's installed by the OS packaging system. pkg-config with its configuration macros is here to stay, whether you like it or not. – Rior

@DimaPasechnik As I said, pkg-config is fine. If your configure script passes but the build fails, then your package is broken. If you use PKG_CHECK_MODULES in configure.ac as a mechanism for invoking pkg-config and the user's system is such that PKG_CHECK_MODULES returns bogus information which causes the build to fail, then your package is broken. To avoid that, your package must validate the information. – Voltage

Answer 2

There is a blog post here that goes into a bit of detail on the bad side of PKG_CHECK_MODULES:

http://tirania.org/blog/archive/2012/Oct-20.html

or this stackoverflow question:

Using the pkg-config macro PKG_CHECK_MODULES failing

It essentially boils down to: It causes very unhelpful errors if someone is trying to run autoconf and doesn't have pkg-config installed. Here's an example of an error I got today running autoconf && ./configure:

./configure: line 5283: syntax error near unexpected token `FFMPEG,'
./configure: line 5283: `   PKG_CHECK_MODULES(FFMPEG, libavutil libavformat libavcodec libswscale, HAVE_FFMPEG=yes)'

To a user/developer just trying to compile a package, this doesn't scream "you need to install pkg-config".

If (as the article suggests) you just call pkg-config directly, you get more helpful errors, e.g.:

AC_SUBST(MONO_LIBS)
AC_SUBST(MONO_CFLAGS)
if pkg-config --atleast-version=2.10 mono; then
   MONO_CFLAGS=`pkg-config --cflags mono`
   MONO_LIBS=`pkg-config --libs mono`
else
   AC_MSG_ERROR(Get your mono from www.go-mono.com)
fi

Edit: in a comment Helmut Grohne says:

Please don't call pkg-config directly. Doing so breaks cross compilation. Use AC_PATH_TOOL(PKG_CONFIG,pkg-config) or better PKG_PROG_PKG_CONFIG to discover which $PKG_CONFIG to use.

I would presume this is correct and you should follow his suggestion, but I've not personally tried it.

Other people suggest not using pkg-config at all; that's a separate issue.

Caputo

I'm confused as to why having pkg-config or not would affect how ./configure, a /bin/sh script, should behave. Does /bin/sh gain the ability to interpret things like PKG_CHECK_MODULES when pkg-config is installed? – Hewes

@Hewes It's the 'autoconf' stage where things go wrong - if pkg-config is not installed, the PKG_CHECK_MODULES macro doesn't get expanded (because that macro is provided by pkg-config). autoconf returns success, but generates an invalid configure script. The issue is not so much that this fails, but that running configure fails with a really unhelpful error message that doesn't make the end-user think "ah, I need to install pkg-config and re-run autoconf". – Caputo

It is normally the responsibility of the maintainer to run auto(re)conf to generate the configure scripts. If pkg-config is necessary, then it should be mentioned in the pre-build instructions (or put into autogen.sh as a check). – Cam

Please don't call pkg-config directly. Doing so breaks cross compilation. Use AC_PATH_TOOL(PKG_CONFIG,pkg-config) or better PKG_PROG_PKG_CONFIG to discover which $PKG_CONFIG to use. – Gilliette

@HelmutGrohne thanks! I was unaware that might be an issue; I've added your comment as an edit to the post. – Caputo

In general, though, regular users are not going to be running auto(re)conf. They'll use a release tarball shipped by the developer that has everything already generated, and they can just run configure, which will error out with an actually-helpful error message if the pkg-config binary is not installed. – Smirch