Does libtool strip all options with -M?

Question

I'm trying to track down a failure to link with a mapfile on Solaris. The missing mapfile causes the following error when I try to run our self tests:

$ ./cryptestcwd v
ld.so.1: cryptestcwd: fatal:
/export/home/cryptopp/.libs/libcryptopp.so.6: hardware capability
(CA_SUNW_HW_1) unsupported: 0x4800000  [ AES SSE4.1 ]
Killed

I've gotten as far as this Automake rule. libcryptopp_la_LINK, which I believe is the shared object, is missing AM_LDFLAGS. AM_LDFLAGS holds the -M cryptopp.mapfile option.

libcryptopp_la_LINK = $(LIBTOOL) --tag=CXX $(AM_LIBTOOLFLAGS) \
    $(LIBTOOLFLAGS) --mode=link $(CXXLD) $(AM_CXXFLAGS) \
    $(CXXFLAGS) $(libcryptopp_la_LDFLAGS) $(LDFLAGS) -o $@

I tried to patch it with sed after configure runs:

libcryptopp_la_LINK = $(LIBTOOL) --tag=CXX $(AM_LIBTOOLFLAGS) \
    $(LIBTOOLFLAGS) --mode=link $(CXXLD) $(AM_CXXFLAGS) \
    $(CXXFLAGS) $(libcryptopp_la_LDFLAGS) -M cryptopp.mapfile $(LDFLAGS) -o $@

I confirmed the sed is successful, but the same test fails again. When the commands are invoked -M <mapfile> is missing.

The libtool manual talks about -M arguments on Cygwin, but not Solaris (and the discussion only applies to GCC, and not other compilers like IBM XL C/C++, Sun C/C++ and LLVM Clang):

Note that you also need to ensure that the standard Unix directories (like /bin, /lib, /usr, /etc) appear in the root of a drive. This means that you must install Cygwin itself into the C:/ root directory (or D:/, or E:/, etc)—instead of the recommended installation into C:/cygwin/. In addition, all file names used in the build system must be relative, symlinks should not be used within the source or build directory trees, and all -M* options to gcc except -MMD must be avoided.

There is no other mention of -M.

And there is no diagnostic, like "Removing -M <mapfile> options to Sun linker" or "Warning: libtool does not understand option -M <mapfile>".

My question is, does libtool discard -M and its arguments for some reason?

Spent

Answer 1

Libtool does drop some link options when creating a library. The manual explains:

When creating a shared library, but not when compiling or creating a program, libtool drops some flags from the command line provided by the user. This is done because flags unknown to libtool may interfere with library creation or require additional support from libtool, and because omitting flags is usually the conservative choice for a successful build.

Personally, I find the justification for this behavior to be a bit cavalier, and I furthermore think it warrants a warning from libtool when it occurs, but unless you care to raise an issue against it, that's pretty much moot.

Experimentation shows that -M is indeed among the options that libtool strips. In particular, if I specify LDFLAGS containing an -M option on the make command line then I can observe it echoed in the make output when it runs the libtool link, but not in libtool's own echo of the link command that is actually executed:

$ make LDFLAGS="-M mapfile"

/bin/sh ./libtool --tag=CC --mode=link gcc -g -O2 -M mapfile -o libmylib.la -rpath /usr/local/lib x.lo y.lo

libtool: link: gcc -shared -fPIC -DPIC .libs/x.o .libs/y.o -O2 -Wl,-soname -Wl,libmylib.so.0 -o .libs/libmylib.so.0.0.0

The libtool docs suggest two workarounds to pass link options that otherwise would be stripped:

• For bona fide linker options, you can use one or more -Wl, or -Xlinker options to pass your options through libtool and the linker driver to the linker itself. For example,

   LDFLAGS=-Wl,-M,cryptopp.mapfile
  

• For options directed specifically to the linker driver, the docs suggest adding the flags to the compiler driver command (CC="gcc -M mapfile"), but this is ineffective because the $(CC) variable is expanded by make to form the libtool command line, leaving any options expressed in it exposed to libtool for stripping.

Additionally, however, there is

• The -XCClinker option by which options can be passed through to the linker driver (as opposed to the linker itself), but its behavior seems a bit quirky: it seems to ignore options that don't start with a hyphen (such as the name of your map file).

Kerby

Thanks John. It tried AM_LDFLAGS=-Wl,-M,cryptopp.mapfile, but it was stripped it, too. (I don't mess with user CXXFLAGS or LDFLAGS. Everything goes in either AM_CXXFLAGS or AM_LDFLAGS). – Spent

Thanks again John. "unless you care to raise an issue against it, that's pretty much moot" - yeah, that was done yesterday. See Issue #29224: Solaris, lost mapfiles and silent failures. The thing is, I'm not sure its a libtool bug. Another component, like Autoconf or Automake, may be stripping it. My guess is, nothing will be done now, just like nothing was done in the past. History has a penchant for repeating itself. – Spent

I don't know what to tell you, @jww. I tested with libtool 2.4.2, and when the libtool command line contained the -Wl, options, they were passed on to the link command, as advertised. Neither autoconf nor automake could be stripping the flags, as the flags are never presented to them in the first place, and I am fairly confident that configure is not doing so. Do be aware, however, that AM_LDFLAGS is ignored for targets that have their own target-specific linker flags (libfoo_la_LDFLAGS) defined. – Kerby

Ack, thanks John. Here is the soup-to-nuts transcript: Crypto++ and Autotools. It was produced by autoreconf --force --install --warnings=all, CXX=/opt/solarisstudio12.4/bin/CC ./configure and then gmake -j 5. I can't find any diagnostic stating requested options are being dropped. It occurs silently, even with --warnings=all. – Spent

Regarding libfoo_la_LDFLAGS, yeah I suspected that early. The actual uniform name (for us) is libcryptopp_la. Our Makefile.am has both libcryptopp_la_SOURCES and libcryptopp_la_LDFLAGS. I also tried adding a lib_LTLDFLAGS for our lib_LTLIBRARIES (is this even legal?), but it produced the same results. – Spent

@jww, your soup-to-nuts example appears to show libtool passing the -M cryptopp.mapfile options through to the linker driver (line 985). That shows the actual command that libtool executes. If libtool were stripping the options then they would still appear on line 984, but not on line 985. Thus, if they are not being honored then it appears that that can only be a function of the linker driver, /opt/solarisstudio12.4/bin/CC. – Kerby

Thanks John. The rule that is failing is the one on line 974, which begins with libtool: link: /opt/solarisstudio12.4/bin/CC -G -hlibcryptopp.so.6 -o .libs/libcryptopp.so.6.0.0.... There is no -M cryptopp.mapfile supplied for the recipe. – Spent

@Spent at lines 973 and 974 we see that the -M cryptopp.mapfile are being presented as arguments to libtool but not passed on to the linker driver. That is, libtool is indeed stripping them. The recommended solution would be to instead present -Wl,-M,cryptopp.mapfile as an argument to libtool. If there is a libcryptopp_la_LDFLAGS Automake variable then that's where that flag should be recorded. – Kerby

If that does not work -- which perhaps it won't -- then the hack you already described may be the best available solution. If it happens, however, that the compiler permits concatenating those arguments into one then you could try instead passing the -XCClinker -Mcryptopp.mapfile option. – Kerby

Thanks again @John. -XCClinker fixed the issue. Here are the former lines 973 and 974: Crypto++ and Autotools (XCClinker). Now, the question for me... Why did libtool not get its options via -XCClibtool so all the -M options could pass unmolested? And why is dropping -M <mapfile> not producing a diagnostic? The dumb engineering decisions make me want to scream. Especially the ones that stay broke for two or three decades rather than being fixed. – Spent

@jww, I cannot answer "why" questions about libtool's design, except by referring you to the manual. I do agree that it seems a poor choice for libtool to silently drop options -- it ought to at least emit a warning about such drops, and ideally the diagnostic would include describing a mechanism for passing the options through. – Kerby

The fix was committed at 91700386556c. – Spent