libtool linkage - global state initalization of convenience libraries

Question

I have a setup that does not work, and I have no idea what I am doing wrong here - I am trying to convert a project from handcrafted Makefiles to autotools, and I think I have most of it set up correctly, as the application and all its convenience libraries builds and links correctly, but there is some trouble with the global state initializers of the convenience libraries.

Some of the libraries follow a pattern like this in the code:

// in global scope of somemodule.cpp
namespace {
  bool registered  = ModuleShare::registerModule<SomeModule>("SomeModule");
}

this code, along with the actual module source, is compiled into a convenience library using libtool

// libsomething Makefile.am
noinst_LTLIBRARIES = libsomething.la

libsomething_la_SOURCES = \
  [ ... ]
  moduleshare.cpp moduleshare.h \
  somemodule.cpp somemodule.h \
  [ ... ] 

and this library is built, and referenced in the application Makefile.am as follows:

// someapp Makefile.am
bin_PROGRAMS = someapp

someapp_SOURCES = someapp.c someapp.h
someapp_CPPFLAGS = -I ${top_srcdir}/something
someapp_LDADD = ${top_srcdir}/something/libsomething.la

I have modified ModuleShare::registerModule to verify it is not called:

template<typename T>
static bool registerModule(const std::string &module){
  printf("%s\n", module.c_str());

  [ ... ]

  return true;
}

What could be the reason for this?

EDIT:

At this point, I have figured out that this problem is related to the linker that is allowed to remove unused symbols during linkage. If I link manually using --whole-archive, everything works as expected.

Coming from a C background, I also tried

static void
__attribute__((constructor))
register (void)
{
  ModuleShare::registerModule<SomeModule>("SomeModule");
}

but that also does not produce the behaviour that I expected, which is wierd, considering that I rely on this construct a lot in my private C projects.

At this point, I am open to suggestions in any direction. I know that libtool does not provide per-library flags in LDADD, and I can't, and simply don't want to compile everything with --whole-archive just to get rid of these symptoms. I have only limited control over the codebase, but I think what I really need to ask here is, what is a good and reliable way to initialize program state in a convenience library using autotools?

EDIT2:

I think I am a step closer - it seems that the application code has no calls to the convenience library, and hence the linker omits it. The application calls into the library only via a templated function defined in a header file of SomeModule, which relies on the static initializers called in the convenience libraries. This dependency reversion is screwing over the whole build.

Still, I am unsure how to solve this :/

Thanks, Andy

Ernaernald

Is registered ever used in the real code? Does it work in the handcrafted Makefiles? IIRC, convenience libs are statically linked by libtool. If registered isn't used, the linker might be throwing out the initialization code as well. – Agretha

@Agretha yes, with the handcrafted makefiles it works. I will check if registered is used in a minute – Ernaernald

@Agretha you were right, registered not used anywhere. However, referencing it in the module code does not help. I also added a function declared __attribute__((constructor)) which is not called as well. – Ernaernald

You could use the xxx_LDFLAGS variable for -Wl,--whole-archive. – Merilyn

If I've understood your need, this may provide a method. It's ugly though. If you defined 'registered' as extern "C" you might be able to tell the linker that the symbol was undefined in your main executable (usually that's a -u option to your linker). – Peralta

It sounds like you're not actually using the libraries from your application code, but you need them to be linked. Are you just trying to get the build to work before you start using the libraries? If so, you might try just using the library and the compiler will figure out that you need it. Otherwise I'm way misunderstanding the problem. – Fad

Answer 1

This answer might require too many code changes for you, but I'll mention it. As I mentioned before, the convenience library model is a kind of static linkage where libraries are collated together before final linkage to an encapsulating library. Only in this case the "library" is an executable. So I'd imagine that stuff in the unnamed namespace (static variables essentially), especially unreferenced variables would be removed. With the above code, it worked as I kind of expected it to.

I was able to get registerModule to print it's message without special linker tricks in a convenience library like this:

somemodule.cpp

// in global scope
namespace registration {
  bool somemodule  = ModuleShare::registerModule<SomeModule>("SomeModule");
}

someapp.cpp

// somewhere
namespace registration {
    extern bool somemodule;
    ... // and all the other registration bools
}

// in some code that does initialization, possibly
if (!(registration::somemodule && ...)) {
    // consequences of initialization failure
}

Agretha

Hey, thanks for your suggestion - however, the solution you propose defeats the purpose that this initialization model has - a kind of a plug-and-play registration of modules without altering the application code. long story short - I wouldn't get this past the code reviewers of the project :) – Ernaernald

I feel this is the most concise and useful answer - I can't use it because of the limitations put upon me, but someone else might find this useful. – Ernaernald

Answer 2

Since you're using autotools, you might be in a situation where exclusive use of gcc is feasible. In that case you can apply the `used' attribute to tell gcc to force the linker to include the symbol:

namespace {
    __attribute__ ((used))
    bool registered  = ModuleShare::registerModule<SomeModule>("SomeModule");
}

Captainship

Thanks for your suggestion, but it does not work. :( please see my updated question. – Ernaernald

Answer 3

Instead of using --whole-archive, have you tried to just add -u registered?

As ld manual states:

-u symbol
   --undefined=symbol
       Force symbol to be entered in the output file as an undefined symbol.  Doing this may, for example, trigger linking of additional modules from standard libraries.  -u may be repeated with different option arguments to
       enter additional undefined symbols.  This option is equivalent to the "EXTERN" linker script command.

Edit: I think that what you try to achieve is quite similar to Qt's plugin management. This article details it a bit. And this is 4.8's official documentation.

Taking into account that convenience libraries are statically build, maybe, it would be enough to create a dummy instance inside this convenience library (or a dummy use of registered) and declare it as extern where you use it (this is what Q_EXPORT_PLUGIN2 and Q_IMPORT_PLUGIN are doing).

Pencel

adding -u registered to LDFLAGS changes nothing, unfortunately :/ – Ernaernald

okay, I rechecked. If I declare the register code as extern C, it does indeed work. however - how does this behave in respect of multiple convenience libraries, each of which defines a symbol registered? would it just guarantee to link at least one of them, or all? Thanks! :) – Ernaernald

@AndreasGrapentin I'm really sorry but, I think, I'm misunderstanding something. I mean: The symbol that you want to make accessible from convenience libraries to other code is registered or registerModule method? – Pencel

I don't actually need the symbols to be accessible, instead, I need the static code of the library to be executed, which does not happen, besause the linker does not realize that the application needs this library. – Ernaernald

@AndreasGrapentin Actually, this is the problem, as static elements are, by default, internally linked and no use is done inside convenience library, linker removes it. I think that what you want to achieve is quite similar to Qt's plugin management, please, check my update. About registered, most probably, you would have redefinition problems. Something that you can try is just create a reference to your static code inside the convenience class and declare it as extern where you really want to use it. – Pencel