How to build *.so module in Automake and a libtool-using project?

Question

I have the same problem as others have:

• I have a *.la file generated by libtool in an Automake project (e.g. module.la),
• but I need the *.so of it to use it for dlopen() (eg. module.so).

But: project is configured and built with --disable-shared to make sure the created main binary is one big statically linked program, e.g. main.x (easier for deployment and debugging). Thus *.so files are not created.

The program main.x is a huge framework-like application which is capable of loading extensions (modules) via dlopen() -- despite it being linked statically.

This works fine when I build module.so by hand. But putting this to work in Makefile.am seems impossible to me. Yes, I can write lib_LTLIBRARIES, but with my standard --disable-shared I do not get a *.so file.

lib_LTLIBRARIES = module.la
module_so_SOURCES = module.cpp

The file module.la is created, which dlopen() refuses to load (of course).

I tried to put rules into Makefile.am building it manually and that works:

# Makefile.am (yes, .am)
all: mm_cpp_logger.so

SUFFIXES = .so

%.so: %.cpp
    $(CXX) $(CXXFLAGS) -fPIC -fpic -c -I $(top_srcdir)/include -o $@  $<

%.so: %.o
    $(CXX) $(LDFLAGS) -shared -fPIC -fpic -o $@  $<

But this can only be a workaround. I do not get all the nice auto-features like dependency-checking and installation.

How can I build module.so with still building the main program with --disable-shared (or with the same effect) in the Makefile.am-way?

• can I postprocess *.la files to *.so files with a special automake rule?
• can I tweak the lib_LTLIBRARIES process to create *.so files in any case?

Nolte

Answer 1

What you are looking for is called a module. You can tell Autotools to create a static binary (executable) by adding -all-static to the LDFLAGS of the application. I think this is the preferred way over using --disable-shared configure flag (which really is aimed at the libraries rather than the executable)

Something like this should do the trick:

AM_CPPFLAGS=-I$(top_srcdir)/include

lib_LTLIBRARIES = module.la
module_la_LDFLAGS = -module -avoid-version -shared
module_la_SOURCES = mm_cpp_logger.cpp

bin_PROGRAMS = application
application_LDFLAGS = -all-static
application_SOURCES = main.cpp

The .so file will (as usual) end up in the .libs/ subdirectory (unless you install it, of course).

And you can build both your application and plugins in one go (even with a single Makefile.am), so there is no need to call configure multiple times.

The use of -fPIC (and friends) should be auto-detected by Autotools.

---

Update: here's a little trick to make the shared-libraries available where you expect them. Since all shlibs end up in .libs/, it is sometimes nice to have them in a non-hidden directory.

The following makefile snippet creates convenience links (on platforms that support symlinks; otherwise they are copied). Simply adding the snippet to your makefile (i usually use an -include convenience-link.mk) should be enough (you might need an AC_PROG_LN_S in your configure.ac)

.PHONY: convenience-link clean-convenience-link

convenience-link: $(lib_LTLIBRARIES)
    @for soname in `echo | $(EGREP) "^dlname=" $^ | $(SED) -e "s|^dlname='\(.*\)'|\1|"`; do  \
        echo "$$soname: creating convenience link from $(abs_builddir)/.libs to $(top_builddir)"; \
        rm -f $(top_builddir)/$$soname ; \
        test -e $(abs_builddir)/.libs/$$soname && \
        cd $(top_builddir) && \
        $(LN_S) $(abs_builddir)/.libs/$$soname $$soname || true;\
    done 

clean-convenience-link:
    @for soname in `echo | $(EGREP) "^dlname=" $(lib_LTLIBRARIES) | $(SED) -e "s|^dlname='\(.*\)'|\1|"`; do  \
        echo "$$soname: cleaning convenience links"; \
        test -L $(top_builddir)/$$soname && rm -f $(top_builddir)/$$soname || true; \
    done 
        
all-local:: convenience-link

clean-local:: clean-convenience-link

Trumpery

Man, I had to jump through hoops by now because of libtool issues. I replaced -all-static at ''application'' with -static-libtool-libs at my other "convenience" libraries (not the one under our discussion) -- I had to do that because dl_blahblah()-stuff crashed the program when I did -all-static (I am quite sure that it was the culprit). Anyway: Alas, I still do get the .libs/mm_cpp_logger.so -- which I find a bit awkward when writing tests for the module. Loading it from .libs/ seems... wrong. Is it normal that it always gets there, no matter what I do? – Nolte

i updated my answer for a possible solution for getting the the libs available outside of .libs – Pharsalus

@umläute: nice solution! Unfotunately, the clean part is not working, since all the files containing "dlname=(.*)" are deleted before the part is started (so grep does not return anything). I have hard-copied your snippet into my Makefile.am file – Gratification

Answer 2

I've solved a similar problem using the noinst_LTLIBRARIES macro.

The noinst_LTLIBRARIES macro creates static, non installable libraries to be only used internally. all noinst_LTLIBRARIES static libraries are created also if you specify the --disable-static configure option.

lib_LTLIBRARIES = libtokenclient.la
noinst_LTLIBRARIES = libtokenclient_static.la 

libtokenclient_la_SOURCES = $(TOKEN_SERVER_CLIENT_SOURCES) cDynlib.c cDynlib.h token_mod.h
libtokenclient_la_CFLAGS = @BASE_CFLAGS@
libtokenclient_la_CXXFLAGS = $(libtokenclient_la_CFLAGS)
libtokenclient_la_LIBADD = @B_BASE_OS_LIBS@
libtokenclient_la_LDFLAGS = @LT_PLUGIN_LIBS_FLAGS@ @LIBS_FLAGS@ $(TOKEN_SERVER_CLIENT_EXPORT_SYMBOLS)

libtokenclient_static_la_SOURCES = $(libtokenclient_la_SOURCES)
libtokenclient_static_la_CFLAGS = $(libtokenclient_la_CFLAGS)
libtokenclient_static_la_CXXFLAGS = $(libtokenclient_static_la_CFLAGS)

token_test_SOURCES = $(TEST_SOURCES)
token_test_LDADD = @B_BASE_OS_LIBS@ libtokenclient_static.la
token_test_CFLAGS = @BASE_CFLAGS@
token_test_CXXFLAGS = $(token_test_CFLAGS)

I use noinst_LTLIBRARIES static libraries for 2 reasons:

1. to speed up compile time I create static libraries to be used as intermediate containers for code that shall be linked against more than once: the code is compiled just once, otherwise automake would compile same source files once for each target
2. to statically link the code to some executable

Electroluminescence

Answer 3

One thing that could work according to the libtool documentation for LT_INIT is to partition your build into two packages: the main application and the plugins. That way you could (in theory) invoke:

./configure --enable-shared=plugins

and things would work the way you would expect.

Shanda

Your answer gave me good keywords to google for and I found a tutorial. Alas, that also seems to assume I build everything with --enable-shared. Has a couple of good hints, though. But the doc you pointed to does not mention "plugins". Packaged? This means an a sub-configure.in, right? Yes, that might work. The directory with the module would get its own configure.in, I guess? – Nolte

Yes, exactly. By "plugin" I mean a dynamically loaded extension. They (the main application and extensions) would both get some kind of sub-configure to configure them. Maybe you could just get away with only one sub-configure for the extensions. – Shanda