dlopen a dynamic library from a static library, when the dynamic library uses symbols of the static one

Question

This question is closely related to dlopen a dynamic library from a static library linux C++, but contains a further complication (and uses C++ instead of C):

I have an application that links against a static library (.a) and that library uses the dlopen function to load dynamic libraries (.so). In addition, the dynamic libraries call functions defined in the static one.

Is there a way to compile this without linking the dynamic libraries against the static one or vice versa?

Here comes what I tried so far, slightly modifying the example from the related question:

app.cpp:

#include "staticlib.hpp"
#include <iostream>
int main()
{
    std::cout << "and the magic number is: " << doSomethingDynamicish() << std::endl;
    return 0;
}

staticlib.hpp:

#ifndef __STATICLIB_H__
#define __STATICLIB_H__

int doSomethingDynamicish();
int doSomethingBoring();
#endif

staticlib.cpp:

#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>
int doSomethingDynamicish()
{
    void* handle = dlopen("./libdynlib.so",RTLD_NOW);
    if(!handle)
    {
        std::cout << "could not dlopen: " << dlerror() << std::endl;
        return 0;
    }

    typedef int(*dynamicfnc)();
    dynamicfnc func = (dynamicfnc)dlsym(handle,"GetMeANumber");
    const char* err = dlerror();
    if(err)
    {
        std::cout << "could not dlsym: " <<err << std::endl;
        return 0;
    }
    return func();
}

staticlib2.cpp:

#include "staticlib.hpp"
#include "dlfcn.h"
#include <iostream>

int doSomethingBoring()

{
    std::cout << "This function is so boring." << std::endl;
    return 0;

}

dynlib.cpp:

#include "staticlib.hpp"

extern "C" int GetMeANumber()
{
    doSomethingBoring();
    return 1337;
}

and build:

g++ -c -o staticlib.o staticlib.cpp
g++ -c -o staticlib2.o staticlib2.cpp
ar rv libstaticlib.a staticlib.o staticlib2.o
ranlib libstaticlib.a
g++ -rdynamic -o app app.cpp libstaticlib.a -ldl 
g++ -fPIC -shared -o libdynlib.so dynlib.cpp

When I run it with ./app I get

could not dlopen: ./libdynlib.so: undefined symbol: _Z17doSomethingBoringv
and the magic number is: 0

Gruver

When it comes to static libraries in a POSIX environment, the only linking involved is when your application links with the static library. The static library itself it not linked. In fact, a static library is nothing more than an archive of object files, that are linked into your application like any other object file. Also, the point of using dlopen is to load a module during runtime, so you don't need to link with the modules you dlopen. – Munsey

Thanks for the clarification, I am pretty new to the world of linking. However, my question remains: How do I make the symbols defined in this archive (or in any object file) available to the dynamically loaded library? – Gruver

Oh by the way, remember that C++ uses name mangling. That means the symbol GetMeANumber will not be named like that in the object file. To skip name mangling you have to make the function extern "C", as in extern "C" int GetMeANumber() { ... } – Munsey

Answer 1

From the dlopen manual page:

If the executable was linked with the flag "-rdynamic" (or, synonymously, "--export-dynamic"), then the global symbols in the executable will also be used to resolve references in a dynamically loaded library.

That means that for the application to export its symbols for use in the dynamic library, you have to link your application with the -rdynamic flag.

---

Besides the problem described above, there is another problem and that has to do with the static library: The problem is namely that since the doSomethingBoring function is not called in your main program, the object file staticlib2.o from the static library is not linked.

The answer can be found in e.g. this old question, which tells you to add the --whole-archive linker flag:

g++ -rdynamic -o app app.cpp -L. \
    -Wl,--whole-archive -lstaticlib \
    -Wl,--no-whole-archive -ldl

Munsey

I had already stumbled upon and tried this flag. Unfortunately, linking the application with g++ -rdynamic -o app app.cpp libstaticlib.a -ldl leads to exactly the same error for my example. – Gruver

@Lea I don't have much experience with dlopen and friends (just some simple experimenting), but have you tried passing the RTLD_LAZY flag to dlopen? – Munsey

@ Joachim Pileborg: Thanks for your continued interest. Yes, I have also tried that, it just makes the error at a different place: ./app: symbol lookup error: ./libdynlib.so: undefined symbol: _Z17doSomethingBoringv. – Gruver

@Lea I figured out the problem, and you're right in a way that it has to do with the static library. Updated answer. – Munsey

Just to note that this is a purely Linux answer. On most Unices, at least in the past, this was the default behavior; there was no need for any special options to the linker. – Stroy

FWIW: Posix actually requires the traditional behavior; Linux varies from Posix/Unix in this regard. – Stroy

--whole-archive flag causes "multiple definition" errors. – Tiberias

@MertMertce Then you need to look at your definitions, to make sure you don't have any duplicates. If you can't figure it out, then perhaps post a question about it? – Munsey

@JoachimPileborg actually, I do have a question about my problem: #32649837 – Tiberias