Boost unit test dynamic linking on Ubuntu
Asked Answered
S

2

5

I am trying to build a unit test using Boost's unit test framework. I would like to dynamically link test suite libraries with the auto generated test module that Boost provides. Here is the basic construction I've been using:

test_main.cpp:

#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MAIN

#include <boost/test/unit_test.hpp>

lib_case.cpp:

#define BOOST_TEST_DYN_LINK
#include <boost/test/unit_test.hpp>


BOOST_AUTO_TEST_SUITE( test_lib )

BOOST_AUTO_TEST_CASE( test_lib_case ) {
    BOOST_ASSERT(true);
}
BOOST_AUTO_TEST_SUITE_END()

Makefile:

    all: unittest unittest2 unittest3

    lib_case.o: lib_case.cpp
        g++ -g -c -Wall -fPIC lib_case.cpp -o lib_case.o

    libcase.so: lib_case.o
        g++ -shared -Wl,-soname,libcase.so -o libcase.so lib_case.o

    unittest: libcase.so
        g++ -o unittest test_main.cpp -L. -lcase -lboost_unit_test_framework

    unittest2: test_main.cpp lib_case.cpp
        g++ -o unittest2 test_main.cpp lib_case.cpp -lboost_unit_test_framework

    unittest3: lib_case.o
        g++ -o unittest3 test_main.cpp lib_case.o -lboost_unit_test_framework

Testing on Ubuntu 14.04, all executables compile and link without error.

'unittest' fails to execute the 'test_lib' suite claiming that the setup failed, but 'unittest2' and 'unittest3' succeed:

$./unittest
Test setup error: test tree is empty
$./unittest2
Running 1 test case...
*** No errors detected
$./unittest3
Running 1 test case...
*** No errors detected

Now for the headache: All unittest* run the test suite on Fedora 20.

In looking at the dependency lists for 'unittest', I do see that 'libcase.so' is not listed in the Ubuntu version, but is in the Fedora 20 version. I have played around with re-ordering the dependencies, using absolute paths for the SO, and changing versions of Boost (1.54 and 1.55). Nothing worked.

Any ideas on what may preventing the 'libcase.so' from be linked on Ubuntu 14.04 but not on Fedora 20? Am I missing some magic compiler/linker flag?

Update:

Sehe's comment and answer have helped to narrow down the problem a bit more. If I'm understanding Boost's dynamically linked UTF implementation (at least as of 1.54/55) correctly, then the framework provides a test case manager singleton. Every test case will automagically register with the manager upon construction.

I think the problem is that, for whatever reason, linking on Ubuntu 'optimizes out' the static global variable used for the singleton instance of the manager during the linking of the library to the binary. In effect, it does not link the two singleton instances despite sharing the same global static variable. It treats them as two separate instances.

I followed the steps described in Multiple instances of singleton across shared libraries on Linux to inspect the library and binary files. Unlike in their case, the -rdynamic option does not solve my problem.

I did some more testing and found this interesting. If you preload libcase.so object, unittest works on Ubuntu. Even though the libcase.so does not appear in its ldd listing. I feel like this is expected because the singleton for the manager was 'preloaded' when unittest runs it will link with it.

$ LD_PRELOAD=/absolute/path/to/libcase.so ./unittest
Running 1 test case ...

Still have no idea why Ubuntu does not want to link as expected/intended, where Fedora does. Reading this tutorial (specifically the 'Comparison to the Microsoft DLL' section) makes me think Ubuntu is following a Windows linking pattern.

Shark answered 17/6, 2015 at 17:31 Comment(6)
Apparently the default version or mode-of-operation of ld on Fedora doesn't optimize unused objects as aggressively.Straphanger
The thing is that lib_case.o is used. So Fedora is correct in not optimizing them out.Shark
Sigh. I don't know what you're hoping to achieve. I can change my answer, but it won't help you. Fact of life: behavior of linkers and tool-chains varies wildly (much more wildly than this) across platforms. I have not taken the time to get to the bottom of this enough to know whether there is a bug. But, you can assume that there aren't any glaring flaws if 47 thousand packages in their repository are built using the tool chain.Straphanger
Long story short, you either file a bug or work with it. Anyways you'll have to work with it for now.Straphanger
The goal is to be able dynamically load unit test modules. Basically, I want to create a shared library of unit tests per feature of my software. I can then create a single binary that dynamically links with all of the shared libraries. Subsequently, I can add new unit tests for a specific feature, rebuild the feature's library, deploy it, and simply re-run the unit test binary without having to recompile the unit test binary or any other feature's unit test library. At least in theory.Shark
Yup. I know all about that goal. I've settled for "oh well. Statically loaded is OK too". Although I specifically remember having it working in both Win32 and AIX using IBM's XlC++ 7/8.0 series (I stopped bothering with later projects). So ... If you are prepared to do what it takes, chances are you will succeed. (Prepare to read about non-portable attributes, linker options, explicit symbol visibility etc.)Straphanger
S
8

Got it!

Ubuntu seems to use the --as-needed linker option by default, where as Fedora may not. Turning it off will add libcase.so library to the needed list for unittest. After deploying the library (or using LD_LIBRARY_PATH) the unittest works now.

unittest: libcase.so
        g++ -o unittest test_main.cpp -Wl,--no-as-needed -L. -lcase -lboost_unit_test_framework

Figures it was something simple ...

Shark answered 20/6, 2015 at 12:48 Comment(1)
Awesome. Let's upvote this a bit because that's one handy option not enough know about :)Straphanger
S
0

The problem is that the lib_case.o is optimized out, because there are no references to anything contained there.

If all the references refer from the test case definition to the unittest framework (for self-registration) but none back, compiling and linking the test main axes the "unused" library.

I could reproduce this on my system (Ubuntu 14). Here's a simple hack to show how you could fix it by forcing a reference (in this case to a global variable called force_reference_this_object_file.

Notes

  • of course you would normally declare that global in a header file
  • you'll find that you need to either deploy the libcase.so or use LD_LIBRARY_PATH to include ./ in the library path

lib_case.cpp

#define BOOST_TEST_DYN_LINK
#include <boost/test/unit_test.hpp>

int force_reference_this_object_file = 42;

BOOST_AUTO_TEST_SUITE( test_lib )

BOOST_AUTO_TEST_CASE( test_lib_case ) {
    BOOST_ASSERT(true);
}
BOOST_AUTO_TEST_SUITE_END()

test_main.cpp

#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MAIN

#include <boost/test/unit_test.hpp>

extern int force_reference_this_object_file;

namespace {
    struct Local
    {
        int& ref_;
        Local() : ref_(force_reference_this_object_file) {}
    };

    static Local hack_;
}

Makefile

all: unittest unittest3

CPPFLAGS=-Wall -fPIC
LDFLAGS+=-L ~/WORK/pocpp/3rdparty/boost_1_58_0/stage/lib/

%.o: %.cpp
    g++ -c $(CPPFLAGS) $^ -o $@

libcase.so: lib_case.o
    g++ $(CPPFLAGS) -shared -Wl,-soname,$@ -o $@ $^

unittest: test_main.o | libcase.so
    #g++ $(CPPFLAGS) -o $@ $< $(LDFLAGS) -L. -lcase -lboost_unit_test_framework
    g++ $(CPPFLAGS) -o $@ $< $(LDFLAGS) ./libcase.so -lboost_unit_test_framework

unittest3: test_main.o lib_case.o
    g++ $(CPPFLAGS) -o $@ $^ $(LDFLAGS) -lboost_unit_test_framework
Straphanger answered 18/6, 2015 at 8:33 Comment(2)
Thanks for the hack. I had figured that there was a difference in the ld optimizations between the OSs. I guess I'm a little confused as to why there though. If the same program is being used by both then the outputs should, in theory, be the same. Furthermore, any difference in operation should be controlled by some configuration parameter. I'm would like to avoid using this hack because I would have to force every library (thinking ahead to a multiple library linking). Seems to defeat the purpose of being able to use the dynamic linking.Shark
I'm not qualified to answer that. I encourage you to find that configuration parameter (keep in mind it could be more complicated than that, but in principle I agree). I'm just happy to understand what's going on.Straphanger

© 2022 - 2024 — McMap. All rights reserved.