How to skip a BOOST unit test?

Question

How can I skip a BOOST unit test? I would like to programatically skip some of my unit tests depending on, (for instance) the platform on which I am executing them. My current solution is:

#define REQUIRE_LINUX char * os_cpu = getenv("OS_CPU"); if ( os_cpu != "Linux-x86_64" ) return;

BOOST_AUTO_TEST_CASE(onlylinux) {
    REQUIRE_LINUX
    ...
    the rest of the test code.
}

(note that our build environment sets the variable OS_CPU). This seems ugly and error-prone, and also like the silent skips could cause users to be skipping tests without knowing about it.

How can I cleanly skip boost unit tests based on arbitrary logic?

Sayer

Answer 1

Instead of skipping them, you can prevent to register them. To achieve that you can use the manual test registration of boost.test:

#include <boost/test/included/unit_test.hpp>
using namespace boost::unit_test;

//____________________________________________________________________________//

void only_linux_test()
{
    ...
}

//____________________________________________________________________________//

test_suite*
init_unit_test_suite( int argc, char* argv[] ) 
{
    if(/* is linux */)
        framework::master_test_suite().
            add( BOOST_TEST_CASE( &only_linux_test ) );

    return 0;
}

See http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/utf/user-guide/test-organization/manual-nullary-test-case.html for more information

Another possibility would be to use #ifdef ... #endif with BOOST_AUTO_TEST_CASE. Therfor you need a definition that is defined if you are compiling the code on the target platform.

#ifdef PLATFORM_IS_LINUX

BOOST_AUTO_TEST_CASE(onlyLinux)
{
    ...
}
#endif

This definition can for example be set by your build environment.

Chandigarh

I can't use ifdefs, some of these criteria must be determine at execute-time. I'll probably use something like your registration suggestion, thanks. – Sayer

Answer 2

BOOST_AUTO_TEST_CASE(a_test_name,*boost::unit_test::disabled())

{
   ...
}

Acrimony

I don't find this second parameter in the documentation, are you sure it is available? – Promiscuity

totally yes boost.org/doc/libs/1_65_1/libs/test/doc/html/boost_test/… – Acrimony

Cool thank you - and now i see that your second parameter was actually a link to the docs, doh. – Promiscuity

Answer 3

Use enable_if / enable / precondition decorators.

boost::test_tools::assertion_result is_linux(boost::unit_test::test_unit_id)
{
  return isLinux;
}


BOOST_AUTO_TEST_SUITE(MyTestSuite)

BOOST_AUTO_TEST_CASE(MyTestCase,
                     * boost::unit_test::precondition(is_linux))
{...}

precondition is evaluated at runtime, enable, enable_if at compile time.

See: http://www.boost.org/doc/libs/1_61_0/libs/test/doc/html/boost_test/tests_organization/enabling.html

Garik

Great! This is exactly what I was looking for (back in the day). I'll check it out with a couple of our tests. – Sayer

@Horus, Is there a way to use a fixture in a precondition? – Baptlsta

@Baptlsta You can have a look at my code github.com/precice/precice/blob/develop/src/testing/Testing.hpp. I use a decorator to remove tests from the test tree, it might work for fixtures, too. Be advised, that my code likely uses unofficial boost API. – Garik

Answer 4

Registering test cases manually is tedious, boring and error-prone. If it is only by platform that you need to distinguish test cases, then I would simply not compile the irrelevant test cases on the platforms where they don't matter by configuring my build system. Alternately, you can use Boost.Predef that defines the necessary preprocessor symbols for everything you might want to know about OS, compiler, etc., that will allow you to #ifdef out certain tests.

Finally, if this criteria can only be known at runtime and is independent of the platform on which you're running then I would group the tests that depend on particular criteria into suites and adjust the command-line used by the build to run only those suites based on the runtime criteria.

Tippett

Answer 5

Here's what I do to disable, for example, interactive test units that aren't suitable to be run in some situations.

I put this into my precompiled header:

// preconditions for running some tests
#define PRECOND_ALLOWED_TO_TAKE_FOCUS \
    *boost::unit_test::precondition([](boost::unit_test::test_unit_id) { return std::getenv("CI_NOFOCUS") == NULL; })

And change the targeted tests like:

BOOST_AUTO_TEST_CASE(MyTestCase, PRECOND_ALLOWED_TO_TAKE_FOCUS)
{ ...

Postgraduate