Unit test that a class is non copyable, and other compile-time properties

Question

Is there a way to test compile-time errors, but without actually generating the error? For example, if I create a class which is non-copyable, I'd like to test the fact that trying to copy it will generate a compiler error, but I'd still like to execute the other runtime tests.

struct Foo {
    int value_;
    Foo(int value) : value_(value) {}
  private:
    Foo(const Foo&);
    const Foo& operator=(const Foo&);
};

int main()
{
    Foo f(12);
    assert(f.value_ == 12);
    assert(IS_COMPILER_ERROR(Foo copy(f);));
} // Would like this to compile and run fine.

I guess this can't be done as simply as that, but is there an idiomatic way to do this, or should I roll my own solution (maybe using scripts compiling separate tests files and testing the results?)?

N.B.: I took non-copyable only to illustrate my point, so I'm not interested in answers about using boost::noncopyable and such.

Marco

I'm guessing static_assert is out of scope too :P (boost.org/doc/libs/1_38_0/doc/html/boost_staticassert.html) – Swell

static_assert won't work. He wants to be able to assert that static_assert actually does cause a compile error. (or that anything else that supposedly triggers a compile error actually does that) – Downstroke

just posted a stupid idea that might help you make a framework of sorts... – Capture

Answer 1

You can do it using make. Each test will be a code snippet. Here's a working example with 2 tests for VC++. (I've used 2 batch files for pass test and fail test). I'm using GNU make here.

Makefile:

FAILTEST = .\failtest.bat
PASSTEST = .\passtest.bat

tests: must_fail_but_passes \
    must_pass_but_fails

must_fail_but_passes:
    @$(FAILTEST) [email protected]

must_pass_but_fails:
    @$(PASSTEST) [email protected]

must_pass_but_fails.cpp

struct Foo {
    int value_;
    Foo(void) : value_(0) {}
  private:
    Foo(const Foo&);
    const Foo& operator=(const Foo&);
};

int main()
{
    Foo f(12);
    return 0;
}

must_fail_but_passes.cpp

struct Foo {
    int value_;
    Foo(int value) : value_(value) {}
  private:
    Foo(const Foo&);
    const Foo& operator=(const Foo&);
};

int main()
{
    Foo f(12);
    return 0;
}

passtest.bat

@echo off
cl /nologo %1 >NUL
if %errorlevel% == 0 goto pass
@echo %1 FAILED
:pass

failtest.bat

@echo off
cl /nologo %1 >NUL
if not %errorlevel% == 0 goto pass
@echo %1 FAILED
:pass

Note that cl.exe (i.e. Visual Studio compiler) need to be in your path for this to "just work"

Have fun!

P.S. I doubt that this would make me famous though :-)

Cheesy

This editor is chopping half the Makefile and I don't know why. It should have 2 other lines: test2: must_pass_but_fails.cpp @$(PASSTEST) $ – Cheesy

Answer 2

BTW the only build system I know that allows such test out-of-the-box is Boost.Build:

Check here" http://beta.boost.org/boost-build2/doc/html/bbv2/builtins/testing.html

For example,

# in your Jamfile
compile-fail crappy.cpp ;

.

int main()
{
  my crappy cpp file
}

For more examples just grep -R compile-fail in your BOOST_TOP_DIR\libs directory.

Waneta

CMake has try_compile which can be used for this purpose as well. – Marco

Answer 3

13 years later I had this need once again and found a very easy way to do it with cmake/ctest:

• create a target to build the code that shouldn't compile. The target can link to the library under test, have compile definitions, etc. Make sure to exclude it from ALL since it is not supposed to compile.
• add a ctest test whose command builds the target
• reverse the test result

The code looks something like this:

add_library(test_foo_not_copyable STATIC EXCLUDE_FROM_ALL)
target_sources(test_foo_not_copyable PRIVATE test_foo_not_copyable.cpp)
target_link_libraries(test_foo_not_copyable PRIVATE foo)
add_test(NAME foo_not_copyable COMMAND "${CMAKE_COMMAND}" --build "${CMAKE_BINARY_DIR}" --target test_foo_not_copyable)
set_tests_properties(foo_not_copyable PROPERTIES WILL_FAIL TRUE)

With this you end up with a ctest test that will fail if test_foo_not_copyable compiles without error.

Of course, you can put that in a helper function to more easily list the various cases you want to test, for instance by generating the source file on the fly.

Marco

Answer 4

Unfortunately there is no easy way to test a compile error in the way you want to, I've also wanted to do this before.

Anyways if your tests are small enough you can write short uncompilable code, like your sample, and verify with a script if the errors generated are correct or not (again you just said it).

An example of this sort of thing would be Unix's configure scripts, in more than a few scripts I've seen them try to compile little samples to verify the version/abilities of the compiler, to configure the makefile correctly.

So at least you can know you're not alone. Now if you wrote a successful test framework for this sort of thing you'd probably become famous :)

Edit: You could possibly also use a #define that either tries or not compile uncompilable code something like this:

#ifdef _COMPILETEST
#define TRY_COMPILE(...) (__VA_ARG__)
#else
#define TRY_COMPILE(...)
#end

Note that this is something I just though about and there are probably many problems with this pattern, but it might serve as a seed for some better ideas.

Capture

Well, that's what I feared...at least I have a new project to work on in my (scarce) spare time :p! – Marco

Yeah its unfortunate. But remember you could become famous ;) – Capture

For the moment, I use something a little similar to your edit: I have a cpp file that either includes test_compile.h or test_runtime.cpp, depending on a compiler constant. – Marco

The problem with the define method is that if some user runs your tests, they first need to think of running two versions, which makes it no automatic testing any more, but second they will see that it fails to compile, but how can they verify that all the places that shouldn't compile have indeed not compiled? – Toland

@ufotds good points. I figured people would keep the builds on an automatic tool that does all build combinations. If you're working without a setup like this it does become a liason – Capture