C++ unit testing framework [closed]

Question

I use the Boost Test framework for my C++ code but there are two problems with it that are probably common to all C++ test frameworks:

• There is no way to create automatic test stubs (by extracting public functions from selected classes for example).
• You cannot run a single test - you have to run the entire 'suite' of tests (unless you create lots of different test projects I guess).

Does anyone know of a better testing framework or am I forever to be jealous of the test tools available to Java/.NET developers?

Vicinity

Answer 1

I just responded to a very similar question. I ended up using Noel Llopis' UnitTest++. I liked it more than boost::test because it didn't insist on implementing the main program of the test harness with a macro - it can plug into whatever executable you create. It does suffer from the same encumbrance of boost::test in that it requires a library to be linked in. I've used CxxTest, and it does come closer than anything else in C++-land to automatically generating tests (though it requires Perl to be part of your build system to do this). C++ just does not provide the reflection hooks that the .NET languages and Java do. The MsTest tools in Visual Studio Team System - Developer's Edition will auto-generate test stubs of unmanaged C++, but the methods have to be exported from a DLL to do this, so it does not work with static libraries. Other test frameworks in the .NET world may have this ability too, but I'm not familiar with any of those. So right now we use UnitTest++ for unmanaged C++ and I'm currently deciding between MsTest and NUnit for the managed libraries.

Lowpressure

Answer 2

I've just pushed my own framework, CATCH, out there. It's still under development but I believe it already surpasses most other frameworks. Different people have different criteria but I've tried to cover most ground without too many trade-offs. Take a look at my linked blog entry for a taster. My top five features are:

• Header only
• Auto registration of function and method based tests
• Decomposes standard C++ expressions into LHS and RHS (so you don't need a whole family of assert macros).
• Support for nested sections within a function based fixture
• Name tests using natural language - function/ method names are generated

It doesn't do generation of stubs - but that's a fairly specialised area. I think Isolator++ is the first tool to truly pull that off. Note that Mocking/ stubbing frameworks are usually independent of unit testing frameworks. CATCH works particularly well with mock objects as test state is not passed around by context.

It also has Objective-C bindings.

[update]

Just happened back across this answer of mine from a few years ago. Thanks for all the great comments! Obviously Catch has developed on a lot in that time. It now has support for BDD style testing (given/ when/ then), tags, now in a single header, and loads of internal improvements and refinements (e.g. richer command line, clear and expressive output etc). A more up-to-date blog post is here.

Belanger

Excellent test suite. – Ginelle

(+1) One the best and easiest to work with, even in its early stages. The only thing missing is mockups, although I require them rarely so I have no complaints. – Lucchesi

+1. Single header include == awesome. – Aliphatic

This rocks. The way you use expression templates to substitute the various assertion macros of other test suites is ingenious. – Advance

Can it do parameterised tests? Also a floating point approximate equality assertion would be nice. – Rutilant

Absolutely great ! Is there a way to speed up the build process? So I can get faster red-green-refactor-cycles? It takes 15sec to build .. I know it's not that much in absolute time but it feels like a lot. Anyways THANK YOU !!! – Cubeb

I didn't use it, but for me it looks like the single best c++ unit test framework. A pity that it's not integrated in something like boost, which people usually use. – Vial

doctest is my reimplementation of Catch with a huge focus on compilation speed - checkout the FAQ to see whats different between them – Cole

@Belanger - how does one mock or stub c++ objects using catch? I cant find any documentation on this. I'm trying to use catch to test where Obj-C objects call into a c++ interface and I would like to assert that certain invocations occurred in the c++ object. – Effluent

Wow, this is an old SO post. @Helium3 there's no mocking facility built in to Catch but HippoMocks and Trompeloeil at both excellent mocking frameworks that work well with Catch. – Belanger

Thanks Phil, will take a look. – Effluent

Answer 3

Take a look at the Google C++ Testing Framework.

It's used by Google for all of their in-house C++ projects, so it must be pretty good.

http://googletesting.blogspot.com/2008/07/announcing-new-google-c-testing.html

http://code.google.com/p/googletest

Jillene

Answer 4

Boost.Test does allow to run test case by name. Or test suite. Or several of them.

Boost.Test does NOT insists on implementing main, though it does make it easy to do so.

Boost.Test is NOT necessary to use as a library. It has single header variant.

Tokyo

Answer 5

I'm a big fan of UnitTest++, it's very lightweight, but does the job. You can run single tests there easily.

Chadburn

The link to UnitTest++ is no longer working. :( – Pneumogastric

It moved from sourceforge to github. – Butler

Answer 6

Great question! A few years ago I looked around forever for something worth using and came up short. I was looking for something that was very lightweight and did not require me to link in some libraries... you know something I could get up and running in minutes.

However, I persisted and ended up running across cxxtest.

From the website:

• Doesn't require RTTI
• Doesn't require member template functions
• Doesn't require exception handling
• Doesn't require any external libraries (including memory management, file/console I/O, graphics libraries)
• Is distributed entirely as a set of header files (and a python script).

Wow... super simple! Include a header file, derive from the Test class and you're off and running. We've been using this for the past four years and I've still yet to find anything that I'm more pleased with.

Cheapen

Maybe you should try CATCH. I say, "maybe" because it does require exceptions and member template functions. The rest is the same as your list (without the Python script). See my answer for more info. – Belanger

That link is no longer valid, they migrated to github. github.com/CxxTest/cxxtest – Member

Answer 7

Try WinUnit. It sounds excellent, and is recommended by John Robbins.

Pecuniary

I'm using it, fairly light and easy to get started. Test suites are organised into DLLs which are then loaded by the winunit exec. Possibility to launch single test, see winunit.codeplex.com/… – Jourdain

Answer 8

I like the Boost unit test framework, principally because it is very lightweight.

• I never heard of a unit-test framework that would generate stubs. I am generally quite unconvinced by code generation, if only because it gets obsolete very quickly. Maybe it becomes useful when you have a large number of classes?

• A proponent of Test Driven Development would probably say that it is fundamental that you run the whole test suite every time, as to make sure that you have not introduced a regression. If running all the tests take too much time, maybe your tests are too big, or make too many calls to CPU intensive functions that should be mocked out? If it remains a problem, a thin wrapper around the boost unit-tests should allow you to pick your tests, and would probably be quicker than learn another framework and port all your tests.

Stressful

Generating stubs is the only way you'll be able to reduce the overhead of writing tests to a few keystrokes. – Pentachlorophenol

Answer 9

http://groups.google.com/group/googletestframework, but it's pretty new

Sanguinolent

Answer 10

I'm using tut-framework

Biflagellate

Answer 11

Aeryn is another framework worth looking at

Sills

Link is dead. Is this the same project? sourceforge.net/projects/aeryn – Member

Answer 12

Visual Studio has a built-in unit testing framework, this is a great link to setting up a test project for win32 console application:

http://codeketchup.blogspot.ie/2012/12/unit-test-for-unmanaged-c-in-visual.html

If you are working on a static DLL project it is much easier to set up as other have pointed out external tesing frameworks like GTest and Boost have extra features.

Cochineal

Answer 13

CppUnit was the original homage to JUnit.

Vilma

Being a fan of JUnit, this looks like it will work for me. – Elainaelaine

So many other answers in this thread have been upvoted more often than mine. You'd do yourself a service by looking into some of the other recommendations. My post is 6 years old, and it's based on my experience w/ CppUnit that's now 13 years old. – Vilma

Correct! There are plenty of great options here. However, I looked up CppUnit and it seemed to be just fine for what I wanted. :-) – Elainaelaine

Answer 14

Eclipse/JUnit is a solid package for java, and there are C++ extensions/equivalents for both. It can do what you're talking about. Of course, you'd have to change IDEs...

Fullfaced

Answer 15

I too am a fan of UnitTest++.

The snag is that the source distribution contains almost 40 seperate files. This is absurd. Managing the source control and build tasks for a simple project is dominated by looking after all these unit testing files.

I have modified UnitTest++ so that it can be integrated with a project by adding one .h and .cpp file. This I have dubbed "cutest". Details are at http://ravenspoint.com/blog/index.php?entry=entry080704-063557

It does not automatically generate test stubs, as asked for in the original question. I cannot help thinking that such a feature would be more trouble than it is worth, generating vast amounts of useless code "testing" accessor functions.

Euchromatin

Answer 16

I would imagine automatically stubbing out test functions would be more of a function (of scripts for the framework or) the development environment in question. Supposedly CodeGear's C++Builder applications will quickly generate test code for user functions.

Dream

Answer 17

Andrew Marlow's Fructose library's worth checking out... http://fructose.sourceforge.net/

I recall his documents containing a fairly detailed analysis and comparison of other offering at the time he wrote Fructose, but can't find a URL direct to that document.

Amon

Answer 18

I'm trying out Igloo, also a header only C++ test suite, even it's two included dependencies are header only.

So, it's pretty straightforward and simple. Besides the included example on github, there's examples and more details at the main site, igloo-testing.org. I'll update this later as I get more experience with it and other frameworks.

Member