A multi-thread implementation of CppUnit?

Question

Could someone point me to a version of CppUnit that would allow to launch the tests in separate threads?

The idea is that, because many of our tests are quite CPU heavy (but are not multi-thread and, of course, are independant one from the other), it would allow us to run the tests much more quickly on today's multi-core machines. Currently, it takes around 5 minutes to run all the tests. It would be great to be able to reduce this to 1 or 2 minutes...

Nalchik

Answer 1

You think five minutes is a long time to wait for tests to complete! Try several hours. I had motivation for the following.

Using Boost threads, CppUnit threading is pretty easy. CppUnit already has some hooks for synchronization so the following should make it thread safe:

class Mutex : public CPPUNIT_NS::SynchronizedObject::SynchronizationObject
{
public:
    void lock() { this->mutex->lock(); }
    void unlock() { this->mutex->unlock(); }
private:
    boost::mutex mutex; 
};

With this, you can modify your test runner to make your TestResult thread safe. Just write something like CPPUNIT_NS::TestResult testResult(new Mutex);. Now here's a threaded test suite:

class TestSuiteThreaded : public CPPUNIT_NS::TestSuite
{
public:
    TestSuiteThreaded(std::string name = "", int nThreads = 0)
        : TestSuite(name)
        , nThreads(nThreads ? nThreads : boost::thread::hardware_concurrency())
    {
    }
    void doRunChildTests(CPPUNIT_NS::TestResult *controller)
    {
        ThreadPool pool(this->nThreads);
        for (int i=0; i < getChildTestCount(); ++i)
        {
            pool.add(
                boost::bind(threadFunction, getChildTestAt(i)
                , controller));
        }
    }
private:
    static void threadFunction(
        CPPUNIT_NS::Test *test, 
        CPPUNIT_NS::TestResult *controller)
    {
        test->run(controller);
    }
    const int nThreads;
};

You may well need a macro for easy use of the threaded test suite. You should be able to use TestSuiteThreaded suite either as a top level suite or a suite comprising multiple methods of the same text fixture. Here's how you do the latter - put this in place of CPPUNIT_TEST_SUITE_END. Some of this is pasted from CppUnit so please respect the license:

#define CPPUNIT_TEST_SUITE_END_THREADED(n)                                     \
    }                                                                          \
    static CPPUNIT_NS::TestSuite *suite()                                      \
    {                                                                          \
      const CPPUNIT_NS::TestNamer &namer = getTestNamer__();                   \
      std::auto_ptr<CPPUNIT_NS::TestSuite> suite(                              \
         new CPPUNIT_NS::TestSuiteThreaded( namer.getFixtureName(), n));       \
      CPPUNIT_NS::ConcretTestFixtureFactory<TestFixtureType> factory;          \
      CPPUNIT_NS::TestSuiteBuilderContextBase context( *suite.get(),           \
                               namer,                                          \
                               factory );                                      \
      TestFixtureType::addTestsToSuite( context );                             \
      return suite.release();                                                  \
    }                                                                          \
  private: /* dummy typedef so that the macro can still end with ';'*/         \
    typedef int CppUnitDummyTypedefForSemiColonEnding__

Now there is the small matter of a ThreadPool. I tried using various publicly available ones with no success. My company has one but I'm unable to publish it here. So roll your own - thread pools are pretty easy and fun to make, with help from Boost. Here is the interface expected by TestSuiteThreaded:

class ThreadPool
{
public:
    // Create thread pool, launching n worker threads
    ThreadPool(unsigned n); 

    // Join all worker threads and clean up
    ~ThreadPool();

    // You can have add() do one of two things.  Both will work:
    // Either: push a new task to the back of the threadpool's work queue
    // Or: block until a worker is free then assign task to that thread
    void add(boost::function0<void> task);
};

I leave this as an exercise for the reader. Have fun!

Vo

Answer 2

Given how many answers to this question you've gotten, especially as compared with the number of upvotes, I doubt anybody has made a good multi-threaded unit testing framework, no matter how great an idea it is. This looks like a great opportunity for someone to make a name for themselves developing something inordinately useful.

Fauna

One could consider a launcher that splits the unit-tests into pools, and then launches a number of test-runner processes with parameters that specifies what pool to execute. This would allow multi-threaded testing of code, without the code needing to be multi-threaded. – Shank

@Rolf Kristensen: There is still a problem though. Tests may depend on external state, on files and such. The tests in the separate pools might stomp on eachother by accident. – Fauna