Force serial execution for specific targets in CMake

Question

In my CMake project I have several targets which simply run a certain set of unit tests (for example, runTestsForA, runTestsForB and runTestsForC). I also have a target, tests, that depends on all of these unit test targets, so I can run them with a single command.

I'm using CLion is my IDE, which tries to use parallel make builds by default (which I want and am also doing on the Continuous Integration server). However, it looks like the tests are running in parallel too now and some tests are not made for this (they use a local loopback to do some magic with sockets), which causes them to fail.. sometimes.

That is why I would like to force serial execution for some/all of the dependencies of my tests target. Unfortunately the CMake documentation did not help me, when I was searching information on how to do this. Which brings me to my questions: is this at all possible and how can it be done if it is?

Sula

Answer 1

Instead of manual tests target declaration you can use CTest tool. Use add_test command to create test targets, then CMake will automatically create tests target that will run all tests:

enable_testing()
add_test(NAME TestsForA COMMAND <command>)
add_test(NAME TestsForB COMMAND <command>)
set_tests_properties(TestsForA TestsForB  PROPERTIES RUN_SERIAL TRUE)

After that you can run make tests or ctest -j8 . in your build tree. The tests will be serialized.

More information can be found at:

• http://www.cmake.org/cmake/help/v3.2/command/add_test.html
• http://www.cmake.org/cmake/help/v3.2/command/enable_testing.html
• http://www.cmake.org/cmake/help/v3.2/command/set_tests_properties.html
• http://www.cmake.org/cmake/help/v3.2/manual/cmake-properties.7.html#properties-on-tests
• http://www.cmake.org/cmake/help/v3.2/manual/ctest.1.html

Reset

In case you haven't used add_test to define your test suite and if you are using something like add_custom_target(runTestsForA) ... Use add_dependencies(runTestsForA runTestsForB) – Penhall

@MuraliPaluru That would be hard to do work is I can not know which test targets have been defined. – Sula

@jet47 This will probably suit my use case, but what if it weren't tests but some other tools. What would the right approach be then? – Sula

You still can use add_test, if this tools can be applied as post-build actions (like static analysis via cppcheck). Otherwise, it seems you will have to use @MuraliPaluru suggestion with add_dependencies. – Reset

We ended up not using this. We are working on adjusting our tests so they can be ran in parallel. In a test project however, this answer did cause the wanted behavior. :) – Sula

I am currently using the add_dependencies() solution, but it has the disadvantage that the build-system will build the runTestsForB target before the runTestsForA when you build the runTestsForA target which is usually not what you want. – Haunt

Answer 2

Perhaps it's not the best solution for controlling test execution, but, generally I believe you can use the CMake JOB_POOLS feature to restrict the amount of parallelism for some of the CMake targets in a project.

First you declare named pool(s) and the number of parallel jobs each pool supports:

set_property(GLOBAL PROPERTY JOB_POOLS two_jobs=2 ten_jobs=10)

You can then assign targets to those pools:

set_property(TARGET myexe PROPERTY JOB_POOL_COMPILE ten_jobs)

Abroms

According to the CMake documentation, JOB_POOL_COMPILE works only when you use Ninja to build. – Valedictorian