CTest with multiple commands

Question

I'm building some tests using CTest. Usually, I can set up the test by simply the line:

ADD_TEST(Test_Name executable args)

However, I've run into a problem, I have some tests that require two commands to be run in order for it to work, is there any way I can run two programs within a single ctest, or am I required to create a new test for each?

Thank you.

Simonize

Adding an add_test for each command worked for me. The tests are executed in order. – Misprize

Answer 1

The add_test command only accepts one executable, but you can run any executable that is really a script. To do this in a cross platform way, write the script in CMake itself. CMake has the -P option for running arbitrary chunks of CMake scripting language when you run make or make test, rather than at Makefile generation time.

Sadly you can't pass arguments to such a script. But you can set variables to values, which is just as good.

This script you can call runtests.cmake, it runs the commands CMD1 and CMD2 and checks each for a non-zero return code, returning out of CMake itself with an error if that happens:

macro(EXEC_CHECK CMD)
    execute_process(COMMAND ${CMD} RESULT_VARIABLE CMD_RESULT)
    if(CMD_RESULT)
        message(FATAL_ERROR "Error running ${CMD}")
    endif()
endmacro()
exec_check(${CMD1})
exec_check(${CMD2})

... and then add your test cases like so:

add_executable(test1 test1.c)
add_executable(test2 test2.c)
add_test(NAME test
    COMMAND ${CMAKE_COMMAND}
            -DCMD1=$<TARGET_FILE:test1>
            -DCMD2=$<TARGET_FILE:test2>
    -P ${CMAKE_CURRENT_SOURCE_DIR}/runtests.cmake)

$<TARGET_FILE:test1> gets expanded to the full path to the executable at build-file generation time. When you run make test or equivalent this will run "cmake -P runtests.cmake" setting the CMD1 and CMD2 variables to the appropriate test programs. The script will then execute your 2 programs in sequence. If either of the test programs fail, the whole test fails. If you need to see the output of the test, you can run make test ARGS=-V

Thole

Hmm...this looks like good, although I need the script to be run during testing time, not build time or makefile generation time. – Simonize

Oh, by "build time" I meant when you run any make command, including make test. As opposed to when you run cmake to generate the Makefiles. I've clarified that in the answer now. – Thole

Answer 2

There is a simple, although not cross platform, way to achieve this.

In Linux you can use bash to execute multiple commands:

add_test(
    NAME
        TestName
    COMMAND
        bash -c "COMMAND1 ; \
            COMMAND2 ; \
            ${CMAKE_CURRENT_BINARY_DIR}/testExecutable"
)

Waechter

Do you know how to do it on Windows? – Angeloangelology

Using bash -c allows for the execution of a script (which contains multiple commands) using a single entry point. I'm not a windows user, but this should be possible there too: https://mcmap.net/q/56045/-what-does-cmd-c-mean-closed – Waechter

I tried and it doesn't work, you should use execute_process – Angeloangelology