Let CMake setup CTtest to print header and footer around the output from single tests

Question

Is there a way, ideally from CMakeLists.txt, to setup ctest as to

• print a header before running the individual tests,
• print a footer after running the individual tests,
• make the footer dependent on whether the tests were all successful or not ?

The footer should appear below the default output

The following tests FAILED:
       76 - MyHardTest
Errors while running CTest

This concretizes and generalizes a somewhat unclear question that is open since more than 2 years (CMakeLists.txt: How to print a message if ctest fails?). Therefore I fear there is no easy solution.

Thence an alternative question: could the desired bevhavior achieved with CDash?

Chericheria

Answer 1

YES, CTest does have macros to achieve exactly this [1]:

• CTEST_CUSTOM_PRE_TEST Command to execute before any tests are run during Test stage
• CTEST_CUSTOM_POST_TEST Command to execute after any tests are run during Test stage

To activate these macros from cmake for use by ctest, they must somehow be placed into the build directory. So it seems, two steps are necessary:

(1) Have a script scriptdir/CTestCustom.cmake.in somewhere in the source tree, which contains

set(CTEST_CUSTOM_POST_TEST "echo AFTER_THE_TEST")

or whatever whatever command instead of "echo"

(2) Let CMakeLists.txt call

configure_file("scriptdir/CTestCustom.cmake.in" ${CMAKE_BINARY_DIR}/CTestCustom.cmake)

so that during configuration stage a CTest configuration file is placed under the preferred name [2] CTestCustom.cmake in the build directory.

[1] https://cmake.org/Wiki/CMake/Testing_With_CTest [2] https://blog.kitware.com/ctest-performance-tip-use-ctestcustom-cmake-not-ctest/

Chericheria

Answer 2

During my research, I found it was extremely difficult to integrate something like this. I am not entirely sure but I believe you can do this in CTestScript, then create a add_custom_target to always allow that Script to execute with ctest. For example, the command make check will now run ctest with the CTestScript that you made... too much work?

Easiest way I can think of for your application is to just add two empty tests at top and bottom as placeholders for header and footers. Ctest already has a "The following tests FAILED:" kind of output at the very end, so you might not have to worry about it. Any sort of conditional logic IF TEST FAILED DO THIS, you cannot do currently in ctest.

add_test(NAME HEADER_GOES_HERE)
add_test(NAME ACTUAL_TEST COMMAND test)
add_test(NAME FOOTER_GOES_HERE)

Maybe someone can give you a better answer, but this is the easiest (not at all good) implementation I can think of.

Catalan