How to let CMake / CTest memcheck exit with status code 1 on failure?

Question

I want to use ctest to run my tests with valgrind. Thus I have written the following in my cmake file:

include(CTest)

find_program(MEMORYCHECK_COMMAND valgrind)
set(MEMORYCHECK_COMMAND_OPTIONS "--leak-check=full --error-exitcode=1")
set(MEMORYCHECK_SUPPRESSIONS_FILE "${PROJECT_SOURCE_DIR}/.valgrind-suppressions")

This seems to work. When I run ctest -D ExperimentalMemCheck . on a leaking program it shows me that it has found memory leaks, however does not exit with status != 0.

How can I get a exit code 1 on failure?

Austen

Do you use MEMORYCHECK_COMMAND_OPTIONS variable in any way? – Subsistent

Only in the way as shown in the example. – Austen

Did you find a solution to this already? – Ordinal

Answer 1

The critical thing is that you put set(MEMORYCHECK_COMMAND_OPTIONS "--error-exitcode=1") ABOVE the include(CTest) call. The variable is apparently only taken into account when first including CTest and has no effect when setting it after including CTest.

When then calling ctest -T memcheck, the command correctly exits with a non-zero exit status.

Also see this questions for reference: How to pass arguments to memcheck with ctest?

Ordinal

Are you sure that CTest is not included further up (or in another) CMakeLists.txt? Maybe try to reduce your CMakeLists.txt as far as possible or try it on a sample project, should work just fine. – Ordinal

Answer 2

By default, valgrind memcheck exits with an error for leak kinds definite and possible.

You might want to have more leak kinds causing an error exit, by using --errors-for-leak-kinds:

--errors-for-leak-kinds=kind1,kind2,..  which leak kinds are errors?
                                        [definite,possible]
    where kind is one of:
      definite indirect possible reachable all none

Etter

The problem is not valgrind. If I run valgrind manually I get the correct exit status. I want ctest to return a non zero exit status. – Austen