gcov produces different results on Clang and GCC

My library

#include "library.h" #include <iostream> void foo(){ std::cout << "Foo!" << std::endl; } void bar(int n){ if (n > 0){ std::cout << "n is grater than 0!" << std::endl; } else if (n < 0){ std::cout << "n is less than 0!" << std::endl; } else{ std::cout << "n is exactly 0!" << std::endl; } } void baz(){ // LCOV_EXCL_START std::cout << "Baz!" << std::endl; } // LCOV_EXCL_STOP

My tests

#ifndef GCOV_TUTORIAL_TEST_LIBRARY_H #define GCOV_TUTORIAL_TEST_LIBRARY_H #include "../src/library.h" #include <gtest/gtest.h> namespace gcov_tutorial::tests { TEST(TestFooSuite,TestFoo){ foo(); } TEST(TestBarSuite,TestBarGreaterThanZero){ bar(100); } TEST(TestBarSuite,TestBarEqualToZero){ //bar(0); } TEST(TestBarSuite,TestBarLessThanZero){ bar(-100); } } #endif //GCOV_TUTORIAL_TEST_LIBRARY_H

CLang Compilation

#!/bin/bash # Rationale: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/ set -euxo pipefail # BASE_DIR is the project's directory, containing the src/ and tests/ folders. BASE_DIR=$PWD COVERAGE_FILE=coverage.info GCOV_PATH=/usr/bin/gcov CLANG_PATH=/usr/bin/clang CLANGPP_PATH=/usr/bin/clang++ rm -rf build mkdir build && cd build # Configure cmake -DCMAKE_C_COMPILER=$CLANG_PATH -DCMAKE_CXX_COMPILER=$CLANGPP_PATH -DCODE_COVERAGE=ON -DCMAKE_BUILD_TYPE=Release .. # Build (for Make on Unix equivalent to `make -j $(nproc)`) cmake --build . --config Release # Clean-up for any previous run. rm -f $COVERAGE_FILE lcov --zerocounters --directory . # Run tests ./tests/RunTests # Create coverage report by taking into account only the files contained in src/ lcov --capture --directory tests/ -o $COVERAGE_FILE --include "$BASE_DIR/src/*" --gcov-tool $GCOV_PATH # Create HTML report in the out/ directory genhtml $COVERAGE_FILE --output-directory out # Show coverage report to the terminal lcov --list $COVERAGE_FILE # Open HTML open out/index.html

Coverage report when compiling with CLang

GCC Compilation

#!/bin/bash # Rationale: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/ set -euxo pipefail # BASE_DIR is the project's directory, containing the src/ and tests/ folders. BASE_DIR=$PWD COVERAGE_FILE=coverage.info GCOV_PATH=/usr/local/bin/gcov-11 GCC_PATH=/usr/local/bin/gcc-11 GPP_PATH=/usr/local/bin/g++-11 rm -rf build mkdir build && cd build # Configure cmake -DCMAKE_C_COMPILER=$GCC_PATH -DCMAKE_CXX_COMPILER=$GPP_PATH -DCODE_COVERAGE=ON -DCMAKE_BUILD_TYPE=Release .. # Build (for Make on Unix equivalent to `make -j $(nproc)`) cmake --build . --config Release # Clean-up for any previous run. rm -f $COVERAGE_FILE lcov --zerocounters --directory . # Run tests ./tests/RunTests # Create coverage report by taking into account only the files contained in src/ lcov --capture --directory tests/ -o $COVERAGE_FILE --include "$BASE_DIR/src/*" --gcov-tool $GCOV_PATH # Create HTML report in the out/ directory genhtml $COVERAGE_FILE --output-directory out # Show coverage report to the terminal lcov --list $COVERAGE_FILE # Open HTML open out/index.html

You are actually asking two questions, here.

Why do the coverage results differ between these two compilers?
How do I structure a CMake project for code coverage?

Answer 1: Coverage differences

The simple answer here is that you are building in Release mode, rather than RelWithDebInfo mode. GCC does not put as much debugging information in by default as Clang does. On my system, adding -DCMAKE_CXX_FLAGS="-g" to your build-and-run-cov-gcc.sh script yields the same results as Clang, as does building in RelWithDebInfo.

For whatever reason, it appears that Clang tracks more debug information either by default or when coverage is enabled. GCC does not have these same guardrails. The lesson to take away is this: collecting coverage information is a form of debugging; you must use a debugging-aware configuration for your compiler if you want accurate results.

Answer 2: Build system structure

It is generally a terrible idea to set CMAKE_CXX_FLAGS inside your build. That variable is intended to be a hook for your build's users to inject their own flags. As I detail in another answer on this site, the modern approach to storing such settings is in the presets

I would get rid of the if (CODE_COVERAGE) section of your top-level CMakeLists.txt and then create the following CMakePresets.json file:

{
  "version": 4,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 23,
    "patch": 0
  },
  "configurePresets": [
    {
      "name": "gcc-coverage",
      "displayName": "Code coverage (GCC)",
      "description": "Enable code coverage on GCC-compatible compilers",
      "binaryDir": "${sourceDir}/build",
      "cacheVariables": {
        "CMAKE_BUILD_TYPE": "RelWithDebInfo",
        "CMAKE_CXX_FLAGS": "-fprofile-arcs -ftest-coverage"
      }
    }
  ],
  "buildPresets": [
    {
      "name": "gcc-coverage",
      "configurePreset": "gcc-coverage",
      "configuration": "RelWithDebInfo"
    }
  ]
}

Then your build script can be simplified considerably.

#!/bin/bash

# Rationale: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
set -euxo pipefail

# Set up defaults for CC, CXX, GCOV_PATH
export CC="${CC:-gcc-11}"
export CXX="${CXX:-g++-11}"
: "${GCOV_PATH:=gcov-11}"

# Record the base directory
BASE_DIR=$PWD

# Clean up old build
rm -rf build

# Configure
cmake --preset gcc-coverage

# Build
cmake --build --preset gcc-coverage

# Enter build directory
cd build

# Clean-up counters for any previous run.
lcov --zerocounters --directory .

# Run tests
./tests/RunTests

# Create coverage report by taking into account only the files contained in src/
lcov --capture --directory tests/ -o coverage.info --include "$BASE_DIR/src/*" --gcov-tool $GCOV_PATH

# Create HTML report in the out/ directory
genhtml coverage.info --output-directory out

# Show coverage report to the terminal
lcov --list coverage.info

# Open HTML
open out/index.html

The key here is the following lines:

# Configure
cmake --preset gcc-coverage
# Build
cmake --build --preset gcc-coverage

This script now lets you vary the compiler and coverage tool via environment variables and the CMakeLists.txt doesn't have to make any assumptions about what compiler is being used.

On my (Linux) system, I can run the following commands successfully:

$ CC=gcc-12 CXX=g++-12 GCOV=gcov-12 ./build-and-run-cov.sh

$ CC=clang-13 CXX=clang++-13 GCOV=$PWD/llvm-cov-13.sh ./build-and-run-cov.sh

Where llvm-cov-13.sh is a wrapper for llvm-cov-13 for compatibility with the --gcov-tool flag. See this answer for more detail.

#!/bin/bash
exec llvm-cov-13 gcov "$@"

As you can see, the results are indistinguishable now that the correct flags are used.