Is it possible to merge coverage data from two executables with gcov/gcovr?

Question

On one project, I'm running the test cases on three different executables, compiled with different options. Depending on the options, some code paths are taken or not. Right now, I'm only using the coverage data from one executable.

I'm using gcovr to generate a XML that is then parsed by Sonar:

gcovr -x -b -r . --object-directory=debug/test > coverage_report.xml

I have three sets of gcda and gcno files, but I don't know how to generate a global report of them.

Is there any way to do that ?

Dalliance

lcov does this . So may be someone from lcov project might help . – Succotash

@k0n3ru, lcov merge data in some intermediate format. – Humfrid

The problem with lcov is that I don't have the XML Cobertura format anymore. – Dalliance

@BaptisteWicht Probably a workaround is that if its possible for your project to use lcov , seems like there are lcov to cobertura xml coverters. – Succotash

When you say you compiled with different options: Do you mean different compiler options (like once with -o2 and another time with -o3), or do you mean with different preprocessor macros defined? Or maybe even something else? – Commix

@DirkHerrmann I mean preprocessor macros that activate different features in my code. For instance, I have a macro enabling GPU acceleration. And I'm interested in the total coverage: the one taken by the normal version and the one taken by the GPU accelerated version. – Dalliance

Answer 1

Assuming that by "compiled with different options" you mean that you compile such that you obtain different outputs after preprocessing, with the help of lcov (as mentioned by k0n3ru) I was able to do it. Here's the sample code in file sut.c:

#include "sut.h"
#include <limits.h>

int foo(int a) {
#if defined(ADD)
    a += 42;
#endif
#if defined(SUB)
    a -= 42;
#endif
    return a;
}

with sut.h only providing the declaration of foo, and a simple main in test.c, which calls foo and prints the results. Then, with this sequence of commands I was able to create a total.info file with 100% coverage for sut.c:

> g++ --coverage -DADD test.c sut.c -o add.out
> ./add.out
> lcov -c -d . -o add.info   # save data from .gdda/.gcno into add.info
> g++ --coverage -DSUB test.c sut.c -o sub.out
> ./sub.out
> lcov -c -d . -o sub.info   # save again, this time into sub.info
> lcov -a add.info -a sub.info -o total.info  # combine them into total.info
> genhtml total.info

Which then for sut.c shows the following results:

EDIT (Thanks to Gluttton for reminding me of adding this part): Going from the total.info file in lcov format to the Cobertura XML output should then be possible with the help of the "lcov to cobertura XML converter" provided here (although I have not tried that): https://github.com/eriwen/lcov-to-cobertura-xml

The fact that merging of coverage information is possible, however, does certainly not mean that it is a good idea to do so: Coverage, IMO, has only limited informative value regarding the quality of a test suite. Merging coverage results from different preprocessor outputs will even further decrease this value.

This is because the possibilities for developers to learn about scenarios they have not considered will be reduced: By using conditional compilation the control structure and data flow of the code can vary tremendously between preprocessor outputs - coverage information that results from 'overlaying' results from test runs for different preprocessor outputs can make a meaningful interpretation of the results impossible.

Commix

But how convert report from html to xml (Cobertura) format? – Humfrid

You would rather convert the .info format of lcov directly to cobertura. A google search brought me here: github.com/eriwen/lcov-to-cobertura-xml – Commix

Thanks. I just tried and it does indeed work for merging the gcov data, but I haven't found a good way to convert the lcov data to cobertura. I tried the mentioned tools but it throws away a lot of information, there is no branch information in the generated xml. – Dalliance

@baptiste-wicht: In that case, I suggest to send them a bug report, because a) the README.md claims they can handle "Line and Branch hits" and b) the CHANGELOG.md claims that Branch "coverage reporting" is available since version 1.1 (the current version is 1.6). – Commix