Aggregating code coverage from different testing frameworks

Question

In modern programming workflow numerous testing frameworks are used at once. For example, in PHP world, it is de-facto standard way to use unit tests, integration tests and functional/acceptance tests at once. Most of the time different frameworks are used for different test type. I am using combination of PHPSpec for unit, PHPunit for integration and CodeCeption for functional tests.

Is it possible to aggregate code coverage results that each of these frameworks return? Is there any tool that aggregates code coverage reports from different frameworks? Or it is only possible to view individual results for each framework while they are incorrect because each code coverage report doesn't take into account other tests.

Gethsemane

Answer 1

It is actually quite simple to perfrom this task. All your frameworks rely on the same library to generate the code coverage.

As you can see the generator in sebastianbergmann/php-code-coverage already supports a merge function (line 335) to merge different aggregates. Since you are part of a team using tests I assume it will be easy for you to change the test execution layer slightly to gather the code coverage in a single php process and just merge em.

Flout

Answer 2

There is a tool for this: phpcov. It allows to merge many coverage files with merge option:

$ parallel --gnu :::                                                 \
'phpunit --coverage-php /tmp/coverage/FooTest.cov tests/FooTest' \
'phpunit --coverage-php /tmp/coverage/BarTest.cov tests/BarTest'

$ phpcov merge /tmp/coverage --clover /tmp/clover.xml
phpcov 2.0.0 by Sebastian Bergmann.

Generating code coverage report in Clover XML format ... done

Tecumseh

Answer 3

I think we are on same boat. How we can tell how much we have converge using this all different testing tool. We start discuss with team and decide to go for

SonarSource. - For PHP Plugin and Live demo

PHP Report Stlyle - I advice you to visit live demo. It will help more.

It is very robust tool. It give us all inside of code.

Sanctify

Your answer looks like an advertisement now. Could you give more information how does it answer the question? Your screenshot is all Java, please tell us more what it can do for PHP. – Garonne

Answer your question just open link that I added in this post or go to this link : nemo.sonarqube.org (DEMO) When you click on PHP, It will show all the component coverage. Sonar Source is design for all technology. – Sanctify

Answer 4

The PHP Test Coverage Tool from Semantic Designs (my company) collects and combines test coverage from any

• framework
• test set
• individual test
• even ad hoc manual tests.

After running some set of tests, our tool is can be easily triggered to dump test coverage vectors to a file; you need to modify the framework slightly to invoke

TCVDump();

when the framework completes, or you can invoke a TCVDDump() by touching an easily found, special web page added by the test coverage tool. Each such call produces a time-stamped or user-named file (e.g, after the framework or test set) so they are easily distinguished

The graphical test coverage display included as part of the tool will interactively select and merge small or large sets of such files to produce a coherent whole, both display and summary. It will also compare test coverage vectors to enable one to decide if coverage from one test set include/intersects another, etc. The test coverage display component will also export text or XML/HTML summaries of the coverage results.

You can even run tests on different subsystems and combine them. This test coverage tool is part of larger family of tools for many languages other than PHP; tests run on a multilingual application system can also be combined to provide an overview of coverage for the multilingual application.

Meanwhile