combine python coverage files?

Question

I'm wondering if it's possible to combine coverage.xml files into 1 file to see global report in HTML output.

I've got my unit/functional tests running as 1 command and integration tests as the second command. That means my coverage for unit/functional tests are overridden by unit tests.

That would be great if I had some solution for that problem, mainly by combining those files into 1 file.

Expostulate

Answer 1

I found a different solution. I used combine feature (read here) So I run my coverage like: coverage run -p and then I do coverage combine.

If you want to keep the old reports, you can use --keep.

That's all. It generates 1 combined report.

Expostulate

coverage combine will merge all .coverage_* files in a directory and create a combined .coverage file and delete the others. Only needed when merging different directories. – Sarcomatosis

With argument --keep the .coverage_* files are not removed. – Patrilineage

Now, it replaces .coverage.* files (. instead of _) – Jordonjorey

Answer 2

You can't combine .xml files, but you can combine the raw data files. Your workflow would look like this:

$ COVERAGE_FILE=.coverage_func coverage run the_functional_tests.py
$ COVERAGE_FILE=.coverage_inte coverage run the_integration_tests.py
$ coverage combine
$ coverage xml

Keshiakesia

A caveat to coverage combine is that files are addressed by path, thus this works fine when coverage is collected and merged in the same directory, but requires extra steps if one step is performed remotely (e.g. in a container) where Python source paths are different. – Twinned

Coverage.py has a feature to specifically help with this: coverage.readthedocs.io/en/v4.5.x/config.html#paths This seems to be often missed by people. What can I do to make it more apparent? – Keshiakesia

I followed above steps but getting error "No data to combine". The directory already have two files to combine ".coverage_ca" and ".coverage_test". – Tusk

@GauravKohli i tried other method. You can find my solution below. – Tusk

This does work when all the raw coverage files are in the same directory, with a couple of tweaks. First, you have to unset COVERAGE_FILE. Second, combine is expecting file names starting with ".coverage.". I was getting the "No data to combine" until I made those two changes. – Hidrosis

Answer 3

You can achieve same result by using appending option. Suppose you ran the coverage on three python scripts.After first coverage use -a for appending.

coverage run first.py
coverage run -a second.py
coverage run -a third.py

Print the report

coverage report -m

Output:Report

Name             Stmts   Miss  Cover   Missing
----------------------------------------------
first.py           97      1    99%   95
second.py            1      0   100%
third.py            10      0   100%
----------------------------------------------
TOTAL               108      1    99%

Tusk

I am using this approach, but they are actually not combined! – Hyalo

@JingpengWu, it works for me. Are you doing something wrong? – Escargot

Answer 4

If your source code is in a directory called my_project, you can also do this if you have included pytest and pytest-cov in your virtual environment:

pytest --cov-report html --cov=my_project unit_tests
pytest --cov-report html --cov=my_project --cov-append functional_tests

The --cov-append will add the functional test coverage info to the coverage file that was created when you ran the unit tests.

Crowns

Answer 5

I had similar case where I had multiple packages and each of them had its tests and they were run using own testrunner. so I could combine all the coverage xml by following these steps.

1. Indivisually generate the coverage report.
   You would need to naviagte to each package and generate the report in that package. This would create .coverage file. You can also add [run]parallel=True in your .coveragerc to create coverage file appended with machine name and processid.

2. Aggregate all the reports.
   You need to copy all the .coverage files to for these packages to a seaparte folder. You might want to run a batch or sh script to copy all the coverage files.

3. Run combine.
   Now naviagte tp the folder when you have all the report files and then run coverage combine. This will delete all the coverage files and combine it to one .coverage file. Now you can run coverage html and coverage xml.

Judicator

Nice, I found extensive documentation of this feature here: coverage.readthedocs.io/en/stable/cmd.html#cmd-combine – Elora