Sonarqube client fails to parse pytest coverage results

Question

I'm trying to set up a gitlab ci pipeline for a python project and I'm having some issues with sonarqube client which is not able to pars the coverage.xml file

The error I'm getting is the following:

INFO: Python test coverage
INFO: Parsing report '/builds/core-tech/tools/nlu/mix-nlu-middleware/server/tests/cov.xml'
WARN: Invalid directory path in 'source' element: /bolt-webserver/bolt
WARN: Invalid directory path in 'source' element: /bolt-webserver/tests
ERROR: Cannot resolve the file path 'base.py' of the coverage report, the file does not exist in all <source>.
ERROR: Cannot resolve 404 file paths, ignoring coverage measures for those files
INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=74ms
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=20ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]

The coverage file (cov.xml) starts with this:

<?xml version="1.0" ?>
<coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="0.3476" lines-covered="10369" lines-valid="29833" timestamp="1564079534753" version="4.4.2">
    <!-- Generated by coverage.py: https://coverage.readthedocs.io -->
    <!-- Based on https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd -->
    <sources>
        <source>/bolt-webserver/bolt</source>
        <source>/bolt-webserver/tests</source>
    </sources>
    <packages>
        <package branch-rate="0" complexity="0" line-rate="0.55" name=".">
            <classes>
                <class branch-rate="0" complexity="0" filename="base.py" line-rate="0.5955" name="base.py">
                    <methods/>
                    <lines>
                        <line hits="1" number="1"/>
                        <line hits="1" number="2"/>
                        <line hits="1" number="3"/>
                        <line hits="1" number="4"/>
                        <line hits="1" number="5"/>
                        <line hits="1" number="7"/>
                        <line hits="1" number="9"/>
                        <line hits="1" number="10"/>
  .......................

Sonar is called like this:

- sonar-scanner -Dsonar.projectKey=mix-nlu-middleware -Dsonar.sources=./server -Dsonar.host.url=$SONAR_SERVER_HOST -Dsonar.login=$SONAR_LOGIN -Dsonar.python.coverage.reportPaths=server/tests/cov.xml -Dsonar.junit.reportPaths=server/tests/junit-report.xml

The project tree looks like this:

.
+-- CONTRIBUTING.md
+-- gen_version.sh
+-- package-lock.json
+-- README.md
+-- scripts
│   +-- .....
+-- server
│   +-- alembic.ini
│   +-- bolt
│   │   +-- .....
│   +-- Bolt.egg-info
│   │   +-- .....
│   +-- conf
│   │   +-- .....
│   +-- dev-requirements.txt
│   +-- Dockerfile
│   +-- Dockerfile-dev
│   +-- http.log
│   +-- MANIFEST.in
│   +-- pytest.ini
│   +-- requirements.txt
│   +-- scripts
│   │   +-- .....
│   +-- sdks
│   │   +-- ....
│   +-- server.log
│   +-- setup.py
│   +-- templates
│   │   +-- .....
│   +-- tests
│   │   +-- .....
│   \-- version.properties
\-- test.txt

Any idea what I'm doing wrong here?

I've also tried to create the path /bolt-webserver/bolt in the root folder of the project and file system but still no luck.

The 'base.py' file and the others mentioned in the conv.xml are located under '/builds/core-tech/tools/nlu/mix-nlu-middleware/server/tests'

Virgel

Does this answer your question? Test/Test Coverage with Python in Sonar not showing up? – Factional

Answer 1

I ran into the same issue with GitHub Actions, was able to resolve by specifying relative_files = True in .coveragerc.

[coverage:run]
relative_files = True
source = my_project/
branch = True

This is also documented on SonarCloud's docs

Note that we specify relative_files = True in the tox.ini file to ensure that GitHub Actions will correctly parse your coverage results.

Hope this helps!

Cigarillo

Also helped with pytest-cov. However, I added the following to pyproject.toml: ``` [tool.coverage.run] source = ['my_project'] relative_files = true ``` – Bunin

Answer 2

You need to assist sonarqube to found proper file, an easy way could be include the following step in your GH actions after running the tests and generate the report, and before calling souncarcloud scan

--> step that run tests

      - name: fix code coverage paths
        run: |
          sed -i 's/\/home\/runner\/work\/<your-repo>\/<your-repo>\//\/github\/workspace\//g' cover/coverage.xml

--> step that call sonar qube report

(I imagine your repo is bolt-webserver)

Pander

Answer 3

OK the problem seem to have come from the cov.xml content. It seems sonarqube is not able to locate the test files based on their name only: filename="base.py" In order to fix the problem I had to update the cov.xml so that the filename fields contained the full file path. filename="/base.py"

Virgel

did you do this manually or were you able to do it in the pytest run? – Residual

I had to do it manually. Well i wrote a script to do it for me – Virgel

Mind to share the script? ;) – Floccule

Bump! How do you find the path of the file from just the filename? I would like to see that script as well. – Mitrewort

...and Bump again! @gheorghievgheniev we need to know :) having the same problem at the moment – Cornstarch

did anyone find how to do this? – Nought

Answer 4

I solved it by doing as stated by Adrian at https://community.sonarsource.com/t/python-coverage-analysis-warning/62629/7

Seems like there is a problem with using source, works when using include instead.

Receptacle

Answer 5

I faced the same issue in python projects that use unittest with coverage and have the integration with SonarQube being executed from both GitLab CI and GitHub Actions.

After spending hours trying each one of the previous answers shared here, the solution for my case was quite simple:

• set the working directory from where SonarQube scanner is executed to the subdirectory where the relative paths are calculated in the test coverage report.

Example:

In my case my repository layout is similar to:

a-file
another-file
project-1/
    app/
    tests/
some-other-dir/
some-other-file
...

Given that I want the scanner to collect test coverage results for the code in project-1, then project-1 should be set as the working directory.

Sample solution, using GitHub actions + sonarqube-scan-action and setting projectBaseDir:

  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          # Disable shallow clones, as recommended by SonarQube for improved analysis and reports
          fetch-depth: 0

      - name: Run tests
        run: # command to run tests and generate coverage report

      - name: SonarQube Scan
        uses: sonarsource/sonarqube-scan-action@master
        with:
          # projectBaseDir: path/to/my/sub_dir/with/app_and_tests_folders
          projectBaseDir: project-1
        env:
          SONAR_TOKEN: ${{ secrets.SONARQUBE_TOKEN }}
          SONAR_HOST_URL: ${{ secrets.SONARQUBE_URL }}

In GitLab CI I achieved the same using the Docker image sonar-scanner-cli-docker and setting the image environment variable SRC_PATH with the path to my project subfolder.

Phyfe

Answer 6

Building on previous answer by @zeshuaro, I added the following to pyproject.toml:

[tool.coverage.run]
source = ['my_project']
relative_files = true

To make it work with pytest-cov and not create additional files.

Bunin