Gitlab test coverage visualization is not working

Question

Hi guys I have a express application and I'm playing around with gitlab to add Gitlab test coverage visualization

Here is my .gitlab-ci.yml

stages:
  - test
  - dockerize
  - staging
  - production

unit-tests:
  stage: test
  script:
    - npm install
    - npm run test
    - npm run test-coverage
    - cat coverage/cobertura-coverage.xml
    - "echo 'Code coverage: 90.90'" 
  coverage: '/Code coverage: \d+\.\d+/'
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "master"'
    - if: '$CI_COMMIT_BRANCH == "release-v1"'
  artifacts:
    reports:
      cobertura: coverage/cobertura-coverage.xml
  tags:
   - demo

dockerize-application:
  stage: dockerize
  script:
    - echo "dockerizing application"
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "master"'
    - if: '$CI_COMMIT_BRANCH == "release-v1"'
  tags:
   - demo


deploy_to_staging:
  stage: staging
  script:
    - echo "deploying to staging"
  rules:
    - if: '$CI_COMMIT_BRANCH == "master"'
  tags:
   - demo

deploy_to_production:
  stage: staging
  script:
    - echo "deploying to production"
  rules:
    - if: '$CI_COMMIT_BRANCH == "release-v1"'
  tags:
   - demo

In my package json, here is the important part:

"scripts": {
    "start": "node ./bin/www",
    "debug": "nodemon ./bin/www",
    "test": "npx nyc --reporter text mocha",
    "test-coverage": "npx nyc --reporter cobertura mocha"
  }

In the logs, I can see that the coverage file is uploaded.

Runtime platform                                    arch=amd64 os=windows pid=19548 revision=775dd39d version=13.8.0
coverage/cobertura-coverage.xml: found 1 matching files and directories 
Uploading artifacts as "cobertura" to coordinator... ok  id=1097622443 responseStatus=201 Created token=6wcrN_d_

I did a lot of research and I still can't figure out why! I figured that was a feature flag turning this feature off by default but not any more:

https://gitlab.com/gitlab-org/gitlab/-/merge_requests/43711

Here is the link to my project: I mainly created to experiment with gitlab ci/cd

Concession

Answer 1

It's now working. The issue was that I needed to wait for the WHOLE pipeline to succeed before I could actually see the visualization.

Here is an issue open in Gitlab:

https://gitlab.com/gitlab-org/gitlab/-/issues/236248

Concession

Answer 2

If you use a Docker container in which you create the coverage.xml, your sources path in the coverage.xml might come out wrong, e.g.

<sources>
    <source>/app/src</source>
</sources> 

However, as described in the docs, the path of the sources tag must follow a specific pattern.

You can use sed to make this correction within the pipeline and use CI_PROJECT_DIR to come up with the correct path.

test:
  stage: test
  script: 
    ...
    - docker cp mycontainer:/app/coverage.xml ./coverage.xml
    - sed -i "s#<source>/app/src</source>#<source>${CI_PROJECT_DIR}/src</source>#g" coverage.xml

Forgave

Answer 3

To see test coverage visualization in Gitlab diff MR views, you need to have a valid Cobertura coverage XML report file (coverage/cobertura-coverage.xml in your case) and configure GitLab to process it:

• Correctly set the artifacts.reports.coverage_report (gitlab-ci.yml) with: coverage_format: cobertura and path: coverage/cobertura-coverage.xml,
• Include the coverage/cobertura-coverage.xml to the artifacts.paths (gitlab-ci.yml),
• Have a valid sources.source (cobertura-coverage.xml) path where you want to collect coverage (a relative path from your repository like src/frontend),
• The pipeline need to be finished, regardless status.

Also, ensure your Jest config correctly define the collectCoverageFrom option like this:

// jest.config.ts:

...
coverageReporters: ["cobertura"],
// if the sources to be covered is in a `/src` directory inside your repository:
collectCoverageFrom: [
    '<rootDir>/src/**/*.{js,jsx,ts,tsx}',
    '!**/*.d.ts'
],

Next, set a correct path to the Cobertura coverage XML report file. In your case, you seem to not running NPM inside a docker container but directly from the shell executor, the sources.source path is probably set to $CI_PROJECT_DIR/src:

// coverage/cobertura-coverage.xml

<sources>
  <source>/builds/my-project/src</source>
</sources>

If you run NPM in a container with a workdir set to "/usr/src/app" like this:

docker run --rm \
        -w /usr/src/app \
        -v `pwd`:/usr/src/app \
        node:lts-alpine sh -c "npm run test:ci"

The resulted path is:

// coverage/cobertura-coverage.xml

<sources>
  <source>/usr/src/app/src</source>
</sources>

To fix this, you can replace the wrong path (/usr/src/app) in the cobertura-coverage.xml file by the relative repository path to the "src" directory:

sed -i "s#<source>/usr/src/app</source>#<source>src</source>#g" coverage/cobertura-coverage.xml

The path is now correct:

// coverage/cobertura-coverage.xml

<sources>
  <source>src</source>
</sources>

Full GitLab 16+ sample, Jest + Junit + Cobertura

Here is a simplified GitLab CI stage as example that can:

• [gitlab-ci.yml job key: coverage] retrieve the coverage score, it displayed in an MR details page in the "overview" tab ; and also in the jobs list page at the "Coverage" column,
• [gitlab-ci.yml job key: artifacts.reports.junit] retrieve all tests results and display it in the "Test summary" report in the MR details page (with a link to the "Full report"),
• [gitlab-ci.yml job key: artifacts.reports.coverage_report] collect coverage information, used to visualize this information inside the file diff view of your merge requests (MRs),

// jest.config.ts

...
reporters: [
    "default",
  [
    "jest-junit",
    {
      outputDirectory: "test_reports",
      outputName: "jest_junit.xml",
    },
  ],
],
coverageReporters: ["cobertura"],
collectCoverageFrom: [
    '<rootDir>/src/**/*.{js,jsx,ts,tsx}',
    '!**/*.d.ts'
],

# gitlab-ci.yml

test-jest:
  stage: test
# the regex pattern retrieve the coverage score from the Jest "text-summary" output
coverage: /All files[^|]*\|[^|]*\s+([\d\.]+)/
  artifacts:
    when: always
    paths:
      - test_reports/jest_junit.xml
      - coverage/cobertura-coverage.xml
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
  script:
    - |
      docker run --rm \
        -w /usr/src/app \
        -v `pwd`:/usr/src/app \
        node:lts-alpine sh -c "npm ci && npm run test:ci"
    - sed -i "s#<source>/usr/src/app</source>#<source>src</source>#g" coverage/cobertura-coverage.xml

GitLab test coverage visualization MR diff view:

GitLab tests coverage score:

GitLab pipeline "Tests" tab:

Sewell