code coverage in gitlab ci/cd for .net applications?

Question

I recently started working with C# and I am working on one of the legacy system we have. I am trying to figure out what is the code coverage for this legacy system. Here is my Sample.UnitTests.csproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="AutoFixture.AutoMoq" Version="4.2.1" />
    <PackageReference Include="AutoFixture.NUnit3" Version="4.2.1" />
    <PackageReference Include="coverlet.msbuild" Version="2.9.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="Moq" Version="4.8.2" />
    <PackageReference Include="nunit" Version="3.9.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
    <PackageReference Include="WireMock.Net" Version="1.0.4.17" />
    <PackageReference Include="Utf8Json" Version="1.3.7" />
    <FrameworkReference Include="Microsoft.AspNetCore.App" />
    <PackageReference Include="System.Buffers" Version="4.5.0" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="../Sample/Sample.csproj" />
  </ItemGroup>

<ItemGroup>
  <DotNetCliToolReference Include="dotnet-reportgenerator-cli" Version="4.2.10" />
</ItemGroup>

</Project>

I did some research and found out we can use coverlet which can generate cobertura style report. I followed exactly as mentioned here on my mac box and everything works fine and I can see the report being generated correctly on my console and also it generates index.html file which we can use to visualize as well.

dotnet add package coverlet.msbuild
dotnet restore
dotnet build
dotnet test /p:CollectCoverage=true
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:Exclude="[xunit*]\*" /p:CoverletOutput="./TestResults/"
dotnet reportgenerator "-reports:TestResults/coverage.cobertura.xml" "-targetdir:TestResults/html" -reporttypes:HTML;

Now since we use gitlab ci/cd pipeline for our project - Is there any way I can make this part of my .gitlab-ci.yml file so that it can generate report automatically for me whenever build happens and everybody in my team can see it successfully. Since as of now it's all manual as I need to run those above commands on my local mac box and I can see it from my console only by clicking index.html file.

These are my stages of .gitlab-ci.yml file as shown below. If needed I can provide my yml file as well but any simple example where it can demonstrate how can I do this then it will be of great help. I tried searching a lot and couldn't find this at all on how can I do it through gitlab pipeline which uses coverlet and cobertura style report for .net applications..

stages:
  - test
  - publish
  - increment
  - deploy
  - integrationTests
  - release

Can this be done through webhook as well if needed?

Sigfried

You can have a look at existing gitlab issues to add the coverlet support gitlab.com/gitlab-org/gitlab-foss/-/issues/62727 and gitlab.com/gitlab-org/gitlab/-/issues/37379 – Burger

ohh so it's not supported yet you mean to say? That's what I understood by reading from that link. Correct me if I am wrong here? – Sigfried

Your link explains how to collect the coverage and create an html report, nothing about CI/CD integration – Burger

well that's what I mentioned too I guess. It doesn't and I did research on my own as well and I don't see anywhere mentioned how to do it through gitlab pipeline and integrate coverlet for .net code coverage. – Sigfried

@Sigfried Did you solved this problem? Can you share what you have done? – Hockett

First, change -reporttypes:HTML to -reporttypes:Cobertura. Then, set artifacts:reports:cobertura to the path of that output xml. – Indecorum

Answer 1

The other answer didn't work for me because of folder confusion. Coverlet puts the test results in folders relative to the respective unit test projects. So

• Either you have to tell gitlab to search for TestResults folders everywhere via wildcards, by setting path to something like ./**/TestResults/**/coverage.cobertura.xml.
• Or you provide the --results-directory option to tell dotnet test where to put those files in the first place.

I went for the second option, gathering all results in a cobertura folder in the repo root. Here is a full, valid .gitlab-ci.yml for running tests for merge requests:

image : mcr.microsoft.com/dotnet/sdk:6.0

stages:
  - test

test:
  stage: test
  only:
    - merge_requests
  script:
    - 'dotnet test DotNetSolution 
        --collect:"XPlat Code Coverage"
        --results-directory cobertura'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: cobertura/*/coverage.cobertura.xml

Luwian

Thanks for sharing, works great for me, with one disclaimer - -r cobertura worked fine for me in .NET 6.0, but when I upgraded to .NET 7.0, looks like -r is for the runtime, and the command fails. When replacing with --results-directory everything works fine. – Macrophysics

@Macrophysics Ah thanks, I adjusted the answer – Luwian

Answer 2

We've set up a GitLab CI/CD pipeline for a .NET application with code coverage. Here's a brief explanation of each section in the configuration:

before_script:
  - 'export DOTNET_CLI_TELEMETRY_OPTOUT=1'
  - 'export PATH=$PATH:$HOME/.dotnet/tools'
  - 'dotnet tool install dotnet-reportgenerator-globaltool --global || echo "DRG already installed."'

test:
  stage: test
  needs: 
    - build
  coverage: '/TOTAL_COVERAGE=(\d+.\d+)/'
  script:
    - dotnet test *.sln --collect:"XPlat Code Coverage" --logger "junit;MethodFormat=Class;FailureBodyFormat=Verbose"
    - reportgenerator -reports:"**/coverage.cobertura.xml" -targetdir:"." -reporttypes:"cobertura"
    - COVERAGE_VALUE=$(grep -oPm 1 'line-rate="\K([0-9.]+)' "./Cobertura.xml")
    - COVERAGE=$(echo "scale=2; $COVERAGE_VALUE * 100" | bc)
    - 'echo "TOTAL_COVERAGE=$COVERAGE%"'
  artifacts:
    when: always
    expire_in: 1 day
    paths:
      - ./**/TestResults.xml
      - ./Cobertura.xml
    reports:
      junit:
        - ./**/TestResults.xml
      coverage_report:
        coverage_format: cobertura
        path: ./Cobertura.xml

We run dotnet test with the XPlat Code Coverage collector and JUnit logger.

All our .csproj has dependencies:

  <ItemGroup>
    <PackageReference Include="coverlet.collector" Version="3.2.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="JunitXml.TestLogger" Version="3.0.124" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.10" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.10" />
  </ItemGroup>

Those will output coverage reports as TestResults/*/coverage.cobertura.xml.

And we used the reportgenerator to merge those results as one file to calculate the total coverage percentage, and echo the result. The artifacts, including test results and coverage reports, are saved and set to expire in one day.

Industry

Answer 3

May need to use the actual GUID instead of *.

stages:
  - test
  - publish
  - increment
  - deploy
  - integrationTests
  - release

build-and-test:
  stage: test
  image: mcr.microsoft.com/dotnet/sdk:6.0
  script:
  - dotnet add package coverlet.msbuild
  - dotnet restore
  - dotnet build
  - 'dotnet test --collect:"XPlat Code Coverage"'
  artifacts:
    reports:
      cobertura: TestResults/*/coverage.cobertura.xml

GitLab can diff with previous Cobertura reports.

If you want HTML instead, simply include it among the artifacts. May also publish it to GitLab Pages.

Indecorum

This worked better for me: ./**/TestResults/**/coverage.cobertura.xml – Pesade