Publish a pipeline Azure Devops code coverage report

Question

I am trying to publish a detailed report online in my Azure DevOps Pipeline, but all I got is a link to download this Coverage file. (That can not be read anymore with the community version since the Visual Studio 2019)

This is my pipeline:

trigger:
  branches:
    include:
    - '*'

pool:
  vmImage: 'windows-2019'

steps:
- task: NuGetToolInstaller@0
  displayName: Instal Nuget
  inputs:
    checkLatest: true

- task: NuGetCommand@2
  displayName: Restore Nuget Packages
  inputs:
    restoreSolution: '**/*.sln'

- task: UseDotNet@2
  displayName: 'Install .NET Core SDK'
  inputs:
    version: 3.1.x
    performMultiLevelLookup: true

- task: DotNetCoreCLI@2
  displayName: Build Tests
  inputs:
    command: 'build'
    projects: '**/OneTienditaUnitTests/*.csproj'
    arguments: '--configuration Release'

- script: dotnet test OneTienditaUnitTests --logger trx --collect "Code coverage"

- task: PublishTestResults@2
  inputs:
    testRunner: VSTest
    testResultsFiles: '**/*.trx'

- task: XamarinAndroid@1
  displayName: Build Android App
  inputs:
    projectFile: '**/*Android*.csproj'
    outputDirectory: '$(build.binariesDirectory)/Release'
    configuration: 'Release'

and if I use Cobertura like this, doesn't work:

- task: DotNetCoreCLI@2
  displayName: Run Tests
  inputs:
    command: 'test'
    projects: '**/OneTienditaUnitTests/*.csproj'
    arguments: '--configuration Release /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=../reports/coverage/'
    
- task: PublishCodeCoverageResults@1
  displayName: 'Publish code coverage results'
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: '$(build.sourcesdirectory)\reports\coverage\coverage.cobertura.xml'
    reportDirectory: '$(build.sourcesdirectory)\reports\coverage'

Please any help? I am not a professional DevOps

Phan

Answer 1

To have there published report you need to use Cobertura. For TRX you will get only link to download file. And to create Cobertura report you need to install in your test projects coverlet.collector nuget package. Here you have code which should fix your problem:

# You just added coverlet.collector to use 'XPlat Code Coverage'
- task: DotNetCoreCLI@2
  displayName: Test
  inputs:
    command: test
    projects: '**/*Tests/*.csproj'
    arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage" -- RunConfiguration.DisableAppDomain=true'
    workingDirectory: $(Build.SourcesDirectory)

- task: DotNetCoreCLI@2
  inputs:
    command: custom
    custom: tool
    arguments: install --tool-path . dotnet-reportgenerator-globaltool
  displayName: Install ReportGenerator tool

- script: ./reportgenerator -reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml -targetdir:$(Build.SourcesDirectory)/coverlet/reports -reporttypes:"Cobertura"
  displayName: Create reports

- task: PublishCodeCoverageResults@1
  displayName: 'Publish code coverage'
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: $(Build.SourcesDirectory)/coverlet/reports/Cobertura.xml  

[2021 UPDATE]

You don't need extra tasks to install/run the custom ReportGenerator tool: it is now the default tool for reading coverage.cobertura.xml files and is included in the dotnet CLI.

By default, it will save the cobertura xml file to the temp directory on the agent, though. So, you just need to update the summaryFileLocation of the PublishCodeCoverageResults task to point to the temp directory and skip the "middle man" steps:

# You just added coverlet.collector to use 'XPlat Code Coverage'
- task: DotNetCoreCLI@2
  displayName: Test
  inputs:
    command: test
    projects: '**/*Tests/*.csproj'
    arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage"'

- task: PublishCodeCoverageResults@1
  displayName: 'Publish code coverage'
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'

If you have multiple test projects which generates multiple coverage files please use these steps after test commad. It will merge files before publishing them:

  - task: reportgenerator@4
    displayName: "Merge code coverage reports"
    inputs:
      reports: "**/coverage.cobertura.xml"
      targetdir: "$(Build.ArtifactStagingDirectory)/coverlet"
      reporttypes: "Cobertura"
      verbosity: "Verbose"

  - task: PublishCodeCoverageResults@1
    displayName: "Publish code coverage results"
    inputs:
      codeCoverageTool: Cobertura
      summaryFileLocation: "$(Build.ArtifactStagingDirectory)/coverlet/Cobertura.xml"

Resorcinol

it works, but a little detail, within the "script" line, I had to take off the "./" in the beginning. Thank you, was totally missing the Install part. – Phan

thanks. Any idea if there is a task to roll all these steps up into one considering XPlat Code Coverage is suggested by the MS docs? learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test – Sniff

Well, you can create a template containing these steps and reusing it across your pipelines. – Resorcinol

As of summer 2021, you don't need extra tasks to install/run the ReportGenerator, as it is now the default tool for reading coverage.covertura.xml files. You simply just need to update the summaryFileLocation of the PublicCodeCoverageResults task to point to the temp directory, like you are already doing with the script task passing in the path to the ReportGenerator – Tenno

I've tried these steps and unfortunately the publish task fails: Multiple file or directory matches were found. Using the first match: C:\agent\_work\38\s\artifacts\test-results\$redacted\In\redacted\coverage.cobertura.xml ##[error]No code coverage results were found to publish. I have no idea what's going on. When I run dotnet test locally I can see all the coverage.cobertura.xml files generates as expected. I can even see them in my pipeline's dotnet test output. – Fairway

Also, I can't use $(Agent.TempDirectory) because I'm running stateful on-premises build agents (not my choice). – Fairway

@NathanAldenSr: I have multiple test projects, and the build pipeline reports the same warning as you reported above - effective only one of the test project's coverage report gets published. Were you able to resolve the issue? – Waring

@Waring if you have multuple files generated you need to merge them first. Please check my edit. – Resorcinol

Until today, the coverage is incluede in dotnet command only when running on Windows. Reportgenerator and Coverlet are necessary to run on non Windows plataforms - learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/… – Romney

Answer 2

I have tried a lot to solve his problem...

So first of all, the problem is the test task of the pipeline (in my case the DotNetCoreCLI@2 task). If this task collects the code coverage it is automatically published with the tests in the .coverage format. For some reason this push of the .coverage file is at the end of the pipeline (or job - this I do not know) and therefore overrides all the previously uploaded files.

The Solution Use a Coverage Tool like OpenCover and collect the code coverage with that instead of the collection via the test task with CodeCoverage collection on.

Example:

OpenCover.Console.exe -target:"dotnet.exe" -targetArgs:"test {solution.sln}" -output:"{outputDir}\OpenCover.xml" {-oldstyle}

Afterwards convert the OpenCover file via the ReportGenerator task into Cobertura format.

And after that you can publish it as usual via the PublishCodeCoverageResult task.

You might have to try arround a little but with that solution in mind it will work.

Fontenot

Answer 3

I had the same problem with Azure DevOps and code coverage report rendering. I tried everything under the sun but to no avail. There were solutions with upvotes but none seems to be working for me. I was always getting that stupid download link only.

After reading @h0p3zZ answer, it gave me a new hint. For some reason, my dotnet test command was generating both coverage.cobertura.xml & *.coverage files and for some weird reason, that file gets precedence over the coverage.cobertura.xml files (which makes devops render the download link).

In my case, I was passing in a .runsettings file to the dontnet test command as shown below, and that made it spit out an additional .coverage file.

- task: DotNetCoreCLI@2
  displayName: Test
  inputs:
    command: test
    projects: '**/*Tests/*.csproj'
    arguments: --configuration $(buildConfiguration) --collect:"XPlat Code Coverage"  --settings CodeCoverage.runsettings

Once I got rid of --settings CodeCoverage.runsettings switch, everything started to work just fine.

Dolan