Azure devops reportgenerator task can't find coverage.cobertura.xml file

Question

as the title suggests, i'm trying to get the code coverage to work on Azure Devops Pipeline.

This is the pipeline:

trigger:
 - master

pool:
   vmImage: 'windows-latest'

variables:
   solution: '**/*.sln'
   buildPlatform: 'Any CPU'
   buildConfiguration: 'Release'

steps:
- task: DotNetCoreInstaller@0
  displayName: 'Installing .NET Core SDK...'
  inputs:
    version: 3.1.101

- script: dotnet build --configuration $(buildConfiguration)
  displayName: 'Building $(buildConfiguration)...'

- task: DotNetCoreCLI@2
  displayName: 'Executing dotnet tests...'
  inputs:
    command: test
    arguments: '--configuration $(BuildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)\TestResults\Coverage\'
    projects: '**/*Test/*.csproj'
    nobuild: true

- script: |
  dotnet tool install -g dotnet-reportgenerator-globaltool
  reportgenerator -reports:$(Build.SourcesDirectory)\TestResults\Coverage\coverage.cobertura.xml - targetdir:$(Build.SourcesDirectory)\CodeCoverage -reporttypes:HtmlInline_AzurePipelines;Cobertura
  displayName: 'Creating code coverage report...'

- task: PublishCodeCoverageResults@1
  displayName: 'Publishing code coverage...'
  inputs:
    codeCoverageTool: Cobertura
    summaryFileLocation: '$(Build.SourcesDirectory)\CodeCoverage\Cobertura.xml'
    reportDirectory: '$(Build.SourcesDirectory)\CodeCoverage'

My solution contains a front-end project and a xunit test project with the name ending in "Test".

Tests are running just fine but i always get an error at this line:

reportgenerator -reports:$(Build.SourcesDirectory)\TestResults\Coverage\coverage.cobertura.xml ...

With the actual pipeline the error is:

The report file 'D:\a\1\s\TestResults\Coverage\coverage.cobertura.xml' is invalid. File does not exist (Full path: 'D:\a\1\s\TestResults\Coverage\coverage.cobertura.xml').

I tried to generate test reports locally on my pc and it works just fine. It means that the packages are ok.

What i'm doing wrong?

Imphal

Answer 1

Just as the error message indicates, it needs the coverage.cobertura.xml but it's not found.(not generated!)

In your script:

reportgenerator -reports:$(Build.SourcesDirectory)\TestResults\Coverage\coverage.cobertura.xml - ...

The coverage.cobertura.xml is expected input file but it's not generated. In my experience, it's because your dotnet test task didn't generate this file successfully.

I believe your dotnet test task may succeed without any error, but if we check the log we can find it didn't generate the expected coverage.cobertura.xml file. For the cause of issue you can refer to this document:

VSTest Integration

If you use dotnet add package coverlet.collector command to add coverlet.collector package to your project, you should use command dotnet test --collect:"XPlat Code Coverage" to generate a coverage.cobertura.xml file containing the results.

MSBuild Integration

And if you add coverlet.msbuild package to your project using dotnet add package coverlet.msbuild. You then need to enable code coverage in this way: dotnet test /p:CollectCoverage=true. And after the above command is run, a coverage.json file containing the results will be generated.

Since your script below dotnet test command wants a coverage.cobertura.xml file, you should use VSTest Integration, and it means you should use command dotnet test --collect:"XPlat Code Coverage" instead of dotnet test /p:CollectCoverage=true. That makes sense to generate the missing file.

Note:

1.In my test, dotnet test task won't throw any error though the code-coverage report is not generated. We can always check the task log for real info we need.

2.Also, check this document you can find:

If you're building on the Windows platform, code coverage metrics can be collected by using the built-in coverage data collector.

and

If you're building on Linux or macOS, you can use Coverlet or a similar tool to collect code coverage metrics.

So according to the content of your script, it's more recommended to run in Linux or macOS agents. If you're just wanting it to run in windows, it has default built-in coverage data collector.

Hope all above helps to resolve your puzzle and original issue.

Mishnah

Thank you very much. I found the issue. I simply forgot to push the package "coverlet.msbuild". Now i found out that i CAN'T use all this stuff for using a code coverage branch policy in a pull request. It only supports .coverage files. – Imphal

Answer 2

In addition to @LoLance's excellent answer, I found that you need certain NuGet Packages otherwise it won't generate the coverage.cobertura.xml file.

Nunit:

<ItemGroup>
      <PackageReference Include="coverlet.collector" Version="3.1.2">
          <PrivateAssets>all</PrivateAssets>
          <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      </PackageReference>
      <PackageReference Include="GenFu" Version="1.6.0" />
      <PackageReference Include="Moq" Version="4.18.2" />
      <PackageReference Include="NUnit" Version="3.13.3" />
      <PackageReference Include="NUnit.Analyzers" Version="3.3.0" />
      <PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
      <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
      <PackageReference Include="NunitXml.TestLogger" Version="3.0.127" />
      <PackageReference Include="ReportGenerator" Version="5.1.10" />
    <PackageReference Include="Polly" Version="7.2.3" />
  </ItemGroup>

Xunit:

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.11.0" />
    <PackageReference Include="Moq" Version="4.16.1" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.1.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

Collum

You dont need all those dependencies. Probably coverlet.collector though. – Soyuz