How to generate code coverage report for asp.net unit tests in Azure DevOps build

Question

I need guidance in generating code coverage report of Asp.net unit tests in azure build pipeline. My project is based on .Net Framework 4.6.

I am able to run all the unit tests using "visual studio test" task.

I tried the "report generator" task, but it require cobertura or jacoco etc xml files, which am unable to generate in the build pipeline.

Expectation - I want to get code coverage report for the runned unit tests which will show complete information like the lines coverage, branch coverage, function coverage etc. same as what "report generator" generates.

Note: I am able to generate the reports using opencover and reportgenerator on my local system but am unable to find a way to do the same in azure build pipeline.

Suspicious

You can use coverlet as explain in this post: Computing code coverage for a .NET Core project with Azure DevOps and Coverlet – Livestock

Thank you for the reply, @meziantou. My project is based on .net framework 4.6 and not .net core. I have updated the same in the description. I apologize for not mentioning the same before. Just want to confirm can I use coverlet in .net framework 4.6. If yes, then which azure build task I need to use. – Suspicious

Answer 1

To get the Code Coverage results in .Net framework you just need to enable it in the "Visual Studio Test" task:

If you are use .yml builds the syntax is:

- task: VSTest@2
  inputs:
    codeCoverageEnabled: true

Results:

Note: if you use Microsoft Hosted Agent you will see the results, if you use Self Hosted Agent you must Visual Studio Enterprise version to see the Code Coverage results.

If you want more detailed code coverage report you can use coverlet in .Net framework by install the tool during the pipeline and then generate the report. you can do with a PowerShell script:

dotnet tool install dotnet-reportgenerator --tool-path . --version 4.0.12
dotnet tool install coverlet.console --tool-path . --version 1.4.1
mkdir .\reports
$unitTestFile = gci -Recurse | ?{ $_.FullName -like "*bin\*test*.dll" }
$coverlet = "$pwd\coverlet.exe"
& $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura"
gci -Recurse |
?{ $_.Name -eq "coverage.cobertura.xml"} |
%{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reportstypes:HTMLInline;HTMLChart" }

Then add "Publish code coverage" task with these parameters:

Results:

Haftarah

Thank you very much @Shayki. Initially, when I tried to run your code in azure devops, I was getting the error "Unable to find package dotnet-reportgenerator". But when I changed "dotnet-reportgenerator" in "dotnet tool install dotnet-reportgenerator --tool-path . --version 4.0.12" to "dotnet-reportgenerator-globaltool". It started working. – Suspicious

However @Shayki, Out of curiosity just want to know do we need gci -Recurse | ?{ $_.Name -eq "coverage.cobertura.xml"} | %{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reportstypes:HTMLInline;HTMLChart" } in powershell script as Publish code coverage results task will publish the reports anyway. – Suspicious

@Suspicious You're welcome! thanks for your comment! I don't really know why, I just saw it in elsewhere and it worked for me :) – Haftarah

No problem, @Shayki. I have one more requirement. I need to display some of the code coverage information in email too. Can you tell me whether there is any api through which I can get the generated coverage information, if not full atleast some part of it. I have tried (learn.microsoft.com/en-us/rest/api/azure/devops/test/…) but not getting any information – Suspicious

Please open a new question exactly what you did you try. – Haftarah

Ok @Shayki. Will do that. In case any more help is required regarding this question will comment the same here. Thanks once again. – Suspicious

Hi @Shayki, we accomplished this task using the .Net Core CLI. Is there a way to accomplish this task (generation of code coverage) without using the .Net Core CLI. I mean without installing .Net Core on the build server. – Suspicious

Do you want the detailed coverage or the basic? – Haftarah

Yes @Shayki, I need the detailed coverage. – Suspicious

@Suspicious You need to include corbertura or Jacoco in your project, google it. in the current situation you can download the basic results and open it in the VS then you will see the all details. – Haftarah

Okay. @Shayki. Actually my development server doesn't have .Net Core installed, that's why I want to avoid installing the same, and also I don't want to make any change in the project for generation of code coverage. – Suspicious

Why now stay with .net core? – Haftarah

It is development server and I can't make changes as per my wish without approval. If at all it is not possible without .Net Core then will think of going with it. – Suspicious

Currently, I am thinking of downloading OpenCover or any other tool on the build server and run the same exe from the build pipeline to generate the cobertura or jacoco file and then consume the output in Report Generator task of the build pipeline. I think this should work. I am interested in knowing if there is any way by which I can download opencover.exe from build pipeline without downloading it externally. – Suspicious

I don’t know how :/ but what is the difference between .net core to open cover? Both you want to download during the build. – Haftarah

Actually, I am unable to find a way to download opercover from build pipeline, so will have to download it once on the build server before triggering build. Whereas, in .Net Core we are able to download coverlet using .Net Core CLI during build. – Suspicious

So why you prefer opencover rather them .net core + coverlet (that works with regular .net framework)? – Haftarah

Only want to avoid any extra installation on the development server. That's it, nothing else. And also want to know, if there is any other way out or not. – Suspicious

But you want to install opencover so it also extra installation.. no? – Haftarah

Yes it is. But for generating code coverage I need to install something, either opencover or coverlet or any other. – Suspicious

Hi @ShaykiAbramczyk and Shubho, I tried the PS script shared by Shayki Abramczyk but it is failing with error "The filename or extension is too long" error. There are multiple test files\dll for which i want to generate the coverage file. I am getting error like below-- Program 'coverlet.exe' failed to run: The filename or extension is too longAt E:\Cache\Agent03\18\s\codecoverage.ps1:8 char:1 + & $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vs – Staffordshire

This is the script i am using --- CD $PSScriptRoot dotnet tool install dotnet-reportgenerator-globaltool --tool-path . --version 4.0.12 dotnet tool install coverlet.console --tool-path . --version 1.4.1 mkdir .\reports1 $unitTestFile = gci -Recurse | ?{ $_.FullName -like "test.dll" } Write-Host "$unitTestFile" $coverlet = "$pwd\coverlet.exe" & $coverlet $unitTestFile.FullName --target "dotnet" --targetargs "vstest $($unitTestFile.FullName) --logger:trx" --format "cobertura" – Staffordshire

Answer 2

For anyone looking for code coverage in Azure Devops (using classic editor, without Yaml), in current .NET (core) 5, with xUnit tests:

1. In your xUnit test project, add following (it generally comes by default in .NET 5, xUnit template now):

   \<PackageReference Include="coverlet.collector" Version="3.0.3" /\>

   Keep checking for new version.

2. Head to Azure devops, create pipeline using classic editor. Do the restore, build steps. (Or you can choose dotnet core template as below):

3. In the test command of dotnet core task, add argument - --collect:"XPlat Code Coverage". Remember "XPlat Code Coverage" is friendly name and case sensitive. Your test command would look like: Check this checkbox if you want to publish your test results: Publish test results and code coverage, but it won't publish code coverage. The functionality is not yet working (at least not in non-windows).

4. Next add - Publish code coverage results task. Choose "Code coverage tool" as "Cobertura" and in "Summary file" field, add $(Agent.TempDirectory)/**/coverage.cobertura.xml. Looks like this:

5. Save and Queue (in any agent, I use Ubuntu) and see the result once pipeline run completes:

Mopup