Login/Register
Why doesn't VSTS Build "Visual Studio Test" step find my NUnit tests?
Asked Answered
Solved azure-devopsnunitazure-pipelinesvstest.console.exe
T

2

7

I've configured a VSTS build job to build my MockingFrameworkExamples .NET 4.7.1 (C#) solution from GitHub. This solution includes five assemblies with NUnit v3.10 unit tests.

I included a Visual Studio Test step in the build definition, but when it runs, the console output says no tests were available in the test assemblies:

No test is available in D:\a\1\s\MSFakesExamples\bin\Release\MSFakesExamples.dll D:\a\1\s\MoqExamples\bin\Release\MoqExamples.dll D:\a\1\s\NSubstituteExamples\bin\Release\NSubstituteExamples.dll D:\a\1\s\RhinoMocksExamples\bin\Release\RhinoMocksExamples.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

What do I need to change to have the test runner to find and run my tests?

My local Visual Studio 2017 instance has no trouble finding and running the tests. The documentation for the Visual Studio Test step says it will run NUnit tests:

Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run.

I've made sure the assemblies are actually building and I've also utilized the Typemock SmartRunner build step to run the tests, which finds and executes them (though, it fails with the MS Fakes assembly, so I can't just use it).

Below are the full build step configuration and the console output from the "Visual Studio Test" step. The build process is configured to run on a "Hosted VS2017" agent.

snapshot of step configuration screen

2018-07-19T18:28:28.1197107Z ##[section]Starting: VsTest - testAssemblies
2018-07-19T18:28:28.1205564Z ==============================================================================
2018-07-19T18:28:28.1205798Z Task         : Visual Studio Test
2018-07-19T18:28:28.1206209Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test runner. Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run. Tests can be distributed on multiple agents using this task (version 2).
2018-07-19T18:28:28.1206596Z Version      : 2.136.10
2018-07-19T18:28:28.1206775Z Author       : Microsoft Corporation
2018-07-19T18:28:28.1207003Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-07-19T18:28:28.1207231Z ==============================================================================
2018-07-19T18:28:28.6970694Z Run the tests locally using vstest.console.exe
2018-07-19T18:28:28.6971006Z ========================================================
2018-07-19T18:28:28.6971725Z Test selector : Test assemblies
2018-07-19T18:28:28.6972103Z Test assemblies : **\release\*examples*.dll,!**\release\typemock*examples*.dll,!**\obj\**
2018-07-19T18:28:28.6972430Z Test filter criteria : null
2018-07-19T18:28:28.6972706Z Search folder : D:\a\1\s
2018-07-19T18:28:28.6972990Z Run settings file : D:\a\1\s
2018-07-19T18:28:28.6973481Z Run in parallel : false
2018-07-19T18:28:28.6973748Z Run in isolation : false
2018-07-19T18:28:28.6975710Z Path to custom adapters : null
2018-07-19T18:28:28.6975982Z Other console options : null
2018-07-19T18:28:28.6976254Z Code coverage enabled : true
2018-07-19T18:28:28.6976788Z Rerun failed tests: false
2018-07-19T18:28:28.6977166Z VisualStudio version selected for test execution : latest
2018-07-19T18:28:29.7035629Z ========================================================
2018-07-19T18:28:30.0191054Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @D:\a\_temp\88c64f61-8b81-11e8-a282-258e7155540f.txt
2018-07-19T18:28:33.5193511Z Microsoft (R) Test Execution Command Line Tool Version 15.7.2
2018-07-19T18:28:33.5198868Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-07-19T18:28:33.5199025Z 
2018-07-19T18:28:33.5440200Z vstest.console.exe 
2018-07-19T18:28:33.5441525Z "D:\a\1\s\MSFakesExamples\bin\Release\MSFakesExamples.dll"
2018-07-19T18:28:33.5442179Z "D:\a\1\s\MoqExamples\bin\Release\MoqExamples.dll"
2018-07-19T18:28:33.5442549Z "D:\a\1\s\NSubstituteExamples\bin\Release\NSubstituteExamples.dll"
2018-07-19T18:28:33.5442909Z "D:\a\1\s\RhinoMocksExamples\bin\Release\RhinoMocksExamples.dll"
2018-07-19T18:28:33.5443312Z /EnableCodeCoverage
2018-07-19T18:28:33.5443575Z /logger:"trx"
2018-07-19T18:28:35.6891585Z Starting test execution, please wait...
2018-07-19T18:28:42.3946126Z Microsoft (R) Coverage Collection Tool Version 15.0.30319.1
2018-07-19T18:28:42.3947458Z 
2018-07-19T18:28:42.3947721Z 
2018-07-19T18:28:42.3948539Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-07-19T18:28:42.3949162Z 
2018-07-19T18:28:42.3949300Z 
2018-07-19T18:28:42.3949487Z 
2018-07-19T18:28:42.3949624Z 
2018-07-19T18:28:42.6811429Z 2.7184
2018-07-19T18:28:53.8206723Z No test is available in D:\a\1\s\MSFakesExamples\bin\Release\MSFakesExamples.dll D:\a\1\s\MoqExamples\bin\Release\MoqExamples.dll D:\a\1\s\NSubstituteExamples\bin\Release\NSubstituteExamples.dll D:\a\1\s\RhinoMocksExamples\bin\Release\RhinoMocksExamples.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
2018-07-19T18:28:53.9442220Z Microsoft (R) Coverage Collection Tool Version 15.0.30319.1
2018-07-19T18:28:53.9443798Z 
2018-07-19T18:28:53.9445199Z 
2018-07-19T18:28:53.9465185Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-07-19T18:28:53.9465983Z 
2018-07-19T18:28:53.9466184Z 
2018-07-19T18:28:53.9466360Z 
2018-07-19T18:28:53.9466554Z 
2018-07-19T18:28:54.6483114Z 
2018-07-19T18:28:54.6491201Z Attachments:
2018-07-19T18:28:54.6491635Z   D:\a\1\s\TestResults\324c9712-62bd-4959-b3b6-40bebf7c05ec\VssAdministrator_factoryvm-az243 2018-07-19 18_28_41.coverage
2018-07-19T18:28:54.6491944Z 
2018-07-19T18:28:54.6574103Z Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.
2018-07-19T18:28:54.6717566Z ##[warning]No results found to publish.
2018-07-19T18:28:54.7119869Z ##[section]Finishing: VsTest - testAssemblies
Torrential answered 20/7, 2018 at 5:41 Comment(1)
I udpated all nuget packages, and then it worked... – Zonate
T
9

It turns out I had to add a NuGet reference to each of the unit test assemblies to the NUnit3TestAdapter NuGet package for this to work. No other config changes were needed.

Torrential answered 20/7, 2018 at 22:21 Comment(1)
This is the correct and recommended way to do this. It should work with only one test project having that reference, but this will change later. I have detailed this in the blogpost here. So you're doing this just fine and should be future-proof too :-) – Disentwine
P
0

You need to specify the path to the NUnit adapter using /TestAdapterPath.

Pucida answered 20/7, 2018 at 9:11 Comment(3)
How do I determine that path on the Hosted VS2017 agent? – Torrential
Ah... my assumption is that you put it where it is by use of the NUnit adapter package. Some versions cause the adapter to be copied to the output dir, so no further action is needed. Others do not, so you have to specify the path. – Pucida
AFAIK, you can't rely on any hosted environment having the NUnit adapter installed. – Pucida

© 2022 - 2024 — McMap. All rights reserved.