Use NUnit Console Runner to run all tests under a folder

Question

I am trying to use NUnit Runners 2.6.4 to run all test assemblies in my test folder. My current command looks like this:

/nologo /noshadow /framework:net-4.0 /xml:.\test\TestResults.xml .\test\*.Test.dll

Unfortunately Nunit just throws a System.ArgumentException: Illegal characters in path.

Is there anyway I can achieve this?

Milkandwater

Answer 1

You can use following PowerShell command (for NUnit3, for NUnit2 change runner name):

PS> nunit3-console (ls -r *\bin\Debug\*.Tests.dll | % FullName | sort-object -Unique)

Command from previous answer runs each test assembly in separate nunit process synchronously. Presented here command runs all test assemblies in single nunit instance, which allows to leverage engine built-in parallel test run.

Remarks

1. Remember to tweak directory search pattern. Given example runs only assemblies ending with .Tests.dll and inside \bin\Debug directories.

2. Be aware of Unique filtering - you may not want to have it.

Threnode

I tried running the script as presented, but I get an error of 'Cannot convert the "FullName" value of type "System.String" to type "System.Management.Automation.ScriptBlock"'.....Any suggestions? – Astonied

Answer 2

It's not possible to use the wildcards for the input files, but you can specify multiple test libraries in the command line:

/nologo /noshadow /framework:net-4.0 /xml:.\test\TestResults.xml .\test\SomeLib1.Test.dll .\test\SomeLib2.Test.dll .\test\SomeLib3.Test.dll

From the official documentation:

An input file may be a managed assembly (.dll or .exe) containing tests or a project file recognized by NUnit. Out of the box, the following project types are recognized:

NUnit project files (.nunit)

Visual Studio solutions (.sln)

Visual Studio projects (.csproj, .vbproj, .csproj)

UPDATE

You could use a batch file to run the command for all files in the folder:

for /f %%f in ('dir .\test\ /b /s *.Test.dll') do nunit-console /nologo /noshadow /framework:net-4.0 /xml:.\test\TestResults.xml "%%f"

The dir command selects names of the files from the .\test\ folder using the *.Test.dll template. The names are passed to the command (nunit-console) one by one.

Solly

I apologise that I haven't specified that I am running nunit console through OpenCover via a FAKE build script, and the command above is put in the arguments attribute for OpenCover to inject into nunit console. Because of this, I can't use a batch script. Specifying the exact names of the dlls is preferably avoided, since adding a new test will result in the test script not detecting it, without any error message. – Milkandwater

'dir /b /s .\test\ *.Test.dll' – Barbecue

Answer 3

Try:

PS> nunit3-console (ls -r *\bin\Debug\*.Tests.dll | % { $_.FullName } | sort-object -Unique)

A little tweak to the answer from one_mile_run to fix the issue that caused an error:

Cannot convert the "FullName" value of type "System.String" to type "System.Management.Automation.ScriptBlock"

Polenta

Answer 4

One method that I've had success with to run tests in a certain folder is to use the where option. The =~ syntax is for using Regex (.NET flavor) to match namespace against. If the namespaces match the folder structure names per convention, you'll get the desired result. You'll want your Regex to match the top-level folder you're targeting.

nunit3-console.exe --where="namespace =~ ^YourBase.Namespace" yourproject.dll

Obreption