How do I use dotnet test to run tests from multiple libraries with a single pass/fail summary

How do I run unit tests with dotnet test if I have multiple test libraries in a code base?

I can run dotnet test, and it will find and run all tests even across multiple libraries, but it runs and reports each test library run independently:

$ dotnet test
Test run for C:\Users\mark\Documents\Redacted.Test\bin\Debug\netcoreapp2.1\Redacted.Test.dll(.NETCoreApp,Version=v2.1)
Test run for C:\Users\mark\Documents\Redacted\Redacted.SqlAccess.Test\bin\Debug\netcoreapp2.1\Redacted.SqlAccess.Test.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Starting test execution, please wait...

Test Run Successful.
Total tests: 59
     Passed: 59
 Total time: 3.1779 Seconds
Test run for C:\Users\mark\Documents\Redacted\Redacted.RestApi.Tests\bin\Debug\netcoreapp2.1\Redacted.RestApi.Tests.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Test Run Successful.
Total tests: 99
     Passed: 99
 Total time: 9.8155 Seconds

Test Run Successful.
Total tests: 25
     Passed: 25
 Total time: 21.2894 Seconds

In this example, there's two test libraries, so I get two test result outputs.

This may work OK if the code has already been compiled, but in a clean build, there's going to be a lot of output from the compiler. This can easily cause one of the test run summaries to scroll past the visible part of the screen.

That's a problem if that test run fails.

How can I collapse all the unit tests to a single pass/fail summary?

On .NET 4.x, I could, for instance, use xUnit.net's console runner to run all test libraries as a single suite:

$ ./packages/xunit.runner.console.2.4.0/tools/net461/xunit.console BookingApi.UnitTests/bin/Debug/Ploeh.Samples.Booking
Api.UnitTests.dll BookingApi.SqlTests/bin/Debug/Ploeh.Samples.BookingApi.SqlTests.dll
xUnit.net Console Runner v2.4.0 (64-bit Desktop .NET 4.6.1, runtime: 4.0.30319.42000)
  Discovering: Ploeh.Samples.BookingApi.UnitTests
  Discovered:  Ploeh.Samples.BookingApi.UnitTests
  Starting:    Ploeh.Samples.BookingApi.UnitTests
  Finished:    Ploeh.Samples.BookingApi.UnitTests
  Discovering: Ploeh.Samples.BookingApi.SqlTests
  Discovered:  Ploeh.Samples.BookingApi.SqlTests
  Starting:    Ploeh.Samples.BookingApi.SqlTests
  Finished:    Ploeh.Samples.BookingApi.SqlTests
=== TEST EXECUTION SUMMARY ===
   Ploeh.Samples.BookingApi.SqlTests   Total:  3, Errors: 0, Failed: 0, Skipped: 0, Time: 3.816s
   Ploeh.Samples.BookingApi.UnitTests  Total:  7, Errors: 0, Failed: 0, Skipped: 0, Time: 0.295s
                                              --          -          -           -        ------
                                 GRAND TOTAL: 10          0          0           0        4.111s (5.565s)

Notice how this produces a single summary at the bottom of the screen, so that I can immediately see if my tests passed or failed.

Use dotnet vstest to run multiple assemblies.

 PS> dotnet vstest --help

Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Usage: vstest.console.exe [Arguments] [Options] [[--] <RunSettings arguments>...]]

Description: Runs tests from the specified files.

Arguments:

[TestFileNames]
      Run tests from the specified files. Separate multiple test file names
      by spaces.
      Examples: mytestproject.dll
                mytestproject.dll myothertestproject.exe
...

Note that this method requires that you point at compiled assemblies (as opposed to dotnet test, which wants you to point at project files, and will optionally build things first for you).

Recommended topics

Hot tags