xUnit - How to select a single Theory when executing from dotnet test CLI

Question

Say I have a test:

  [Theory]
  [InlineData("one")]
  [InlineData("two")]
  public void ShouldSelectSingleTheoryFromDotnetTest(string s)
  {
      Assert.True(true);
  }

I want to execute just the test with Theory data = "one" from the command line using dotnet test.... Is this possible?

I know we can select tests using Traits, but I cannot see how to associate a single trait with each single line of Theory data.

As a workaround I thought maybe I could use the technique of dynamically skipping tests, but still to do that I'd need to read an argument from the command line and can't see how to do that either. One workaround to that, might be to set an environment variable on the command line, then run the tests using the dynamic skipping pattern, where the dynamic skip logic would read the command line argument. But that is very clunky.

Konyn

You can always split into two tests. To avoid duplication you can wrap test with the method and call it in both tests. Then you will be able to run test based on it's name from command line – Issykkul

Thanks Fabio that'd work well for small cases. Unfortunately in my case there are hundreds of tests, and the Theory data is being provided by a class which dynamically produces many 10s of rows of data (Browser/OS combinations). I tried to simplify the problem for posting here. – Konyn

Answer 1

You can use --filter parameter of dotnet test.

First of all use -t|--list-tests to see the test names, then you will get something like:

dotnet test -t

The following Tests are available:
    xUnitSkip.UnitTest1.ShouldSelectSingleTheoryFromDotnetTest(s: "one")
    xUnitSkip.UnitTest1.ShouldSelectSingleTheoryFromDotnetTest(s: "two")

Then you can use this information to filter on the DisplayName.

Was not able to figure out how to escape double quotes and parenthesis in the name so used "workaround" to combine filtering on FullyQualifiedName and DisplayName:

dotnet test --filter "FullyQualifiedName~ShouldSelectSingleTheoryFromDotnetTest&DisplayName~one"

Which gives me the following output:

Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: < 1 ms - xUnitSkip.dll (net7.0)

Compared to one without filter:

Passed! - Failed: 0, Passed: 2, Skipped: 0, Total: 2, Duration: 4 ms - xUnitSkip.dll (net7.0)

UPD

Was able to figure out the double quotes escaping, so parameter names can be used - replace double quotes with %22:

dotnet test --filter "DisplayName~s: %22one%22"

UPD2

Courtesy of @Jan Zakrzewski:

Parenthesis can be escaped by using \ i.e. \( and \).

Which makes the full filter looking like the following:

dotnet test --filter "DisplayName=xUnitSkip.UnitTest1.ShouldSelectSingleTheoryFromDotnetTest\(s: %22two%22\)"

Garett

Answer 2

you could separately run tests using XUnit Traits i.e. something like that

  [Trait("Category", "one")]
  [Theory]
  [InlineData("one")]
  public void ShouldSelectSingleTheoryFromDotnetTest(string s)
  {
      myTestImpl(s);
  }

  [Trait("Category", "two")]
  [Theory]
  [InlineData("two")]
  public void ShouldSelectSingleTheoryFromDotnetTest(string s)
  {
      myTestImpl(s);
  }

  private void myTestImpl(string s)
  {
      Assert.True(true);
  }

to run only tests for "one" inline data - dotnet test --filter Category=one and for second - dotnet test --filter Category=two

Softener

You don't need categories for this, you can filter just by test name. – Garett

Categories are more convenient than filters, at least because you could rename test function name and inline data, with categories you don't need to change cmd line to run tests – Softener

While I completely agree with the general approach (personally extensively use categories) in this particular case I don't see any major advantages. – Garett