Skipping test cases (be it facts or cases from a theory) dynamically, based on a run-time choice, can be done by extending xUnit's features.
I recommend adapting the DynamicSkipExample from xunit.samples (licensed Apache 2.0, copyrighted by their respective authors), which includes this sample usage:
[SkippableTheory]
[InlineData(null)]
[InlineData("Hello, world!")]
public void DynamicallySkipped(string value)
{
if (value == null)
throw new SkipTestException("I don't like to run against null data.");
Assert.StartsWith("foo", value);
}
So basically, in usage, you can dynamically throw a specific exception that leads to Skipped test cases instead of failed ones.
The code can be found in linked repository under its own license. It wires things together with an IXunitTestCaseDiscoverer
but the interesting bit are in the TestCase
(one for Fact
and one for Theory
):
public override async Task<RunSummary> RunAsync(IMessageSink diagnosticMessageSink,
IMessageBus messageBus,
object[] constructorArguments,
ExceptionAggregator aggregator,
CancellationTokenSource cancellationTokenSource)
{
// Duplicated code from SkippableFactTestCase. I'm sure we could find a way to de-dup with some thought.
var skipMessageBus = new SkippableFactMessageBus(messageBus);
var result = await base.RunAsync(diagnosticMessageSink, skipMessageBus, constructorArguments, aggregator, cancellationTokenSource);
if (skipMessageBus.DynamicallySkippedTestCount > 0)
{
result.Failed -= skipMessageBus.DynamicallySkippedTestCount;
result.Skipped += skipMessageBus.DynamicallySkippedTestCount;
}
return result;
}
And its associated IMessageBus
:
public bool QueueMessage(IMessageSinkMessage message)
{
var testFailed = message as ITestFailed;
if (testFailed != null)
{
var exceptionType = testFailed.ExceptionTypes.FirstOrDefault();
if (exceptionType == typeof(SkipTestException).FullName)
{
DynamicallySkippedTestCount++;
return innerBus.QueueMessage(new TestSkipped(testFailed.Test, testFailed.Messages.FirstOrDefault()));
}
}
// Nothing we care about, send it on its way
return innerBus.QueueMessage(message);
}