Skip unit tests while running on the build server

Question

We have some UI integration tests that can't be run on the build server as launching test GUI app requires running the build agent as an user (instead of a service which is how it's currently setup).

This causes the build pipeline to get stuck. So I'd like to run these tests locally but not on the build server.

Is there a way to achieve this using xUnit or MSTests and Azure DevOps build pipeline?

Ingather

Answer 1

You sure can.

Setup an Environment variable to indicate if it's running on the build server in your build.yml file.

variables:
- name: IsRunningOnBuildServer
  value: true

Answer 1: Using xUnit

Now create a custom fact attribute to use this:

// This is taken from this SO answer: https://mcmap.net/q/57627/-how-do-i-skip-specific-tests-in-xunit-based-on-current-platform
public class IgnoreOnBuildServerFactAttribute : FactAttribute
{
    public IgnoreOnBuildServerFactAttribute()
    {
        if (IsRunningOnBuildServer())
        {
            Skip = "This integration test is skipped running in the build server as it involves launching an UI which requires build agents to be run as non-service. Run it locally!";
        }
    }
    /// <summary>
    /// Determine if the test is running on build server
    /// </summary>
    /// <returns>True if being executed in Build server, false otherwise.</returns>
    public static bool IsRunningOnBuildServer()
    {
        return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
    }
}

Now use this FactAttribute on your test methods that you want to skip running on build server. For eg:

[IgnoreOnBuildServerFact]
public async Task Can_Identify_Some_Behavior_Async()
{
   // Your test code...
}

Answer 2: Using MSTests

Create a custom test method attribute to override the Execute method:

public class SkipTestOnBuildServerAttribute : TestMethodAttribute
{
    public override TestResult[] Execute(ITestMethod testMethod)
    {
        if (!IsRunningOnBuildServer())
        {
            return base.Execute(testMethod);
        }
        else
        {
            return new TestResult[] { new TestResult { Outcome = UnitTestOutcome.Inconclusive } };
        }
    }

    public static bool IsRunningOnBuildServer()
    {
        return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
    }
}

Now use this TestMethodAttribute on your test methods that you want to skip running on build server. For eg:

[SkipTestOnBuildServer]
public async Task Can_Identify_Some_Behavior_Async()
{
   // Your test code...
}

Ingather

For GitHub actions env: IsRunningOnBuildServer: 'true' – Ceasar

In azure devops you can use the build in TF_BUILD env variable. return !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("TF_BUILD")); – Virnelli

Answer 2

You can filter out tests based on namespace etc.

dotnet test --filter FullyQualifiedName!~IntegrationTests

This will run all tests NOT containing "IntetegrationTests" in it's namespace.

You can read more about test filtering here: https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests

Maddux