Ignore IgnoreAttribute

Question

We have MSTest tests which automatically run in hourly production. One of these tests is marked with [Ignore] attribute because it is not yet ready to run it in our production environment.
Now I want to start that test (only) on my local environment because my local environment is ready for that test.
I try to run that test by clicking on ReSharper's icon on the left side of test code or by clicking Run Selected Tests icon in Unit Test Sessions window and nothing happens.

I fix it currently by commenting out the [Ignore] line. But now I need to be aware to remove the comment characters (//) before checking-in the code.

Is there another way to temporarly run an [Ignore]'d test ?

Buskined

The strange thing is that this never happened before to us. With VS2010 and R#6 we could execute an ignored test clicking on the left ball over the code and executing "Run". – Serenata

Answer 1

Recently when I have encountered problems such as this, I add a new Build Configuration to the visual studio project named something such as "Local Developer Debug" and use the settings from the existing Debug configuration. Then I go to "Project -> MyProjectName Properties -> Build", make sure "Local Developer Debug" is the selected configuration and add "LOCALDEVBUILD" to "Conditional compliation symbols". This allows for the use of preprocessor directives to 'toggle' code at compile time:

#if (!LOCALDEVBUILD)
    [Ignore]
#endif

Not sure if this is what you're looking for... but it allows you to run/utilize specific code depending on the intentions of the build (via the build configuration)... With this method you can leave the test ignored for more 'official' builds, but still execute it at your leisure if you so desire.

Squad

Answer 2

You could also use a test category to mark the test methods you do not want to be included in your automated test, e.g.

[TestCategory("IgnoreOnBuild")]

Thus not using the [Ignore] attribute. And combine this with a filter in your build definition, under Basic -> Automated Test -> Test Source -> Test Case Filter:

TestCategory!=IgnoreOnBuild

Gatehouse

And inside the Visual Studio Test Manager window, you can filter for Trait:"IgnoreOnBuild" (use -Trait:"IgnoreOnBuild" to exclude them) – Headrest

Where is Basic -> Automated Test -> Test Source -> Test Case Filter: ? What top-level menu or context menu is this under ? – Welcy

Answer 3

It looks like there are other ways to enable/disable tests using the testrunconfig file. However, if you use the IgnoreAttribute it compiles in with the code and therefore cannot be easily toggled.

From the documentation:

The Ignore attribute for a unit test resides in the source file of the unit test, together with the other attributes and source code of the test. This means that, if you disable a unit test in the Test Manager window, and later run the test by using the MSTest.exe command-line utility, the test still runs. But if you mark a unit test with the Ignore attribute, compile the assembly, and then run the test by using MSTest.exe, the test does not run. It does not run because the Ignore attribute has become part of the test.

Dree

OP wants the opposite of this feature. This only allows one to ignore a test locally, but not ignore it during automated testing. OP wishes to continue to ignore the test during automated testing, but not ignore it locally. – Deterrence

@Deterrence that's not true. Decorating a test with [Ignore] compiles the decoration into the assembly and it doesn't matter where or how the test is run ( local machine, manual kick-off or test runner, automated pipeline deploy to shared environment, etc. ) the ignored test will be excluded. – Welcy

Answer 4

As @Lasse mentioned, it is possible to set [TestCategory("DockerEnvironmentRequired")]. In addition to that, you can set a filter in your local .runsettings that includes or excludes the tests with the specified TestCategory.

Example Test:

[TestClass]
[TestCategory("DockerEnvironmentRequired")]
public class MyTests
{
    [TestMethod]
    public async Task XXX_ShouldSuccess()
    {
        ...
    }
}

Example external build environment .runsettings:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <TestRunParameters>
        ...
    </TestRunParameters>
    <RunConfiguration>
        <TestCaseFilter>TestCategory!=DockerEnvironmentRequired</TestCaseFilter>
    </RunConfiguration>
</RunSettings>

Locally, you would remove the TestCaseFilter node in your .runsettings to run them.

Alternative with CLI: dotnet test --filter TestCategory!=DockerEnvironmentRequired

Gazelle

Answer 5

I have another approach. This automatically removes [Ignore] attributes on the developer side.

1. Add Directory.Build.targets to your root tests folder

<Project>
    
    <PropertyGroup Condition="'$(CI)' == 'true'">
        <DefineConstants>$(DefineConstants);CONTINUOUS_INTEGRATION_BUILD</DefineConstants>
    </PropertyGroup>

</Project>

2. Add this code to local tests:

#if CONTINUOUS_INTEGRATION_BUILD
    [Ignore]
#endif

You can also check possible CI environment variables here: https://github.com/dotnet/reproducible-builds/blob/f41ae4c0a4400acce7797691d69134f2562b26b0/src/DotNet.ReproducibleBuilds/DotNet.ReproducibleBuilds.props#L32

Signal

This works, but I personally hate having directives littered around in general code like this. – Grass