Suppress surefire warning for skipped tests (due to @Disabled/@Enabled annotations)?

Question

When disabling tests with a @Disabled* / @Enabled* annotation, those tests will be skipped as expected, but the surefire test runner also shows a [WARNING] in front of the result line for the affected class. My understanding is a dev team should only see warnings for things requiring further attention, hence i agree having a warning for certain tests (i.e. that were temporarily disabled due to an unresolved bug) can be a good thing.

Now: The test suite i'm writing covers code that is specific to different operating system environments – some of the tests only make sense to be executed when run in a windows environment, for example. Hence there is no point in issuing a warning for such tests (which are annotated with @EnabledOnOs(OS.WINDOWS)) as they are absolutely fine and skipping is expected (mandatory actually) – so there is simply no ToDo or issue here.

How can we control which skipped tests will result in a warning (i.e. via the @SuppressWarnings annotation or by some surefire configuration option)?

Blenny

Answer 1

There is no way to suppress warnings for specific tests.

However, you can at least reduce the amount of warnings which will be printed by Maven Surefire Plugin. For this, you can use the configuration property printSummary. See documentation for more info. So if you add such property in your pom.xml:

<properties>
    <surefire.printSummary>false</surefire.printSummary>
</properties>

you will not see a summary for each test, but only final aggregated summary in the end of your build.

[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ demo ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1                   
[INFO]                                                                       
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

As an alternative solution to conditions, I can suggest you use Tags.

So you can add 'os' tags for you test classes and/or methods

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;


class TaggingDemo {

    @Test
    @Tag("windows")
    void testingForWidows() {
    }

    @Test
    @Tag("linux")
    void testingForLinux() {
    }

}

Then when executing a Maven command to run tests you can specify which tags you can include or exclude. For example:

mvn clean test -Dgroups=!windows

So you will not see any warnings for tests which were 'excluded' from the build.

[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ demo ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0                   
[INFO]                                                                       
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Pneumo

+1 for the -Dgroups=!windows negation trick, did not know about this. Combined with tags that may provide a feasible way to reduce the unwanted warnings. However, i'm still looking for a fine-grained control over warning vs. info, b/c a warning will also be shown if an assumption is not met (i.e. headless vs. non-headless tests), which may be expected behaviour and shouldn't be graced with a warning. Apart from that do you know if one can also supply a default value for groups in pom.xml? – Blenny