Is there a concept of test suites in Gradle/Spock land?

Question

Groovy/Gradle project here that uses Spock for unit testing.

Does Spock and/or Gradle support test suites or named sets of tests? For reasons outside the scope of this question, there are certain Spock tests (Specifications) that the CI server just can't run.

So it would be great to divide all my app's Spock tests into two groups:

1. "ci-tests"; and
2. "local-only-tests"

And then perhaps we could invoke them via:

./gradlew test --suite ci-tests

etc. Is this possible? If so, what does the setup/config look like?

Reinareinald

Spock is based on JUnit, so I would think you could use the standard approach to developing suites. For the Gradle integration take a look at the Gradle unit test plugin documentation for junit. – Orang

Answer 1

You can annotate the tests that should not run in your CI server with the Spock annotation @IgnoreIf( ).

See the documentation here: https://spockframework.github.io/spock/docs/1.0/extensions.html#_ignoreif

All you need to do is let the CI server set an environment variable, and exclude the test class if that variable is set.

Spock even have properties inside the closure to make it easy:

@IgnoreIf({ sys.isCiServer })

Piatt

Answer 2

I would set up a submodule my-app-ci-test, with the following in build.gradle:

test {
    enabled = false
}
task functionalTest(type: Test) {
}

Then you place your tests in src/test/groovy and run ./gradlew functionalTest.

Alternatively, you could include them in the same module and configure the test and functionalTest tasks with includes / excludes

test {
    exclude '**/*FunctionalTest.groovy'
}
task functionalTest(type: Test) {
    include '**/*FunctionalTest.groovy'
}

Ohmage

Answer 3

If you use Junit test-runner for Spock tests, you may use @Category annotation. Example by article and official documentation:

 public interface FastTests {
 }

 public interface SlowTests {
 }

 public interface SmokeTests
 }

 public static class A {
     @Test
     public void a() {
         fail();
     }

     @Category(SlowTests.class)
     @Test
     public void b() {
     }

     @Category({FastTests.class, SmokeTests.class})
     @Test
     public void c() {
     }
 }

 @Category({SlowTests.class, FastTests.class})
 public static class B {
     @Test
     public void d() {
     }
 }

test {
    useJUnit {
        includeCategories 'package.FastTests'
    }
    testLogging {
        showStandardStreams = true
    }
}

Dermatosis

Categories are no longer supported in JUnit 5, see https://mcmap.net/q/1511051/-is-there-a-concept-of-test-suites-in-gradle-spock-land for a workaround – Goosander

@JakubBochenski, I think in Junit 5 it should be just replaced by @Tag annotation as by of the migration guide: arhohuttunen.com/junit-5-migration/#categories – Dermatosis

if you want to take the time to edit/sed all your tests then sure, that's the way to go – Goosander

Answer 4

You can use the following SpockConfiguration.groovy to allow passing include/exclude via system properties

    runner {
        exclude {
            System.properties['spock.exclude.annotations']
                ?.split(',')
                *.trim()
                ?.each {
                    try {
                        annotation Class.forName(it)
                        println "Excluding ${it}"
                    } catch (ClassNotFoundException e) {
                        println "Can't load ${it}: ${e.message}"
                    }
                }
        }
        include {
            System.properties['spock.include.annotations']
                ?.split(',')
                *.trim()
                ?.each {
                    try {
                        annotation Class.forName(it)
                        println "Including ${it}"
                    } catch (ClassNotFoundException e) {
                        println "Can't load ${it}: ${e.message}"
                    }
                }
        }
    }

Goosander