Login/Register
Run a specific test in a single test class with Spock and Maven
Asked Answered
Solved maventestinggroovyspockmaven-surefire-plugin
P

3

13

I am using Spock framework for testing (1.0-groovy-2.4 release). Junit offers this option to run a specific test using the command line (with Maven):

mvn -Dtest=TestCircle#mytest test

Question: How can I do this with Spock?

This version has a dependency on Junit 4.12, it is stated in Junit documentation that this feature is supported only for Junit 4.x, basically Spock should propose something similar.

Pouliot answered 27/1, 2016 at 8:7 Comment(0)
W
10

After a further investigation, the answer is most probably: No, you can't invoke specific test methods with Spock using that Surefire Plugin feature. Below the reason.

Given the following Spock test case:

import spock.lang.Specification

class HelloSpec extends Specification {

    def hello = new Main();

    def sayHello() {
        given: "A person's name is given as a method parameter."
        def greeting = hello.sayHello("Petri");

        expect: "Should say hello to the person whose name is given as a method parameter"
        greeting == "Hello Petri";

        println "hello from HelloSpec"
    }
}

And given the following plugins configuration:

<plugin>
    <groupId>org.codehaus.gmavenplus</groupId>
    <artifactId>gmavenplus-plugin</artifactId>
    <version>1.5</version>
    <executions>
        <execution>
            <goals>
                <goal>addTestSources</goal>
                <goal>testCompile</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <sources>
            <fileset>
                <directory>${pom.basedir}/src/test/java</directory>
                <includes>
                    <include>**/*.groovy</include>
                </includes>
            </fileset>
        </sources>
    </configuration>
</plugin>

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
            <include>**/*Spec.java</include>
        </includes>
    </configuration>
</plugin>

It runs fine as part of the Maven test phase executing mvn clean test:

-------------------------------------------------------   
 T E S T S   
-------------------------------------------------------   
Running com.sample.HelloSpec  
hello from HelloSpec   
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.314 sec - in com.sample.HelloSpec

Even executing mvn clean test -Dtest=HelloSpec you get exactly the same result as above, successfully execution.

So, why we can't run a single method?

If we execute the javap command on the HelloSpec sample test above we get the following output:

Compiled from "HelloSpec.groovy"
public class com.sample.HelloSpec extends spock.lang.Specification implements groovy.lang.GroovyObject {
  public static transient boolean __$stMC;
  public com.sample.HelloSpec();
  public void $spock_feature_0_0();
  protected groovy.lang.MetaClass $getStaticMetaClass();
  public groovy.lang.MetaClass getMetaClass();
  public void setMetaClass(groovy.lang.MetaClass);
  public java.lang.Object invokeMethod(java.lang.String, java.lang.Object);
  public java.lang.Object getProperty(java.lang.String);
  public void setProperty(java.lang.String, java.lang.Object);
  public java.lang.Object getHello();
  public void setHello(java.lang.Object);
}

So in the generated bytecode there is no sayHello method, because Spock changes methods name in order to allow spaces in them. So the method name you write is never actually the real method name as part of the compiled class.

In this case, the method name is most probably $spock_feature_0_0, not something really friendly.

This is also confirmed in this answer and the comments of Peter Niederwieser, author of Spock actually, so a more than reliable source.

You could try running the following: 

mvn clean test -Dtest=HelloSpec#*spock*

which should indeed match the real method name, but you would however get an error

-------------------------------------------------------   
 T E S T S   
-------------------------------------------------------   
Running com.sample.HelloSpec   
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.007 sec <<< FAILURE! - in com.sample.HelloSpec  
initializationError(org.junit.runner.manipulation.Filter)  Time elapsed: 0.006 sec  <<< ERROR!  
java.lang.Exception: No tests found matching Method $spock_feature_0_0(com.sample.HelloSpec) from org.junit.internal.requests.ClassRequest@282ba1e  
    at org.junit.internal.requests.FilterRequest.getRunner(FilterRequest.java:35)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:275)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:149)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)  
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)  
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)  
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)  

Results :  

Tests in error:   
  Filter.initializationError »  No tests found matching Method $spock_feature_0_...   

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

This is most likely because with the direct invocation name we are bypassing the glue between JUnit and Spock and as such the execution fails.

Watkin answered 28/1, 2016 at 8:57 Comment(2)
Nice proof. Here Spock is paying dearly for nice naming conventions it supports. – Pouliot
@Bill Chen has a solution! – Civvies
D
14

I use: 

Windows: mvn -Dtest="TestCircle#my test" test 
*Nix: mvn "-Dtest=TestCircle#my test" test

It works perfectly with surefire 2.19.1 and junit 4.8.1. The following is the code example and the execution output:

  import spock.lang.Specification

class HelloSpec extends Specification {
    def sayHello() {
        given: "A person's name is given as a method parameter."
        def greeting =  "Hello Petri";


        expect: "Should say hello to the person whose name is given as a method parameter"
        greeting == "Hello Petri";

        println "hello from HelloSpec"
    }

    def sayHi() {
        expect:
        1==1

        println "sayHi from HelloSpec"

    }

    def "say hi to spock" () {
        expect:
        true
        println "say hi to spock from HelloSpec"

    }
}

# mvn test "-Dtest=HelloSpec#say hi to spock"

[INFO] Scanning for projects...                                                                                  
[INFO]                                                                                                           
[INFO] ------------------------------------------------------------------------                                  
[INFO] Building  Auto Test 1                                                                              
[INFO] ------------------------------------------------------------------------                                  
[INFO]                                                                                                           
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ auto-test ---                      
[INFO] Using 'UTF-8' encoding to copy filtered resources.                                                        
[INFO] Copying 5 resources                                                                                       
[INFO]                                                                                                           
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ auto-test ---                         
[INFO] No sources to compile                                                                                     
[INFO]                                                                                                           
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ auto-test ---              
[INFO] Using 'UTF-8' encoding to copy filtered resources.                                                        
[INFO] Copying 122 resources                                                                                     
[INFO]                                                                                                           
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ auto-test ---                 
[INFO] No sources to compile                                                                                     
[INFO]                                                                                                           
[INFO] --- gmaven-plugin:1.4:testCompile (compile-test) @ auto-test ---                                  
[INFO] Compiled 42 Groovy classes                                                                                
[INFO]                                                                                                           
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ auto-test ---                              

-------------------------------------------------------
 T E S T S                                             
-------------------------------------------------------
Running HelloSpec                                      
say hi to spock from HelloSpec                         
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.558 sec - in HelloSpec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS                                                           
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.169s                                                     
[INFO] Finished at: Fri Jul 01 14:19:20 CST 2016                               
[INFO] Final Memory: 31M/736M                                                  
[INFO] ------------------------------------------------------------------------

# mvn test "-Dtest=HelloSpec#sayHi"

[INFO] Scanning for projects...                                                                        
[INFO]                                                                                                 
[INFO] ------------------------------------------------------------------------                        
[INFO] Building  Auto Test 1                                                                    
[INFO] ------------------------------------------------------------------------                        
[INFO]                                                                                                 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ auto-test ---            
[INFO] Using 'UTF-8' encoding to copy filtered resources.                                              
[INFO] Copying 5 resources                                                                             
[INFO]                                                                                                 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ auto-test ---               
[INFO] No sources to compile                                                                           
[INFO]                                                                                                 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ auto-test ---    
[INFO] Using 'UTF-8' encoding to copy filtered resources.                                              
[INFO] Copying 122 resources                                                                           
[INFO]                                                                                                 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ auto-test ---       
[INFO] No sources to compile                                                                           
[INFO]                                                                                                 
[INFO] --- gmaven-plugin:1.4:testCompile (compile-test) @ auto-test ---                        
[INFO] Compiled 42 Groovy classes                                                                      
[INFO]                                                                                                 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ auto-test ---                    

-------------------------------------------------------
 T E S T S                                             
-------------------------------------------------------
Running HelloSpec                                      
sayHi from HelloSpec                                   
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.539 sec - in HelloSpec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS                                                           
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.187s                                                     
[INFO] Finished at: Fri Jul 01 14:19:49 CST 2016                               
[INFO] Final Memory: 31M/736M                                                  
[INFO] ------------------------------------------------------------------------

# mvn test "-Dtest=HelloSpec#sayHello"

[INFO] Scanning for projects...                                                                           
[INFO]                                                                                                    
[INFO] ------------------------------------------------------------------------                           
[INFO] Building Auto Test 1                                                                       
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ auto-test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ auto-test ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ auto-test ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 122 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ auto-test ---
[INFO] No sources to compile
[INFO]
[INFO] --- gmaven-plugin:1.4:testCompile (compile-test) @ auto-test ---
[INFO] Compiled 42 Groovy classes
[INFO]
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ auto-test ---

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running HelloSpec
hello from HelloSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.515 sec - in HelloSpec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.170s
[INFO] Finished at: Fri Jul 01 14:20:14 CST 2016
[INFO] Final Memory: 31M/736M
[INFO] ------------------------------------------------------------------------

Hope this help.

Dimity answered 1/7, 2016 at 6:34 Comment(3)
Cool. This seems to disprove the other answer that claimed it was impossible. – Holifield
My tests used version 2.18.1 of the maven-surefire-plugin, newer versions 2.19 and 2.19.1 apparently fixed several issues on the -Dtest property which may explain why it worked for you on 2.19.1. – Watkin
Yes, it works on 2.19.1. This way seems more natural to me: mvn test -Dtest="TestCircle#my test" – Fiveandten
W
10

After a further investigation, the answer is most probably: No, you can't invoke specific test methods with Spock using that Surefire Plugin feature. Below the reason.

Given the following Spock test case:

import spock.lang.Specification

class HelloSpec extends Specification {

    def hello = new Main();

    def sayHello() {
        given: "A person's name is given as a method parameter."
        def greeting = hello.sayHello("Petri");

        expect: "Should say hello to the person whose name is given as a method parameter"
        greeting == "Hello Petri";

        println "hello from HelloSpec"
    }
}

And given the following plugins configuration:

<plugin>
    <groupId>org.codehaus.gmavenplus</groupId>
    <artifactId>gmavenplus-plugin</artifactId>
    <version>1.5</version>
    <executions>
        <execution>
            <goals>
                <goal>addTestSources</goal>
                <goal>testCompile</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <sources>
            <fileset>
                <directory>${pom.basedir}/src/test/java</directory>
                <includes>
                    <include>**/*.groovy</include>
                </includes>
            </fileset>
        </sources>
    </configuration>
</plugin>

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
            <include>**/*Spec.java</include>
        </includes>
    </configuration>
</plugin>

It runs fine as part of the Maven test phase executing mvn clean test:

-------------------------------------------------------   
 T E S T S   
-------------------------------------------------------   
Running com.sample.HelloSpec  
hello from HelloSpec   
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.314 sec - in com.sample.HelloSpec

Even executing mvn clean test -Dtest=HelloSpec you get exactly the same result as above, successfully execution.

So, why we can't run a single method?

If we execute the javap command on the HelloSpec sample test above we get the following output:

Compiled from "HelloSpec.groovy"
public class com.sample.HelloSpec extends spock.lang.Specification implements groovy.lang.GroovyObject {
  public static transient boolean __$stMC;
  public com.sample.HelloSpec();
  public void $spock_feature_0_0();
  protected groovy.lang.MetaClass $getStaticMetaClass();
  public groovy.lang.MetaClass getMetaClass();
  public void setMetaClass(groovy.lang.MetaClass);
  public java.lang.Object invokeMethod(java.lang.String, java.lang.Object);
  public java.lang.Object getProperty(java.lang.String);
  public void setProperty(java.lang.String, java.lang.Object);
  public java.lang.Object getHello();
  public void setHello(java.lang.Object);
}

So in the generated bytecode there is no sayHello method, because Spock changes methods name in order to allow spaces in them. So the method name you write is never actually the real method name as part of the compiled class.

In this case, the method name is most probably $spock_feature_0_0, not something really friendly.

This is also confirmed in this answer and the comments of Peter Niederwieser, author of Spock actually, so a more than reliable source.

You could try running the following: 

mvn clean test -Dtest=HelloSpec#*spock*

which should indeed match the real method name, but you would however get an error

-------------------------------------------------------   
 T E S T S   
-------------------------------------------------------   
Running com.sample.HelloSpec   
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.007 sec <<< FAILURE! - in com.sample.HelloSpec  
initializationError(org.junit.runner.manipulation.Filter)  Time elapsed: 0.006 sec  <<< ERROR!  
java.lang.Exception: No tests found matching Method $spock_feature_0_0(com.sample.HelloSpec) from org.junit.internal.requests.ClassRequest@282ba1e  
    at org.junit.internal.requests.FilterRequest.getRunner(FilterRequest.java:35)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:275)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:149)  
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)  
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)  
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)  
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)  

Results :  

Tests in error:   
  Filter.initializationError »  No tests found matching Method $spock_feature_0_...   

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

This is most likely because with the direct invocation name we are bypassing the glue between JUnit and Spock and as such the execution fails.

Watkin answered 28/1, 2016 at 8:57 Comment(2)
Nice proof. Here Spock is paying dearly for nice naming conventions it supports. – Pouliot
@Bill Chen has a solution! – Civvies
D
1

Have you tried this (https://groups.google.com/forum/#!topic/spockframework/KVDkA9QxC4U):

Windows: mvn -Dtest="TestCircle#my test" test 
*Nix: mvn "-Dtest=TestCircle#my test" test
Deathful answered 27/1, 2016 at 13:9 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.