Kotlin - Maven not executing tests

Question

I have a Kotlin application which I want to test. My tests (.kt) files execute with success in Eclipse. (The test itself is a h2 mock jdbc test).

Now when running mvn test -X it says:

 releases: [enabled => true, update => never]
]
[DEBUG]   (s) reportFormat = brief
[DEBUG]   (s) reportsDirectory = C:\Users\Ivar\workspace\anotherworkspace\newrestservice\complete\target\surefire-reports
[DEBUG]   (f) rerunFailingTestsCount = 0
[DEBUG]   (f) reuseForks = true
[DEBUG]   (s) runOrder = filesystem
[DEBUG]   (f) shutdown = testset
[DEBUG]   (s) skip = false
[DEBUG]   (f) skipAfterFailureCount = 0
[DEBUG]   (s) skipTests = false
[DEBUG]   (s) suiteXmlFiles = []
[DEBUG]   (s) testClassesDirectory = C:\Users\Ivar\workspace\anotherworkspace\newrestservice\complete\target\test-classes
[DEBUG]   (s) testFailureIgnore = false
[DEBUG]   (s) testNGArtifactName = org.testng:testng
[DEBUG]   (s) testSourceDirectory = C:\Users\Ivar\workspace\anotherworkspace\newrestservice\complete\src\test\java
[DEBUG]   (s) threadCountClasses = 0
[DEBUG]   (s) threadCountMethods = 0
[DEBUG]   (s) threadCountSuites = 0
[DEBUG]   (s) trimStackTrace = true
[DEBUG]   (s) useFile = true
[DEBUG]   (s) useManifestOnlyJar = true
[DEBUG]   (s) useSystemClassLoader = true
[DEBUG]   (s) useUnlimitedThreads = false
[DEBUG]   (s) workingDirectory = C:\Users\Ivar\workspace\anotherworkspace\newrestservice\complete
[DEBUG]   (s) project = MavenProject: org.springframework:gs-rest-service:0.1.0 @ C:\Users\Ivar\workspace\anotherworkspace\newrestservice\complete\pom.xml

And it doesn't execute any test (It can't find em)

Here's my pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.springframework</groupId>
    <artifactId>gs-rest-service</artifactId>
    <version>0.1.0</version>
    <packaging>war</packaging>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.4.0.RELEASE</version>
    </parent>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jdbc</artifactId>
        </dependency>
        <!--  <dependency>
            <groupId>com.jayway.jsonpath</groupId>
            <artifactId>json-path</artifactId>
            <scope>test</scope>
        </dependency>-->
        <dependency>
            <groupId>org.jetbrains.kotlin</groupId>
            <artifactId>kotlin-stdlib</artifactId>
            <version>1.0.3</version>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <version>9.4.1211</version><!--$NO-MVN-MAN-VER$ -->
        </dependency>
        <dependency>
            <groupId>commons-dbutils</groupId>
            <artifactId>commons-dbutils</artifactId>
            <version>1.6</version>
        </dependency>
        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <version>1.4.191</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-reflect -->


    </dependencies>

    <properties>
        <java.version>1.8</java.version>
    </properties>


    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <artifactId>kotlin-maven-plugin</artifactId>
                <groupId>org.jetbrains.kotlin</groupId>
                <version>1.0.3</version>
                <executions>
                    <execution>
                        <id>compile</id>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                        <configuration>
                            <sourceDirs>
                                <sourceDir>${project.basedir}/src/main/java</sourceDir>
                            </sourceDirs>
                        </configuration>
                    </execution>
                    <execution>
                        <id>test-compile</id>
                        <goals>
                            <goal>test-compile</goal>
                        </goals>
                        <configuration>
                            <sourceDirs>
                                <sourceDir>${project.basedir}/src/main/java</sourceDir>
                            </sourceDirs>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.5.1</version><!--$NO-MVN-MAN-VER$ -->
                <executions>
                    <!-- Replacing default-compile as it is treated specially by maven -->
                    <execution>
                        <id>default-compile</id>
                        <phase>none</phase>
                    </execution>
                    <!-- Replacing default-testCompile as it is treated specially by maven -->
                    <execution>
                        <id>default-testCompile</id>
                        <phase>none</phase>
                    </execution>
                    <execution>
                        <id>java-compile</id>
                        <phase>compile</phase>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>java-test-compile</id>
                        <phase>test-compile</phase>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.19.1</version><!--$NO-MVN-MAN-VER$-->
                    <configuration>
                        <includes>
                            <include>**/Test*.kt</include>
                            <include>**/*Test.kt</include>
                            <include>**/*TestCase.kt</include>
                        </includes>
                    </configuration>
                </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>spring-releases</id>
            <url>https://repo.spring.io/libs-release</url>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>spring-releases</id>
            <url>https://repo.spring.io/libs-release</url>
        </pluginRepository>
    </pluginRepositories>
</project>

Pedicle

Maven documentation is often not very clear about settings, their use and what they mean. You hit a typical example of where Maven docs fall short. My answer shows the problem... – Wideeyed

Answer 1

Two problems, the first issue you found yourself but I'll document here. In the Kotlin-Maven plugin you have the settings for test-compile goal as:

<execution>
      <id>test-compile</id>
      <goals>
          <goal>test-compile</goal>
      </goals>
      <configuration>
          <sourceDirs>
               <sourceDir>${project.basedir}/src/main/java</sourceDir>
          </sourceDirs>
      </configuration>
</execution>

Which is compiling the src/main/java directory instead of src/test/java so your tests are not being compiled at all. Should be:

          <sourceDirs>
               <sourceDir>${project.basedir}/src/test/java</sourceDir>
          </sourceDirs>

Or if your files are both in the java and kotlin directories:

          <sourceDirs>
               <sourceDir>${project.basedir}/src/test/java</sourceDir>
               <sourceDir>${project.basedir}/src/test/kotlin</sourceDir>
          </sourceDirs>

The second issue is likely you trying to work around the first one. And it looks fine on the surface but is incorrect configuration of the Surefire plugin. The documentation for the Maven Surefire plugin isn't quite accurate (or it isn't complete about how it works). If you remove your Surefire plugin configuration completely it will work because the defaults already do what you want. Or you can change from listing the include file patterns from having the .kt suffix instead to .class or .java suffixes and those will work fine even for Kotlin, Clojure, Scala, ...

Why? here is the story...

This configuration:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <includes>
                    <include>**/Test*.java</include>
                    <include>**/*Test.java</include>
                    <include>**/*TestCase.java</include>
                    <include>**/RandomName.java</include>
                </includes>
            </configuration>
        </plugin>

Doesn't do what you think it does. This is actually converted in Surefire by doing a search and replace on .java to .class, and internally becomes:

            <configuration>
                <includes>
                    <include>**/Test*.class</include>
                    <include>**/*Test.class</include>
                    <include>**/*TestCase.class</include>
                    <include>**/RandomName.class</include>
                </includes>
            </configuration>

But if you use the .kt extension you break this hard coded magic. You can see it in the source code of Surefire plugin where it replaces files that end with .java with .class. Ooops, it has nothing to do with source files at all and is looking for compiled classes.

In Surefire plugin 2.19 they added the ability to have regex and also fully qualified classname patterns. So the way the plugin decides which you are using seems to be by the filename extension .java. If it sees .java it knows that each .java file becomes a class of the same name, so instead it looks for the classes that match the pattern, not the source code. The source is already compiled by the compiler plugins, not the test runner. It has no business looking for source code.

So this setting is misleading and is really using "magic" to figure out a classname instead of just asking for classnames. Of course the samples and everything completely make you believe it is related to source files. It isn't.

It is better to use no configuration and follow the default naming convention. Or if you must specify something, use the more modern regex or class name version of the include configuration:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <includes>
                    <include>Test*</include>
                    <include>*Test</include>
                    <include>*TestCase</include>
                    <include>RandomName</include>
                </includes>
            </configuration>
        </plugin>

Which is now comparing to any class with those names in any package.

Wideeyed

Turns out this wasn't the problem, the problem was that the test weren't compiled (maven plugin compiler sourceDir). Your (very detailed and explanatory) answer helped me resolve the problem so I'll accept it! – Pedicle

@Ivaro18 if compiled does the includes of with *.kt actually work? I think that still fails yes? – Wideeyed

Not sure, I changed it to the class name version before adding the sourceDir – Pedicle

Updated the answer including the src/main/java vs. src/main/test issue – Wideeyed

I'm pretty sure both issues were a problem. glad you found the fix – Wideeyed

Was facing with similar issues, a that solution doesn't helped unless update surface plugin up to 2.20.1.. I'm using java + kotlin (in both main and test dirs and Gradle compiles that with no problem and additional configurations at all – Knife

After going through these troubleshooting steps my java tests were discovered but kotlin tests were still not found. I verified that the class files were being generated. In my case I was using JUnit5 and was missing the junit-jupiter-engine dependency. With that set make sure your versions of maven-surefire-plugin and junit-jupiter-* dependencies are compatible. We've reached a point that the latest versions will probably be good to go. Today I'm using 3.0.0-M3 and 5.5.0. – Tristram

Thanks @Winder, I'm also using JUnit 5. Upgrading the maven-surefire-plugin from 2.22.2 to 3.0.0-M5 solved my problem. No extra <configuration> tags needed anywhere. – Seward

My issue was that I copy and renamed a Java test from SomeUnitTest to SomeUnitTestKt. That no longer matched the patterns surefire is looking for (starting or ending in test), so it wasn't found. Renaming it to KtSomeUnitTest worked – Skiagraph