Java 9 + maven + junit: does test code need module-info.java of its own and where to put it?

Question

Let's say I have a Java project using Maven 3 and junit. There are src/main/java and src/test/java directories which contain main sources and test sources, respectively (everything is standard).

Now I want to migrate the project to Java 9. src/main/java content represents Java 9 module; there is com/acme/project/module-info.java looking approximately like this:

module com.acme.project {
    require module1;
    require module2;
    ...
}

What if test code needs module-info.java of its own? For example, to add a dependence on some module that is only needed for tests, not for production code. In such a case, I have to put module-info.java to src/test/java/com/acme/project/ giving the module a different name. This way Maven seems to treat main sources and test sources as different modules, so I have to export packages from the main module to the test module, and require packages in the test module, something like this:

main module (in src/main/java/com/acme/project):

module prod.module {
    exports com.acme.project to test.module;
}

test module (in src/test/java/com/acme/project):

module test.module {
    requires junit;
    requires prod.module;
}

This produces

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:testCompile (default-testCompile) on project test-java9-modules-junit: Compilation failure: Compilation failure:
[ERROR] /home/rpuch/git/my/test-java9-modules-junit/src/test/java/com/acme/project/GreeterTest.java:[1,1] package exists in another module: prod.module

because one package is defined in two modules. So now I have to have different projects in main module and test module, which is not convenient.

I feel I follow wrong path, it all starts looking very ugly. How can I have module-info.java of its own in test code, or how do I achieve the same effects (require, etc) without it?

Sitter

First forget Maven 2...use Maven 3+...a module-info in the test are does from my point of view not make sense? Special requirement / achievement behind that? – Edmon

It is Maven 3, of course – Sitter

Answer 1

The module system does not distinguish between production code and test code, so if you choose to modularize test code, the prod.module and the test.module cannot share the same package com.acme.project, as described in the specs:

Non-interference — The Java compiler, virtual machine, and run-time system must ensure that modules that contain packages of the same name do not interfere with each other. If two distinct modules contain packages of the same name then, from the perspective of each module, all of the types and members in that package are defined only by that module. Code in that package in one module must not be able to access package-private types or members in that package in the other module.

As indicated by Alan Bateman, the Maven compiler plugin uses --patch-module and other options provided by the module system when compiling code in the src/test/java tree, so that the module under test is augmented with the test classes. And this is also done by the Surefire plugin when running the test classes (see Support running unit tests in named Java 9 modules). This means you don't need to place your test code in a module.

Wyly

Changing the package to test has the downside that your tests won't reach default and protected modifiers anymore. – Disquiet

The module provides provides --patch-module and other options to support the compilation and execution of tests that are in the same package/module as the module under test. The maven-compiler-plugin uses these options when compiling code in the src/test tree. Ditto for the surefire-plugin. – Epiphysis

@AlanBateman Thanks for this info. I didn't know Maven does that. I will update the answer with your info. – Wyly

@AlanBateman curious to know what tag in surefire supports this? Any sources you can link would be helpful to read for me. – Bugbee

@nullpointer It seems added in issues.apache.org/jira/browse/SUREFIRE-1420. – Wyly

@manouti Ya saw that eventually after making the comment. Since argLine has already been in use previously for surefire, that was a useful existing option that could be extended. And the combination with classpathDependencyScopeExclude serves a good purpose there. :) – Bugbee

IMO, this is very correct answer. gradle has adopted the same pattern. – Cartomancy

Answer 2

You might want to rethink the project design you're trying to implement. Since you are implementing a module and its test into a project, you shall refrain from using different modules for each of them individually.

There should just be one single module-info.java for a module and its corresponding tests.

Your relevant project structure might look like this:-

Project/
|-- pom.xml/
|
|-- src/
|   |-- test/
|   |   |-- com.acme.project
|   |   |        |-- com/acme/project
|   |   |        |      |-- SomeTest.java
|   |   
|   |-- main/
|   |   |-- com.acme.project
|   |   |    |-- module-info.java
|   |   |    |-- com/acme/project
|   |   |    |    |-- Main.java

where the module-info.java could further be:-

module com.acme.project {
    requires module1;
    requires module2;
    // requires junit; not required using Maven
}

---

Just to sum all of the above as per your questions --

I feel I follow wrong path, it all starts looking very ugly. How can I have module-info.java of its own in test code, or how do I achieve the same effects (require, etc) without it?

Yes, you should not consider managing different modules for test code making it complex.

You can achieve similar effect by treating junit as a compile-time dependency using the directives as follows-

requires static junit;

Using Maven you can achieve this following the above-stated structure and using maven-surefire-plugin which would take care of patching the tests to the module by itself.

Bugbee

I would suggest not to use a module name in the directory structure cause there is no advantage in it ....? – Edmon

@Edmon I believe you mean the com.acme.project/com/acme/project. Just followed the quick-start guide there. Though I agree, it doesn't provide any advantage as such. – Bugbee

Requiring junit in the module descriptor does not look very nice for me – Biz

Also, IDEA highlights requires static junit as an error (since junit has a test scope in pom.xml) – Biz

@Biz (1) Why junit in module descriptor does not look good? (2) the scope in maven can be controlled and not necessarily be test(though that's a good practice agreed) – Bugbee

@nullpointer (1) Because if my module depends on junit, someone can accidentally use its classes – Biz

@Biz Ya maybe I am on the other side of the audience, who preferably dislike the approach of patching modules. Updating the answer with Maven's approach though. Thanks for suggesting. – Bugbee

Intellij IDEA requires a different dir structure, which doesn't seem to allow the test or main directories. – Landmass

this advice of a single module is not even possible, at times. here is an example – Cartomancy

Answer 3

Also note that maven-surefire-plugin now has useModulePath false as a configuration option.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>3.0.0-M6</version>
  <configuration>
    <useModulePath>false</useModulePath>  <!-- tests use classpath -->
  </configuration>
</plugin>

This is an option where a project uses module-path for main but classpath for tests and testing. It is probably not a bad option for people to go to this approach if "patching" the module-path gets painful.

Edit: We can also set this via property - surefire.useModulePath e.g.

<properties>
  <surefire.useModulePath>false</surefire.useModulePath>
</properties>

Deflect

Answer 4

Adding some details.

In Java since 9, a jar file (or a directory with classes) may be put on classpath (as earlier), or on module path. If it is added to classpath, its module-info is ignored, and no module-related restrictions (what reads what, what exports what, etc) are applied. If, however, a jar is added to module path, it is treated as a module, so its module-info is processed, and additional module-related restrictions will be enforced.

Currently (version 2.20.1), maven-surefire-plugin can only work in the old way, so it puts the classes being tested on classpath, and module-path is ignored. So, right now, adding module-info to a Maven project should not change anything with tests being run using Maven (with surefire plugin).

In my case, the command line is like the following:

/bin/sh -c cd /home/rpuch/git/my/test-java9-modules-junit && /home/rpuch/soft/jdk-9/bin/java --add-modules java.se.ee -jar /home/rpuch/git/my/test-java9-modules-junit/target/surefire/surefirebooter852849097737067355.jar /home/rpuch/git/my/test-java9-modules-junit/target/surefire 2017-10-12T23-09-21_577-jvmRun1 surefire8407763413259855828tmp surefire_05575863484264768860tmp

The classes under test is not added as a module, so they are on classpath.

Currently, a work is under way in https://issues.apache.org/jira/browse/SUREFIRE-1262 (SUREFIRE-1420 is marked as a duplicate of SUREFIRE-1262) to teach surefire plugin to put code under test on module path. When it is finished and released, a module-info will be considered. But if they will make the module under test to read junit module automatically (as SUREFIRE-1420 suggests), module-info (which is a main module descriptor) will not have to include a reference to junit (which is only needed for tests).

A resume:

1. module-info just needs to be added to the main sources
2. for the time being, surefire ignores new module-related logic (but this will be changed in the future)
3. (when modules will work under surefire tests) junit will probably not need to be added to the module-info
4. (when modules will work under surefire tests) if some module is required by tests (and only by them), it may be added as a compile-only dependence (using require static), as suggested by @nullpointer. In this case, the Maven module will have to depend on an artifact supplying that test-only module using compile (not test) scope which I don't like much.

Sitter

Answer 5

I just want to add my 0.02$ here on the general testing approach, since it seems no one is addressing gradle and we use it.

First thing first, one needs to tell gradle about modules. It is fairly trivial, via (this will be "on" since gradle-7):

plugins.withType(JavaPlugin).configureEach {
    java {
        modularity.inferModulePath = true
    }
}

Once you need to test your code, gradle says this:

If you don’t have a module-info.java file in your test source set (src/test/java) this source set will be considered as traditional Java library during compilation and test runtime.

In plain english, if you do not define a module-info.java for testing purposes - things "will just work" and in the majority of cases this is exactly what we want.

---

But, that is not the end of story. What if I do want to define an JUnit5 Extension, via ServiceLocator. That means I need to go into module-info.java, from tests; one that I yet do not have.

And gradle has that solved again:

Another approach for whitebox testing is to stay in the module world by patching the tests into the module under test. This way, module boundaries stay in place, but the tests themselves become part of the module under test and can then access the module’s internals.

So we define a module-info.java in src/test/java, where I can put :

 provides org.junit.jupiter.api.extension.Extension with zero.x.extensions.ForAllExtension;

we also need to do --patch-module, just like maven plugins do it. It looks like this:

def moduleName = "zero.x"
def patchArgs = ["--patch-module", "$moduleName=${tasks.compileJava.destinationDirectory.asFile.get().path}"]
tasks.compileTestJava {
    options.compilerArgs += patchArgs
}
tasks.test {
    jvmArgs += patchArgs
}

The only problem is that intellij does not "see" this patch and thinks that we also need a requires directive (requires zero.x.services), but that's not really the case. All the tests run just fine from command line and intellij.

The example is here

Cartomancy

Thank you for sharing. I am fairly a novice in Gradle and just have a basic understanding such that I can relate a few concepts to that of Maven. What I understand with this will be "on" since gradle-7 in your answer is that Gradle would build a capability to identify the java modulepath versus classpath awareness and choose wisely on what code resides where on its own. As a user, specifying the java source and target in the test plugins should be sufficient from my point of view. – Bugbee

But yeah, something like provides org.junit.jupiter.api.extension.Extension with __; seems to be outpost with that. Honestly, I would have to look into Junit5's recommendation over defining the extension and aligning that with the Java module system. (time to look back at some answers from Sormuras here I guess, specifically this one!) – Bugbee

Answer 6

I was not able to make it work also with the latest Maven surefire plugin version (3.0.0-M5). It seems if the main sources are using a module, the compiler plugin when using Java 11 is also expecting referenced packages to be in a module.

My solution was to place an own module-info.java inside the test sources (src/test/java in Maven) for the test module with the below contents. I my case I had to use the keyword open (See Allowing runtime-only access to all packages in a module) because I'm using Mockito in my test, which requires reflective access.

// the same module name like for the main module can be used, so the main module has also the name "com.foo.bar"
open module com.foo.bar {
// I use junit4
    requires junit;
// require Mockito here
    requires org.mockito;
// very important, Mockito needs it
    requires net.bytebuddy;
// add here your stuff
    requires org.bouncycastle.provider;
}

Olympic

As simple as it can be: Module and test should not have the same package, for example: Module package: com.acme.project Test package can be: test.com.acme.project – Exanimate

Apart from simplicity, the big advantage of having test code in the same package (and Maven project) as the main code is the ability to access non-exported/non-public code. See my answer above for a solution that does not require separate modules. – Clevie

Answer 7

You do not need a secondary module-info.java, you can specify everything in the same file:

For any dependency that is only required for testing,

1. add an entry requires static <module-name> in the projects module-info.java under src/main/java
2. set the Maven dependency scope (in pom.xml) to <scope>provided</scope>

Why does this work?

require static indicates to the compiler that these dependencies aren't required upon runtime. They're also not transitive, so they will not show up in other projects

<scope>provided</scope> is like compile+runtime+test, but not-transitive, so it will also not show up in other projects.

This approach also works just fine in Eclipse, for what it's worth.

If it's not working for you, try updating your Maven plugin dependencies (maven-compiler-plugin, maven-surefire-plugin) and/or your IDE.

Wordage