Using different versions of dependencies in separated Java platform modules

Question

I expected it's possible to use i.e. Guava-19 in myModuleA and guava-20 in myModuleB, since jigsaw modules have their own classpath.

Let's say myModuleA uses Iterators.emptyIterator(); - which is removed in guava-20 and myModuleB uses the new static method FluentIterable.of(); - which wasn't available in guava-19. Unfortunately, my test is negative. At compile-time, it looks fine. In contrast to runtime the result is a NoSuchMethodError. Means that, the class which was the first on the classloader decides which one fails.

The encapsulation with the underlying coupling? I found a reason for myself. It couldn't be supported because of transitive dependencies would have the same problem as before. If a guava class which has version conflicts occurred in the signature in ModuleA and ModuleB depends on it. Which class should be used?

But why all over the internet we can read "jigsaw - the module system stops the classpath hell"? We have now multiple smaller "similar-to-classpaths" with the same problems. It's more an uncertainty than a question.

Ethridge

Java 9 solves the classpath hell a bit radically. It justs forbids split packages (i.e. multiple modules having the same package). Pre Java 9 versions don't forbid split packages and it can cause surprises because classes are loaded unpredictably (which is not most people want). – Corps

The Jigsaw project is a long-term project (the original blog post was in Dec 2008). I would check the date on any articles re: classpath hell. The version problem was often used as a motivating concern back in the day. – Stefanstefanac

Just wondering: any feedback to my answer? Helpful in any way? – Scherle

Answer 1

Version Conflicts

First a correction: You say that modules have their own class path, which is not correct. The application's class path remains as it is. Parallel to it the module path was introduced but it essentially works in the same way. Particularly, all application classes are loaded by the same class loader (by default at least).

That there is only a single class loader for all application classes also explains why there can't be two versions of the same class: The entire class loading infrastructure is built on the assumption that a fully qualified class name suffices to identify a class with a class loader.

This also opens the path to the solution for multiple versions. Like before you can achieve that by using different class loaders. The module system native way to do that would be to create additional layers (each layer has its own loader).

Module Hell?

So does the module system replace class path hell with module hell? Well, multiple versions of the same library are still not possible without creating new class loaders, so this fundamental problem remains.

On the other hand, now you at least get an error at compile or launch due to split packages. This prevents the program from subtly misbehaving, which is not that bad, either.

Extracellular

Sorry; I overlooked that link. – Scherle

Thanks for pointing it out, though - I just made sure that the fourth paragraph is more exact. – Extracellular

So what does the module system accomplish, if it doesn't solve this fundamental problem? – Devitt

Among others, reliable configuration and strong encapsulation. Also note that any solution to the multiple versions problem would either be backwards incompatible or require a massive rework of entire class loading mechanism (and I'm not even sure whether that would be possible while keeping backwards compatible). – Extracellular

I was very excited to hear about modules some number of years ago because on the surface I thought it would solve the problem of classpath hell. Then I learned that you can't have different versions of the same library across modules. To me this somewhat defeats the purpose of modules, but I hadn't heard about Layers at that time. To me, this functionality should be built directly into the modules using the visibility options without introducing another concept, but at least there's a way. – Geyserite

@Geyserite A layers is more or less just a class loader plus a module graph, so they provide no fundamentally new features. You could have done the same with just class loaders in Java <9. – Extracellular

Hi @NicolaiParlog Just to try-out I have created a layer of a module (with it's own class loader) and executed a method using reflection, until now everything is perfect. Could able to get Object from that method but now question is how can I cast that object? You can find more details in my question here – Forceps

Answer 2

Theoretically it is possible to use different versions of the same library within your application. The concept that enables this: layering!

When you study Jigsaw under the hood you find a whole section dedicated to this topic.

The idea is basically that you can further group modules using these layers. Layers are constructed at runtime; and they have their own classloader. Meaning: it should be absolutely possible to use modules in different versions within one application - they just need to go into different layers. And as shown - this kind of "multiple version support" is actively discussed by the people working on java/jigsaw. It is not an obscure feature - it is meant to support different module versions under one hood.

The only disclaimer at this point: unfortunately there are no "complete" source code examples out there (of which I know), thus I can only link to that Oracle presentation.

In other words: there is some sort of solution to this versioning problem on the horizon - but it will take more time until to make experiences in real world code with this new idea. And to be precise: you can have different layers that are isolated by different class loaders. There is no support that would allow you that "the same object" uses modV1 and modV2 at the same time. You can only have two objects, one using modV1 and the other modV2.

( German readers might want to have a look here - that publication contain another introduction to the topic of layers ).

Scherle

Layers support multiple versions of the same JAR because each layer has its own class loader. They add no new features here, though, and the exact same results can be achieved in Java 8 with blank class loaders (like OSGi does). In summary: the module system does not improve support for multiple versions. – Extracellular

It is insofar different that there is a "first class" construct (layers) now. Before Java9, you have to built things yourself or rely on 3rd party solutions. Now you have the concept being part of the language. – Scherle

any updates in 2020 as to whether there are complete source code examples for using different versions of the same library under the hood without resorting to a 3rd party solution? – Messy

I am not aware of any, but I am not an active person in such "areas". I think you might be more lucky asking user stackoverflow.com/users/2525313/nicolai for his views. – Scherle

You might find Layrry interesting, as described in my other answer. Granted, it is a "3rd party solution", but it's based on JPMS layers, and if nothing else, you could use it as inspiration for working with layers themselves directly. – Solnit

Answer 3

Java 9 doesn't solve such problems. In a nutshell what was done in java 9 is to extend classic access modifiers (public, protected, package-private, private) to the jar levels.

Prior to java 9, if a module A depends on module B, then all public classes from B will be visible for A.

With Java 9, visibility could be configured, so it could be limited only to a subset of classes, each module could define which packages exports and which packages requires.

Most of those checks are done by the compiler.

From a run time perspective(classloader architecture), there is no big change, all application modules are loaded by the same classloader, so it's not possible to have the same class with different versions in the same jvm unless you use a modular framework like OSGI or manipulate classloaders by yourself.

Harragan

Answer 4

As others have hinted, JPMS layers can help with that. You can use them just manually, but Layrry might be helpful to you, which is a fluent API and configuration-based launcher for running layered applications. It allows you to define the layer structure by means of configuration and it will fire up the layer graph for you. It also supports the dynamic addition/removal of layers at runtime.

Disclaimer: I'm the initial creator of Layrry

Solnit