Publish a bom from a multi-module-project

Question

We are a large company with about 2000 separate Java projects. For historic reasons, we do not have multi-module projects, but we would like to introduce them.

Logically, we already have "groups" of projects, i.e. someone responsible for (say) 50 projects which are closely related. This someone regularly publishes a BOM which contains recent, coherent versions of these 50 projects.

Now it would make a lot of sense to grab these 50 projects and put them into one large multi-module project. Still, it would be necessary to publish a BOM because other projects (outside our group) should have coherent versions.

So, summarised, we need a BOM that contains the versions of all 50 projects that are part of the multi-module project. I wonder what would be the "Maven way" to create such a BOM. What I can think of:

• The bom is the 51st project of the multi-module project. The versions of the dependencies are set by properties in the parent pom.
• The bom is generated from the information present in the multi-module project and published as side artifact (this probably requires us to write a Maven plugin for this).

What would be advisable?

Paralysis

By projects you mean anything - a standalone app or a lib? And you reference those libs (but not apps) in other projects (both apps and libs)? And you want to group multiple libs into 1 project and publish them all as one entity (BOM)? – Hewe

It is unclear what exactly you want to achieve from what you have. – Sheer

@ThorbjørnRavnAndersen I would like to clarify, but could you tell me which parts are unclear to you? – Paralysis

@StanislavBashkyrtsev This is more or less correct. Most of the projects are libraries that are used by other libraries. We use the BOM mechanism to make sure that the jars that are used are coherent, at least inside one "group". – Paralysis

Answer 1

We are using BOMs as well for our multi-modules projects, but we are not tying their generation or update to the build of those modules.

A BOM is only updated when our release management process completes the delivery of a built module (or group of modules): once delivered, then the BOM is updated and pushed to Nexus (stored as a 1.0-SNAPSHOT version, constantly overridden after each delivery)

The BOM is then included within our POM (for mono or multi-module projects) and use for dependency management only, meaning our projects depends on artifact without the version: the dependency management from the BOM provides with the latest delivered version of other dependent modules.

In other words, we separate the build aspect (done here with maven) from the release part: the "bills of materials" represent what has been delivered, and ensure all projects are building with versions deemed working well together (since they have been delivered into production together).

Giraudoux

Thank you for this insightful answer. It is a good thought to postpone the BOM creation after the testing (and possiblly: delivery stage). But I would be careful with your "misuse" of SNAPSHOT because exchanging the versions in this way may cause build instability (unless, of course, this is clearly announced and happens rarely). – Paralysis

@JFMeier Exactly: that is why I have developed a recorder: by default the build does not use the BOM, but a special dependency management section in which versions are set. On demand, the recorded plugin reads the BOM, and replace that special section with the BOM version. Since the pom.xml is in a Git repo, one can follow the version changes recorded after each call to that plugin. But by default, no plugin is called, and the "special dependency management section" is reused as is, with a fixed set of version, ensuring build stability. – Giraudoux

@JFMeier The goal is to answer the question: with which versions am I supposed to build my project with? On demand, you can get those versions as "the one currently deployed" (reading the BOM, modifying for you the pom.xml). But for all the other build instance, you are just using what was previously set (again: build stability) – Giraudoux

Answer 2

I've never seen 2K of commercial Java projects, so will base my answer on how open source works:

• Libraries shouldn't be grouped by people - they should be grouped by the problems that they solve. Often open source projects have multiple libs e.g. Jackson has jackson-databind, jackson-datatype-jsr310, etc. These libs tightly relate to each and may depend on each other.
• Such groups shouldn't be too big. Some projects may have 1, others - 5 or 10. 50 libs in a group sounds way too much.
• It's easier if libs in a group are released all at the same time (even if only one is updated). This makes it straightforward to keep track of versions in the apps that use multiple libs from a group.
• There should be no dependencies between groups! And this is probably the most important rule. Deep hierarchy of libraries that depend on each other is not acceptable because now you need to keep compatibility between many projects and libs. This just doesn't scale. Which means there will be occasional copy-paste code between libs - this is the lesser evil.
• There could be some exceptions to the last rule (maybe a lib that is used everywhere) but those must keep backward compatibility of the public API until there are no projects that depend on the old API. Such libs are very hard to maintain and it's better to opensource them.

Standalone projects now can depend on libraries from the same or different groups, but because the version within the group is the same, it's easy to set it as a property just once. Alternatively:

• You can look at <scope>import</scope> which allows importing <dependencyManagement> sections from other POM files like parent POMs within a group (for some reason never worked for me).
• Or at xxx-all modules - a module that depends on all other modules within group and thus when you depend on it, you also depend on others transitively.

Hewe

Thank you for you answer. But the truth is: Our company has 2000 Java projects (each of them creating a jar, war or ear), most of them are kind of libraries (some are ears or wars using the libraries), and they already depend on each other. These dependencies are inside the "groups" (which are based on topics actually, but have a responsible guy each), but they also go between groups in many different ways. – Paralysis

We cannot untie this big knot - we can only manage it. This is why we want to put the projects of one group into a multi-module project. Still, the different multi-module projects will have various dependencies between each other. – Paralysis

dependencies between groups have a tree structure other than any cycles should be ok – Thacher