How to continuously build and deploy feature branches with Maven?

Question

My team is using feature branches to implement new features and continuously deploys snapshot builds into a remote repo for our users to use. Thus 'deploy' really only means 'distributing to a remote Maven repository'. We're currently only running continuous integration builds for the master branch and not the feature branches for the following reason: we're using Maven to build our projects and distribute the JavaDoc and sources alongside the JAR.

My plan was now to add a classifier to each feature branches build and expected that one to be used when creating and deploying the artifacts like this:

• Branch: master
• Classifier: none

• Artifacts: foo-${version}.jar, foo-${version}-sources.jar, foo-${version}-javadoc.jar

• Branch: feature-X

• Classifier: myfeature
• Artifacts: foo-${version}-feature.jar, foo-${version}-sources-feature.jar, foo-${version}-javadoc-feature.jar

I don't really care about the exact naming of the artifact, I just need separate main, source and JavaDoc artifacts for the feature branch. It turns out, neither the JavaDoc plugin nor the source plugin consider the classifier configured and thus effectively overwrite the artifacts created for my master build.

I don't really want to change the artifactId although this would probably solve the issue. How do you approach feature branches and continuous integration with Maven?

Cavicorn

How static are your topoc branches? How often do you expect to setup a new job and how often are they going to be teared down? What do you use in the CI Server to help you with that? This is one of the things which stops me from thinking about such a build. Maybe a gatekeeper model or developer local CI server is better suited. – Overstrain

you should not use the classifier to reflect the difference in branches, as you will have nasty side effect with some other plugins. Classifiers are supposed to be sources, javadocs, etc... For your need you should either change the artifactId or version. – Mythicize

@Overstrain - We use Bamboo which supports automatically triggering a build job against a different branch based on a regular expression on the branch name. As soon as it detects a branch matching that expression, it pretty much clones a build job if generally instructed to do so. – Cavicorn

@Farid - Yeah, that seems the only solution right now. Your comment would make a good answer, even if I wished there was a better solution :). – Cavicorn

This question deserves many more views, and up votes. – Wilkey

We're having similar issues in our development lifecycle that we are trying to resolve. Did you find a functional solution finally? Did you add the qualifier to the version number? Is there a specific version # format that you found that works best? – Cozmo

I just stick to 1.2.0.$issuenumber-SNAPSHOT for now and that hasn't created any issues so far. – Cavicorn

Answer 1

I would suggest to add the branch-qualifier into the version component, as it is more related to that part. This also allows your snapshot dependencies on those versions alongside the main branch.

Overstrain

While this seems like a good approach, it leads to problems when using version number ranges (f.e. for continuous integration). See my answer. – Antre

Not only version number ranges, putting alphas into the version and also putting snapshot, does not make this a snapshot, as noted in my comment a feature-SNAPSHOT is no longer a snapshot, the qualifier is now the feature name and SNAPSHOT. – Bickart

Answer 2

I would suggest to use an appropriate version which represents the branch as well as the version things like:

1.0.0-SNAPSHOT for master

and

1.0.0-F1-SNAPSHOT for feature F1

etc.

This give also an indicator from which release 1.0.0 the feature branch has been made.

Euphoria

1.0.0* is equivalent to <MajorVersion [> . <MinorVersion [> . <IncrementalVersion ] ] and -F1 is equivalent to [> - <BuildNumber | Qualifier ]>. -SNAPSHOT is part of Maven. Furthermore is there any problem with that? Simply no. – Euphoria

how would you instruct mercurial or maven to not include pom version when merge back to master? I mean I don't think you want to resolve this manually every time you merge, right? – Geelong

We have a feature branch setup script that creates a first commit with only the changes to the version numbers. On merging we simply drop this commit and squash and merge all subsequent ones on this branch (or simply cherry pick it if there's only a second commit). – Cavicorn

I've been using this approach. using maven versions plugin. mvn versions:set -DallowSnapshots=true -DnewVersion=0.1-$branch-SNAPSHOT -Pall – Role

@OliverGierke I am just wondering if you can share that script with the public? We also ran into the same issue. Merging branches with pom files containing different component version numbers has become very challenging. For e.g. my master might have 1.4, and my release branch might have 1.1., but when we try to merge release branch with master, this becomes a sticky point. – Malonis

@Malonis — The script can be found here. It creates a dedicated commit for the setup which we simply drop when merging the branch. That way, the version customization never makes it into master. – Cavicorn

@OliverGierke thanks for sharing that. We will customize this to our needs and let you know how it goes. – Malonis

@khmarbaise, in your comment F1 is not a qualifier, the qualifier is actually F1-SNAPSHOT. Maven versions have multiple schemes depending on if you have a qualifier or not. 1.2.3-SNAPSHOT is <major>.<minor>.<incremental>-<qualifier>, 1.2.3-4 is <major>.<minor>.<incremental>-<build>, and; what I see a lot that's not valid is 1.2.3.4 becomes <incremental>. – Bickart

@BrettRyan This schema is based on Maven 2... in Maven 3 you can define versions more or less you like (but usually it does not make sense). I can simply define a version like 1.0.0-F1-SNAPSHOT and it will work.. A version like 1.2.3.4 is simply a version ? I don't see any relationship to qualifier etc. See also blog.soebes.de/blog/2017/02/04/… and also maven.apache.org/pom.html#version-order-specification apart from that I have not claimed that F1 was an qualifier?...The comment was at 2012... ? related to Maven 2. – Euphoria

@khmarbaise, you said -F1 is equivalent to [> - <BuildNumber | Qualifier ]>. -SNAPSHOT is part of Maven, if after the - the remaining part as a whole is a qualifier (when alpha), otherwise it's a build number if numeric. Therefore -F1-SNAPSHOT is a qualifier and is NOT treated as a snapshot. – Bickart

@BrettRyan While it's technically true that -F1-SNAPSHOT is the qualifier, every version that ends with -SNAPSHOT is in fact treated as a snapshot. See org.apache.maven.artifact.versioning.DefaultArtifactVersion and org.apache.maven.artifact.ArtifactUtils. – Descant

Answer 3

Using the version number to store the branch name, as suggested by others, is a quick win, but leads to problems if you use version ranges. The version number was not meant to be used like that. We use them in a continuous integration process to make the integration tests depend on the tested artifact:

[1.8-SNAPSHOT,1.9-SNAPSHOT)

The qualifier part inside the version number denotes different incremental stages of the same code base:

1.8-alpha1-SNAPSHOT
1.8-alpha1-SNAPSHOT
1.8-beta1-SNAPSHOT

This is why the version range above will capture the above and Maven will order them in this order:

1.8-SNAPSHOT
1.8-alpha1-SNAPSHOT
1.8-alpha1-SNAPSHOT
1.8-beta1-SNAPSHOT

Any artifact carrying the feature branch name in the version number (1.8-featureA-SNAPSHOT) will be ordered newer than the snapshots without qualifier. But a feature branch is a 'different' code base, not a newer representation of the same code base. For our integration test scenario this led to useless test failures. The feature branch was not ready yet to be tested by integration tests.

We now follow this rule: if you have to change something anyway, why not the artifact id? We change the artifact id for feature branches and it works just fine.

Antre

while it's true that the feature branch is a 'different' code base, it is still the same project as the main build. It seems there should be a solution other than modifying the artifact id. I don't have one though =( Maybe maven versions are either too feature-full (supporting ranges) or too inflexible (only supporting 3 dotted versions a.b.c as numbers and defaulting to strings for a.b.c.d) – Jd

FWIW: I'm going with completely different versions on feature branches. 1, 2, 3, 4... on master and featureA-001, featureA-002, ... on feature branches. Considered featureA-SNAPSHOT also, but seemed like a more complicated approach. – Jd

This is true - however, using version ranges in Maven has various problems anyway (such as the requirement to stick to Maven's version number format), so I don't think they are used a lot in practice. If you never use version ranges, this problem does not apply. – Machicolation

@Eduard Wirch - exactly how do you update the artifactId? I agree with your statements (feature branches are different code bases) and therefore it is appropriate to update artifactId, but I'm struggling to find a way to do this automatically at build time (so that feature branch builds generate new/unique artifacts) – Pride

We modified them manually, which had its own drawbacks, but worked out for a while. My answer is pretty old now. In the meantime we moved to not installing artifacts into the artifact repository but build Docker images intstead to test feature branches. – Antre

I would stay away from version ranges. If you use mutable version snapshots, only change them if different dependencies are required. – Overstrain

This is absolutely the correct answer. As noted on my comment above the versioning scheme offered by maven simply does not allow for features in the version. – Bickart

Answer 4

Instead of altering the maven coordinates of the artifact you could use maven-branch-extension to effectively create a separate SNAPSHOT namespace for each of the feature branches. Quote from the project page:

Instead of changing the version number when on a feature branch, we change the repository. Each feature is deployed into a subdirectory, based on their branch name, in a remote repository that is only for feature branches. There is no risk of artifacts being overwritten. Version numbers do not change. Branching and merging with Git stays simple (like it was meant to be!).

The extension gets the current Git branch and resolves a property in the URL of the repository so that artifacts can be stored and retrieved correctly. It also manages caching and fetching of artifacts to the local repository so that artifacts are taken from the feature branch repository, if they exist, when working from a feature branch.

The beauty of this is that external users of the SNAPSHOT dependencies are completely isolated from the internal work on topic branches.

Birdbath

I think this is the best approach, much cleaner than messing around witch changing the GAVs of artifacts. Also, the "feature" repo can be optimized specifically for the feature branch lifecycle (auto-cleaning when the feature branch has been deleted...etc) – Colis

Answer 5

I know this is an old question, but for those who are searching a better solution for CI, starting with maven 3.5.0, there are 3 variables introduced to satisfy this type of demand : https://maven.apache.org/maven-ci-friendly.html In general, you can use <version>${revision}${sha1}${changelist}</version> in your pom.xml. Then you can change sha1 value for feature branch.

   <properties>
    <revision>1.3.1</revision>
    <changelist>-SNAPSHOT</changelist>
    <sha1/>
  </properties>

Epispastic