Continuous integration and continuous delivery with git-flow

Question

We have been doing continuous integration and continuous delivery since a while with Subversion commits as the pipelines triggers. Recently, we started using git in some projects with git-flow and we are trying to decide which of the branches of git-flow should we use to trigger the continuous integration and continous delivery pipelines.

Here are two approaches:

1. Use develop branch

Problem: With git-flow we are supposed to deploy the release (or master) branch in production, so we would have to build two different pipelines, one for continuous integration (branch develop) and one for continuous delivery (branch master). This could introduce bugs in production because the version in production will not be the same that the one in other environments (integration, test, staging).

2. Use master branch:

Problem: This way, we would not have a truly continuous integration, since changes to these branches are pushed not very frequently.

Which is the rigth branch to use in the pipelines?

Heymann

In my experience git-flow is more suitable for packaged software (something you release on occasion; with discrete version numbers; having possibly several older versions in the wild). It that your case? For a more web based approach (released constantly; the only version that matters is the one live) I’ve found the simpler feature branch workflow or github flow approaches much more suitable. – Tolu

Is it possible to do integration, test and staging in other branch then master? – Sexpartite

Answer 1

Git-flow and continuous integration are, by definition, incompatible. Branches are a mechanism for delaying integration: when you commit to a branch other than master (or trunk, if you come from Subversion), you are avoiding continuous integration. Doing continuous integration is simple, but not easy.

Payer

You are only avoiding CI of your untested commits. Using CI to manage the master branch and only committing to the develop branch is just fine. – Schaumberger

No Alex. The point is that a mature team is able to test changes well enough before committing them, so there are no "untested commits". Feature branches are a band-aid that allows you to do low-quality commits. Remove the band-aid, and learn to do high-quality commits! – Payer

Me and my team commit more than 20 times a day every day to a branch for a specific feature. Only when the feature is complete do I merge back or create a pull request. That is literally equivalent to tour so-called high-quality commits. Except we can collaborate remotely and preserve work if an engineer's hard disk fails. – Schaumberger

Well, if your merges from feature branch to master always work quickly and flawlessly, then you're doing fine. – Payer

I agree with @xpmatteo's answer, but to me CI only applies for small features. You cannot integrate untested code. But you cannot test an unfinished feature. So if the feature is not small, i.e. cannot be finished and tested within the day, then by definition you cannot use CI, and git-flow should be better fit. – Slocum

I am curious though, if there is an established workaround to big un-chunkable features when using CI. – Slocum

@Slocum unfinished features are usually hidden from production users with feature toggles, while they can be tested both automatically and manually in test environments – Payer

Answer 2

The truth lies between the two. If you want to adopt git-flow in a strict CD pipeline, you should use the release-branch for your CI:

1. For every (batch of) develop-branch commit(s), let your CI server automatically create a release-branch and run all your tests on it.
2. If it fails, let it report and/or delete the branch, otherwise merge it to master.

The basic idea comes from John Ferguson Smart's slide about CI/CD in Java Maven projects (BDD in Action, Jenkins Definite Guide).

Detoxicate

With git-flow you can never do CD, because when your code lives in different branches you are not continuously ready for a release. You always need to merge everything to trunk before you can release. If you want CD, you should not use branches. – Willin

But isn't your regular review-process a break anyway? Whether you're using pull- or merge-request your feature lives in different branch and becomes ready after approval, i.e. a merge into master. – Detoxicate

@BastianVoigt Sorry but I disagree with this. If you imagine CD as a strict 'this is the only way you can ever do it' definition, maybe you're right. But if you view CD as a guide to improve software and its delivery, and take into account different projects have different nuances, surely there is room for flexibility. Branches help add focus during development, make it easy to open pull requests for discussing changes, improving them, etc. They don't have to be huge changes, and we can ensure the develop branch is always stable / ready for release, and really isn't that the main goal of CD? – Ybarra

Answer 3

In my view, if you want to apply git-flow in Continuous Delivery, you should have two different pipelines as you said in your first approach.

I'd suggest this approach:

1. Develop branch

• Develop branch will trigger the Commit Build: As soon a feature is added to the develop branch (on merge or pull request), the CI will build, test (Unit Testing & Code revision) and package the solution (with a "-develop-vX" suffix). So the team is able to react fast in case of failure.
• Once the Commit Build has finished successfully, the task is done (otherwise, the change is reverted and the developer who committed the change must fix it immediately). In parallel, the Acceptance Test Stage starts deploying the previous build to the Development environment for executing the Acceptance Test Suits (e.g. Functional & Regression testing) without blocking the developer's work. Once it's finished, the develop branch status is communicated to the team. Therefore, the team is aware of the solution's stability during the current Sprint: if the Acceptance Test Stage finishes successfully, the product is ready for merging with the Master branch (Otherwise, it'll be fixed).

2. Master branch

• Once the Sprint is finished, the stable Developer branch (it's stable) is merged and tagged to Master Branch. So the Master Branch will trigger the Trunk Commit Build that will build the solution, test it and package for deployment (the package now is stored with a release candidate or master suffix).
• If the Trunk Commit Build finishes successfully (it should work), the Acceptance Test Stage will deploy and validate the acceptance tests against an Integration environment. And in case of success, the new version is ready for production. Otherwise, in case of error at Commit Build or Acceptance Test Stage, the merge is reverted.

Runny

What you suggest is not CD. CD means that you are always ready for a release that contains the latest commits of all developers, which is not the case here. It is not possible to achieve CD as long as you use branches. – Willin

+1 to @BastianVoigt as the OP clearly states he wants to do CD. This question is really split into two: what CD really is and whether it's possible to use git-flow and CD altogether. – Mireillemireles

but it's CI as you integrate the latest development continuously, isn't it ? – Audile