What branching strategy should I use during the development/maintenance of a web application?

Question

I am trying to decide on the best branching strategy for a web application project. Here is what I have come up with until now and I would greatly appreciate any comments and experiences.

The way I see it there are two main branching strategies: "branch by release" and "branch by feature".

"Branch by release": Development takes place on the trunk. When the time for a release is near, a branch is made for that release. This branch is then stabilized/tested and finally a release is made. After the release, the branch is merged back into the trunk, while keeping the release branch alive for bug-fixing. Is a bug-fix applied, then it is merged into the trunk ( if the development on the trunk hasn't eclipsed the bug in other means). New features are added to the trunk and don't affect the release branch. When a new release time nears, a new release branch is created a.s.o.

"Branch by feature": The trunk is always the "production" trunk (the code that is live). Bugfixes are commited directly to the trunk. Features for the next release are developed in feature branches.Bugfixes are merged from time-to-time into the feature branches. When release time comes, the features branches are merged into the trunk and the cycle of life continues.

Now the way I see it the big practical difference between these two strategies is that "by release" allows you to maintane different production versions of you software ( when client A has version 1 and client B version 1.5, client is the paying customer in this case). In contrast using the "by feature" strategy you can only support the current production version ( all clients are using latest version).

Since in a typical web application all clients are using the same "latest" version (since they all access the same server), I would assume that the "by feature" approach is the most commonly used. It removes the need to merge "across the hierarchy", say when a bugfix has to be applied to all 3 release versions.

So my current status is that I should go with "branch by feature". If it matters, my team is not very skilled.

Twitch

Thanks for asking this question - i'm really enjoying everyone's replies. – Jot

Answer 1

If you only have one release live at any time, and you do all development in a single feature branch, then these approaches are effectively the same.

If branch-by-feature to you means having a several branches on the go at once, i'd avoid it like the plague. More branches means more merging, which is a pain in itself, and more integration hell. It's far better to do continuous integration to a single codeline.

If your deployment process is any more involved than branch, test, go live, then an advantage of branch-by-release is that you can have multiple release branches going at once, in different phases: one live and being bugfixed as necessary, and another one being stabilised, tested, going through acceptance, etc, while development continues on the trunk. If you have a live trunk, on the other hand, once you merge a feature branch in with a view to taking it live, you've lost your ability to do bugfixes to the current live system. The feature branch merge becomes a point of no return.

Jot

i've not thought of it like that before, interesting view point 1+ from me :-) – Nitrosyl

Hmm.. you mention "merge a feature with a view to taking it live". In such a case have a problem because, as you said, you just lost the ability to do bugfixes on the live system. But to me merging a feature means: The merging of the features and the release of a new version is an atomic operation. In other words the merging of a new feature means that directly after the merge the trunk goes live as a new release. This also implies that the client has tested and accepted the new feature before merging. But now that I am reading my comment it does sound kind-of weird... – Twitch

@BernardMarx: i think you've found the problem: you can't test the code that will go live until you've merged, because it doesn't exist until then. But once you've merged, you're committed. I suppose there is a narrow window: you could freeze the trunk, do the merge on a development machine, run pre-production QA without checking in, then commit the merge if it passes, taking the committed code live straight away. If you desperately needed to bugfix live during that time, you could unfreeze the trunk, fix, then start the feature merge all over again. Don't think i'd want to try that though! – Jot

Merging isn't a pain with Git or Mercurial, I hear. – Gallon

@Ziplin: better tools - like Git and Mercurial - cut away some of the reducible complexity of merging. But there is still irreducible complexity - the fact that your code over here knows how to deal with credit cards and PayPal, but the code you're merging over there now lets customers pay with gift cards. No tool can help with that. Merging sooner and more often can, because it lets you deal with the divergence while it's still small. – Jot

@Tom Anderson: Yeah freezing the trunk shortly before release was what I was thinking and I agree it just doesn't sound elegant. – Twitch

Answer 2

What kind of software are you developing? Shrink wrap? Open Source Project? If so, then go with the "branch by release" or "unstable trunk" approach. Especially if your release cycles are every six months to a year apart.

But if your maintaining a web-based project that has changes going out on shorter frequencies, like once every few weeks or less, then go with the "branch by feature" or "stable trunk" approach. The problem with this approach is integrating multiple feature changes that have sweeping changes make the merge process less than fun. It just really gets difficult.

But both of those work well, but what if you need both? That is, you have a project that deploys says once every couple of weeks with big feature changes, but you find that you have a number of bug fixes that you cannot wait for those feature changes to be ready. Trunk is your release branch with the "branch by feature" approach. What if you could get both releases and features their own branch?

Check out this blog entry by CollabNet's Bob Archer. His Agile Release strategy gives you best of both. I've used this. It's extremely flexible. Even though Bob doesn't show it in his diagram, you can have multiple release branches going at the same time. This means you could have one release branch that is ready for rollout to production, and another that is being prepared for final QA checks. But two things to consider:

First, how good are your developers at merging? You cannot do the the agile release strategy approach by yourself, even if it is a small team. Everyone has to do their part and they really have to understand merging and the tools they are using to do the merge.

Secondly, you are going to need a good grasp on changes that ready and those that are about to be. Release management is key to making this work like clock work. Each feature when ready will need to get assigned to a release branch and merged to it.

Whatever approach you choose, it comes down to what you are developing and the frequency of changes you are releasing for that development.

Mayence

That 'agile release' strategy (which runs in diametric opposition to the release strategy of agile development, but never mind) looks like it involves an eye-watering amount of merging. How often are the merges into release branches? If every few days, then it works quite like an unstable trunk, but where the role of trunk passes between different branches over time. That should work, and be more flexible than a stable trunk. But if the merges are far apart, i just can't see this working - if large-scale refactoring is done on two or more task branches, then you get a seven-circle merge hell. – Jot

@Tom Anderson: Merges to the release branches only occur when the feature is done. So ideally, there should only be one merge from the feature branch to the release branch. Now that said, not all issues are worked out in a feature branch the first time. So there can be subsequent merges from the feature branch to the release branch for patch-fixes if better resolved in the feature branch rather than the release branch. – Mayence

@Tom Anderson: You are correct. If merges are far apart (more than a few weeks) for a release, you might as well reduce the merging overhead and go with a branch by release or branch by feature approach. – Mayence

Thx for bringing up the skills perspective. I have edited the original post to state that skills are...limited – Twitch

Answer 3

These choices are not mutually exclusive - use both. They solve different problems:

"Branch by release" - release branch is used to ensure you can back to the source used to produce the current live version (or previous released versions) while the next version is in development. E.g. This is in order to make changes to the release version for bug fixes or backpatch features from the current development trunk.

"Branch by feature" - used to keep a stable development trunk at all times, which is particularly useful for multiple developers and for experimental "maybe" features.

I would use both, but you can forego one of the approaches if the problem it solves doesn't apply to you or if you have a different solution that problem.

I strongly recommend using a modern DVCS like git or Mercurial. They are geared at parallel development, so they store changes differently than older systems, which makes merging much saner.

Bridget

Answer 4

At the risk of confusing you further: you can have release branches and make all changes on feature branches. These things are not mutually exclusive.

That being said, it sounds like you don't need parallel release families and you want to deploy often, perhaps even continuously. So you would want to have a "stable trunk" that you can release at any time. Feature branches help to keep the trunk stable, because you only merge back to the trunk when the changes are done and have proven themselves.

So I'd say that your choice is a good fit.

Lineate

One approach to continuous deployment is to develop features on branches, and only merge to trunk when they're ready to go live. That keeps the addition of features atomic - they're absent or complete - but means you have large, infrequent merges. Another is to integrate continuously, but to somehow hide the features until they're ready to go live, with some sort of configuration switch that lets you expose them in internal environments. That means smoother integration, but is a bit of a bodge, and complicates testing, because you have to test both the features-off and features-on modes. – Jot

Although maybe you want to make the in-progress features optional - a Google Labs sort of approach. Maybe. – Jot

Anyway (and sorry for making three comments), i get the impression that the continuous deployment guys are strongly in favour of a single codeline with continuous integration (this comes out of the lean manufacturing model which is the intellectual origin of that movement), and i think their approach is that features need to be developed incrementally. I'm by no means an expert on any kind of lean, though - i'd love to read more about this particular question. – Jot

@Tom: you don't have to postpone the whole merge - you can regularly merge the changes from the trunk to the feature branch to stay in sync while you develop, so that the reintegration at the end is trivial. Of course, if the changes entering the trunk are large then even these merges can be a pain. There isn't really any solution for that other than splitting big changes into smaller maneagable changes. – Lineate

True, that helps keep the merges smaller. Still, if you only merge a feature branch into the trunk when the feature is done, then the changes that are coming out of the trunk to the feature branches will necessarily be feature-sized. Whether this is painful or not all depends on how big the features are, i suppose. – Jot

Answer 5

I tend to use Git for my projects, but the process I tend to follow goes like this (and should work for Subversion as well):

• for each new feature, create a branch for that feature.
• When everything works, merge it into the staging branch, and deploy it to the staging server (you do have one of those, right?)
• Once we're sure the client is happy with what's on staging, we merge the staging branch into the production branch, tag it as something like production_release_22 or production_release_new_feature_x, and then deploy that tag to the production server.

Tags are never, ever updated - once something gets deployed, it stays that way until more changes are built, tested, and tagged - and then the new tag is deployed. By making sure that it's tags getting deployed and not branches, I keep myself (or others) from doing things like "I'll just commit this one quick change and update the server without testing it".

It's worked pretty well for me so far.

Blepharitis

I don't really understand what your staging branch is. To me it sounds like a feature branch with the trunk's changed merged into it. So basically the same as a feature branch that is kept up-to-date with the trunk. I am probably arguing semantics here... – Twitch

The staging branch is where all of the things that are finished (but not necessarily fully tested/confirmed by the client) go - the staging server runs code off of the staging branch, but the production server only runs tags from the production branch (which staging gets merged into when we confirm that it's time to deploy the changes). – Blepharitis

If you called the staging branch 'trunk' and the production branch 'release', is this basically the same as the branch-by-feature-and-release strategy that others are also suggesting? Or are there subtle differences? – Jot

Basically the same, yeah - I'd say the only major difference is using tags instead of branches for production deploys. – Blepharitis