SVN: Release branch and externals?

Question

We have two websites for the same client (main www site and another for the ecommerce site which sits on a seperate server) that use shared portion of code (various features/styles/javascript etc.). We currently manage this by having the shared code as seperate projects (in the same repos) in SVN and using svn:externals to pull the branch of each of these into the two website projects.

We have just created two release branches, one each for the two sites. Everything gets commited to trunk for each of the sites as normal when working and when "ready for live" we merge it into the release branch for that site. Works a treat except today we modified the some of the shared code and noticed that the release branch pulled it in straight away when we did an update on the release branch. This is not what we wanted :(

So any ideas how we can iron out this problem. We used externals to DRY up the sharing of the code but is there another way? Notice that in this question (How can I branch in SVN and have it branch my svn:external folders as well?) they mention externals are bad and we should be using a different build process but no mention of what that should be.

We have CruiseControl.net running our builds and are keen to get this nut cracked. Has anyone any ideas on a better process?

Cheers

Pete

UPDATE: We've since moved to using Mercurial (Fogcreek's Kiln) for our source control. Mercurial also has the idea of sub-repos so we followed that route with our project there too. However it comes at a cost, branching is made really difficult as is tagging and simply keeping everything up to date. Our latest method is to bring both projects into one repos including all the shared repos too. This gives us one "mega project" which slows clone time (check out in SVN) but we do that so rarely that the gains of not having to deal with sub-repos makes it well worth while. We now use feature switching/toggling rather than branching which also simplifies our development greatly.

Ebneter

Answer 1

If I understand correctly you have something like the following structure:

• project1
  • trunk
    • library (via svn:externals library svn://repo/library)
  • branches
    • release1
      • library (via svn:externals library svn://repo/library)
    • release2
      • library (via svn:externals library svn://repo/library)
• project2
  • trunk
    • library (via svn:externals library svn://repo/library)
  • branches
    • release1
      • library (via svn:externals library svn://repo/library)
• library

I'm assuming you are settings svn:externals on /project1/trunk to /library. If you then merge the revision with the svn:externals to /project1/branches/release1 and update that branch, you will automatically get the latest version from the library. By default svn:externals will get the HEAD revision of the link.

You could define the svn:externals to link to a specific revision, like this: svn:externals -r123 library svn://repo/library. This means that the externals link will always point to that specific revision. So you can safely use this type of svn:externals link in your release branch without worrying that the shared library code will ever be updated unless you manually change the svn:externals

Probably a better way would be to use tags for the shared library and link to a specific tag. In this case you would get the following repository structure:

• project1
  • trunk
    • library (via svn:externals library svn://repo/library/tags/version3)
  • branches
    • release1
      • library (via svn:externals library svn://repo/library/tags/version1)
    • release2
      • library (via svn:externals library svn://repo/library/tags/version2)
• project2
  • trunk
    • library (via svn:externals library svn://repo/library/tags/version2)
  • branches
    • release1
      • library (via svn:externals library svn://repo/library/tags/version1)
• library
  • tags
    • version1
    • version2
    • version3

Huey

Ahhh now thats more like it. Great write up btw. Thanks. I suppose it would be safe to leave trunk referencing the HEAD of the externals but have my releases point to a tagged version. Would that be safe I wonder? I've been going around and around with this problem of late and been getting my self in a mess. Thanks for the clarity. – Ebneter

@Pete Duncanson: I've experimented with this (leaving the project trunk referencing the HEAD of the externals but have my releases point to a tagged version) and haven't noticed any major problems. – Nonprofit

Whether you will have problems or not will depend entirely on the code you are referencing as the svn:external. If the code is backwards compatible (same methods, classes, behaviours, only new stuff is different), you won't have any issues. However, if you have made changes to classes, methods, or the behavior in general of the code, you will be lucky if your project even compiles. – Brady

To add to this answer: @PeteDuncanson Think of your "library" as having its own development lifecycle (thus own tags/branches/trunk and releases). You can theoretically have a separate team maintaining it on their own schedule (but in reality it would mostly be the same team). Using their HEAD in your trunk may be unstable and you will not know at what stable release version that "HEAD" is at. Rather use externals to "release branches" so that you know you are using v1.9 of the library in you trunk, and consciously update it to new (stable) releases when available. – Exodus

Another more practical suggestion. Integrate that library into the project that mostly control it. Then use svn:externals in other projects to the release branches of that main project, but directly onto the library eg. project1/branches/release1/library this couples the library version directly to the main project versions, but ensures that breaking changes in the library inside project1/trunk does not affect the other projects, also releases of them then track a specific release of the main project's library. – Exodus

Answer 2

Otherside has some good advice, but it really is using svn:externals as a poor man's dependency management system. It is making a hacky anti-pattern somewhat workable for the disciplined.

You're absolutely correct that that svn:externals is not the path.

One thing more to think of, if you stay on that path - unless your svn tags are atomic (via a pre-commit hook), you'll want to specify the revision as well as the tag.

I'm currently suffering shell shock at having inherited some .NET stuff, makes me miss maven so much. I'd even settle for a mess of ant/ivy.

You may want to check out https://www.nuget.org/

Nannana

Answer 3

Yes, using externals pointed to a specific revision or tag is the way to go. You could easily found this out just by reading the svn manual on externals... RFD! also, is just about one page long ... :)

http://svnbook.red-bean.com/en/1.0/ch07s03.html

Inferno

I've been reading up so much of late, all while under preasure to get projects out the door, update our build server to handy release branches, while taking on new staff (taking us to a total of 3!), inbetween breaking two sites because we've not had sufficent prcoesses in place. Its easy to get burnt out with all the info. I posted here not out of lazyness but frustration, exhaustion and confusion. So far its been worth the bounty hit. Thanks for your answer ;) – Ebneter

Answer 4

You can do svn update --ignore-externals if you don’t want the externals to be updated.

Tympanitis

I'm thinking externals might not be the way to go at all. My trouble is getting changed from the externals into the release branch when I want them to be, currently they are added regardless when we do an update. The only solution with the current setup is to have release branches for each of the externals and that quickly becomes a knightmare to manage, too much going on. Hence me thinking externals are a deadend in this case. – Ebneter

Answer 5

I am making release/stable branches by copying also the externals so that the branch becomes completely independant from the trunk. Externals are svn-copied using

svn copy --parent

... to their respective mount point. un-revisionned externals use revision you want to root your branch on; revisionned externals used their specified revision at copy. svn:externals property(ies) must then be removed in the new branch.

Some scripts (perl for me) may help to automate this. Ping me if you need more info on this.

Huth

But this does not help my problem. We make changes to the externals as part of our work on the main branches. This way you'll loose the history and won't be able to work out what you changed with out manual diff'ing. We solved the problem using the method in my edit. – Ebneter

Ok, I thought you were looking for release branches that are isolated from everything and can evolve independently from externals and trunk. Btw, I don't lose history with "svn copy" and hence, automated merged is still possible. Good you found a solution anyway. – Huth