Shared components throughout all projects, is there a better alternative than svn:externals?

Question

My situation: I have several components, which sometimes have changes to them, and are shared across a lot of different projects. Each project puts these in a subfolder called /depends. The depends holds a bunch of svn externals for all of our common components.

svn:externals is causing me a lot of time and pain.

• Show log on the project root folder will not show changes for svn:external folders (yet funny enough commit and update will work with svn:externals)
• When you branch, the svn:externals are not branched.
• Because of no branching on svn:externals, any change usually breaks the trunk.
• Tags do not freeze their externals. This really defeats the purpose of tagging.

Remember that I have several projects (Let's say 10 for this discussion each using the same externals), so keeping normal committed directories for each project would cost me a lot of merging time.

Is there a better alternative for my situation?

Exchequer

Answer 1

I believe part of the problem is that the release cycles for the shared components don't necessarily match the release cycles for the projects.

Shared components as described here have their own release cycle. In other words each one could be managed as a separate project (or perhaps the collection of them managed as a separate project) with a release/version number all its own.

Note that the svn:externals definition can include a specific revision.

This allows each of the projects that makes use of a shared component to be developed against a specific release/revision of that shared component (or the collection of shared components), providing a stable set of dependencies for the project. Changes to the components won't break the projects because each project is looking at a specific revision of the components that isn't necessarily the HEAD on the trunk.

While this may seem like more work up front, I believe in the long run this practice provides a better change management process for this type of situation.

Cosine

Answer 2

I agree with @Ken.

I'd very strongly recommend against using svn:externals at all without a specific revision. Without a revision, it is impossible to recreate old checkouts. If you only pin your externals when tagging, you will only be able to recreate what you've tagged. If you want to recreate some intermediate revision in trunk, you're on your own.

One reason for not branching externals is that it is not clear how that should be done. If your externals into project A points to tags/2.0.0 and you're creating a 3.4.x branch for your project, what should the external for project A point to? Should project A branch as well? If so, to what version?

If the projects have different release cycles, it is in general impossible to define a reasonable behavior for externals when branching.

(You might want to take a look at the svncopy.pl script if you haven't already (included in the Subversion source distribution) which allows you to pin externals during tagging.)

We've found that svn:externals work very poorly when used to keep together a set of components you're actively developing. Externals works nice to bring in external components which don't move very much, and where you don't have the issue of branching.

Caskey

it's not impossible, just look at the commit date in your main repo and revert the external to the revision # it was at that time. – Bargeboard

s/impossible/prohibitively difficult/, then. – Caskey

you haven't provided an alternative to svn:externals in your answer – Elysium

SVN 1.9 has a new --pin-externals option for svn copy which will "pin externals with no explicit revision to their current revision (recommended when tagging)" – Marcel

Answer 3

I stated this on a similar question: You should use svn:externals as external references from different repositories. So svn:externals should refer to components, modules, 3rd party tools, etc. which reside in different repositories.

You should not use svn:externals to emulate a "symbolic link"-behaviour by using externals to point into the same repository.

You can solve such issues most of the time by modifying your build-structure, or use checkout-scripts and sparse checkout feature.

svn:externals have a lot of problems most of them are difficult to see, track and repair: see an example here

• commits cannot span over externals(no atomic commits)
• branches will not branch their externals
• tags will not "freeze" their externals so latter builds may result in different/broken builds
• merging and reeintegrate merge will not work on externals

If you use externals to point to other repositories, you will most of the time do not have these problems.

Hetzel

Are you certain that all these limitations still apply now that svn:externals supports relative paths? After switching my svn:externals to use relative paths (for example: ../../shared_dir), I'm able to create branches which also branch their externals. Reintegration also works without any issues. See: svnbook.red-bean.com/en/1.7/svn.advanced.externals.html – Snowfall

@Avian00: relative externals are working only in specific repolayouts and in single repository. Also if you do this and decide to restructure your repository(for example join multiple or separate them with dumpfilter) you will run into big trouble. – Hetzel

Answer 4

You could try using the so-called vendor branches:

http://svnbook.red-bean.com/en/1.5/svn-book.html#svn.advanced.vendorbr

It's a useful strategy when you deal with third-party libraries, but it can also be useful in a situation like yours.

Sanity

Answer 5

svncopy.pl (mentioned in this question) will rewrite the paths in the svn:externals to their new location in the branch.

Cobbs