Best way to organize a subversion repository of many small projects

Question

To start out, I have looked at the following pages and don't quite have my answer: how-would-you-organize-a-subversion-repository-for-in-house-software-projects and how-do-you-organize-your-version-control-repository

I have also looked at chapter 8 of Pragmatic Version Control using Subversion.

They all have good advice, but I'm having a hard time relating it to my needs.

Basically, I want to organize the code for our web server. We have a $WEBROOT/htdocs and $WEBROOT/cgi-bin. Under our htdocs dir we have $WEBROOT/htdocs/js and $WEBROOT/htdocs/css for java script and style sheets.

Our "projects" are not really projects, but small bits of code - maybe a Perl script, java script file, and a style sheet. We have maybe a hundred or so of these small "projects" that are all pretty much independent of each other, but all live under the same $WEBROOT on the same webserver.

Our code is not in subversion yet, but I want it to be - I am just having trouble organizing it efficiently. We can have multiple svn repositories if needed, but if each repository was only 3-10 elements, that seems like a waste to me.

What I thought could work is something like this: If I write a script to count the running processes on the webserver (for the sake of an example). Let's say I have a perl script, a js file, and a css file. I could name the "project" webserver_processes, and check it into the repository as:

/svnrepo/webserver_processes/trunk

Under trunk, I could have:

htdocs/html/webserver_processes
htdocs/js/webserver_processes
htdocs/css/webserver_processes
cgi-bin/webserver_processes

I don't have any static html docs in this "project" but if i did, they would go in the "html" directory.

The benefit I see in this structure is that I can checkout one "project" at a time without really affecting anything else on the web server. The disadvantage (and maybe it isn't really a disadvantage) is deploying. I would have to deploy 1 project at a time from the repository. I don't see how it would be possible to create a working copy with my structure of $WEBROOT/htdocs and $WEBROOT/cgi-bin using this method.

Another option:

I could create a svn repository like this:

/svnrepo/webcode/trunk

Under trunk would be all of the code on my web server, in these two directories:

htdocs
cgi-bin

The huge disadvantage would be, for a small code change to 1 element, I would have to checkout every piece of code in my web environment. The benefit (somewhat) would be I could do an "svn update" on our web server to pick up any changes committed to the repository.

Maybe I am just making this more complex than it should be, but does anyone have any advice on how I could efficiently organize my code in subversion?

Many thanks in advance!

Brian

Arri

Seems like your two options are the same thing, except you've named the top level directory webserver_processes in one and webcode in the other? – Chaumont

The second option is everything under my $WEBROOT is a single project in subversion. The first option is that every "project" that we work on is it's own project in the subversion repository. – Arri

Answer 1

I think you're making the right call by keeping a single repository for your many projects, so I would only make a change to your deployment process:

Your svn repo would look like (your first option.)

/svnrepo/project1/trunk
/svnrepo/project1/trunk/htdocs
/svnrepo/project1/trunk/css
...
/svnrepo/project1/branches/branch1
/svnrepo/project1/tags/blah
/svnrepo/project2/trunk
/svnrepo/project3/trunk

When you want to deploy use a script to copy the file(s) to where they ought to be deployed.

This way, you're keeping an artificial barrier (a folder) up to organize your thoughts between projects rather than having just a big mess of files.

edit:accidentally saved & additional directories added for clarity

Strage

Answer 2

I think your best bet is the second option: having all code under single repo with htdocs and cgi-bin directories. That's true that you'll have to check out all your code, but you do it once and the rest of changes will be much smaller. If it's a production server you obviously would need to check that all your code is production-ready: keep trunk green, so to speak.

In future it might as well help with eliminating duplicate functionality.

Nombril

Answer 3

Since you're not committed to Subversion yet, consider using Bazaar. It's particularly well suited to version control of many small projects, since it has very little setup overhead.

Riocard

Bazaar does look nice, but unfortunately we don't have python here, and svn is the standard here - there is already an svn server for us to use - I'm just trying to get our code into it. – Arri

I was going to make the same recommendation for Mercurial. :) – Antipas

Answer 4

First, use a single repository. For an example of how well a single repository can scale, look at the Apache svn repository. Everything is one repository.

If you use a single trunk that's directly deployed onto the webserver, then you'll need to make sure trunk stays deployment ready. That means all development should first happen on a branch that gets merged back in. You don't want to end up in a situation where one project is half-finished but in trunk and another project needs to do a bug fix deploy in trunk.

I wouldn't worry about requiring people to check out the whole trunk, but if it's a real issue, then you might consider a hybrid approach:

/svnrepo/trunk/project1/...
/svnrepo/trunk/project2/...
/svnrepo/deploy/htdocs
/svnrepo/deploy/cgi-bin

In this case, developers would have to do an svn copy from their project in trunk to the appropriate place in the deploy directory. You could then automatically release everything under deploy.

Ashes

I like the direction you are heading. In your hybrid example you mention developers wouldn't have to checkout the whole trunk. But how would a developer get their correct perl script (under cgi-bin) and js/css files (under htdocs)? And wouldn't the hybrid example be a lot of duplication? – Arri

@BrianH: The hybrid example would require duplication. Each file would be in 2 places: it's project home and the deployment directory. That's the downside of it. That opens up the possibility of someone directly editing under "deploy" instead of the project. – Ashes