Subversion: Is there anything faster than "svnsync"?

Question

So I have my subversion repository stored on some cloud (for example code.google.com) but due to various reasons I need to make my code non-public.

I decided I needed to download the entire repository and migrate to my own svn server.

So I went about using:

svnsync init DEST SRC
svnsync sync DEST

And it took about 0.5 seconds for each revision of the repo!

Luckily my repo only had like 200 revisions... so a couple of minutes to wait. But what about mature projects that have 200,000 or 2,000,000 revisions!

... 2e6 * 0.5 / 60 / 60 / 24 ~ about 11 days!

---

Is there anything faster than "svnsync" to download your repo from a cloud?

Pauly

My snarky answer is to say "git is faster" (because it is). But you'll probably also see better speeds using "svnadmin dump" and "svnadmin load". – Guenzi

how do you do svnadmin dump on something like code.google.com/? – Pauly

@DietrichEpp your comment was snarky in 2010. but git is now the defacto standard. given that git is the defacto standard, suggesting "use git not svn" is a good/great answer. – Pauly

Answer 1

Well, obviously you could back it up yourself on the server and then zip it and download it. Or you could just not download all the history.

But what's the point of this question? It's a bit academic, as your problem is solved.

Farrago

"zip it and download it". That is indeed the fastest way, but it cannot be done with the subversion command (over the subversion network protocols), and is probably not possible with Google Code. – Houghton

A little academic curiosity never hurts. ;-) – Berdichev

Answer 2

I have this same problem in my collection of repositories that have hundreds of thousands of revisions. Here is how I get around it:

1. Create empty repository on mirror.
2. Create a gziped dump file of my repository. (my backup system already does this) (note: this step took overnight to send my giant repository across continent)
3. scp (or your favorite remote file copy technique), the dump file to the mirror server.
4. Load repository, making sure to specify --force-uuid.
5. Set up the revprops on revision 0. I just took a normally configured blank repository and looked at its rev 0.

Now you are ready to run svnsync on your master server. This will continue from wherever your dump left off at.

Rowlett

Also, I use the write-thru mirror technique to use this as a local mirror for my remote offices. Also I run one in the local office as an "emergency backup". – Rowlett

Answer 3

Yes, there is. Replication performed by VisualSVN Distributed File System (VDFS) is at least 10 times faster than replication via svnsync. Moreover mirrored VDFS repositories are writable.

The technology enables distributed teams to work with Subversion repositories at the same speed as if they all were in the same local network. Employing the VDFS technology, the main repository can be replicated across multiple sites and locations which results in up to 1000% read operations speedup. For instance, the 50Gb working copy can be checked out in as little as 10 minutes. The same task performed over a regular internet connection would take an hour at the very least.

Genius

Answer 4

In the OP's case, where you do not have console access to the svn server, svnsync (which is essentially svn checkout from URL 1 combined with svn commit to URL 2) is as good as you are going to get.

But if you do have access to the server, there are faster ways than svnsync. One good method to build a mirror is use svnadmin hotcopy to make the initial copy of the repository, then use the svnsync init --allow-non-empty option added in subversion 1.7 to turn it into a mirror. This also gives you a backup of your hooks etc. which svnsync will not otherwise do.

Note that you will want to move your hooks directory to hooks-original or something so that they will not be used by the mirror---especially if you have a post-commit hook on the original repo that calls svnsync sync!

Fanaticism