Is it possible to have a Subversion repository as a Git submodule?

Question

Is there a way to add a Subversion repository as a Git submodule in my Git repository?

Something like:

git-svn submodule add https://svn.foo.com/svn/proj --stdlayout svn-project

Where https://svn.foo.com/svn/proj points to a Subversion repository.

I know there is git-svn which allows one to interact with a Subversion repository. So I am thinking, maybe there is a way to checkout a Subversion repository with git-svn and then use it as a submodule.

Barley

Answer 1

No. Your best bet would be to set up a mirror of the svn repository in a dedicated git repository.

git svn clone -s http://subversion.example.com/ mysvnclone
cd mysvnclone
git remote add origin [email protected]:project.git
git push origin master

Then you can add the git repository as a submodule to the original project

cd /path/to/gitproject
git submodule add git://example.com/project.git -- svn-project
git add svn-project
git commit -m "Add submodule"

There is one conceptual difference between svn:externals and git submodule that may trip you up if you approach this from a subversion point of view. The git submodule is pegged to the revision that you give it. If "upstream" changes, then you have to update your submodule's reference.

So when we resync with the upstream subversion:

cd /path/to/mysvnclone
git svn rebase
git push

... the git project will still use the original revision that we committed earlier. To update to the svn HEAD, you would have to use

cd /path/to/gitproject/svn-project
git checkout master
git pull
cd ..
git add svn-project
git commit -m"Update submodule"

Procter

Have you tried these codes before post it to here? submodules can't work properly at git svn. – Tun

@Tun yes, and I am not advocating mixing git-svn and submodules in the same repository. The clone that uses git-svn is only a bridge to creating a native git clone of the svn repository. – Procter

sorry. I haven't found you using two folders to acts as submodules. nice trick. – Tun

Nice technique. Don't you need a 'git init --bare' step on the server, before pushing the git-svn repo there? I had to do this. – Honig

It's probably not the default but you can bind svn:externals to a specific revision just like git submodules do. – Vendor

I get complaints (warning) about a missing --prefix flag for the first command. Should I care? – Barto

@Barto have a look here blog.tfnico.com/2013/08/always-use-git-svn-with-prefix.html – Sophistication

If something ever happens to the original svn/git project, is it possible to rebuild it using git svn again? – Dunc

@Dunc you have all the history (and svn metadata info) in your git svn copy, so yes in theory it would be possible. Haven't tried it though so YMMV :) – Procter

I discovered that when cloning from sourceforge.net/p/slam6d/code/HEAD/tree that removing the -s flag was needed to get svn to clone the actual repository. – Chaqueta

@MatthewHoggan -s tells git-svn to use the "standard layout" that includes trunk, branches and tags. The respository you linked to doesn't use the traditional tags directory, so I think that's why it didn't work. Maybe you could pass in -b if you need the branches, or just nothing as you say, if all you need is trunk. – Procter

Answer 2

I just went through this. I'm doing something similar to rq, but slightly different. I setup one of my servers to host these git clones of the svn repos I need. In my case I only want read-only versions, and need a bare repo on the server.

On the server I run:

GIT_DIR=<projectname>.git git init
cd <projectname>.git/
GIT_DIR=. git svn init svn://example.com/trunk
GIT_DIR=. git svn fetch
git gc

This sets up my bare repo, then I have a cron script to update it:

#!/usr/bin/python

import os, glob

GIT_HOME='/var/www/git'

os.chdir(GIT_HOME)
os.environ['GIT_DIR']='.'
gits = glob.glob('*.git')
for git in gits:
  if not os.path.isdir(git):
    continue
  os.chdir(os.path.join(GIT_HOME, git))
  if not os.path.isdir('svn/git-svn'):
    #Not a git-svn repo
    continue

  #Pull in svn updates
  os.system('git svn fetch && git gc --quiet')
  #fix-svn-refs.sh makes all the svn branches/tags pullable
  os.system('fix-svn-refs.sh')
  #Update the master branch
  os.system('git fetch . +svn/git-svn:master && git gc --quiet')`

This also requires fix-svn-refs.sh from http://www.shatow.net/fix-svn-refs.sh This was mostly inspired by: http://gsocblog.jsharpe.net/archives/12

I'm not sure why the git gc is needed here, but I wasn't able to do a git pull without it.

So after all this you can then use git submodule following rq's instructions.

Salpa

One would think you could even do this as a commit hook. – Manuelmanuela

Answer 3

Currently git-svn doesn't support svn:externals. But there are two other tools which can help you:

1. SubGit

   SubGit is server-side solution, it enables Git access to Subversion repository and vice versa. You may refer to documenation for more details, but in general it is fairly easy to use SubGit:

   $ subgit configure --layout auto $SVN_URL $GIT_REPO
   

   Above command will detect branches layout in the SVN project and then will create empty bare Git repository ready to mirror SVN project. You may be asked for credentials unless those are already stored in the SVN credentials cache at ~/.subversion directory. You can also adjust $GIT_REPO/subgit/authors.txt to map SVN author names to Git identities.

   $ subgit install $GIT_REPO
   $ ... let initial translation complete ... 
   $ TRANSLATION SUCCESSFUL
   

   At this moment you have Subversion repository connected to newly created Git repository. SubGit translates SVN revision into Git commit on every svn commit and Git commit into SVN revision on every git push.

Everything you need further is to make Git repository available to committers. Take a look at git-http-backend for that. Then you can add created Git repository as a usual submodule. SubGit is also availale as an add-on for the Bitbucket Server, to find out more check out here. So, there is no need to use any external tools like git-svn or any other.

SubGit is proprietary software but it's free for small companies (up to 10 committers), academic and open-source projects.

2. SmartGit

   SmartGit replaces git-svn on client-side. More information on its features you may find here.

   In particular SmartGit does support both git submodules and svn:externals, you can mix them in your repository.

   SmartGit is proprietary software but it's free for non-commercial usage.

Overspend

Both the subgit (subgit.com/documentation/…) and the smartgit supports the svn:externals in the same way by an explicit .gitsvnextmodules file in a working copy. Which means you still have to use these software to checkout externals and you can not use basic git utility to checkout those externals directly from an external git hub server like github or gitlab. So the sources releated to an snv:externals on an external git hub server won't be observable and downloadable without these software which is still a significant issue. – Eteocles

Answer 4

In addition to what rq said, another method would be to use the third-party "externals" project (http://nopugs.com/ext-tutorial), which better mimics how svn external references work. With externals you can track either git or svn repositories, and it looks easier to push your changes upstream to those repos. However, it requires project members to download and install the separate package.

I haven't used submodules or externals yet; however, I've spent a few hours reading about all alternatives and it looks like externals will be a better fit for my needs. There is an excellent discussion about these and other custom methods in Chapter 15 of "Version Control with Git", by Jon Loeliger (http://oreilly.com/catalog/9780596520120), which I strongly recommend.

Guideline

Answer 5

Piston is being rewritten to support this, and the converse, plus the existing Subversion URL in a Subvresion repoistory and git+git.

Check out the piston Github repository.

Unfortunately it doesn't seem to have been released.

Comose

Piston will fail in your face when you need it the most though ;), so I don't recommend that. Plus there aren't any bugfixes for piston anymore. – Fdic

Answer 6

Well, there is git-remote-testsvn, so I guess something like

git submodule add testsvn::http://www.telegraphics.com.au/svn/bzquips/trunk/ \
    module/bzquips

should work. Does it?

Ergo