How do I add Git submodule to a sub-directory?

Question

I have a Git repo in ~/.janus/ with a bunch of submodules in it. I want to add a submodule in ~/.janus/snipmate-snippets/snippets/, but when I run git submodule add <[email protected]:...> in the snipmate-snippets directory, I get the following error message:

You need to run this command from the top level of the working tree.

So the question is: How do I add a submodule to the snipmate-snippets directory?

Phelgen

Going to the root directory of a git repo for submodule commands won't be a requirement anymore (soon). See my answer below – Overthecounter

git submodule add -b <branch> <url> <relative_path_4m_root> – Minier

Answer 1

You go into ~/.janus and run:

git submodule add <git@github ...> snipmate-snippets/snippets/

If you need more information about submodules (or git in general) ProGit is pretty useful.

Grunenwald

it seems a good idea to add branch when adding otherwise HEAD gets easily detached: git submodule add -b <branch> <repository> [<submodule-path>] – Meow

For me this was causing 'subprojects' already exists in the index (I was using subprojects as the directory name). Instead what helped, is the VonC's answer below, i.e. doing cd subprojects, and then git submodule add <get@github …> without the path. – Riot

I upvoted this too soon, and now can't remove my upvote, unfortunately. I'm using git version 2.25.1 and am getting the 'dir_name' already exists in the index error as well. @VonC's answer worked instead. – Hyalo

I figured it out! Please update your answer to explain these subtleties. Let's say I want to add my ripgrep_replace repo as a subrepo inside my eRCaGuy_dotfiles repo, at the path eRCaGuy_dotfiles/useful_scripts/ripgrep_replace. To do that using the command style shown in your answer, you must do cd eRCaGuy_dotfiles... – Hyalo

...and then git submodule add https://github.com/ElectricRCAircraftGuy/ripgrep_replace.git useful_scripts/ripgrep_replace, NOT git submodule add https://github.com/ElectricRCAircraftGuy/ripgrep_replace.git useful_scripts! See the difference in the path I specified? I tried the latter command and it gives me the error 'useful_scripts' already exists in the index. I mistakenly thought that I only had to specify the directory I want the ripgrep_replace dir inside, but rather, I must specify the full path I want that repo's actual files inside! – Hyalo

Answer 2

Note that starting git1.8.4 (July 2013), you wouldn't have to go back to the root directory anymore.

 cd ~/.janus/snipmate-snippets
 git submodule add <git@github ...> snippets

(Bouke Versteegh comments that you don't have to use /., as in snippets/.: snippets is enough)

See commit 091a6eb0feed820a43663ca63dc2bc0bb247bbae:

submodule: drop the top-level requirement

Use the new rev-parse --prefix option to process all paths given to the submodule command, dropping the requirement that it be run from the top-level of the repository.

Since the interpretation of a relative submodule URL depends on whether or not "remote.origin.url" is configured, explicitly block relative URLs in "git submodule add" when not at the top level of the working tree.

Signed-off-by: John Keeping

Depends on commit 12b9d32790b40bf3ea49134095619700191abf1f

This makes 'git rev-parse' behave as if it were invoked from the specified subdirectory of a repository, with the difference that any file paths which it prints are prefixed with the full path from the top of the working tree.

This is useful for shell scripts where we may want to cd to the top of the working tree but need to handle relative paths given by the user on the command line.

Overthecounter

I'm on git version 2.7.4 but still I'm getting this error message Relative path can only be used from the toplevel of the working tree . I'm doing git submodule add ../../../functest – Scots

@user3426358 yes, that is expected: the answer above os about the capability of doing a git submoduel add from any subfolder of the main repo, not just from its root folder. It is not about referencing the submodule remote repo with a relative path. If you do, you will get the error message that you see. – Overthecounter

@user3426358 And by the way, that error message (that you see: "Relative path can only be used from the toplevel of the working tree") is not the one from the original question ("You need to run this command from the toplevel of the working tree") – Overthecounter

So what is the way around my situation? How do we reference the submodule remote repo with a relative path? – Scots

@user3426358 The error message comes from github.com/github/git-msysgit/blob/master/…, which check that your path starts with $wt_prefix (the working tree prefix): here, as I mention in my previous answer (https://mcmap.net/q/81945/-can-39-t-add-git-submodule-when-specified-as-a-relative-path), it is the location relative to the superproject's origin repository. See this comment: #1974681. But an absolute path would be easier. – Overthecounter

Answer 3

For those of you who share my weird fondness of manually editing config files, adding (or modifying) the following would also do the trick.

.git/config (personal config)

[submodule "cookbooks/apt"]
    url = https://github.com/opscode-cookbooks/apt

.gitmodules (committed shared config)

[submodule "cookbooks/apt"]
    path = cookbooks/apt
    url = https://github.com/opscode-cookbooks/apt

See this as well - difference between .gitmodules and specifying submodules in .git/config?

Bridgework

Answer 4

I had a similar issue, but had painted myself into a corner with GUI tools.

I had a subproject with a few files in it that I had so far just copied around instead of checking into their own git repo. I created a repo in the subfolder, was able to commit, push, etc just fine. But in the parent repo the subfolder wasn't treated as a submodule, and its files were still being tracked by the parent repo - no good.

To get out of this mess I had to tell Git to stop tracking the subfolder (without deleting the files):

proj> git rm -r --cached ./ui/jslib

Then I had to tell it there was a submodule there (which you can't do if anything there is currently being tracked by git):

proj> git submodule add ./ui/jslib

Update

The ideal way to handle this involves a couple more steps. Ideally, the existing repo is moved out to its own directory, free of any parent git modules, committed and pushed, and then added as a submodule like:

proj> git submodule add [email protected]:user/jslib.git ui/jslib

That will clone the git repo in as a submodule - which involves the standard cloning steps, but also several other more obscure config steps that git takes on your behalf to get that submodule to work. The most important difference is that it places a simple .git file there, instead of a .git directory, which contains a path reference to where the real git dir lives - generally at parent project root .git/modules/jslib.

If you don't do things this way they'll work fine for you, but as soon as you commit and push the parent, and another dev goes to pull that parent, you just made their life a lot harder. It will be very difficult for them to replicate the structure you have on your machine so long as you have a full .git dir in a subfolder of a dir that contains its own .git dir.

So, move, push, git add submodule, is the cleanest option.

Copartner

Answer 5

one-liner bash script to help facility Chris's answer above, as I had painted myself in a corner as well using Vundle updates to my .vim scripts. DEST is the path to the directory containing your submodules. Do this after doing git rm -r $DEST

DEST='path'; for file in `ls ${DEST}`; do git submodule add `grep url ${DEST}/${file}/.git/config|awk -F= '{print $2}'` ${DEST}/${file}; done

cheers

Tol