Git repository in a Git repository

Question

I have a Git repository including a Git repository.

repo1/
     .git/
     files
     repo2/
          .git/
          files
     files

Is it possible to work with this architecture?

Awe

I should seriously consider using submodules. – Awe

Why are you thinking of doing this? There may be alternatives. – Knelt

This 'problem' occurs naturally when you're working with frameworks and framework components that are hosted as separate repos. For example, if you check out a WordPress or Vagrant+WordPress projects, and then you check out a WordPress theme and maybe some Plugins inside of the file structure that is part of the first repo. Each of these are typically their own git repos, and not submodules. I just add the root folder of each of these 3-rd part repos to the gitignore file of their parent repos. – Seventeenth

Answer 1

You can have nested git repos:
The parent repo will simply ignore nested repo.

jleedev comments and illustrates with this gist script that the parent repo would track the nested repo state through a gitlink.
(gitlink = SHA-1 of the object refering to a commit in another repository. Git links can only be specified by SHA or through a commit mark.
A gitlink has a special mode '160000', used for submodules, but also present for simple nested repos).

However, usual commands would not acknowledge the nested repo: add or commit would apply only in one repo, not the other.

git submodule would allow to reference the nested repo from the parent repo, and keep an exact reference of the child repo.

Another alternative could involve:

• two separate Git repos (not nested)
• a symlink from one to a specific part of the other (both Unix, but also Windows Vista+ have symlinks)

Demagogic

The parent repo won’t ignore it; it’ll track the HEAD with a gitlink. – Maxon

@jleedev: interesting, I wasn't aware of that. Do you have a reference to a documentation? After a (very quick) search, I didn't find that information. – Demagogic

It’s unfortunately sparse; most of the docs talk about submodules and not gitlinks. Grep for 160000 or gitlink in Documentation/. Here’s an example of how gitlinks behave. – Maxon

Most notably, gitlink is the tree/index entry, while submodule is the UI that makes it possible to clone and sync your gitlinks. The documentation doesn’t really explain this. – Maxon

@jleedev: in your example, repo2 is a nested repo? not a submodule? I thought that a file mode 160000 was only to implement submodules, as mentioned in kernel.org/pub/software/scm/git/docs/… – Demagogic

Thanks ! I have to check on that. Submodules seems to fit my needs better since I need to be able to tag both repo or only the repo2 – Awe

@Demagogic Yes, I just created the repo inside. A submodule consists of a nested repo and some config options to tell it how to fetch. – Maxon

@jleedev: excellent. I have integrated your comment in my answer, adding the link to git fast-import man page, which is the only doc in Git mentioning clearly what a gitlink is. – Demagogic

If you do git add . or git add inner-repo the result is a git link, as described here. However, if you do git add inner-repo/ (trailing slash), then it adds the files inside the repo, and starts tracking them as it would any other file (except for inner-repo/.git, which it ignores). So the result is that the two repos track the files separately, and they don't know about each other. – Reddin

Someone left a comment on the above-linked gist: "I cannot reproduce that. Did the behavior change?" – Exposed

@Exposed it depends on how the someone attempted to reproduce that. I can confirm that adding a folder which is the root of a nested git repo would be registered in the main parent repo as a gitlink. – Demagogic

How would this symlink approach work to make the nested .git repo not visible in the parent repo? – Repeated

Answer 2

You are trying to accomplish something called a "submodule".

Please check out Git Tools - Submodules to find out how it's working.

Anglicist

Yup I'm on that, more on the experiments tomorrow :3 – Awe

Answer 3

I've used that structure for quite a while, with the sub-repo directories specified in .gitignore in the outer repo.

It confuses the git tool in my editor (PhpStorm), which always wants to commit to the outer repo, but otherwise works fine. I load the whole outer repo (which includes all innner repos) as a single project in the editor. That allows me to easily search for and examine code in the outer repo while working on an inner one.

I do all Git operations from Git bash in whatever repo I'm working on.

Submodules might be a better approach. I haven't had time to investigate whether they would work better with PhpStorm.

Bough

Answer 4

Yes, you can use this pattern. I've used it in the past to bring in SVN externals into a git-svn clone. Submodules may handle this better now, but didn't suit my needs at the time.

You'll want to add the following to repo1/.git/info/exclude to ensure changes in repo2 don't mix with repo1:

repo2

Wilheminawilhide

Answer 5

I also agree with Ronald William's answer. The main purpose of Git submodules is updating the code taken from the outside world without need to commit changes if that code were modified by the update.

The Composer package management system does the same. Actually they don't recommend to commit those changes either and ignore the vendor folder in .gitignore in the root of project.

It is a nightmare if you'd try to commit this folder because some of the vendor/some_repo can be of a development version, and consequently they have a .git folder which results in all those packages become submodules even if you don't add them with git submodule add. You could see something like this if you modify some_file in a nested .git repository:

~/project_root $ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#   (commit or discard the untracked or modified content in submodules)
#
#    modified:   vendor/nested_repo (modified content)

Note the modified content in submodules entry and that you don't see the some_file name in the output. Instead you see the (modified content) notice, because the root_project .git sees the vendor/nested_repo as a submodule and does not track individual files in that folder.

If you run git add --all you'll get no result until you commit changes in vendor/nested_repo and only after that can you commit changes in the root repository.

Don't do this. Instead if you want to keep your project as a whole .git repository (any, not only Composer built repository), which is very convenient sometimes, add this entry to the root .gitignore BEFORE the initial commit:

.git
!/.git

Unfortunately, for the whole recipe to work you need to run the git add command for each of the nested repositories you want later to modify individually. Notice that trailing slash in paths of repositories is a MUST.

~/project_root $ git add vendor/some_repo/ vendor/another_repo/

Then modify some_file in the vendor/some_repo and see the difference:

~/project_root $ git status
# On branch master
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#    modified:   vendor/some_repo/some_file

That way you can run git add --all and then git commit "Changes ..." in the project_root as usual.

Restorative

Answer 6

There are also package management solutions.

It is true that Git submodules would allow you to develop with the architecture you described, and Git subtrees provide a similar solution that many people prefer.

In my opinion, package management software is an integral part of any complex project. I like Composer because of the intuitive workflows it supports.

Unfortunately Git submodules are not supported by PhpStorm:

Git submodules should be supported (IDEA-64024)

Pervade