Update 2 Sept. 2023: the "proper" way to do it is through git submodules. But, they take some learning and experience. If in a hurry or just archiving projects, use the brute-force methods below, instead. If wanting to learn git submodules to do proper "repo-within-repo" software development with git, see the bottom of this answer in the section titled "More on git submodule".
Manual, brute-force method:
For anyone landing on this page whose goal is just to archive a bunch of git repos inside a bigger parent repo or something, the simplest brute-force solution is to just rename all nested .git folders to anything else--ex: to ..git. Now, git add -A will add them all just like any other normal folder inside the parent git project, and you can git commit everything inside the parent repo easily. Done.
Automatic, brute-force method:
Use git-disable-repos.sh
(Part of https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles).
I just wrote this script over the weekend and have already used it on a number of projects. It works very well! See the comments in the top of the file for details and installation, and run git disable-repos -h for the help menu.
Installation:
git clone https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles.git
cd eRCaGuy_dotfiles/useful_scripts
mkdir -p ~/bin
ln -si "${PWD}/git-disable-repos.sh" ~/bin/git-disable-repos
# If this is the first time using your ~/bin dir, log out and
# log back in now. Otherwise, just re-source your .bashrc file:
. ~/.bashrc
Here is the standard usage pattern:
cd path/to/parent/repo
# Do a dry-run to see which repos will be temporarily disabled
git disable-repos --true_dryrun
# Now actually disable them: disable all git repos in this dir and below
git disable-repos --true
# re-enable just the parent repo
mv ..git .git
# quit tracking the subrepo as a single file (required
# if you previously tried to add it to your main repo before
# disabling it as a git repo)
git rm --cached path/to/subrepo
# add all files, including the now-disabled sub-repos, to the parent repo
git add -A
# commit all files
git commit
That will commit all sub-repos, including their (now ..git) .git folders and all git artifacts, as regular files, to the parent git repo. You have 100% of the control! Want to update just 1 subrepo? Then cd into it and rename its one ..git folder back to .git, manually, then use that sub-repo like normal, then when done run git disable-repos --true on it again (or manually do the rename from .git back to ..git), and commit it into the parent repo. The beauty of my git disable-repos script is that it can quickly and seemlessly disable or enable 100s of subrepos at once if necessary, whereas this would be impractical to do manually.
Perhaps my use-cases is strange: I need to just commit a ton of stuff into one repo until I can clean up and separate out each subrepo individually at a later date, but it does what I need it to do.
And here is the full help menu output of git disable-repos -h:
$ git disable-repos -h
'git disable-repos' version 0.3.0
- Rename all ".git" subdirectories in the current directory to "..git" to temporarily
"disable" them so that they can be easily added to a parent git repo as if they weren't
git repos themselves (".git" <--> "..git").
- Why? See my StackOverflow answer here: https://mcmap.net/q/183491/-git-how-to-make-outer-repository-and-embedded-repository-work-as-common-standalone-repository
- See also the "Long Description" below.
- NB: if your sub-repo's dir is already being tracked in your git repo, accidentally, stop
tracking it with this cmd: 'git rm --cached path/to/subrepo' in order to be able to
start tracking it again fully, as a normal directory, after disabling it as a sub-repo
with this script. To view all tracked files in your repo, use 'git ls-files'.
- References:
1. https://mcmap.net/q/11542/-how-do-i-make-git-forget-about-a-file-that-was-tracked-but-is-now-in-gitignore/1274447#1274447
2. https://mcmap.net/q/183680/-add-subproject-as-usual-folder-to-repository/27416839#27416839
3. https://mcmap.net/q/14569/-list-files-in-local-git-repo/14406253#14406253
Usage: 'git disable-repos [positional_parameters]'
Positional Parameters:
'-h' OR '-?' = print this help menu, piped to the 'less' page viewer
'-v' OR '--version' = print the author and version
'--true' = Disable all repos by renaming all ".git" subdirectories --> "..git"
So, once you do 'git disable-repos --true' **from within the parent repo's root directory,**
you can then do 'mv ..git .git && git add -A' to re-enable the parent repo ONLY and
stage all files and folders to be added to it. Then, run 'git commit' to commit them.
Prior to running 'git disable-repos --true', git would not have allowed adding all
subdirectories since it won't normally let you add sub-repos to a repo, and it recognizes
sub-repos by the existence of their ".git" directories.
'--true_dryrun' = dry run of the above
'--false' = Re-enable all repos by renaming all "..git" subdirectories --> ".git"
'--false_dryrun' = dry run of the above
'--list' = list all ".git" and "..git" subdirectories
Common Usage Examples:
1. To rename all '.git' subdirectories to '..git' **except for** the one immediately in the current
directory, so as to not disable the parent repo's .git dir (assuming you are in the parent
repo's root dir when running this command), run this:
git disable-repos --true # disable all git repos in this dir and below
mv ..git .git # re-enable just the parent repo
Be sure to do a dry run first for safety, to ensure it will do what you expect:
git disable-repos --true_dryrun
2. To recursively list all git repos within a given folder, run this command from within the
folder of interest:
git disable-repos --list
3. Assuming you tried to add a sub-repo to your main git repo previously, BEFORE you deleted or
renamed the sub-repo's .git dir to disable the sub-repo, this is the process to disable
the sub-repo, remove it from your main repo's tracking index, and now re-add it to your
main repo as a regular directory, including all of its sub-files and things:
Description: remove sub-repo as a sub-repo, add it as a normal directory, and commit
all of its files to your main repo:
Minimum Set of Commands (just gets the job done without printing extra info.):
git disable-repos --true # disable all repos in this dir and below
mv ..git .git # re-enable just the main repo
# quit tracking the subrepo as a single file
git rm --cached path/to/subrepo
# start tracking the subrepo as a normal folder
git add -A
git commit
Full Set of Commands (let's you see more info. during the process):
git disable-repos --true # disable all repos in this dir and below
mv ..git .git # re-enable just the main repo
git ls-files path/to/subrepo # see what is currently tracked in the subrepo dir
# quit tracking the subrepo as a single file
git rm --cached path/to/subrepo
git status
# start tracking the subrepo as a normal folder
git add -A
git status
git commit
Long Description:
I want to archive a bunch of small git repos inside a single, larger repo, which I will back up on
GitHub until I have time to manually pull out each small, nested repo into its own stand-alone
GitHub repo. To do this, however, 'git' in the outer, parent repo must NOT KNOW that the inner
git repos are git repos! The easiest way to do this is to just rename all inner, nested '.git'
folders to anything else, such as to '..git', so that git won't recognize them as stand-alone
repositories, and so that it will just treat their contents like any other normal directory
and allow you to back it all up! Thus, this project is born. It will allow you to quickly
toggle the naming of any folder from '.git' to '..git', or vice versa. Hence the name of this
project: git-disable-repos.
See my answer here:
https://mcmap.net/q/183491/-git-how-to-make-outer-repository-and-embedded-repository-work-as-common-standalone-repository/62368415#62368415
This program is part of: https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles
Other, More-Sophisticated Tools:
For anyone looking for a more "professional" solution, these seem to be the most popular solutions, in order with the most-popular (and seemingly, therefore, most-supported?) first:
git submodule - https://git-scm.com/docs/git-submodule - the canonical, officially-supported tool built into git.git subtree - https://www.atlassian.com/git/tutorials/git-subtreegit subrepo - https://github.com/ingydotnet/git-subrepo
Which of those is the best? I cannot say, but they all look confusing to me so I'm choosing the manual, brute-force option I described above, as that meets my intended purposes best in this case until I can find the time to break out each of the sub-repos into their own individually-maintained repos on GitHub someday.
More on git submodule:
Update 2 Sept. 2023: I've been using git submodules for a couple years now. To learn the basic commands and things, see my section titled "Git submodules and Git LFS: how to clone this repo and all git submodules and git lfs files" in my eRCaGuy_dotfiles repo here. (Also, don't use git lfs. See my question here and answer here, both of which contain explanations and reasons why.)
Update 21 Sept. 2020: this article by Martin Owen in May 2016 ("Git Submodules vs Git Subtrees") contains a good comparison of git submodule vs git subtree, and generally favors git submodule. However, the author was not even aware of git subrepo at the time, and made no mention of it except when it was brought up in the comments.
git submodule seems to be the canonical, officially-supported tool built into git. Although it looks like it has a learning curve for sure, I plan on using it in my next project, now that I'm ready to open that project up and begin working on it again, and it depends on sub-git repos. I plan on beginning by learning about it here:
- A brief intro by Atlassian's Bitbucket: https://www.atlassian.com/git/tutorials/git-submodule
- The official
git submodule documentation here: https://git-scm.com/book/en/v2/Git-Tools-Submodules
See also:
- My answer on How to update all git submodules in a repo (two ways to do two very different things!)
Additional References:
- https://medium.com/@porteneuve/mastering-git-subtrees-943d29a798ec
- When to use git subtree?
- https://webmasters.stackexchange.com/questions/84378/how-can-i-create-a-git-repo-that-contains-several-other-git-repos
- Git treat nested git repos as regular file/folders
- Git: How to make outer repository and embedded repository work as common/standalone repository?
- https://www.atlassian.com/git/tutorials/git-subtree
Keywords: git add subrepo; git add sub repository; git add nested repository; git add .git folder and files
