How to un-submodule a Git submodule?

Asked 18/11, 2009 at 22:23 Answered 25/11, 2021 at 19:44

490

What are the best practices for un-submoduling a Git submodule, bringing all the code back into the core repository?

Sf answered 18/11, 2009 at 22:23 Comment(4)

Note: with git1.8.3, you can now try a git submodule deinit, see my answer below – Transcurrent 23/4, 2013 at 6:13

I may misunderstand, but git submodule deinit seems to remove the code. – Befitting 21/9, 2013 at 12:50

Since git 1.8.5 (November 2013), a simple git submodule deinit asubmodule ; git rm asubmodule is enough, as illustrated in my answer below – Transcurrent 2/3, 2014 at 11:8

consider using git subtree – Doubtless 13/12, 2019 at 12:57

664

If all you want is to put your submodule code into the main repository, you just need to remove the submodule and re-add the files into the main repo:

git rm --cached submodule_path # delete reference to submodule HEAD (no trailing slash)
git rm .gitmodules             # if you have more than one submodules,
                                # you need to edit this file instead of deleting!
rm -rf submodule_path/.git     # make sure you have backup!!
git add submodule_path         # will add files instead of commit reference
git commit -m "remove submodule"

If you also want to preserve the history of the submodule, you can do a small trick: “merge” the submodule into the main repository, so that the result will be the same as it was before, except that the submodule files are now in the main repository.

In the main module you will need to do the following:

# Fetch the submodule commits into the main repository
git remote add submodule_origin git://url/to/submodule/origin
git fetch submodule_origin

# Start a fake merge (won't change any files, won't commit anything)
git merge -s ours --no-commit submodule_origin/master

# Do the same as in the first solution
git rm --cached submodule_path # delete reference to submodule HEAD
git rm .gitmodules             # if you have more than one submodules,
                                # you need to edit this file instead of deleting!
rm -rf submodule_path/.git     # make sure you have backup!!
git add submodule_path         # will add files instead of commit reference

# Commit and cleanup
git commit -m "removed submodule"
git remote rm submodule_origin

The resulting repository will look a bit weird: there will be more than one initial commit. But it won’t cause any problems for Git.

A big advantage of this second solution is that you can still run git blame or git log on the files which were originally in submodules. In fact, what happens here is just a renaming of many files inside one repository, and Git should automatically detect this. If you still have problems with git log, try some options (e.g., --follow, -M, -C) which do better rename and copy detection.

Alcestis answered 24/11, 2009 at 11:9 Comment(20)

I think I need to do your second method (history preserving) on some git repos I have. Could you please explain which part of the above commands causes the files from the submodule to end up in the subdirectory? Is it that you when you do the merge git brings in the file in the top level directory (with its history) but when you do the git add submodule_path it implicity does a git mv for every file? – Olimpiaolin 20/12, 2011 at 4:43

Basically, yes. The trick is that git does not store rename operations: instead, it detects them by looking at the parent commits. If there is a file content that was present in the previous commit, but with a different filename, it is considered a rename (or copy). In the steps above, git merge ensures that there will be a "previous commit" for every file (at one of the two "sides" of the merge). – Alcestis 26/12, 2011 at 13:44

And yes, since the files were in the root directory in the "submodule side" of the history, git will see a merge AND a bunch of renames in the commit. – Alcestis 26/12, 2011 at 14:11

Thanks gyim, I started a project where I thought it made sense to split things into a couple of repositories and link them back together with submodules. But now it seems over engineered and I want to combine them back together without losing my history. – Olimpiaolin 27/12, 2011 at 23:32

After following the instructions exactly, his does not work for me anymore in git 1.8. Git does not detect the files as moved, but says they are new! Any idea why? – Burroughs 27/11, 2012 at 16:52

a git alias for this would be of great help! – Rattler 2/9, 2013 at 19:29

@Burroughs I also had this problem. It can be fixed by, before following these steps, moving all of the files from your submodules repository into a directory structure with the same path as the repository that you are about to merge into: ie. if your submodule in the main repository is in foo/, in the submodule, perform mkdir foo && git mv !(foo) foo && git commit. – Ulrich 11/10, 2013 at 7:27

On Git 1.9.3 here, and sadly I can't get these instructions to work properly. I am just seeing one git log entry where in one commit i added every single file from the submodule. I think maybe it breaks somewhere around the git merge command. For me it just gave an immediate "merge failed" message, and the two git rm commands after it did not succeed either. – Iormina 23/1, 2015 at 2:55

This answer makes a lot more sense to me: https://mcmap.net/q/13849/-how-to-import-existing-git-repository-into-another – Iormina 23/1, 2015 at 3:5

About detecting renames: it is normal that the merged files appear as newly added. Git does not have a notion of moving/renaming, git mv is exactly the same as deleting a file and adding it under a new name. If you use git log --follow <filename>, it will detect renames and show the history properly. – Alcestis 14/4, 2015 at 14:49

By the way, this solution results in almost the same thing as https://mcmap.net/q/13849/-how-to-import-existing-git-repository-into-another (referenced by @StevenLu). The main differences are: 1. we need to "un-submodule" here as well in addition to pulling the files from the other repo; 2. we do the merging, file renaming and "un-submoduling" in a single commit (this is why we use git merge --no-commit) – Alcestis 14/4, 2015 at 15:2

While this does preserve history, I can't checkout or pull the SHA from a SHA with the submodule instead of the folder. git complains that The following untracked working tree files would be overwritten by merge and lists all files in my submodule. git reset --hard does seem to work, but normal checkouts and pulls do not work. Is this expected? – Railing 27/10, 2015 at 15:30

Note that the non-history preserving approach (at the top) in recent version of git does not remove .git/modules/packages/submodule_name -- this is where the submodule's git repository is stored (not in submodule_path/.git as it used to be in earlier versions of git). I just removed this folder manually. (This folder will be removed by the "git rm" step in the "Since git 1.8.5" answer.) – Nena 21/6, 2016 at 15:54

Needed to add --allow-unrelated-histories to force the merge at the fake merge as I was getting fatal: refusing to merge unrelated histories, more here: github.com/git/git/blob/master/Documentation/RelNotes/… – Predominate 15/8, 2016 at 13:23

I think git submodule deinit would be better than git rm .gitmodules since you may have other submodules. – Thermolysis 30/6, 2017 at 4:54

it does not preserve history, if I type after `git log subdirectory_where_submodule" it does not show all history. – Fenestrated 3/7, 2017 at 11:39

Same for me. This does not seem to be preserving the history – Talton 14/3, 2022 at 13:36

What if there are other submodules inside the submodule being removed? – Casals 6/6, 2022 at 10:12

...and you still want to preserve those children as submodules. – Casals 6/6, 2022 at 11:28

git log --name-status shows the paths of files in the preserved-, merged history remaining relative to their former submodule root dir. Would it be possible to rewrite that history making the paths reflect their files' new root? – Triatomic 19/9, 2023 at 8:58

104

I've created a script that will translate a submodule to a simple directory, while retaining all file history. It doesn't suffer from the git log --follow <file> issues that the other solutions suffer from. It's also a very easy one-line invocation that does all of the work for you. G'luck.

It builds on the excellent work by Lucas Jenß, described in his blog post "Integrating a submodule into the parent repository", but automates the entire process and cleans up a few other corner cases.

The latest code will be maintained with bugfixes on github at https://github.com/jeremysears/scripts/blob/master/bin/git-submodule-rewrite, but for the sake of proper stackoverflow answer protocol, I've included the solution in its entirety below.

Usage:

$ git-submodule-rewrite <submodule-name>

git-submodule-rewrite:

#!/usr/bin/env bash

# This script builds on the excellent work by Lucas Jenß, described in his blog
# post "Integrating a submodule into the parent repository", but automates the
# entire process and cleans up a few other corner cases.
# https://x3ro.de/2013/09/01/Integrating-a-submodule-into-the-parent-repository.html

function usage() {
  echo "Merge a submodule into a repo, retaining file history."
  echo "Usage: $0 <submodule-name>"
  echo ""
  echo "options:"
  echo "  -h, --help                Print this message"
  echo "  -v, --verbose             Display verbose output"
}

function abort {
    echo "$(tput setaf 1)$1$(tput sgr0)"
    exit 1
}

function request_confirmation {
    read -p "$(tput setaf 4)$1 (y/n) $(tput sgr0)"
    [ "$REPLY" == "y" ] || abort "Aborted!"
}

function warn() {
  cat << EOF
    This script will convert your "${sub}" git submodule into
    a simple subdirectory in the parent repository while retaining all
    contents and file history.

    The script will:
      * delete the ${sub} submodule configuration from .gitmodules and
        .git/config and commit it.
      * rewrite the entire history of the ${sub} submodule so that all
        paths are prefixed by ${path}.
        This ensures that git log will correctly follow the original file
        history.
      * merge the submodule into its parent repository and commit it.

    NOTE: This script might completely garble your repository, so PLEASE apply
    this only to a fresh clone of the repository where it does not matter if
    the repo is destroyed.  It would be wise to keep a backup clone of your
    repository, so that you can reconstitute it if need be.  You have been
    warned.  Use at your own risk.

EOF

  request_confirmation "Do you want to proceed?"
}

function git_version_lte() {
  OP_VERSION=$(printf "%03d%03d%03d%03d" $(echo "$1" | tr '.' '\n' | head -n 4))
  GIT_VERSION=$(git version)
  GIT_VERSION=$(printf "%03d%03d%03d%03d" $(echo "${GIT_VERSION#git version}" | tr '.' '\n' | head -n 4))
  echo -e "${GIT_VERSION}\n${OP_VERSION}" | sort | head -n1
  [ ${OP_VERSION} -le ${GIT_VERSION} ]
}

function main() {

  warn

  if [ "${verbose}" == "true" ]; then
    set -x
  fi

  # Remove submodule and commit
  git config -f .gitmodules --remove-section "submodule.${sub}"
  if git config -f .git/config --get "submodule.${sub}.url"; then
    git config -f .git/config --remove-section "submodule.${sub}"
  fi
  rm -rf "${path}"
  git add -A .
  git commit -m "Remove submodule ${sub}"
  rm -rf ".git/modules/${sub}"

  # Rewrite submodule history
  local tmpdir="$(mktemp -d -t submodule-rewrite-XXXXXX)"
  git clone "${url}" "${tmpdir}"
  pushd "${tmpdir}"
  local tab="$(printf '\t')"
  local filter="git ls-files -s | sed \"s/${tab}/${tab}${path}\//\" | GIT_INDEX_FILE=\${GIT_INDEX_FILE}.new git update-index --index-info && mv \${GIT_INDEX_FILE}.new \${GIT_INDEX_FILE}"
  git filter-branch --index-filter "${filter}" HEAD
  popd

  # Merge in rewritten submodule history
  git remote add "${sub}" "${tmpdir}"
  git fetch "${sub}"

  if git_version_lte 2.8.4
  then
    # Previous to git 2.9.0 the parameter would yield an error
    ALLOW_UNRELATED_HISTORIES=""
  else
    # From git 2.9.0 this parameter is required
    ALLOW_UNRELATED_HISTORIES="--allow-unrelated-histories"
  fi

  git merge -s ours --no-commit ${ALLOW_UNRELATED_HISTORIES} "${sub}/master"
  rm -rf tmpdir

  # Add submodule content
  git clone "${url}" "${path}"
  rm -rf "${path}/.git"
  git add "${path}"
  git commit -m "Merge submodule contents for ${sub}"
  git config -f .git/config --remove-section "remote.${sub}"

  set +x
  echo "$(tput setaf 2)Submodule merge complete. Push changes after review.$(tput sgr0)"
}

set -euo pipefail

declare verbose=false
while [ $# -gt 0 ]; do
    case "$1" in
        (-h|--help)
            usage
            exit 0
            ;;
        (-v|--verbose)
            verbose=true
            ;;
        (*)
            break
            ;;
    esac
    shift
done

declare sub="${1:-}"

if [ -z "${sub}" ]; then
  >&2 echo "Error: No submodule specified"
  usage
  exit 1
fi

shift

if [ -n "${1:-}" ]; then
  >&2 echo "Error: Unknown option: ${1:-}"
  usage
  exit 1
fi

if ! [ -d ".git" ]; then
  >&2 echo "Error: No git repository found.  Must be run from the root of a git repository"
  usage
  exit 1
fi

declare path="$(git config -f .gitmodules --get "submodule.${sub}.path")"
declare url="$(git config -f .gitmodules --get "submodule.${sub}.url")"

if [ -z "${path}" ]; then
  >&2 echo "Error: Submodule not found: ${sub}"
  usage
  exit 1
fi

if ! [ -d "${path}" ]; then
  >&2 echo "Error: Submodule path not found: ${path}"
  usage
  exit 1
fi

main

Indoiranian answered 22/4, 2017 at 0:52 Comment(16)

Does not work on Ubuntu 16.04. I sent a pull request to the Github repo. – Sunfast 3/5, 2017 at 11:9

Good catch, @qznc. This was tested on OSX. I'll happily merge that when it passes on both platforms. – Indoiranian 3/5, 2017 at 15:9

@Sunfast Ubuntu 16.04 support merged and answer updated. – Indoiranian 12/5, 2017 at 21:0

Thank you so much for this! I got it to work perfectly in Windows 10 by grabbing your latest script from GitHub and running it under WSL. – Bruce 3/9, 2017 at 2:11

Still works on macOS 10.12.6 with git version 2.10.1 (Apple Git-78), to check the name of the submodule, just type cat .gitmodules. – Plourde 3/9, 2017 at 11:11

This worked great for me but not on WSL - got an error suggesting a mix of forward- and back-slashes going into the reconstructed paths? So ran it on macOS instead and it worked perfectly. – Alviani 1/1, 2019 at 12:31

Failed on Cygwin with: fatal: Unable to create '/tmp/submodule-rewrite-9J3x5U/.git/index.lock': No such file or directory It worked however on CentOS 7 like a charm. Good job !!!! – Magnetron 2/7, 2019 at 17:40

Thanks for the awesome solution! I wish it included an explanation as to what'll happen once this is merged into master and someone pulls it. Will the submodule be properly replaced by git? – Om 7/7, 2019 at 15:7

Do all work without errors in Git Bash 2.20.1.1 on Windows 10 with latest version from github: curl https://raw.githubusercontent.com/jeremysears/scripts/master/bin/git-submodule-rewrite > git-submodule-rewrite.sh and ./git-submodule-rewrite.sh <submodule-name> – Eskridge 6/12, 2019 at 15:35

is it possible to do this with only one commit? right now there is one for removing submodule and the other for creating the folder. By the way the second commit is a merge commit and I could not rebase them. – Doubtless 7/12, 2019 at 13:23

In my experience with submodules I had a hard time doing anything related to removing an existing version of a submodule unless I did a full proper removal (even just renaming a submodule, actually) – Incurve 30/12, 2019 at 4:1

works great on mac os 10.14 mojave+latest git (2.24.1) – Incurve 30/12, 2019 at 6:8

@Indoiranian hi, quick question: do I have to run this script from master branch? – Substandard 18/4, 2023 at 10:6

I run the script from develop on my windows pc but it failed "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-branch: line 468: ../message: Permission denied" – Substandard 18/4, 2023 at 10:34

Do you know the reason why I get the following error: ´ Rewrite d791f0933e0f4a58ff8c2c06d17f69455a35a09e (2/221) (0 seconds passed, remaining 0 predicted) mv: cannot stat '/tmp/submodule-rewrite-bDhx95/.git-rewrite/t/../index.new': No such file or directory index filter failed: git ls-files -s | sed "s/ / mysubmodule\//" | GIT_INDEX_FILE=${GIT_INDEX_FILE}.new git update-index --index-info && mv ${GIT_INDEX_FILE}.new ${GIT_INDEX_FILE} ´ – Chantilly 9/12, 2023 at 13:36

After using the updated script from the github to ignore those above: I get a bunch following: ! [rejected] 1.0.0 -> 1.0.0 (would clobber existing tag) ! [rejected] 1.0.0-rc1 -> 1.0.0-rc1 (would clobber existing tag) ! [rejected] 1.0.0-rc2 -> 1.0.0-rc2 (would clobber existing tag) ! [rejected] 1.0.0-rc3 -> 1.0.0-rc3 (would clobber existing tag) ! [rejected] 1.0.1 -> 1.0.1 (would clobber existing tag) ! [rejected] 1.0.2 -> 1.0.2 (would clobber existing tag) – Chantilly 9/12, 2023 at 14:56

Since git 1.8.5 (Nov 2013) (without keeping the history of the submodule):

mv yoursubmodule yoursubmodule_tmp
git submodule deinit yourSubmodule
git rm yourSubmodule
mv yoursubmodule_tmp yoursubmodule
git add yoursubmodule

That will:

unregister and unload (ie delete the content of) the submodule (deinit, hence the mv first),
clean up the .gitmodules for you (rm),
and remove the special entry representing that submodule SHA1 in the index of the parent repo (rm).

Once the removal of the submodule is complete (deinit and git rm), you can rename the folder back to its original name and add it to the git repo as a regular folder.

Note: if the submodule was created by an old Git (< 1.8), you might need to remove the nested .git folder within the submodule itself, as commented by Simon East

If you need to keep the history of the submodule, see jsears's answer, which uses git filter-branch.

Transcurrent answered 23/4, 2013 at 6:13 Comment(42)

This actually does delete it from the working tree in 1.8.4 (my entire submodule dir was cleared). – Ulrich 11/10, 2013 at 7:7

@ChrisDown you mean, the deinit alone cleaned the working tree from your submodule? – Transcurrent 11/10, 2013 at 7:40

Yeah, it removes all the content in the submodule directory. – Ulrich 11/10, 2013 at 7:47

I can confirm it does clear the submodule in the working tree. Any way to keep it? – Beth 20/11, 2013 at 12:55

@Beth it (git submodule deinit) does clear it, but it doesn't remove the special entry from the index, does it? – Transcurrent 20/11, 2013 at 13:18

@VonC: no, and it still has to exist as an index entry since you deinit without remove? It still shows an entry in git ls-tree HEAD path/to/sub – Beth 20/11, 2013 at 13:22

@Beth just checkin. So the problem remains: how to do a git submodule deinit without having the working tree of that submodule disappear, right,. – Transcurrent 20/11, 2013 at 13:25

git config --remove-section submodule.name, as in https://mcmap.net/q/13850/-can-i-make-a-quot-deep-copy-quot-of-a-git-repository-with-submodules? – Beth 20/11, 2013 at 13:26

@Beth yes, that would mimic what git submodule deinit does (minus maybe the rm -Rf .git/modules/xxx), without the all "removing the working tree" part. – Transcurrent 20/11, 2013 at 13:28

@Transcurrent Does this maintain the history of all of the commits made to the submodule or does it just make a single commit of the files as-is with no history? – Wershba 4/4, 2014 at 23:46

@Wershba it does maintain the history of the files within the submodule. This only affects the parent repo. – Transcurrent 5/4, 2014 at 6:5

@Transcurrent So if the submodule has 5 commits to it, and I then use this method to move its files into the parent repo as the parent repo's files instead of as a submodule, the 5 commits history will be preserved for the files after they are in the parent repo? Thanks! – Wershba 6/4, 2014 at 21:51

That means to really unsubmodule a submodule, you will do (git 1.8.5 or 1.9, Q4 2013) (...) - But that is wrong. git submodule deinit SubModule does remove the files from the working tree. If you do git rm --cached SubModule your files are already gone from the working tree. – Daytime 22/1, 2015 at 14:1

@Transcurrent Well the goal is to un-submodule a submodule. This means to add the submodule files to the parent repository without the reference to a submodule. I haven't found an easy solution using git submodule deinit SubModule. When using this command the files are gone from disk. Or do I get something wrong? – Daytime 22/1, 2015 at 14:53

@and-dev I haven't tested it recently, but at the time, gone from the disk didn't mean gone from the index. – Transcurrent 22/1, 2015 at 14:56

@Transcurrent And which command should restore it from the index? git rm --cached yourSubmodule does only delete the last traces of yourSubModule being a submodule right? So what would be the complete setup to un-submodule a submodule while keeping the files on disk? (that need to be added as normal tracked files to the parent project) – Daytime 22/1, 2015 at 15:7

@and-dev simply rename the submodule folder before doing the git submodule deinit yourSubmodule && git rm --cached yourSubmodule. Then rename it back. – Transcurrent 22/1, 2015 at 15:21

This answer is terrifying. It doesn't clearly explain which process to use, or what to avoid. – Hoedown 26/5, 2015 at 6:19

@TimOgilvy I have simplified the answer. – Transcurrent 26/5, 2015 at 6:32

Given that the original question asks how to keep the files in the working tree, but convert the sub-module to integrated code, it's now apparent that this is both a) an incomplete answer, and b) dangerous. At least please add a warning, that this will remove the subodule files? – Hoedown 26/5, 2015 at 6:58

@TimOgilvy I have clarified the steps to preserve the content of the submodule. – Transcurrent 26/5, 2015 at 7:2

Sure. Why is everyone so obsessed with the word unload? It's somewhat jargony. What it does, in plain terms, is delete the submodule. I like the word delete - it's unambiguous, and given that SO is for people trying to learn stuff, I reckon being unambiguous is a nifty concept. – Hoedown 26/5, 2015 at 7:38

@TimOgilvy OK. I have added the word delete. – Transcurrent 26/5, 2015 at 7:40

Am i missing something? This leaves the submodule with a .git folder in it and adding it back into the repo just creates a subproject. Not sure how this is getting upvoted as it's not great advice and doesn't explain to people at all what you are doing (even if it was the right way to do this) just that they should bash these commands into the terminal. – Hickerson 13/3, 2016 at 19:53

@mschuett no, you are not missing anything: a submodule does not have a .git in it in the first place. If that was the case for you, it was a nested repo, not a submodule. That explains why this answer above would not apply in your case. For the difference between the two, see https://mcmap.net/q/13851/-difference-between-nested-git-repos-and-submodules. – Transcurrent 13/3, 2016 at 20:14

Not sure I follow. I have this listed in my .gitsubmoules file ` [submodule "phacility/phabricator"] path = phacility/phabricator url = github.com/phacility/phabricator.git ` and when I run git submodule init && git submodule update to pull them in they all contain a .git folder. Sorry for formatting. – Hickerson 13/3, 2016 at 20:24

@mschuett what version of git are you using? – Transcurrent 13/3, 2016 at 20:36

git version 1.9.1 which is the current stock version shipped on ubuntu. – Hickerson 13/3, 2016 at 20:38

@mschuett that would explain why, submodules have changed a lot since that old version. Try upgrading git first: askubuntu.com/a/579591/5470 – Transcurrent 13/3, 2016 at 20:40

Sounds good. Just wanted to make sure I wasn't doing something completely stupid. Although using stock ubuntu packages is in that realm. – Hickerson 13/3, 2016 at 20:41

I think this is missing a step. The resulting submodule folder still has a .git file there which prevents it from being committed to the parent repo. – Thermolysis 30/6, 2017 at 3:25

@SimonEast What Git version are you using? – Transcurrent 30/6, 2017 at 4:25

@Transcurrent I'm currently on 2.9.0.windows.1, however the submodules may have been created several years ago on a much earlier version of git, I'm not sure. I think the steps seem to work as long as I remove that file before doing the final add + commit. – Thermolysis 30/6, 2017 at 5:12

@SimonEast OK that would make sense. I will edit the steps shortly – Transcurrent 30/6, 2017 at 5:20

It does not preserve history. – Fenestrated 3/7, 2017 at 11:40

@Fenestrated Yes, that is on purpose. For preserving the history, see the answer above. – Transcurrent 3/7, 2017 at 11:44

Looks like stackoverflow reoder content based on user, because of no answer above yours. May be you mean answer with >300 votes? It is also wrong, it need at least call to git filter-branch to get right answer. But anyway, it is version control system, and answer that throw away history is very bad, it should at least mention damage of history with bold somehwere at the top. – Fenestrated 3/7, 2017 at 12:58

@Fenestrated Good point. I have added the warning (in bold), and added a link to the answer which uses git filter-branch and preserves the history. – Transcurrent 3/7, 2017 at 13:17

I had to commit after removing, remove .git, and commit again after adding - mv, deinit, git rm -r, commit, rm -rf <submod>_tmp/.git, mv, add, commit. – Ngocnguyen 9/9, 2019 at 18:39

@Ngocnguyen with which version of Git? – Transcurrent 9/9, 2019 at 19:0

Git 2.17.1 on Ubuntu 18.04.2 x64 (shell bash 4.4.20(1), in case that matters). – Ngocnguyen 9/9, 2019 at 19:10

@Ngocnguyen OK. I would be curious to know if this is necessary with Git 2.23. – Transcurrent 9/9, 2019 at 20:37

git rm --cached the_submodule_path
remove the submodule section from the .gitmodules file, or if it's the only submodule, remove the file.
do a commit "removed submodule xyz"
git add the_submodule_path
another commit "added codebase of xyz"

I didn't find any easier way yet. You can compress 3-5 into one step via git commit -a - matter of taste.

Cookson answered 18/11, 2009 at 23:58 Comment(4)

Shouldn't it be .gitmodules instead of .submodules? – Rok 18/2, 2015 at 3:32

It should be .gitmodules not .submodules – Unhurried 9/7, 2018 at 15:56

I had to remove the .git directory of the submodule before git add would work on the submodule folder – Brood 12/12, 2018 at 15:48

Seconding Carson Evans, You definitely have to remove the .git file at the root of the submodule. This should be step 2.5. – Pope 14/7, 2020 at 12:48

Lots of answers here but all of them seem to be overly complex and likely do not do what you want. I am sure most people want to keep their history.

For this example the main repo will be [email protected]:main/main.git and the submodule repo will be [email protected]:main/child.git. This assumes that the submodule is located in the root directory of the parent repo. Adjust the instructions as needed.

Start by cloning the parent repo and removing the old submodule.

git clone [email protected]:main/main.git
git submodule deinit child
git rm child
git add --all
git commit -m "remove child submodule"

Now we will add the child repos upstream to the main repo.

git remote add upstream [email protected]:main/child.git
git fetch upstream
git checkout -b merge-prep upstream/master

The next step assumes that you want to move the files on the merge-prep branch into the same location as the submodule was above although you can easily change the location by changing the file path.

mkdir child

move all folders and files except the .git folder into the child folder.

git add --all
git commit -m "merge prep"

Now you can simply merge your files back into the master branch.

git checkout master
git merge merge-prep # --allow-unrelated-histories merge-prep flag may be required

Look around and make sure everything looks good before running git push

The one thing you have to remember now is that git log does not by default follow moved files however by running git log --follow filename you can see the full history of your files.

Hickerson answered 13/3, 2016 at 19:37 Comment(7)

I got all the way to the final git merge merge-prep and received the error fatal: refusing to merge unrelated histories. Workaround is this: git merge --allow-unrelated-histories merge-prep. – Weidar 28/12, 2016 at 18:46

@Weidar thanks i added a little comment in case others run into this as well. – Hickerson 29/12, 2016 at 0:35

The best answer to keep history of submodule. Thank you @mschuett – Montserrat 7/11, 2017 at 10:48

In the example here, is there any way to fetch the upstream's files into the child directory, so you don't have to move them later? I have the same file names in a submodule and the main repo... so I just get a merge conflict since it's trying to merge the two files together. – Voyeur 1/1, 2018 at 23:24

Possibly but I don't know it offhand. I would personally just make a commit moving the files in the repo you are trying to move in so they reside in the directory you want before pulling them in. – Hickerson 3/1, 2018 at 16:54

hi. if I do this operation on the master branch I guess all the branches in the repo will reflect the new structire, correct? – Substandard 18/4, 2023 at 19:29

@Substandard if you rebase/merge master into them then yes. – Hickerson 18/4, 2023 at 22:33

It happened to us that we created 2 repositories for 2 projects that were so coupled that didn't make any sense to have them separated, so we merged them.

I'll show how to merge the master branches in each first and then I will explain how you can extend this to every branches you got, hope it helps you.

If you got the submodule working, and you want to convert it to a directory in place you can do:

git clone project_uri project_name

Here we do a clean clone to work. For this process you don't need to initialize or update the submodules, so just skip it.

cd project_name
vim .gitmodules

Edit .gitmodules with your favorite editor (or Vim) to remove the submodule you plan to replace. The lines you need to remove should look something like this:

[submodule "lib/asi-http-request"]
    path = lib/asi-http-request
    url = https://github.com/pokeb/asi-http-request.git

After saving the file,

git rm --cached directory_of_submodule
git commit -am "Removed submodule_name as submodule"
rm -rf directory_of_submodule

Here we remove the submodule relation completely so we can create bring the other repo to the project in-place.

git remote add -f submodule_origin submodule_uri
git fetch submodel_origin/master

Here we fetch the submodule repository to merge.

git merge -s ours --no-commit submodule_origin/master

Here we start a merge operation of the 2 repositories, but stop before commit.

git read-tree --prefix=directory_of_submodule/ -u submodule_origin/master

Here we send the content of master in the submodule to the directory where it was before prefixing a directory name

git commit -am "submodule_name is now part of main project"

Here we complete the procedure doing a commit of the changes in the merge.

After finishing this you can push, and start again with any other branch to merge, just checkout the branch in you repository that will receive the changes and change the branch you bringing in the merge and read-tree operations.

Eaten answered 17/1, 2012 at 21:1 Comment(6)

this doesn't seemed to have preserved history of the submodule files, I just see a single commit in the git log for the files added under directory_of_submodule – Ferdie 21/1, 2013 at 12:47

@Ferdie Sorry for the delay to reply. I just did the full procedure again (with a small fix). The procedure keeps the whole history, but it has a merge point, maybe that is why you don't find it. If you want to see the submodule history just do a "git log", lookup for the merge commit (in the example is the one with message "submodule_name is now part of main project"). It will have 2 parent commits (Merge: sdasda asdasd), git log the second commit and you got all your submodule/master history there. – Eaten 24/4, 2013 at 9:31

my memory is hazy now but I think I was able to get the history of the merged submodule files by doing git log original_path_of_file_in_submodule i.e. the path registered in the git repo for the file (which doesn't any longer exist on the filesystem) even though the submodule file now lives at submodule_path/new_path_of_file – Ferdie 24/4, 2013 at 10:0

This doesn't preserve the history very well, and also the paths are wrong. I feel that something like a tree-filter is needed but I'm out of my depth... trying what I've found here: x3ro.de/2013/09/01/… – Ferrocyanide 29/8, 2014 at 12:46

This answer is obsolete, https://mcmap.net/q/13595/-how-to-un-submodule-a-git-submodule (VonC answer) does the same but better – Beth 8/1, 2018 at 11:28

git log --follow $file_from_read_tree or with any other option will not work after this. I ended up doing the same as this with the exception of having a local branch of the submodule, renaming all of its files under the new "subtree path", doing the "ours" merge and then reading the tree with empty prefix and merging. This allows the git log --follow to find the commits. You need to remove/add any nested submodules temporarily. Quite the hassle! – Expostulatory 14/11, 2019 at 16:42

Here's a slightly improved version (IMHO) of @gyim's answer. He is doing a bunch of dangerous changes in the main working copy, where I think it's much easier to operate on separate clones and then merge them together at the end.

In a separate directory (to make mistakes easier to clean up and try again) check out both the top repo and the subrepo.

git clone ../main_repo main.tmp
git clone ../main_repo/sub_repo sub.tmp

First edit the subrepo to move all files into the desired subdirectory

cd sub.tmp
mkdir sub_repo_path
git mv `ls | grep -v sub_repo_path` sub_repo_path/
git commit -m "Moved entire subrepo into sub_repo_path"

Make a note of the HEAD

SUBREPO_HEAD=`git reflog | awk '{ print $1; exit; }'`

Now remove the subrepo from the main repo

cd ../main.tmp
rmdir sub_repo_path
vi .gitmodules  # remove config for submodule
git add -A
git commit -m "Removed submodule sub_repo_path in preparation for merge"

And finally, just merge them

git fetch ../sub.tmp
# remove --allow-unrelated-histories if using git older than 2.9.0
git merge --allow-unrelated-histories $SUBREPO_HEAD

And done! Safely and without any magic.

Hagiography answered 30/1, 2014 at 22:38 Comment(5)

... which answer is that? Might want to reference the username as well as the top answer can change over time. – Stale 22/10, 2019 at 10:5

@Stale answer updated. but the top answer is still the top answer by a 400 point lead ;-) – Hagiography 23/10, 2019 at 19:8

Does this work if the subrepo already contains a directory named subrepo with stuff in it? – Critical 17/12, 2019 at 23:4

In the last step I get following error: git merge $SUBREPO_HEAD fatal: refusing to merge unrelated histories Should I use git merge $SUBREPO_HEAD --allow-unrelated-histories in this case? Or should it work without and I made a mistake? – Itacolumite 15/1, 2020 at 10:54

@Itacolumite Yes, this is exactly the case of merging two histories that do not share any commits. The guard against unrelated histories seems to be new in git since I first wrote this; I'll update my answer. – Hagiography 16/1, 2020 at 17:6

The best answer to this I have found is here:

http://x3ro.de/2013/09/01/Integrating-a-submodule-into-the-parent-repository.html

This article explains the procedure very well.

Ferrocyanide answered 29/8, 2014 at 13:4 Comment(0)

For when

git rm [-r] --cached submodule_path

returns

fatal: pathspec 'emr/normalizers/' did not match any files

Context: I did rm -r .git* in my submodule folders before realizing that they needed to be de-submoduled in the main project to which I had just added them. I got the above error when de-submoduling some, but not all of them. Anyway, I fixed them by running, (after, of course, the rm -r .git*)

mv submodule_path submodule_path.temp
git add -A .
git commit -m "De-submodulization phase 1/2"
mv submodule_path.temp submodule_path
git add -A .
git commit -m "De-submodulization phase 2/2"

Note that this doesn't preserve history.

Cytochrome answered 20/9, 2012 at 20:37 Comment(0)

Based on VonC's answer, I have created a simple bash script that does this. The add at the end has to use wildcards otherwise it will undo the previous rm for the submodule itself. It's important to add the contents of the submodule directory, and not to name the directory itself in the add command.

In a file called git-integrate-submodule:

#!/usr/bin/env bash
mv "$1" "${1}_"
git submodule deinit "$1"
git rm "$1"
mv "${1}_" "$1"
git add "$1/**"

Daphie answered 18/10, 2016 at 18:25 Comment(0)

In main repo

git rm --cached [submodules_repo]
git commit -m "Submodules removed."
git push origin [master]

In submodules repo

rm -rf .git

Again main repo

git add [submodules_repo]
git add .
git commit -m "Submodules repo added into main."
git push origin [master]

Malaguena answered 25/11, 2021 at 19:44 Comment(2)

There are already several answers here, and most of them include lots of explanation. Does this answer introduce new information? Please read How to Answer. – Klarrisa 25/11, 2021 at 20:31

This answer worked for me. – Nakashima 19/1, 2022 at 16:3

I found it more convenient to (also?) fetch local commit data from the submodule, because otherwise I would loose them. (Could not push them as I have not access to that remote). So I added submodule/.git as remote_origin2, fetched it commits and merged from that branch. Not sure if I still need the submodule remote as origin, since I am not familiar enough with git yet.

Lesser answered 19/3, 2013 at 16:53 Comment(0)

Here's what I found best & simplest.

In submodule repo, from HEAD you want to merge into main repo:

git checkout -b "mergeMe"
mkdir "foo/bar/myLib/" (identical path as where you want the files on main repo)
git mv * "foo/bar/myLib/" (move all into path)
git commit -m "ready to merge into main"

Back in main repo after removing the submodule and clearing the path "foo/bar/myLib":

git merge --allow-unrelated-histories SubmoduleOriginRemote/mergeMe

boom done

histories preserved

no worries

Note this nearly identical to some other answers. But this assumes you own submodule repo. Also this makes it easy to get future upstream changes for submodule.

Strathspey answered 4/2, 2020 at 16:54 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags