Why is my Git Submodule HEAD detached from master?

Question

I am using Git submodules. After pulling changes from server, many times my submodule head gets detached from master branch.

Why does it happen?

I have to always do:

git branch
git checkout master

How can I make sure that my submodule is always pointing to master branch?

Repartition

Did you read this answer? #1778354 – Marder

@bitoiu I looked at subtree and Google Repo. I haven't got perfect solution yet :( – Repartition

my experience with gitsubmodules, in a CI environment is awful, maybe some other people have better experiences. – Guatemala

@JohnnyZ Thanks. I understood that submodule points to a commit and not head of the tree. But Why detached from branch. If you have one branch shouldn't it be attached to it by default – Repartition

Submodule is actually working pretty well, we have an enterprise environment with 16 submodules and 400 developers and multiple product branches running and it actually works pretty fine so keep up the good work – Centre

Don't be too quick to dismiss submodules just because you heard they're bad. They are a poor solution if you want continuous integration, but they are a near-perfect solution if you want to embed code from an external project and you explicitly manage all pulls. This is often the best practice if you are integrating with a non-forked module that's not controlled by your organization. The problem is that they are a tempting solution in all kinds of other situations where they don't work very well at all. The best advice is to read up on how they work and evaluate your scenario. – Jugendstil

@SarahG Thanks. We are still using submodules. I have one main project with 2 submodules. If I were to have 3 branches say Master, Prod and Dev then I would end up managing 9 branches which makes the workflow very complex. Right now everybody checks in the code to master so we are not having any issue. But I am trying to find a way so that other developers dont even need to know if there is any submodule in the solution. Just like SVN Externals. I haven't found such solution yet :( – Repartition

Answer 1

EDIT:

See @Simba Answer for valid solution

submodule.<name>.update is what you want to change, see the docs - default checkout
submodule.<name>.branch specify remote branch to be tracked - default master

---

OLD ANSWER:

Personally I hate answers here which direct to external links which may stop working over time and check my answer here (Unless question is duplicate) - directing to question which does cover subject between the lines of other subject, but overall equals: "I'm not answering, read the documentation."

So back to the question: Why does it happen?

Situation you described

After pulling changes from server, many times my submodule head gets detached from master branch.

This is a common case when one does not use submodules too often or has just started with submodules. I believe that I am correct in stating, that we all have been there at some point where our submodule's HEAD gets detached.

• Cause: Your submodule is not tracking correct branch (default master).
  Solution: Make sure your submodule is tracking the correct branch

$ cd <submodule-path>
# if the master branch already exists locally:
# (From git docs - branch)
# -u <upstream>
# --set-upstream-to=<upstream>
#    Set up <branchname>'s tracking information so <upstream>
#    is considered <branchname>'s upstream branch.
#    If no <branchname> is specified, then it defaults to the current branch.
$ git branch -u <origin>/<branch> <branch>
# else:
$ git checkout -b <branch> --track <origin>/<branch>

• Cause: Your parent repo is not configured to track submodules branch.
  Solution: Make your submodule track its remote branch by adding new submodules with the following two commands.
  • First you tell git to track your remote <branch>.
  • you tell git to perform rebase or merge instead of checkout
  • you tell git to update your submodule from remote.

    $ git submodule add -b <branch> <repository> [<submodule-path>]
    $ git config -f .gitmodules submodule.<submodule-path>.update rebase
    $ git submodule update --remote

• If you haven't added your existing submodule like this you can easily fix that:
  • First you want to make sure that your submodule has the branch checked out which you want to be tracked.

    $ cd <submodule-path>
    $ git checkout <branch>
    $ cd <parent-repo-path>
    # <submodule-path> is here path releative to parent repo root
    # without starting path separator
    $ git config -f .gitmodules submodule.<submodule-path>.branch <branch>
    $ git config -f .gitmodules submodule.<submodule-path>.update <rebase|merge>

In the common cases, you already have fixed by now your DETACHED HEAD since it was related to one of the configuration issues above.

fixing DETACHED HEAD when .update = checkout

$ cd <submodule-path> # and make modification to your submodule
$ git add .
$ git commit -m"Your modification" # Let's say you forgot to push it to remote.
$ cd <parent-repo-path>
$ git status # you will get
Your branch is up-to-date with '<origin>/<branch>'.
Changes not staged for commit:
    modified:   path/to/submodule (new commits)
# As normally you would commit new commit hash to your parent repo
$ git add -A
$ git commit -m"Updated submodule"
$ git push <origin> <branch>.
$ git status
Your branch is up-to-date with '<origin>/<branch>'.
nothing to commit, working directory clean
# If you now update your submodule
$ git submodule update --remote
Submodule path 'path/to/submodule': checked out 'commit-hash'
$ git status # will show again that (submodule has new commits)
$ cd <submodule-path>
$ git status
HEAD detached at <hash>
# as you see you are DETACHED and you are lucky if you found out now
# since at this point you just asked git to update your submodule
# from remote master which is 1 commit behind your local branch
# since you did not push you submodule chage commit to remote. 
# Here you can fix it simply by. (in submodules path)
$ git checkout <branch>
$ git push <origin>/<branch>
# which will fix the states for both submodule and parent since 
# you told already parent repo which is the submodules commit hash 
# to track so you don't see it anymore as untracked.

But if you managed to make some changes locally already for submodule and commited, pushed these to remote then when you executed 'git checkout ', Git notifies you:

$ git checkout <branch>
Warning: you are leaving 1 commit behind, not connected to any of your branches:
If you want to keep it by creating a new branch, this may be a good time to do so with:

The recommended option to create a temporary branch can be good, and then you can just merge these branches etc. However I personally would use just git cherry-pick <hash> in this case.

$ git cherry-pick <hash> # hash which git showed you related to DETACHED HEAD
# if you get 'error: could not apply...' run mergetool and fix conflicts
$ git mergetool
$ git status # since your modifications are staged just remove untracked junk files
$ rm -rf <untracked junk file(s)>
$ git commit # without arguments
# which should open for you commit message from DETACHED HEAD
# just save it or modify the message.
$ git push <origin> <branch>
$ cd <parent-repo-path>
$ git add -A # or just the unstaged submodule
$ git commit -m"Updated <submodule>"
$ git push <origin> <branch>

Although there are some more cases you can get your submodules into DETACHED HEAD state, I hope that you understand now a bit more how to debug your particular case.

Walkthrough

HEAD detached is the default behavior of git submodule update --remote. Please have a look at Simba's answer, I think that should be the right answer. – Stereo

Your instructions are too complicated. Please simplify. – Herrmann

Answer 2

From git submodule --help, HEAD detached is the default behavior of git submodule update --remote. This has nothing to do with which branch is being tracked in a submodule.

For anyone only want a solution, jump to the 2nd part directly.

Reason

We need to understand what a submodule is.

Submodule is a way to include another project into your current project. It's not really adding these files into the commit history of your main project, but by referring a snapshot (commit) of the submodule.

Quote from Starting with Submodules section in book Pro Git

Although submodule DbConnector is a subdirectory in your working directory, Git sees it as a submodule and doesn’t track its contents when you’re not in that directory. Instead, Git sees it as a particular commit from that repository.

Every commit of a repo is a snapshot/state of your code at that time. The state of submodule at that time has to be deterministic too. You can't say in this commit, I include the master (or another) branch of another repo. You have to specify the state of submodule by a commit id.

Including another repos as a submodule is basically

git clone uri://another-repo path/to/submodule
cd path/to/submodule
git checkout <commit-id>

# git submodule system will add the reference commit id but not the files

When anyone uses your repo with submodule, it will clone the submodule and checkout the specified commit as well.

And checking out a commit results HEAD detached. Why did my Git repo enter a detached HEAD state?

Solution

If you want the submodule merged with remote branch automatically, use --merge or --rebase.

man git-submodule

--merge

This option is only valid for the update command. Merge the commit recorded in the superproject into the current branch of the submodule. If this option is given, the submodule's HEAD will not be detached.

--rebase

Rebase the current branch onto the commit recorded in the superproject. If this option is given, the submodule's HEAD will not be detached.

If your submodule has already been detached, fix the detached state before using following 2 solutions.

cd path/to/submodule
# Assuming you're tracking the 'master' in the submodule
git checkout master

Solution 1: Use options in command line

# cd back to project root
git submodule update --remote --merge
# or
git submodule update --remote --rebase

Recommended alias:

git config alias.supdate 'submodule update --remote --merge'

# do submodule update with
git supdate

Solution 2: Add options into config file

Another solution is to change submodule update behavior in the gitmodule file by by setting submodule.$name.update to merge or rebase. It basically means you can do git submodule update --remote without passing --merge or --rebase explcitly, but read from config file automatically.

Here's an example about how to config the default update behavior of submodule update in .gitmodule.

[submodule "bash/plugins/dircolors-solarized"]
    path = bash/plugins/dircolors-solarized
    url = https://github.com/seebi/dircolors-solarized.git
    update = merge # <-- this is what you need to add

Or configure it through command line,

# replace $name with a real submodule name
git config -f .gitmodules submodule.$name.update merge

Others

Adding a branch option in .gitmodule is NOT related to the detached behavior of submodules at all. The old answer from mkungla is incorrect, or obsolete.

Let's make it clear there's no need to specify a branch to be tracked. origin/master is the default branch to be tracked.

--remote

Instead of using the superproject's recorded SHA-1 to update the submodule, use the status of the submodule's remote-tracking branch. The remote used is branch's remote (branch.<name>.remote), defaulting to origin. The remote branch used defaults to master.

References

• git submodule --help
• Submodules tutorial from book Pro Git

Adduction

I use git submodule update --remote --merge, and it pulls down the submodule in a detached state. Also tried --rebase with the same result. – Revolute

@JoeStrout If your submodule has been detached already, fix the detached state before you do an update with the above commands. cd into the submodule, checkout the submodule to a specific branch with, git checkout master. – Adduction

Or - if this is too much of a hassle for multiple (recursive) submodules - simply do git submodule foreach --recursive git checkout master. – Shaeffer

I only partially understand the "how git works" descriptions. TBH I not really am interested in understanding how git works, I just want to use it. Now I understand that I can fixed detached submodules with git submodule foreach --recursive git checkout master. But how can I prevent git from always detaching them? Setting config options for each submodule is not an option! – Substation

For me, running git submodule update --remote --merge did not leave the submodule in a detached HEAD state, but running git submodule update after editing my .gitmodule file as you indicated DID leave the submodule in a detached HEAD state. – Searcy

@Searcy I can't make sense of it. My submodules are on some branch not master, then after updating .gitmodules to have update = merge for each submodule and running git submodule update --remote, then it appears to merge in master to all of those submodules instead of merging from the remote that the submodules are checked out to. This all makes zero sense. – Alwyn

@Searcy It's .gitmodules not .gitmodule. And please provide related lines in your .gitmodules if you're really trying to get help. – Adduction

@Alwyn As what I wrote in the answer, git tracks "master" branch of your submodule by default. If your submodule are not on the master branch. You had to specify a branch in .gitmodules under the section of this submodule, with branch = branch-name. – Adduction

Does that make it alway use the latest commit of that branch when for example using git clone --recursive? – Alwyn

@Adduction my edits were made to .gitmodules, I mistakenly omitted the 's' in my comment. Here are the relevant lines: pastebin.com/vgJiiFeA . I think I may be getting this problem because I make changes to the submodule in-place. I haven't tried to clone the submodule's repo elsewhere, make changes, and update. I've also noticed that when I make changes, git status reports the submodule directory as changed and running git diff in the root module gives something like this: pastebin.com/VCBBZiWa . Hope that helps bring some clarity. – Searcy

oh, also my current workaround is to just commit the submodule directory in the root project every time I make changes in the submodule. So my workflow goes something like 1) edit submodule 2) commit changes to submodule repo 3) go back to root module and commit the new hash – Searcy

@Searcy You are right. It's caused by the change in the submodule. The default behavior repo is to track a specific commit of a submodule. Once a commit is checkout, the submodule is detached from the head. The answer here is to always track the head of a submodule when you do submodule update. And your workaround is the correct workflow in the case where submodule is modified directly. – Adduction

@Searcy The repo doesn't contain files from a submodule. It just record a URL to a specific commit of the submodule. Once you modified a submodule, you have to push the changes to a git server. Cause the repo checks the accessibility of URL to the submodule before you push the repo to a remote place. – Adduction

When I clone a repo and do git submodule update --init --recursive, the head is detached. This answer did not help fix it. I tried going in to submodule folder and doing git checkout master and coming out of subfolder and git status and git has not recorded any update so its not possible to update parent repo with details you have given. So any user who clones a parent repo and does init is still going to face the head detached issue. This answer is not fixing it or I am missing something. plz advice, thanks – Dichlorodiphenyltrichloroethane

Why doesn't this work - git submodule foreach --recursive 'git config -f .gitmodules submodule.$name.update merge' – Eventful

@Dichlorodiphenyltrichloroethane I had the same issue and ended up writing this macro. It inits modules and undetaches ones having update = merge (add it as an [alias] in ~/.gitconfig and call git init-all from topmost repo dir): init-all = "! f() { (cd ./$1; [ -z $(git branch --show-current) ] && echo Undetaching $1 && git checkout $2); for smdir in $(git config --file $1.gitmodules --get-regexp update | grep \".update merge\" | awk -F '.' '{ print $2 }'); do f $1$smdir/ $(git config --file $1.gitmodules --get submodule.$smdir.branch || echo master); done }; git submodule update --init --recursive; f" – Preoccupied

When I clone, the submodule is in a detached state. Doing git submodule update --remote --rebase doesn't change anything, it stays in a detached state. For which git version is this answer working?! Definitely not 2.31 – Ahron

Great post, but how do I replicate this behavior when doing git pull in the repo that contains the submodules? – Eczema

Answer 3

I got tired of it always detaching so iIjust use a shell script to build it out for all my modules. I assume all submodules are on master: here is the script:

#!/bin/bash
echo "Good Day Friend, building all submodules while checking out from MASTER branch."

git submodule update 
git submodule foreach git checkout master 
git submodule foreach git pull origin master 

execute it from your parent module

Disharoon

git submodule foreach git pull origin master -- this is what i was looking for.. kudos – Vandyke

simple and concise! Thanks! – Merriemerrielle

in my world we say "ADAMSIN" whenever we see such an answer like this! appreciated! – Gratitude

Dang... too bad some of my submodules use master, and some use main... – Eczema

Answer 4

Check out my answer here: Git submodules: Specify a branch/tag

If you want, you can add the "branch = master" line into your .gitmodules file manually. Read the link to see what I mean.

EDIT: To track an existing submodule project at a branch, follow VonC's instructions here instead:

Git submodules: Specify a branch/tag

Marder

Answers are suppose to be inline; IIRC linking to answers is a Stack Overflow faux pas. – Punt

@TonyTopper Even when just linking to another SO answer? IIRC only external links are frowned about as these may go away and then the link is dead and the answer is, well, useless. Yet there's no such danger with SO answers, they won't ever go away, unless SO goes away (and can be restored no matter what happens). Also he has answered the question, as branch = master" line into your .gitmodule is in fact the full answer, solved that problem for me. – Irita

Answer 5

The other way to make your submodule to check out the branch is to go the .gitmodules file in the root folder and add the field branch in the module configuration as following:

branch = <branch-name-you-want-module-to-checkout>

Phylactery

For me this does not work. I have correctly set branch = my_wanted_branch. But running git submodule update --remote it still checks out as detached head. – Benzidine

Do this, then cd sudmodule&git co thebranche & cd.., then git submodule update --remote and it works! – Sorcha

Isn't it that way '.gitmodules' is under active use (is being read) only while superproject is being cloned in submodules-recursive way or submodule being initialized? In other words your own repository you update the file in which is not always profiting from submodule config updates put to '.gitmodules'. In my understanding '.gitmodules' is a template for config created while repo is being cloned or so. – Titty

Answer 6

As other people have said, the reason this happens is that the parent repo only contains a reference to (the SHA1 of) a specific commit in the submodule – it doesn't know anything about branches. This is how it should work: the branch that was at that commit may have moved forward (or backwards), and if the parent repo had referenced the branch then it could easily break when that happens.

However, especially if you are actively developing in both the parent repo and the submodule, detached HEAD state can be confusing and potentially dangerous. If you make commits in the submodule while it's in detached HEAD state, these become dangling and you can easily lose your work. (Dangling commits can usually be rescued using git reflog, but it's much better to avoid them in the first place.)

If you're like me, then most of the time if there is a branch in the submodule that points to the commit being checked out, you would rather check out that branch than be in detached HEAD state at the same commit. You can do this by adding the following alias to your gitconfig file:

[alias]
    submodule-checkout-branch = "!f() { git submodule -q foreach 'branch=$(git branch --no-column --format=\"%(refname:short)\" --points-at `git rev-parse HEAD` | grep -v \"HEAD detached\" | head -1); if [[ ! -z $branch && -z `git symbolic-ref --short -q HEAD` ]]; then git checkout -q \"$branch\"; fi'; }; f"

Now, after doing git submodule update you just need to call git submodule-checkout-branch, and any submodule that is checked out at a commit which has a branch pointing to it will check out that branch. If you don't often have multiple local branches all pointing to the same commit, then this will usually do what you want; if not, then at least it will ensure that any commits you do make go onto an actual branch instead of being left dangling.

Furthermore, if you have set up git to automatically update submodules on checkout (using git config --global submodule.recurse true, see this answer), you can make a post-checkout hook that calls this alias automatically:

$ cat .git/hooks/post-checkout 
#!/bin/sh
git submodule-checkout-branch

Then you don't need to call either git submodule update or git submodule-checkout-branch, just doing git checkout will update all submodules to their respective commits and check out the corresponding branches (if they exist).

Pedicab

$ cat .git/hooks/post-checkout -- should that be vim etc ? – Laski

Sure, you can use vim, or whatever other text editor you prefer. That code snippet was just intended to show what the post-checkout hook file should look like. – Pedicab

Answer 7

The simplest solution is:

git clone --recursive [email protected]:name/repo.git

Then cd in the repo directory and:

git submodule update --init
git submodule foreach -q --recursive 'git checkout $(git config -f $toplevel/.gitmodules submodule.$name.branch || echo master)'
git config --global status.submoduleSummary true

Additional reading: Git submodules best practices.

Mertiemerton

The git config trick inside submodule-foreach is very good, thanks! – Garda

Answer 8

I am also still figuring out the internals of git, and have figured out this so far:

1. HEAD is a file in your .git/ directory that normally looks something like this:

% cat .git/HEAD
ref: refs/heads/master

2. refs/heads/master is itself a file that normally has the hash value of the latest commit:

% cat .git/refs/heads/master 
cbf01a8e629e8d884888f19ac203fa037acd901f

3. If you git checkout a remote branch that is ahead of your master, this may cause your HEAD file to be updated to contain the hash of the latest commit in the remote master:

% cat .git/HEAD
8e2c815f83231f85f067f19ed49723fd1dc023b7

This is called a detached HEAD. The remote master is ahead of your local master. When you do git submodule --remote myrepo to get the latest commit of your submodule, it will by default do a checkout, which will update HEAD. Since your current branch master is behind, HEAD becomes 'detached' from your current branch, so to speak.

Cutler

Answer 9

Since git 1.8.2 you can use -b branch_name option when adding git submodule. Or for existing submodule you can add info about tracking branch:

git submodule set-branch -b branch_name submodule_name

For more info you can refer this article.

Torin