Git submodule head 'reference is not a tree' error

Question

I have a project with a submodule that is pointing to an invalid commit: the submodule commit remained local and when I try to fetch it from another repo I get:

$ git submodule update
fatal: reference is not a tree: 2d7cfbd09fc96c04c4c41148d44ed7778add6b43
Unable to checkout '2d7cfbd09fc96c04c4c41148d44ed7778add6b43' in submodule path 'mysubmodule'

I know what the submodule HEAD should be, is there any way I can change this locally, without pushing from the repo that does have commit 2d7cfbd09fc96c04c4c41148d44ed7778add6b43 ?

I'm not sure if I'm being clear... here's a similar situation I found.

Horowitz

"fatal: reference is not a tree" in reference to submodules appears to generally mean the submodule commit that the parent repo expects has not yet been pushed, or is screwed up in some other way. For us this confusing error message was resolved by just pushing a submodule someone forgot to push. – Bergamo

@ChrisMoschini - I just had that issue, and that was my "solution", I pushed and pull the main repo., but I forgot to push my last commit to the submodule's repo. Thanks! – Slugabed

Maybe you forgot to push the latest submodule commits – Petes

Answer 1

Assuming the submodule's repository does contain a commit you want to use (unlike the commit that is referenced from the current state of the super-project), there are two ways to do it.

The first requires you to already know the commit from the submodule that you want to use. It works from the “inside, out” by directly adjusting the submodule then updating the super-project. The second works from the “outside, in” by finding the super-projects commit that modified the submodule and then resetting the super-project's index to refer to a different submodule commit.

Inside, Out

If you already know which commit you to want the submodule to use, cd to the submodule, check out the commit you want, then git add and git commit it back in the super-project.

Example:

$ git submodule update
fatal: reference is not a tree: e47c0a16d5909d8cb3db47c81896b8b885ae1556
Unable to checkout 'e47c0a16d5909d8cb3db47c81896b8b885ae1556' in submodule path 'sub'

Oops, someone made a super-project commit that refers to an unpublished commit in the submodule sub. Somehow, we already know that we want the submodule to be at commit 5d5a3ee314476701a20f2c6ec4a53f88d651df6c. Go there and check it out directly.

Checkout in the Submodule

$ cd sub
$ git checkout 5d5a3ee314476701a20f2c6ec4a53f88d651df6c
Note: moving to '5d5a3ee314476701a20f2c6ec4a53f88d651df6c' which isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
  git checkout -b <new_branch_name>
HEAD is now at 5d5a3ee... quux
$ cd ..

Since we are checking out a commit, this produces a detached HEAD in the submodule. If you want to make sure that the submodule is using a branch, then use git checkout -b newbranch <commit> to create and checkout a branch at the commit or checkout the branch that you want (e.g. one with the desired commit at the tip).

Update the Super-project

Checkout in the submodule is reflected in the super-project as a change to the working tree. So we need to stage the change in the super-project's index and verify the results.

$ git add sub

Check the Results

$ git submodule update
$ git diff
$ git diff --cached
diff --git c/sub i/sub
index e47c0a1..5d5a3ee 160000
--- c/sub
+++ i/sub
@@ -1 +1 @@
-Subproject commit e47c0a16d5909d8cb3db47c81896b8b885ae1556
+Subproject commit 5d5a3ee314476701a20f2c6ec4a53f88d651df6c

The submodule update was silent because the submodule is already at the specified commit. The first diff shows that the index and work tree are the same. The third diff shows that the only staged change is moving the sub submodule to a different commit.

Commit

git commit

This commits the fixed-up submodule entry.

---

Outside, In

If you are not sure which commit you should use from the submodule, you can look at the history in the superproject to guide you. You can also manage the reset directly from the super-project.

$ git submodule update
fatal: reference is not a tree: e47c0a16d5909d8cb3db47c81896b8b885ae1556
Unable to checkout 'e47c0a16d5909d8cb3db47c81896b8b885ae1556' in submodule path 'sub'

This is the same situation as above. But this time we will focus on fixing it from the super-project instead of dipping it into the submodule.

Find the Super-project's Errant Commit

$ git log --oneline -p -- sub
ce5d37c local change in sub
diff --git a/sub b/sub
index 5d5a3ee..e47c0a1 160000
--- a/sub
+++ b/sub
@@ -1 +1 @@
-Subproject commit 5d5a3ee314476701a20f2c6ec4a53f88d651df6c
+Subproject commit e47c0a16d5909d8cb3db47c81896b8b885ae1556
bca4663 added sub
diff --git a/sub b/sub
new file mode 160000
index 0000000..5d5a3ee
--- /dev/null
+++ b/sub
@@ -0,0 +1 @@
+Subproject commit 5d5a3ee314476701a20f2c6ec4a53f88d651df6c

OK, it looks like it went bad in ce5d37c, so we will restore the submodule from its parent (ce5d37c~).

Alternatively, you can take the submodule's commit from the patch text (5d5a3ee314476701a20f2c6ec4a53f88d651df6c) and use the above “inside, out” process instead.

Checkout in the Super-project

$ git checkout ce5d37c~ -- sub

This resets the submodule entry for sub to what it was at commit ce5d37c~ in the super-project.

Update the Submodule

$ git submodule update
Submodule path 'sub': checked out '5d5a3ee314476701a20f2c6ec4a53f88d651df6c'

The submodule update went OK (it indicates a detached HEAD).

Check the Results

$ git diff ce5d37c~ -- sub
$ git diff
$ git diff --cached
diff --git c/sub i/sub
index e47c0a1..5d5a3ee 160000
--- c/sub
+++ i/sub
@@ -1 +1 @@
-Subproject commit e47c0a16d5909d8cb3db47c81896b8b885ae1556
+Subproject commit 5d5a3ee314476701a20f2c6ec4a53f88d651df6c

The first diff shows that sub is now the same in ce5d37c~. The second diff shows that the index and work tree are the same. The third diff shows the only staged change is moving the sub submodule to a different commit.

Commit

git commit

This commits the fixed-up submodule entry.

Flatter

In the "Outside, In" approach, could you elucidate on why "it looks like it went bad in ce5d37c?" What fingers that one as the bad commit? – Medullary

@Garrett: The assumption is e47c0a is a commit that does not exist in the local repository for sub, yet the super-project’s sub points to that commit. This might have happened because someone else created e47c0a in their copy of sub, updated their super-project to point to that commit and pushed the super-project without pushing e47c0a to the central/shared repository for sub. When we pull from the central/shared super-project we get a commit that points sub to e47c0a, but we can not “see” that commit. ce5d37c is suspect because, based on the diff, it introduced e47c0a. – Flatter

It is still left rather vague where is the specific hash of the sub kept in the parent repo which has it as a submodule, and whether or not it can be manipulated directly to the current HEAD of sub directly, without relying on an older state of the parent repo, which may not always help. – Oates

@matanster: It is stored in Git's object database directly – Blader

Answer 2

try this:

git submodule sync
git submodule update

Stpierre

Not for me unfortunately, one of our submodules was targeted by the main git repository with an add command, now having trouble undoing it – Bergh

Worked for me too. Would love to know why though. – Astrobiology

Turns out that doing a git submodule sync is necessary in scenarios where the URL of the remote for a given submodule has changed. In our case we had added our submodule from a public repo and then changed the URL to a private fork - and got ourselves into this particular pickle. – Cheyenne

For example: I had a repo (A) set up with a submodule pointing to my github repo (B). I created a branch in the repo A because I wanted to point B at someone else's github repo. After a bit of struggling with that and committing the branch, I switched my repo A back to master and had this problem with repo B. @Lonre Wang's solution fixed it. – Word

Assuming nobody REALLY screwed up (in which case you'd need Chris Johnsen excellent answer) the answer by Lonre Wang should fix the problem,... UNLESS your submodules have submodules of their own (and the problem is inside a submodule). In that case you need to cd into the submodule that has the submodule with the problem and execute the above commands. Note that update has a --recursive option (git submodule update --recursive), but sync doesn't; you really have to manually run 'git submodule sync' inside the submodule that has the problematic sub(sub)module. This was my problem ;). – Denten

Answer 3

This error can mean that a commit is missing in the submodule. That is, the repository (A) has a submodule (B). A wants to load B so that it is pointing to a certain commit (in B). If that commit is somehow missing, you'll get that error. Once possible cause: the reference to the commit was pushed in A, but the actual commit was not pushed from B. So I'd start there.

Less likely, there's a permissions problem, and the commit cannot be pulled (possible if you're using git+ssh).

Make sure the submodule paths look ok in .git/config and .gitmodules.

One last thing to try - inside the submodule directory: git reset HEAD --hard

Donegan

I already explained that in the question... the question itself was how to solve it. And it has already been successfully answered almost two years ago... Permissions have nothing to do with this. – Horowitz

You stated it, you certainly didn't explain it. – Donegan

this helped me out: "inside the submodule directory: git reset HEAD --hard" – Middleton

the "git reset HEAD --hard" helped me too... nothing else worked. I tried the previous solutions too, no dice. Thanks! – Beebe

Answer 4

Your branch may not be up to date, a simple solution but try git fetch

Christianly

This is what solved it for me. The commit was visible in the submodule repo, I just didn't know about it locally. – Seaboard

Answer 5

Possible cause

This can happens when:

1. Submodule(s) have been edited in place
2. Submodule(s) committed, which updates the hash of the submodule being pointed to
3. Submodule(s) not pushed.

e.g. something like this happened:

$ cd submodule
$ emacs my_source_file  # edit some file(s)
$ git commit -am "Making some changes but will forget to push!"

Should have submodule pushed at this point.

$ cd .. # back to parent repository
$ git commit -am "updates to parent repository"
$ git push origin master

As a result, the missing commits could not possibly be found by the remote user because they are still on the local disk.

Solution

Informa the person who modified the submodule to push, i.e.

$ cd submodule
$ git push

Stob

Answer 6

I got this error when I did:

$ git submodule update --init --depth 1

but the commit in the parent project was pointing at an earlier commit.

Deleting the submodule folder and running:

$ git submodule update --init

did NOT solve the problem. I deleted the repo and tried again without the depth flag and it worked.

This error happens in Ubuntu 16.04 git 2.7.4, but not on Ubuntu 18.04 git 2.17.

@pavan kumar notes in comment:

I just increased the depth count to include the old commit and it worked.

Baugher

my team has since abandoned submodules in our code way too much hassle lol – Baugher

what was your alternative? – Kinfolk

@nuzzolilo we added username/repo#sha to package.json, a much more flexible option is to organize your system with a set of docker containers – Baugher

This is so annoying. --depth=1 saves so much bandwidth when I don't need the repo history. If anyone ever finds or knows why this is happening I'd love to know. – Connelley

@Connelley Although I can't explain why, I've written a cmake helper which helps mitigate the problem here: github.com/LMMS/lmms/blob/…. It uses a deinit approach which fixes the problem the majority of the time. When bundled with a build system, the end-user can just let the build system fetch the submodules and ditch the broken recursive command altogether. There are still scenarios where this breaks, such as the submodule has done a force push and wiped out the commit altogether. – Jeffery

THIS WORKED: Same happened for me with depth=1. Worked well when on master branch, but the issue occurred when switched to an old release branch. When checked, the submodule commit referenced in release branch was older. I just increased the depth count to include the old commit and it worked. – Incorporated

Answer 7

This may also happen when you have a submodule pointing to a repository that was rebased and the given commit is "gone". While the commit may still be in the remote repository, it is not in a branch. If you can't create a new branch (e.g. not your repository), you're stuck with having to update the super project to point to a new commit. Alternatively you can push one of your copies of the submodules elsewhere and then update the super-project to point to that repository instead.

Haleyhalf

Answer 8

This answer is for users of SourceTree with limited terminal git experience.

Open the problematic submodule from within the Git project (super-project).

Fetch and ensure 'Fetch all tags' is checked.

Rebase pull your Git project.

This will solve the 'reference is not a tree' problem 9 out of ten times. That 1 time it won't, is a terminal fix as described by the top answer.

Obau

Answer 9

Your submodule history is safely preserved in the submodule git anyway.

So, why not just delete the submodule and add it again?

Otherwise, did you try manually editing the HEAD or the refs/master/head within the submodule .git

Heraldry

This won't work, because somewhere there's a reference to 2d7cfbd09fc96c04c4c41148d44ed7778add6b43 which is only in the local repo somewhere else, but not published – Horowitz

Answer 10

Just to be sure, try updating your git binaries.

GitHub for Windows has the version git version 1.8.4.msysgit.0 which in my case was the problem. Updating solved it.

Kauai

Answer 11

In my case, none of answer above solve the problem even thoungh they are good answers. So I post my solution (in my case there are two git clients, client A and B):

1. go to submodule's dir:

   cd sub
   

2. checkout to master:

   git checkout master
   

3. rebase to a commit code which both client can see

4. go back to the parent's dir:

5. commit to master

6. change to the other client， do rebase again.

7. finally it works fine now! Maybe lose a couple of commits but it works.

8. FYI, don't try to remove your submodule, it will remain .git/modules there and cannot readd this submodule again, unless reactive local one.

Prong

Answer 12

To sync the git repo with the submodule's head, in case that is really what you want, I found that removing the submodule and then readding it avoids the tinkering with the history. Unfortunately removing a submodule requires hacking rather than being a single git command, but doable.

Steps I followed to remove the submodule, inspired by https://gist.github.com/kyleturner/1563153:

1. Run git rm --cached
2. Delete the relevant lines from the .gitmodules file.
3. Delete the relevant section from .git/config.
4. Delete the now untracked submodule files.
5. Remove directory .git/modules/

Again, this can be useful if all you want is to point at the submodule's head again, and you haven't complicated things by needing to keep the local copy of the submodule intact. It assumes you have the submodule "right" as its own repo, wherever the origin of it is, and you just want to get back to properly including it as a submodule.

Note: always make a full copy of your project before engaging in these kinds of manipulation or any git command beyond simple commit or push. I'd advise that with all other answers as well, and as a general git guideline.

Oates

Answer 13

Just stumbled upon this problem, and none of these solutions worked for me. What turned out to be the solution for my issue is actually much simpler: upgrade Git. Mine was 1.7.1, and after I upgraded it to 2.16.1 (latest), the problem went away without a trace! Guess I'm leaving it here, hope it helps someone.

Jorgan

Answer 14

For those who see this on CI side: probably you reference a commit in the non-merged PR and then merge it with squash

Beedon