Git commit to common submodule (master branch)

Question

I've two or more projects (let's call them ProjectFoo and ProjectBar) having some common code that I put in a submodule.

My understanding is that if I commit changes to a submodule from within ProjectFoo it'll be in a detached head that only all ProjectFoo clones can see:

(master) $ cd ProjectFooBarCommoneSubmodule/
(master) $ git commit -am "Common code fix."
(56f21fb0...) $ git push
Everything up-to-date

That's probably because the master branch hasn't changed. I could probably do something like git checkout master && git merge Everything up-to-date but that seem pretty ugly. May be a git reset --hard master would do the same but it seems even uglier.

How to have a common code shared by project, updated from within those projects using it? In other words, committing to that submodule should update all various repositories (repositories, not just clones) that use this same submodule.

---- EDIT ----

Visibly my checked-out repository was messed up and broken. It should have worked right from the start like that (on ProjectFoo in this example):

(master) $ cd ProjectFooBarCommoneSubmodule/
(master) $ git commit -am "Common code fix."
(master) $ git push
....
   fbfdd71..0acce63  master -> master
(master) $ cd ..
(master) $ git add ProjectFooBarCommoneSubmodule
(master) $ git commit -m "Submodule update."

Then to get that change from on other projects, like ProjectBar:

(master) $ cd ProjectFooBarCommoneSubmodule/
(master) $ git pull

Would update to the latest common code. A git checkout master may be required if it's on a detached head.

Lowrie

Answer 1

Short answer:

cd ProjectFooBarCommoneSubmodule
git checkout master
<Do your editing>
git commit --all -m "Lots of fixes"
git push submodule_origin master
cd ..

git add ProjectFooBarCommoneSubmodule
git commit -m "Bumped up the revision of ProjectFooBarCommoneSubmodule"
git push origin master

The longer one:

Git submodules are a dependency mechanism, where the main project (say A) defines a specified revision in a subproject (say B), which will be used in building project A. In order for the tool to be useful the behavior has to be predictable from A:s point of view. Dependencies cannot change, unless somebody decides to incorporate the change to project A. All kinds of nasty things could happen, were project B:s changes automatically imported, of which compile errors are probably the best ones, as A would notice the failures immediately. This is why B:s head is kept in detached state.

The state of B is store in A (check out git submodule status), and a revision change has to be done and committed in A, in order for it to have any effect. This is what happens in the example above, A changes the revision number stored in the repo, and bumps up the version to the latest one. The process will have to be repeated in the other main repo as well, so no automatic "use master" switch AFAIK.

BTW. The Git book chapter on submodules and the submodule man page contain lots of useful info about submodules, as normal usage and typical pitfalls as well. Worth checking out.

---

EDIT: I'll try to explain this better

I took the liberty to create example projects on my github account. The commits are meaningless and contain junk, but the setup should be fine. Please check it out to follow.

Both ProjectFoo and ProjectBar share the code in the common submodule.

ProjectFooBarCommoneSubmodule:master is 6850e4e4c1fac49de398

In ProjectFoo:

git submodule status

-6850e4e4c1fac49de39890703f21486ca04b87a0 common

In ProjectBar:

git submodule status

-6850e4e4c1fac49de39890703f21486ca04b87a0 common

So both point to the same revision, right? The trick here is to see, that ProjectFoo and ProjectBar point to the revision (6850e4e4c1fac49de39890703f21486ca04b87a0) not the branch (master), although they are the same thing. The first one is a detached head, and the other a named branch.

If you want to do some fixing on ProjectFooBarCommoneSubmodule, you can go to the subdir in e.g. ProjectFoo, and choose the branch instead of the revision:

git checkout master 
<Do your coding and pushing here>

Then go one directory up, and check git submodule status. It should tell you, that you are now out of sync. E.g

git submodule status

+e24bd2bf45d52171a63b67ac05cd4be0ac965f60 common (heads/master-1-ge24bd2b)

Now you can do a git add, to set the reference to this particular commit(ge24bd...), do a commit, and after this the submodule reference points to this revision, which also happens to be master on ProjectFooBarCommoneSubmodule.

Now you need to update the reference in ProjectBar as well. Go to ProjectBar/common, and do git fetch origin (this is a fast forward merge), do

git checkout master 
cd ..
git add common
git commit -m "Bumped up the revision"
git push origin master # to publish the revision bump to everybody else

So, as with any git repository, you don't need to work on a detached head. You can either work on master, or create a named branch. Either way, make sure that upstream contains the ProjectFooBarCommoneSubmodule changes, or you will break both ProjectFoo and ProjectBar, if they reference something that doesn't exist. Hope this explained it better

Charity

Thanks, I saw that chapter. Sadly I didn't find a clear answer my question. You say there is no automatic use master, so what if a common subproject, let's say a library, updates? Isn't a git submodule update going to update all projects relying on that library? I'd like to do the same but update that library directly from any of my main project. If that's not possible, what would be the alternative? – Lowrie

Git repositories do not contain any info about who has cloned them. This is why it's impossible to push out changes to all clones. Making things simple, I think you're expecting something like symlinks, so anything would be propagated immediately, while clones are more like "tar, gzip, copy, unzip". The key is probably to just publish the changed revision number in Foo and Bar. Anyone fetching changes will see the changed revision number the next time. Anything better than that, I would say, no. I would be glad to be proven wrong in this, though. – Charity

Sure clones have to make a update or pull manually. My question is, how to? Two projects, not clones, sharing a common code in a Git submodule. Is there any other choice than ProjectFoo commits submodule to detached head, then ProjectBar (NOT a clone) merges the changes (if that's even possible, I don't know where those changes are stored). – Lowrie

Seem much more clear. Could you just write some details about your section git checkout master <Do your coding and pushing here>? It's the part where after a git commit I'm in a detached head and cannot update "master" branch simply. – Lowrie

If you do a checkout master, you say that you want to work on this current branch. I.e. the next commit will be the head of master, when you commit, and it will also show up on upstream. Compare this to doing a checkout on a tag. In this case the tag will not move, if you change and commit, so it's a detached head. So the submodule detached head is basically a tag. It points to revision, and will move only if explicitly asked to. When you do the checkout master, you actually select a branch to work on. The commit in the main project again "tags" the revision to be used in the module. – Charity

I did this on ProjectFoo: cd ProjectFooBarCommoneSubmodule/, (fe2530...) git checkout master, (master) git commit -am "Common code fix.", (8395f8b...) cd .., (master) git add ProjectFooBarCommoneSubmodule && git commit -am "Update submodule." && git push. So far so good. Now on ProjectBar: git pull && git submodule update && cd ProjectFooBarCommoneSubmodule/ && git pull && git checkout master none of those commands did actually bring the changes I committed before (in ref 8395f8b). – Lowrie

Possible issues: 1) On ProjectBar you do pull and submodule update, but you have not yet updated ProjectBar or the submodule version, i.e. you get back the previous version (fe2530...) 2) Submodule is on a detached head, meaning that pull shouldn't be able to merge int the changes cleanly. The detached head should not change 3) Submodule master has not been changed during pull, because of 2. The submodule master still points to , without a warning. Submodule master still points to fe2530 4) A fetch could get all the changes, and then you could move master to remotes/master – Charity

It would most probably be beneficial to have a look at what's going on using gitk --all. The graph can be a great tool for problem solving in these cases. – Charity

I seems that even though I committed locally it didn't update the remote. Meaning even clones of ProjectFoo cannot correctly update the submodule. I'm trying to find out why... – Lowrie

I'll mark it a accepted as you actually helped me quiet a lot even though the problem wasn't visibly my commands but the messed up repository which didn't work as expected. – Lowrie

Its such a pain that the procedure is this long. btw, you are missing a git push from sub module after git commit --all -m "Lots of fixes". otherwise it wont be reflected in other projects – Amando

@Amando Yes, submodules often seem like a nice thing to go for, but usually end up causing more confusion, than truly solving the problem. And about your comment: True. Long time since I last had a look at this. You're right. Thanks! – Charity

If, like me, you've stupidly committed your changes in the submodule to detached head, then you can git checkout master and then git merge 61c9bf6 or whatever the ref of your commit is. – Desdamonna

Could you please clarify git push submodule_origin master, is that only if you want to keep your code merged with upstream changes or is there some other reason. – Park

Why submodule_origin? when I get into the submodule directory the remotes have automatically changed to point to submodule's repo. – Diffract

Answer 2

On the submodule: git push origin HEAD:master

Vindicable

Answer 3

I just do:

git submodule foreach git push -u origin master

Ronen

Answer 4

Notice:

git submodule foreach 'git commit -a'

will fail if one of the submodules containt no commit to do.

To get rid of this, you have to force the command result to 0.

git submodule foreach "git commit -am 'your comment' || echo ' '"

By using the echo piped you force the whole command to return with 0 and continue executing the commit command on the other submodules

Cacie

»By using the echo piped« -- The || is not a pipe. It does not do anything with the output of its left-hand side. || is a short-circuiting OR operation: The right-hand side is executed if and only if the left-hand side fails. Instead of || echo ' ' a clearer choice to hide the failure would be || true. – Riser

Answer 5

If you want to commit and push all submodules at once do:

git submodule foreach 'git commit -a' ;
git submodule foreach 'git push --all' ;
git commit -a && \
git push --all --recurse-submodules=on-demand

Audrieaudris

Answer 6

To merge changed from detached HEAD into master, run:

git rebase HEAD master

then checkout master (use -f for force):

git checkout master

If you've got multiple submodules to deal with, use: git submodule foreach, e.g.

git submodule foreach git pull origin master -r

Valorievalorization

Answer 7

If one don't have push rights to the submodule and/or just wants to commit submodule contents to the parent repository:

mv /path/to/submodule /path/to/submodule-old
git submodule deinit /path/to/submodule
git rm --cached -r /path/to/submodule
mv /path/to/submodule-old /path/to/submodule
git add /path/to/submodule

Polygamist