recursively git pull for all of the git submodules simply

Question

My personal repository has some repositories as submodules. And the following command

$ git submodule foreach git pull origin master

was faced with the following result right after entering ruby repository because ruby repository seems that it does not have a master branch and "git pull" stopped.

Entering 'rails'
From git://github.com/rails/rails
 * branch            master     -> FETCH_HEAD
Already up-to-date.
Entering 'roo'
From git://github.com/hmcgowan/roo
 * branch            master     -> FETCH_HEAD
Already up-to-date.
Entering 'ruby'
fatal: Couldn't find remote ref master
Stopping at 'ruby'; script returned non-zero status.

So my question is what should I do to git pull for all of submodules only by git command? Should I do make a script to this? I hope just ONE command line provided from git will make this.

Causalgia

If you don't want git submodule foreach ...to stop when an error occurs in one of the sub-repos just add ||: to the command, i.e. git submodule foreach 'git pull origin master || :' – Furthermore

Answer 1

git submodules are typically in detached-HEAD states, and thus git pull on them can't figure out what you mean when it comes to the merge phase. If all you are trying to do is get the latest changes into the repository, try git submodule foreach git fetch instead. If you want to get each submodules master updated to its respective origin/master, then you can follow up with git submodule foreach git checkout master; git submodule foreach git merge origin/master.

Then, of course, you need to decide what version of each submodule you want your main repository to use (I would not recommend blindly going with origin/master all the time - it may be unstable - better to pick a known-good tag or something), check out those versions in the submodules, and follow up with the appropriate git adds and git commit in your main repository.

Drees

Answer 2

Just add || true to your submodule command:

git submodule foreach 'git commit -m "my commit message" || true'

Photocompose

Answer 3

Git 2.18 (Q2 2018) might avoid that error, by improving git submodule update, which influences git submodule pull.

"git submodule update" attempts two different kinds of "git fetch" against the upstream repository to grab a commit bound at the submodule's path, but it incorrectly gave up if the first kind (i.e. a normal fetch) failed, making the second "last resort" one (i.e. fetching an exact commit object by object name) ineffective.
This has been corrected.

See commit e30d833 (15 May 2018) by Stefan Beller (stefanbeller).
^{(Merged by Junio C Hamano -- gitster -- in commit a173ddd, 30 May 2018)}

git-submodule.sh: try harder to fetch a submodule

This is the logical continuum of fb43e31 (submodule: try harder to fetch needed sha1 by direct fetching sha1, 2016-02-23, Git 2.8.0) and fixes it as some assumptions were not correct.

The commit states:

If $sha1 was not part of the default fetch ... fail ourselves here assumes that the fetch_in_submodule only fails when the serverside does not support fetching by sha1.

There are other failures, why such a fetch may fail, such as

fatal: Couldn't find remote ref HEAD

which can happen if the remote side doesn't advertise HEAD and we do not have a local fetch refspec.

Not advertising HEAD is allowed by the protocol spec and would happen, if HEAD points at an unborn branch for example.

Not having a local fetch refspec can happen when submodules are fetched shallowly, as then git-clone doesn't setup a fetch refspec.

So do try even harder for a submodule by ignoring the exit code of the first fetch and rather relying on the following is_tip_reachable to see if we try fetching again.

---

Note: the error message on submodule fetch has been improved with Git 2.22 (Q2 2019)

See commit bd5e567 (13 Mar 2019) by Jonathan Tan (jhowtan).
^{(Merged by Junio C Hamano -- gitster -- in commit 32414ce, 09 Apr 2019)}

submodule: explain first attempt failure clearly

When cloning with --recurse-submodules a superproject with at least one submodule with HEAD pointing to an unborn branch, the clone goes something like this:

Cloning into 'test'...
<messages about cloning of superproject>
Submodule '<name>' (<uri>) registered for path '<submodule path>'
Cloning into '<submodule path>'...
fatal: Couldn't find remote ref HEAD
Unable to fetch in submodule path '<submodule path>'
<messages about fetching with SHA-1>
From <uri>
    * branch            <hash> -> FETCH_HEAD
Submodule path '<submodule path>': checked out '<hash>'

In other words, first, a fetch is done with no hash arguments (that is, a fetch of HEAD) resulting in a "Couldn't find remote ref HEAD" error; then, a fetch is done given a hash, which succeeds.

This commit improves the notice to be clearer that we are retrying the fetch, and that the previous messages (in particular, the fatal errors from fetch) do not necessarily indicate that the whole command fails.

---

Note that, with Git 2.34 (Q4 2021), reimplementation of parts of "git submodule"^(man) in C continues.

See commit c51f8f9 (24 Aug 2021) by Atharva Raykar (tfidfwastaken).
^{(Merged by Junio C Hamano -- gitster -- in commit e78db9d, 20 Sep 2021)}

submodule--helper: run update procedures from C

^{Mentored-by: Christian Couder
Mentored-by: Shourya Shukla
Signed-off-by: Atharva Raykar}

Add a new submodule--helper subcommand run-update-procedure that runs the update procedure if the SHA1 of the submodule does not match what the superproject expects.

That means the is_tip_reachable shell method mentioned above does no longer exist.

---

The same rewrite in C goes on with Git 2.36 (Q2 2022), and "git submodule update"^(man).

See commit c9d2562, commit 104744f, commit 97cb977, commit 29a5e9e, commit 1012a5c, commit e441966, commit 1a0b78c, commit f7bdb32 (04 Mar 2022) by Glen Choo (chooglen).
See commit 3ce52cb, commit 5312a85, commit a77c3fc (04 Mar 2022) by Atharva Raykar (tfidfwastaken).
See commit ed9c848, commit aca8568 (04 Mar 2022) by Ævar Arnfjörð Bjarmason (avar).
^{(Merged by Junio C Hamano -- gitster -- in commit 7649bfb, 23 Mar 2022)}

And the rewrite foes on with Git 2.39 (Q4 2022), which includes preparation to remove git-submodule.sh and replace it with a builtin.

See commit 69d9446, commit 1b6e200, commit 64f48ad, commit 82ff877, commit 46e87b5, commit d50d848, commit 435285b, commit 44874cb, commit cc74a4a (08 Nov 2022) by Ævar Arnfjörð Bjarmason (avar).
^{(Merged by Junio C Hamano -- gitster -- in commit 1107a39, 23 Nov 2022)}

Aleron