Creating multiple remote branches with git submodules

Question

I am working with git submodules for the first time. Struggling with understanding how to create branches across the board and adding them to all the remote repos.

Currently my file structure resembles the following:

-parent_repo
  |
  |_ submodule_1
  |_ submodule_2
  |_ submodule_3
  |_ submodule_4
  |_ submodule_5
  |_ submodule_6
  |_ submodule_7

If I create a branch on parent repo:

(master) $ git checkout -b feature/my_feature
(feature/my_feature) $ git commit -m "created my_feature"
(feature/my_feature) $ git push -u origin feature/my_feature

I would like to create a branch across all submodules including the parent. After which all branches are pushed remote to each submodules and their respected repos.

Tried the following steps:

$ git submodule foreach -b branch_name
$ git push --recurse-submodules=on-demand
$ git submodule foreach "(git checkout branch_name; git pull)&"

..just fails. First command not found.

..and if I do:

$ git config -f .gitmodules submodule.submodule_1.branch branch_name
$ git submodule update --remote

git returns:

fatal: Needed a single revision
Unable to find current origin/branch_name revision in submodule path 'submodule_1'

Greenfield

git submodule foreach -b branch_name should be git submodule foreach 'git checkout -b branch_name' – Skipjack

Git 2.36 (Q1 2022): git -c submodule.propagateBranches=true branch --recurse-submodules topic origin/main. See my answer below – Kidwell

Answer 1

See Submodule tips:

For example, let’s say we want to start a new feature or do a bugfix and we have work going on in several submodules.
We can easily stash all the work in all our submodules:

$ git submodule foreach 'git stash'
Entering 'CryptoLibrary'
No local changes to save
Entering 'DbConnector'
Saved working directory and index state WIP on stable: 82d2ad3 Merge from origin/stable
HEAD is now at 82d2ad3 Merge from origin/stable

Then we can create a new branch and switch to it in all our submodules:

$ git submodule foreach 'git checkout -b featureA'
Entering 'CryptoLibrary'
Switched to a new branch 'featureA'
Entering 'DbConnector'
Switched to a new branch 'featureA'

Then you still need to create the same branch in the parent repo, add, commit and push, as all submodule repos will have changed.

But remember: those are different branches (even if they have the same name), each one specific to its own repo (either the parent repo, or a submodule repo).

Kidwell

Once I create and checkout a branch as you mentioned above. Which works. Are you familiar with being able to push -u origin featureA across all repos without having to go into each one and creating a remote branch. Something like $ git submodule foreach 'git push -u origin featureA'. Since submodules are created using http reference to a repo and not ssh, I am prompted for credentials. – Greenfield

@Greenfield you can try that in one submodule, and cache those credentials once and for all: see the beginning of https://mcmap.net/q/13472/-how-to-use-git-with-gnome-keyring-integration. – Kidwell

Answer 2

I would like to create a branch across all submodules including the parent

This will be officially supported (in an experimental mode at first) with Git 2.36 (Q2 2022).

"git branch"^(man) learned the --recurse-submodules option.

See commit 09e0be1 (31 Jan 2022) by Junio C Hamano (gitster).
See commit 679e369, commit 961b130, commit 6e0a2ca, commit 3f3e760, commit bc0893c, commit e89f151 (28 Jan 2022) by Glen Choo (chooglen).
^{(Merged by Junio C Hamano -- gitster -- in commit 5cc9522, 18 Feb 2022)}

branch: add --recurse-submodules option for branch creation

^{Helped-by: Jonathan Tan
Signed-off-by: Glen Choo
Reviewed-by: Jonathan Tan}

To improve the submodules UX, we would like to teach Git to handle branches in submodules.
Start this process by teaching "git branch"^(man) the --recurse-submodules option so that git branch --recurse-submodules^(man) topic will create the topic branch in the superproject and its submodules.

Although this commit does not introduce breaking changes, it does not work well with existing --recurse-submodules commands becausegit branch --recurse-submodules" writes to the submodule ref store, but most commands only consider the superproject gitlink and ignore the submodule ref store.

For example, "git checkout --recurse-submodules"^(man) will check out the commits in the superproject gitlinks (and put the submodules in detached HEAD) instead of checking out the submodule branches.

Because of this, this commit introduces a new configuration value, submodule.propagateBranches.
The plan is for Git commands to prioritize submodule ref store information over superproject gitlinks if this value is true.

Because "git branch --recurse-submodules" writes to submodule ref stores, for the sake of clarity, it will not function unless this configuration value is set.

This commit also includes changes that support working with submodules from a superproject commit because "branch --recurse-submodules" (and future commands) need to read .gitmodules and gitlinks from the superproject commit, but submodules are typically read from the filesystem's .gitmodules and the index's gitlinks.

These changes are:

• add a submodules_of_tree() helper that gives the relevant information of an in-tree submodule (e.g. path and oid) and initializes the repository
• add is_tree_submodule_active() by adding a treeish_name parameter to is_submodule_active()
• add the "submoduleNotUpdated" advice to advise users to update the submodules in their trees

Incidentally, fix an incorrect usage string that combined the 'list' usage of git branch (-l) with the 'create' usage; this string has been incorrect since its inception, a8dfd5e ("Make builtin-branch.c use parse_options.", 2007-10-07, Git v1.5.4-rc0 -- merge).

git config now includes in its man page:

submodulesNotUpdated

Advice shown when a user runs a submodule command that fails because git submodule update --init was not run.

git config now includes in its man page:

submodule.recurse:

A boolean indicating if commands should enable the --recurse-submodules option by default.
Defaults to false.

When set to true, it can be deactivated via the --no-recurse-submodules option. Note that some Git commands lacking this option may call some of the above commands affected by submodule.recurse; for instance git remote update will call git fetch but does not have a --no-recurse-submodules option. For these commands a workaround is to temporarily change the configuration value by using git -c submodule.recurse=0.

The following list shows the commands that accept --recurse-submodules and whether they are supported by this setting.

• checkout, fetch, grep, pull, push, read-tree, reset, restore and switch are always supported.
• clone and ls-files are not supported.
• branch is supported only if submodule.propagateBranches is enabled

submodule.propagateBranches

[EXPERIMENTAL] A boolean that enables branching support when using --recurse-submodules or submodule.recurse=true. Enabling this will allow certain commands to accept --recurse-submodules and certain commands that already accept --recurse-submodules will now consider branches.

git branch now includes in its man page:

'git branch' [--track[=(direct|inherit)] | --no-track] [-f]
[--recurse-submodules] <branchname> [<start-point>]

git branch now includes in its man page:

--recurse-submodules

THIS OPTION IS EXPERIMENTAL! Causes the current command to recurse into submodules if submodule.propagateBranches is enabled. See submodule.propagateBranches in git config.
Currently, only branch creation is supported.

When used in branch creation, a new branch <branchname> will be created in the superproject and all of the submodules in the superproject's .

In submodules, the branch will point to the submodule commit in the superproject's <start-point> but the branch's tracking information will be set up based on the submodule's branches and remotes e.g. git branch --recurse-submodules topic origin/main will create the submodule branch "topic" that points to the submodule commit in the superproject's "origin/main", but tracks the submodule's "origin/main".

---

With Git 2.40 (Q1 2023), the advice message given when "git branch --recurse-submodules"^(man) fails is improved.

See commit 97cf0c7 (16 Jan 2023) by Philippe Blain (phil-blain).
^{(Merged by Junio C Hamano -- gitster -- in commit 7d4d34f, 27 Jan 2023)}

branch: improve advice when --recurse-submodules fails

^{Signed-off-by: Philippe Blain
Reviewed-by: Glen Choo}

'git branch --recurse-submodules'^(man) start from-here fails if any submodule present in 'from-here' is not yet cloned (under submodule.propagateBranches=true).
We then give this advice:

You may try updating the submodules using 'git checkout from-here && git submodule update --init'

If 'submodule.recurse' is set, 'git checkout'^(man) from-here will also fail since it will try to recursively checkout the submodules.

Improve the advice by adding '--no-recurse-submodules' to the checkout command.

You may try updating the submodules using 'git checkout --no-recurse-submodules %s && git submodule update --init

Kidwell