Git: maintaining many topic branches on a frequently-moving base

Question

In my day-to-day git workflow, I have many topic branches, like so:

              o--o--o (t2)
             /
         o--o (t1)
        /
 o--o--o (master)
        \
         o--o--o (t3)

When I pull from upstream,

              o--o--o (t2)
             /
         o--o (t1)
        /
 o--o--o--n--n--n (master)
        \
         o--o--o (t3)

I want to rebase all my topic branches on top of the new master:

                        o'--o'--o' (t2)
                       /
                  o'--o' (t1)
                 /
 o--o--o--n--n--n (master)
                 \
                  o'--o'--o' (t3)

Currently I do this by hand, using git rebase --onto. In this scenario, the whole update process would be:

$ git checkout master
$ git pull
$ git rebase master t1
$ git rebase --onto t1 t2~3 t2
$ git rebase master t3

This gets even hairier when jumping between various topic branches and adding commits.

Dependencies between topic branches in my case are purely tree-like: no branch depends on more than a single other branch. (I have to eventually upstream dependent patches in some particular order, so I choose that order a priori.)

Are there any tools that can help me manage this workflow? I've seen TopGit, but it seems to be tied quite heavily to the tg patch email-based workflow, which isn't relevant to me.

Jocelynjocelyne

Why are you rebasing t2 onto master instead of just rebasing it onto t1, as you show in your graph? Similarly, with t3, the --onto flag isn't appropriate. – Leopoldeen

Also see the question git rebase branch with all subbranches – Juror

Kevin: whoops, yes -- typo :) – Jocelynjocelyne

See also: github.com/goncalopp/git-utilities – Sharpedged

Answer 1

Pretty much the same question was asked on the git mailing list: Rebasing Multiple branches at once... The linked response has a perl script attached that generates the commands you would need.

If you want this script to be fast and avoid having it tread on your toes, also consider using git-new-workdir to set up a working copy just for automatic rebasing.

If you find yourself resolving the same conflicts over and over, consider enabling git rerere.

Having said all that, here is an alternate recipe:

# Construct a placeholder commit that has all topics as parent.
HEADS="$(git for-each-ref refs/heads/\*)" &&
MAGIC_COMMIT=$(echo "Magic Octopus"$'\n\n'"$HEADS" |
  git commit-tree \
    $(git merge-base $(echo "$HEADS" | sed 's/ .*//' ))^{tree} \
    $(echo "$HEADS" | sed 's/ .*//;s/^/-p /')) &&
git update-ref refs/hidden/all $MAGIC_COMMIT

# Rebase the whole lot at once.
git rebase --preserve-merges master refs/hidden/all

# Resolve conflicts and all that jazz.

# Update topic refs from the rebased placeholder.
PARENT=
echo "$HEADS" |
while read HASH TYPE REF
do
  let ++PARENT
  git update-ref -m 'Mass rebase' "$REF" refs/hidden/all^$PARENT "$HASH"
done

Spacetime

Awesome, thanks! :D I used a slightly modified version of this technique to make a script that rebases just a subtree: github.com/nornagon/git-rebase-all – Jocelynjocelyne

Note that since Git 2.18, git rebase now has --rebase-merges, which is an improved version of --preserve-merges and should generally be used in its place. – Vowel

Answer 2

There's a new tool that allows to automate such tasks: git-assembler

Here's a direct link to the example in the documentation for rebasing local branches

You shouldn't dismiss TopGit immediately though. The ability to generate email patches is entirely optional.

Qadi

Answer 3

Since git v2.44 (feb 2024), there is a new experimental command git replay that allows you to do it relatively easily with the command (based on your example):

git replay --onto master old_master..t1 old_master..t2 old_master..t3 | git update-ref --stdin

There is even a simpler syntax where commit ranges are expressed differently:

git replay --onto master ^old_master t1 t2 t3 | git update-ref --stdin

Notes:

• This command doesn't use the working directory to operate (it's an advantage because you don't have to checkout the branch but it could have "strange" side effects in specific cases)
• and so is not able at the moment to handle conflicts (i.e is doing nothing if one is encountered and return an exit code different than 0). And so this solution is maybe the 1st to try and if it fails, fall back on a less convenient work around.
• is creating empty commits if a commit has been already applied (i.e. no more changes to apply)
• replay command doesn't update refs but is just creating git objects and outputs in the console reference updates that need to be done in a format suitable by git update-ref --stdin that effectively does the ref update.
• if one branch replayed is checked out, working directory is not synced (so you end up with changes in the working directory that kind of "revert" work of commits rebased onto). So maybe better to have a clean working directory (and reset working directory) or launch it from another checked out branch or commit and then checkout the working branch again after.

Related: a script finding the references to replay for you: https://mcmap.net/q/276760/-git-equivalent-to-mercurial-39-s-39-hg-evolve-39

Culbreth

Answer 4

Don't rebase. Start your features from a common point. Merges are way less work in the end.

This is what we do:

http://dymitruk.com/blog/2012/02/05/branch-per-feature/

Coussoule

That's nice, but I'm integrating with the Chromium workflow, which is linear. These topic branches are local only, small (typically 1-5 commits each) and short-lived. – Jocelynjocelyne

Wow, this was a good article to read. +1 Sensible advice about rebasing as well, although there will be reasons to do so in specific circumstances. – Epithelium

@Jocelynjocelyne in the workflow I have outlined, it applies to small features. There is no difference. Merging is better in general as you will take into consideration code changes and where they came from including moving files. – Coussoule