Git's documentation for the rebase command is quite brief:
--preserve-merges
Instead of ignoring merges, try to recreate them.
This uses the --interactive machinery internally, but combining it
with the --interactive option explicitly is generally not a good idea
unless you know what you are doing (see BUGS below).
So what actually happens when you use --preserve-merges? How does it differ from the default behavior (without that flag)? What does it mean to "recreate" a merge, etc.
As with a normal git rebase, git with --preserve-merges first identifies a list of commits made in one part of the commit graph, and then replays those commits on top of another part. The differences with --preserve-merges concern which commits are selected for replay and how that replaying works for merge commits.
To be more explicit about the main differences between normal and merge-preserving rebase:
git checkout <desired first parent>), whereas normal rebase doesn't have to worry about that.First I will try to describe "sufficiently exactly" what rebase --preserve-merges does, and then there will be some examples. One can of course start with the examples, if that seems more useful.
The Algorithm in "Brief"
If you want to really get into the weeds, download the git source and explore the file git-rebase--interactive.sh. (Rebase is not part of Git's C core, but rather is written in bash. And, behind the scenes, it shares code with "interactive rebase".)
But here I will sketch what I think is the essence of it. In order to reduce the number of things to think about, I have taken a few liberties. (e.g. I don't try to capture with 100% accuracy the precise order in which computations take place, and ignore some less central-seeming topics, e.g. what to do about commits that have already been cherry-picked between branches).
First, note that a non-merge-preserving rebase is rather simple. It's more or less:
Find all commits on B but not on A ("git log A..B")
Reset B to A ("git reset --hard A")
Replay all those commits onto B one at a time in order.
Rebase --preserve-merges is comparatively complicated. Here's as simple as I've been able to make it without losing things that seem pretty important:
Find the commits to replay:
First find the merge-base(s) of A and B (i.e. the most recent common ancestor(s))
This (these) merge base(s) will serve as a root/boundary for the rebase.
In particular, we'll take its (their) descendants and replay them on top of new parents
Now we can define C, the set of commits to replay. In particular, it's those commits:
1) reachable from B but not A (as in a normal rebase), and ALSO
2) descendants of the merge base(s)
If we ignore cherry-picks and other cleverness preserve-merges does, it's more or less:
git log A..B --not $(git merge-base --all A B)
Replay the commits:
Create a branch B_new, on which to replay our commits.
Switch to B_new (i.e. "git checkout B_new")
Proceeding parents-before-children (--topo-order), replay each commit c in C on top of B_new:
If it's a non-merge commit, cherry-pick as usual (i.e. "git cherry-pick c")
Otherwise it's a merge commit, and we'll construct an "equivalent" merge commit c':
To create a merge commit, its parents must exist and we must know what they are.
So first, figure out which parents to use for c', by reference to the parents of c:
For each parent p_i in parents_of(c):
If p_i is one of the merge bases mentioned above:
# p_i is one of the "boundary commits" that we no longer want to use as parents
For the new commit's ith parent (p_i'), use the HEAD of B_new.
Else if p_i is one of the commits being rewritten (i.e. if p_i is in R):
# Note: Because we're moving parents-before-children, a rewritten version
# of p_i must already exist. So reuse it:
For the new commit's ith parent (p_i'), use the rewritten version of p_i.
Otherwise:
# p_i is one of the commits that's *not* slated for rewrite. So don't rewrite it
For the new commit's ith parent (p_i'), use p_i, i.e. the old commit's ith parent.
Second, actually create the new commit c':
Go to p_1'. (i.e. "git checkout p_1'", p_1' being the "first parent" we want for our new commit)
Merge in the other parent(s):
For a typical two-parent merge, it's just "git merge p_2'".
For an octopus merge, it's "git merge p_2' p_3' p_4' ...".
Switch (i.e. "git reset") B_new to the current commit (i.e. HEAD), if it's not already there
Change the label B to apply to this new branch, rather than the old one. (i.e. "git reset --hard B")
Rebase with an --onto C argument should be very similar. Just instead of starting commit playback at the HEAD of B, you start commit playback at the HEAD of C instead. (And use C_new instead of B_new.)
Example 1
For example, take commit graph
B---C <-- master
/
A-------D------E----m----H <-- topic
\ /
F-------G
m is a merge commit with parents E and G.
Suppose we rebased topic (H) on top of master (C) using a normal, non-merge-preserving rebase. (For example, checkout topic; rebase master.) In that case, git would select the following commits for replay:
and then update the commit graph like so:
B---C <-- master
/ \
A D'---E'---F'---G'---H' <-- topic
(D' is the replayed equivalent of D, etc..)
Note that merge commit m is not selected for replay.
If we instead did a --preserve-merges rebase of H on top of C. (For example, checkout topic; rebase --preserve-merges master.) In this new case, git would select the following commits for replay:
Now m was chosen for replay. Also note that merge parents E and G were picked for inclusion before merge commit m.
Here is the resulting commit graph:
B---C <-- master
/ \
A D'-----E'----m'----H' <-- topic
\ /
F'-------G'
Again, D' is a cherry-picked (i.e. recreated) version of D. Same for E', etc.. Every commit not on master has been replayed. Both E and G (the merge parents of m) have been recreated as E' and G' to serve as the parents of m' (after rebase, the tree history still remains the same).
Example 2
Unlike with normal rebase, merge-preserving rebase can create multiple children of the upstream head.
For example, consider:
B---C <-- master
/
A-------D------E---m----H <-- topic
\ |
------- F-----G--/
If we rebase H (topic) on top of C (master), then the commits chosen for rebase are:
And the result is like so:
B---C <-- master
/ | \
A | D'----E'---m'----H' <-- topic
\ |
F'----G'---/
Example 3
In the above examples, both the merge commit and its two parents are replayed commits, rather than the original parents that the original merge commit have. However, in other rebases a replayed merge commit can end up with parents that were already in the commit graph before the merge.
For example, consider:
B--C---D <-- master
/ \
A---E--m------F <-- topic
If we rebase topic onto master (preserving merges), then the commits to replay will be
The rewritten commit graph will look like so:
B--C--D <-- master
/ \
A-----E---m'--F'; <-- topic
Here replayed merge commit m' gets parents that pre-existed in the commit graph, namely D (the HEAD of master) and E (one of the parents of the original merge commit m).
Example 4
Merge-preserving rebase can get confused in certain "empty commit" cases. At least this is true only some older versions of git (e.g. 1.7.8.)
Take this commit graph:
A--------B-----C-----m2---D <-- master
\ \ /
E--- F--\--G----/
\ \
---m1--H <--topic
Note that both commit m1 and m2 should have incorporated all the changes from B and F.
If we try to do git rebase --preserve-merges of H (topic) onto D (master), then the following commits are chosen for replay:
Note that the changes (B, F) united in m1 should already be incorporated into D. (Those changes should already be incorporated into m2, because m2 merges together the children of B and F.) Therefore, conceptually, replaying m1 on top of D should probably either be a no-op or create an empty commit (i.e. one where the diff between successive revisions is empty).
Instead, however, git may reject the attempt to replay m1 on top of D. You can get an error like so:
error: Commit 90caf85 is a merge but no -m option was given.
fatal: cherry-pick failed
It looks like one forgot to pass a flag to git, but the underlying problem is that git dislikes creating empty commits.
git rebase --preserve-merges is much slower than a rebase without --preserve-merges. Is that a side-effect of finding the right commits? Is there anything one can do to speed it up? (By the way…thanks for the very detailed answer!) –
Mirk --preserve-merges. Generally speaking, since rebasing is rewriting things, if you are applying it in such a way that things outside your local repository will be affected, then there's probably a better approach to the overall workflow. Granted, there are times you have to break things due to circumstances you can't control, though... –
Turnbow git config --global pull.rebase preserve, but for classical rebase? Would this work? git config --global branch.rebase preserve –
Manille --preserve-merges is deprecated. Use --rebase-merges or -r mode, which is similar in spirit and it works faster. –
Hyksos Git 2.18 (Q2 2018) will improve considerably the --preserve-merge option by adding a new option.
"git rebase" learned "--rebase-merges" to transplant the whole
topology of commit graph elsewhere.
(Note: Git 2.22, Q2 2019, actually deprecates --preserve-merge, and Git 2.25, Q1 2020, stops advertising it in the "git rebase --help" output)
See commit 25cff9f, commit 7543f6f, commit 1131ec9, commit 7ccdf65, commit 537e7d6, commit a9be29c, commit 8f6aed7, commit 1644c73, commit d1e8b01, commit 4c68e7d, commit 9055e40, commit cb5206e, commit a01c2a5, commit 2f6b1d1, commit bf5c057 (25 Apr 2018) by Johannes Schindelin (dscho).
See commit f431d73 (25 Apr 2018) by Stefan Beller (stefanbeller).
See commit 2429335 (25 Apr 2018) by Phillip Wood (phillipwood).
(Merged by Junio C Hamano -- gitster -- in commit 2c18e6a, 23 May 2018)
pull: accept--rebase-mergesto recreate the branch topology
Similar to the
preservemode simply passing the--preserve-mergesoption to therebasecommand, themergesmode simply passes the--rebase-mergesoption.This will allow users to conveniently rebase non-trivial commit topologies when pulling new commits, without flattening them.
git rebase man page now has a full section dedicated to rebasing history with merges.
Extract:
There are legitimate reasons why a developer may want to recreate merge commits: to keep the branch structure (or "commit topology") when working on multiple, inter-related branches.
In the following example, the developer works on a topic branch that refactors the way buttons are defined, and on another topic branch that uses that refactoring to implement a "Report a bug" button.
The output ofgit log --graph --format=%s -5may look like this:* Merge branch 'report-a-bug' |\ | * Add the feedback button * | Merge branch 'refactor-button' |\ \ | |/ | * Use the Button class for all buttons | * Extract a generic Button class from the DownloadButton oneThe developer might want to rebase those commits to a newer
masterwhile keeping the branch topology, for example when the first topic branch is expected to be integrated intomastermuch earlier than the second one, say, to resolve merge conflicts with changes to theDownloadButtonclass that made it intomaster.This rebase can be performed using the
--rebase-mergesoption.
See commit 1644c73 for a small example:
rebase-helper--make-script: introduce a flag to rebase merges
The sequencer just learned new commands intended to recreate branch structure (similar in spirit to
--preserve-merges, but with a substantially less-broken design).Let's allow the
rebase--helperto generate todo lists making use of these commands, triggered by the new--rebase-mergesoption.
For a commit topology like this (where the HEAD points to C):- A - B - C (HEAD) \ / Dthe generated todo list would look like this:
# branch D pick 0123 A label branch-point pick 1234 D label D reset branch-point pick 2345 B merge -C 3456 D # C
What is the difference with --preserve-merge?
Commit 8f6aed7 explains:
Once upon a time, this here developer thought: wouldn't it be nice if, say, Git for Windows' patches on top of core Git could be represented as a thicket of branches, and be rebased on top of core Git in order to maintain a cherry-pick'able set of patch series?
The original attempt to answer this was:
git rebase --preserve-merges.However, that experiment was never intended as an interactive option, and it only piggy-backed on
git rebase --interactivebecause that command's implementation looked already very, very familiar: it was designed by the same person who designed--preserve-merges: yours truly.
And by "yours truly", the author refers to himself: Johannes Schindelin (dscho), who is the main reason (with a few other heroes -- Hannes, Steffen, Sebastian, ...) that we have Git For Windows (even though back in the day -- 2009 -- that was not easy).
He is working at Microsoft since Sept. 2015, which makes sense considering Microsoft now heavily uses Git and needs his services.
That trend started in 2013 actually, with TFS. Since then, Microsoft manages the largest Git repository on the planet! And, since Oct. 2018, Microsoft acquired GitHub.
You can see Johannes speak in this video for Git Merge 2018 in April 2018.
Some time later, some other developer (I am looking at you, Andreas! ;-)) decided that it would be a good idea to allow
--preserve-mergesto be combined with--interactive(with caveats!) and the Git maintainer (well, the interim Git maintainer during Junio's absence, that is) agreed, and that is when the glamor of the--preserve-mergesdesign started to fall apart rather quickly and unglamorously.
Here Jonathan is talking about Andreas Schwab from Suse.
You can see some of their discussions back in 2012.
The reason? In
--preserve-mergesmode, the parents of a merge commit (or for that matter, of any commit) were not stated explicitly, but were implied by the commit name passed to thepickcommand.This made it impossible, for example, to reorder commits.
Not to mention to move commits between branches or, deity forbid, to split topic branches into two.Alas, these shortcomings also prevented that mode (whose original purpose was to serve Git for Windows' needs, with the additional hope that it may be useful to others, too) from serving Git for Windows' needs.
Five years later, when it became really untenable to have one unwieldy, big hodge-podge patch series of partly related, partly unrelated patches in Git for Windows that was rebased onto core Git's tags from time to time (earning the undeserved wrath of the developer of the ill-fated
git-remote-hgseries that first obsoleted Git for Windows' competing approach, only to be abandoned without maintainer later) was really untenable, the "Git garden shears" were born: a script, piggy-backing on top of the interactive rebase, that would first determine the branch topology of the patches to be rebased, create a pseudo todo list for further editing, transform the result into a real todo list (making heavy use of theexeccommand to "implement" the missing todo list commands) and finally recreate the patch series on top of the new base commit.
(The Git garden shears script is referenced in this patch in commit 9055e40)
That was in 2013.
And it took about three weeks to come up with the design and implement it as an out-of-tree script. Needless to say, the implementation needed quite a few years to stabilize, all the while the design itself proved itself sound.With this patch, the goodness of the Git garden shears comes to
git rebase -iitself.
Passing the--rebase-mergesoption will generate a todo list that can be understood readily, and where it is obvious how to reorder commits.
New branches can be introduced by insertinglabelcommands and callingmerge <label>.
And once this mode will have become stable and universally accepted, we can deprecate the design mistake that was--preserve-merges.
Git 2.19 (Q3 2018) improves the new --rebase-merges option by making it work with --exec.
The "--exec" option to "git rebase --rebase-merges" placed the exec
commands at wrong places, which has been corrected.
See commit 1ace63b (09 Aug 2018), and commit f0880f7 (06 Aug 2018) by Johannes Schindelin (dscho).
(Merged by Junio C Hamano -- gitster -- in commit 750eb11, 20 Aug 2018)
rebase --exec: make it work with--rebase-merges
The idea of
--execis to append anexeccall after eachpick.Since the introduction of
fixup!/squash!commits, this idea was extended to apply to "pick, possibly followed by a fixup/squash chain", i.e. an exec would not be inserted between apickand any of its correspondingfixuporsquashlines.The current implementation uses a dirty trick to achieve that: it assumes that there are only pick/fixup/squash commands, and then inserts the
execlines before anypickbut the first, and appends a final one.With the todo lists generated by
git rebase --rebase-merges, this simple implementation shows its problems: it produces the exact wrong thing when there arelabel,resetandmergecommands.Let's change the implementation to do exactly what we want: look for
picklines, skip any fixup/squash chains, and then insert theexecline. Lather, rinse, repeat.Note: we take pains to insert before comment lines whenever possible, as empty commits are represented by commented-out pick lines (and we want to insert a preceding pick's exec line before such a line, not afterward).
While at it, also add
execlines aftermergecommands, because they are similar in spirit topickcommands: they add new commits.
Git 2.22 (Q2 2019) fixes the usage of the refs/rewritten/ hierarchy to store a rebase intermediate states, which inherently makes the hierarchy per worktree.
See commit b9317d5, commit 90d31ff, commit 09e6564 (07 Mar 2019) by Nguyễn Thái Ngọc Duy (pclouds).
(Merged by Junio C Hamano -- gitster -- in commit 917f2cd, 09 Apr 2019)
Make sure refs/rewritten/ is per-worktree
a9be29c (sequencer: make refs generated by the label command
worktree-local, 2018-04-25, Git 2.19) adds refs/rewritten/ as per-worktree
reference space.
Unfortunately (my bad) there are a couple places that
need update to make sure it's really per-worktree.
add_per_worktree_entries_to_dir()is updated to make sure ref listing look at per-worktreerefs/rewritten/instead of per-repo one.
common_list[]is updated so thatgit_path()returns the correct location. This includes "rev-parse --git-path".This mess is created by me.
I started trying to fix it with the introduction ofrefs/worktree,where all refs will be per-worktree without special treatments.
Unfortunate refs/rewritten came before refs/worktree so this is all we can do.
With Git 2.24 (Q4 2019), "git rebase --rebase-merges" learned to drive different merge strategies and pass strategy specific options to them.
See commit 476998d (04 Sep 2019) by Elijah Newren (newren).
See commit e1fac53, commit a63f990, commit 5dcdd74, commit e145d99, commit 4e6023b, commit f67336d, commit a9c7107, commit b8c6f24, commit d51b771, commit c248d32, commit 8c1e240, commit 5efed0e, commit 68b54f6, commit 2e7bbac, commit 6180b20, commit d5b581f (31 Jul 2019) by Johannes Schindelin (dscho).
(Merged by Junio C Hamano -- gitster -- in commit 917a319, 18 Sep 2019)
With Git 2.25 (Q1 2020), the logic used to tell worktree local and repository global refs apart is fixed, to facilitate the preserve-merge.
See commit f45f88b, commit c72fc40, commit 8a64881, commit 7cb8c92, commit e536b1f (21 Oct 2019) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit db806d7, 10 Nov 2019)
path.c: don't call thematchfunction without value intrie_find()Signed-off-by: SZEDER Gábor
'logs/refs' is not a working tree-specific path, but since commit b9317d55a3 (Make sure refs/rewritten/ is per-worktree, 2019-03-07, v2.22.0-rc0) '
git rev-parse --git-path' has been returning a bogus path if a trailing '/' is present:$ git -C WT/ rev-parse --git-path logs/refs --git-path logs/refs/ /home/szeder/src/git/.git/logs/refs /home/szeder/src/git/.git/worktrees/WT/logs/refs/We use a
triedata structure to efficiently decide whether a path belongs to the common dir or is working tree-specific.As it happens b9317d55a3 triggered a bug that is as old as the
trieimplementation itself, added in 4e09cf2acf ("path: optimize common dir checking", 2015-08-31, Git v2.7.0-rc0 -- merge listed in batch #2).
According to the comment describing
trie_find(), it should only call the given match function 'fn' for a "/-or-\0-terminated prefix of the key for which the trie contains a value".
This is not true: there are three places where trie_find() calls the match function, but one of them is missing the check for value's existence.b9317d55a3 added two new keys to the
trie:'
logs/refs/rewritten', and'
logs/refs/worktree', next to the already existing 'logs/refs/bisect'.
This resulted in atrienode with the path 'logs/refs/', which didn't exist before, and which doesn't have a value attached.
A query for 'logs/refs/' finds this node and then hits that one callsite of thematchfunction which doesn't check for the value's existence, and thus invokes thematchfunction withNULLas value.When the
matchfunctioncheck_common()is invoked with aNULLvalue, it returns 0, which indicates that the queried path doesn't belong to the common directory, ultimately resulting the bogus path shown above.Add the missing condition to
trie_find()so it will never invoke the match function with a non-existing value.
check_common()will then no longer have to check that it got a non-NULL value, so remove that condition.I believe that there are no other paths that could cause similar bogus output.
AFAICT the only other key resulting in the match function being called with a
NULLvalue is 'co' (because of the keys 'common' and 'config').However, as they are not in a directory that belongs to the common directory the resulting working tree-specific path is expected.
Make sure to use Git 2.34 (Q4 2021), to avoid a memory leak.
See commit 6e65854, commit 0c52cf8 (13 Oct 2021), and commit e5a917f (07 Oct 2021) by Ævar Arnfjörð Bjarmason (avar).
See commit 9d05b45 (07 Oct 2021) by Junio C Hamano (gitster).
(Merged by Junio C Hamano -- gitster -- in commit bfa646c, 25 Oct 2021)
sequencer: fix a memory leak indo_reset()Signed-off-by: Ævar Arnfjörð Bjarmason
Fix a memory leak introduced in 9055e40 ("
sequencer: introduce new commands to reset the revision", 2018-04-25, Git v2.18.0-rc0 -- merge listed in batch #6), which calledsetup_unpack_trees_porcelain()without a corresponding call toclear_unpack_trees_porcelain().
Git 2.41 (Q2 2023) streamlines --rebase-merges command line option handling and introduces rebase.merges configuration variable.
See commit 6605fb7, commit 33561f5, commit 7e5dcec (25 Mar 2023) by Alex Henrie (alexhenrie).
(Merged by Junio C Hamano -- gitster -- in commit 9142fce, 04 Apr 2023)
rebase: add a config option for--rebase-mergesSigned-off-by: Alex Henrie
The purpose of the new option is to accommodate users who would like
--rebase-mergesto be on by default and to facilitate turning on--rebase-mergesby default without configuration in a future version of Git.Name the new option
rebase.rebaseMerges, even though it is a little redundant, for consistency with the name of the command line option and to be clear when scrolling through values in the[rebase]section of.gitconfig.Support setting
rebase.rebaseMergesto the nonspecific value "true" for users who don't need to or don't want to learn about the difference between rebase-cousins and no-rebase-cousins.Make
--rebase-mergeswithout an argument on the command line override any value ofrebase.rebaseMergesin the configuration, for consistency with other command line flags with optional arguments that have an associated config option.
git config now includes in its man page:
rebase.rebaseMergesWhether and how to set the
--rebase-mergesoption by default. Can berebase-cousins,no-rebase-cousins, or a boolean. Setting to true or tono-rebase-cousinsis equivalent to--rebase-merges=no-rebase-cousins, setting torebase-cousinsis equivalent to--rebase-merges=rebase-cousins, and setting to false is equivalent to--no-rebase-merges.
Passing--rebase-mergeson the command line, with or without an argument, overrides anyrebase.rebaseMergesconfiguration.
git rebase now includes in its man page:
countermand both the
rebase.rebaseMergesconfig option and a previous--rebase-merges.
--preserve-merges doesnt actually "preserve" the merges as you want, it is very naive. This allows you to preserve the merge commits & their parent commit relationships while giving you the flexibility of an interactive rebase. This new feature is awesome & if not for this well written S.O. answer, i wouldnt have known! –
Bell git config pull.rebase option: it can set either policy by default. –
Slr --rebase-merges ofc, hadn't had my coffee yet... Ty for the link. –
Frown rebase itself. Maybe you can specify it for that branch you want to rebase (using branch.<name>.rebase), and then try your rebase -i, but I am not sure it would work. –
Slr --rebase-merge I think I will need to look further into it –
Frown For those ending here just because they pulled and got that message :
git pull
(...)
warning: git rebase --preserve-merges is deprecated. Use --rebase-merges instead.
Have a look into your ~/.gitconfig and /etc/gitconfig and search for this option :
[pull]
rebase = preserve
Then head to that doc to understand and fix according to your needs : https://www.git-scm.com/docs/git-config#Documentation/git-config.txt-pullrebase
preserve was replaced by merges in v2.32.0 git-scm.com/docs/git-config/2.32.0#Documentation/… –
Downall © 2022 - 2024 — McMap. All rights reserved.
git --rebase-mergeswill ultimately replace the oldgit --preserve-merges. See my answer below – Slrrebase-mergeswas removed and will result in a number of differentfatal:messages, depending on how you chance upon the option. There are edge cases. Visual Studio has yet to notice, particularly for thegit pullconfig options – Leisaleiser--preserve-mergeshas been removed? – Pretension