How does git merge after cherry-pick work?

Question

Let's imagine that we have a master branch.

Then we create a newbranch

git checkout -b newbranch

and make two new commits to newbranch: commit1 and commit2

Then we switch to master and make cherry-pick

git checkout master
git cherry-pick hash_of_commit1

Looking into gitk we see that commit1 and its cherry-picked version have different hashes, so technically they are two different commits.

Finally we merge newbranch into master:

git merge newbranch

and see that these two commits with different hashes were merged without problems although they imply that the same changes should be applied twice, so one of them should fail.

Does git really do a smart analysis of commit's content while merging and decide that changes shouldn't be applied twice or these commits are marked internally as linked together?

Newsmagazine

Answer 1

Short answer

Don't worry, Git will handle it.

Long answer

Unlike e.g. SVN¹, Git does not store commits in delta format, but is snapshot-based^2,3. While SVN would naively try to apply each merged commit as a patch (and fail, for the exact reason you described), Git is generally able to handle this scenario.

When merging, Git will try to combine the snapshots of both HEAD commits into a new snapshot. If a portion of code or a file is identical in both snapshots (i.e. because a commit was already cherry-picked), Git won't touch it.

Sources

¹ Skip-Deltas in Subversion
² Git Basics
³ The Git object model

Egger

Actually, I'd say you should worry about merging and "git will handle it" is not a good rule of thumb. – Thematic

In fact the merge CAN result in duplicate content in some cases. Git does handle it sometimes, but sometimes it does not. – Amsterdam

This is horribly wrong. Got pretty much stores the file files in all condevable forms. And if I recall correctly SVN used to store snapshots. – Clevie

@he_the_great, no. SVN's skip-delta storage format (!= snapshots) is well documented in the manual. And I really don't get what you mean by condevable. I'm not a native speaker, but I'm pretty sure that's not a real word. – Egger

@helmbert, sorry 'conceivable' is what I wanted. I unfortunately can't find any historic records about SVN's backend. But my main point is Git does use deltas. – Clevie

@he_the_great, again, no. Git storage internals are documented in-depth in various documentations, as for example here, here or here. – Egger

@Clevie But even as packfiles any given hash for a file results in the full file. Yes it compresses using deltas, but it is not a delta for the changes in a commit, but instead a delta between hashes for a file. As far as the commit object is concerned it is referencing a tree that is referencing the hash for a full file. That under the hood the data is compressed does not effect the way git works. Git stores complete files as far as a commit is concerned, SVN stores deltas for commits as I understand. – Foeman

@PeterOlson Sorry Locking my computer committed a started message. A hash is made from the whole file content, but you don't delta the hash because it doesn't contain all the data of the file. But as quoted, git does store deltas for similar files, "When Git packs objects, it looks for files that are named and sized similarly, and stores just the deltas from one version of the file to the next." – Clevie

Why did you condemn SVN's "apply each merged commit as a patch" method as "naive"? I find that sometimes git aligns the wrong parts of files, and I really hope that it were patch-based, so that at least the merge conflicts are easier to understand. Disclaimer: I don't actually use SVN. – Suppository

@ImperishableNight "naive" is this context was meant to be read as: "using the easy, obvious solution of applying each change from the to-be-merged branch as a patch (instead of Git's more complex three-way-merge) that would work most of the time, but not in the scenario that was stated in the original question (merging a branch from which individual changes have already been cherry-picked)." – Egger

As mentioned in other comments, this can go horribly wrong (It sucks that Git apparently lacks any kind of option to mitigate this). I've seen several time the following situation: branch A periodically merges branch B, new commit b1 from B makes it into A and breaks things, in B it's already reverted and then reapplied with the fix, if you manually revert commit in A, it'll likely silently drop the reapplied fix from B. It's hard to detect. – Touber

Answer 2

After such merge you may have cherry-picked commits in history twice.

Solution to prevent this I quote from article which recommends for branches with duplicate(cherry-picked) commits use rebase before merge:

git merge after git cherry-pick: avoiding duplicate commits

Imagine we have the master branch and a branch b:

   o---X   <-- master
    \
     b1---b2---b3---b4   <-- b

Now we urgently need the commits b1 and b3 in master, but not the remaining commits in b. So what we do is checkout the master branch and cherry-pick commits b1 and b3:

$ git checkout master
$ git cherry-pick "b1's SHA"
$ git cherry-pick "b3's SHA"

The result would be:

   o---X---b1'---b3'   <-- master
    \
     b1---b2---b3---b4   <-- b

Let’s say we do another commit on master and we get:

   o---X---b1'---b3'---Y   <-- master
    \
     b1---b2---b3---b4   <-- b

If we would now merge branch b into master:

$ git merge b

We would get the following:

   o---X---b1'---b3'---Y--- M  <-- master
     \                     /
      b1----b2----b3----b4   <-- b

That means the changes introduced by b1 and b3 would appear twice in the history. To avoid that we can rebase instead of merge:

$ git rebase master b

Which would yield:

   o---X---b1'---b3'---Y   <-- master
                        \
                         b2'---b4'   <-- b

Finally:

$ git checkout master
$ git merge b

gives us:

   o---X---b1'---b3'---Y---b2'---b4'   <-- master, b

EDIT Corrections supposed by David Lemon's comment

Pankhurst

great hint about rebase! it will 'skip' all cherry-picked commits automatically. – Cassidy

Honestly, it sounds too good to be true, I have to see it with my eyes. Also rebase modifies commits, your last timeline should be ---Y---b2'---b4' – Tagalog

Works perfectly. Very helpful if you don't want to have the cherry-picked commits twice in history. – Scramble

Shouldn't it be noted that while rebase is beautiful, the danger with using it is that any forks or branches created from the old b will be out of sync, and users may need to resort to stuff like git reset --hard and git push -f? – Monotheism

@Monotheism thats why we rebase local branch here – Pankhurst

@Monotheism That is correct, but is it a danger? or just part of the workflow? I think it's not usually a big issue, I can imagine if the repo was super large and distributed then it might be but usually you are rebasing your own branches. E.g., here b is your branch that you are putting out of sync, but the branch that is usually shared, master, doesn't go out of sync at any time – Speedometer

Jose, for reviews it can be a pain since tracking differences between two reviewed versions could be difficult. Github, Gerrit etc usually deals with it though. – Monotheism

@Pankhurst There is nothing associating b1 and b3 on b to b1' and b3' on master. How can git know to eliminate b1 and b3 while rebasing? – Bogor

@OfekShilon they do not bring any changes so rebase removes them as empty ones – Pankhurst

@Pankhurst the diff is taken from the parent commit, so to git it looks like they do bring changes. – Bogor