git-apply fails mysteriously, how do I troubleshoot/fix?

Question

I'm currently trying to to code-style checking on the PRs of a (github) repository, and I want to deliver patches to the submitters with which they can easily fix the codestyle. To this end, I'm pulling down their PR, run our uncrustify script over it to fix any style errors, and want to create a .patch file they can easily apply. However, it consistently breaks on some files.

I do (git version 1.7.10.4 with core.autocrlf=input, core.filemode=false):

$ git checkout pr-branch
$ git log -1 (shows: commit dbb8d3f)
$ git status (nothing to commit, working directory clean)
$ <run the code styler script, which modifies some files>
$ git diff > ../style.patch (so the patch file lands outside the repo)
$ git reset --hard HEAD (to simulate the situation at the submitter's end)
$ git log -1 (shows: commit dbb8d3f)
$ git status (nothing to commit, working directory clean, so we are where we started)
$ git apply ../style.patch
error: patch failed: somefile.cpp:195
error: somefile.cpp: patch does not apply (same output using the --check option)

This only applies to some files, not all of them. I don't know how to troubleshoot this, i.e. how to get git to tell me exactly where it goes wrong - it only tells me a hunk# when I dig, but that's still pretty huge.

What I've tried so far (without success):

1. apply --reverse, apply --whitespace=nowarn
2. diff HEAD instead of diff alone
3. make a dummy commit (committing works without problem!), use format-patch, delete the dummy commit, apply patch with git-am with or without -3, or apply with git-apply
4. Have the patch file in the local dir instead of one up (grasping at straws, here)
5. Check the man-pages of git-diff, -apply, -format-patch, -am for anything useful
6. patch with the linux patch command
7. ....

I don't know what could be wrong with the diff. Whitespace things should only warn, right? In any case, I won't want to ignore them, since it's a style fix which obviously involves whitespace.

How can I fix/diagnose this or even find out where it bails exactly? Would it help if I posted the diff of one of the culprit files? What baffles me also is that the committ works without problem, but the patch created from the commit does not??

After wrestling with this for several hours I'm at the end of my knowledge...

Squiffy

git apply --reject to see the rejected changes. – Troat

Tried that already...this only shows you the whole hunk which fails (about 2-300 lines in that one case), so not very informative – Squiffy

Another guess - there could be some file renames and something might have become 'gitignored' as the result. – Troat

nope, it's just style changes within text files, no renames. The --index option by Franci seems to have fixed the problem. – Squiffy

Had the same problem. What helped me was --whitespace=fix but I still don't understand why, as I've just created the patch and haven't edited it manually. – Quench

I landed here, for a different reason, related to CRLF characters. this answer helped me to fix it – Leeuwarden

See also, for silent failures: – Thermosetting

@Leeuwarden can you shed more light on what changes from this comment shall one make to prevent git fail during git apply method ? like one should choose which End of line sequence (CRLF or LF) and what encoding method to avoid this error during git apply ? Also how to set them – Westley

hey @humbleCodes, it's been a while... I don't remember the details, but the link that I provided above still works and they provide a lot of details on how/when to set them. I hope it helps you! – Leeuwarden

Answer 1

Update:

You can use git apply -v to see more detailed info about what's going on, git apply --check to just verify the operation, or git apply --index to rebuild the local index file.

Based on your comment, it seems your local index was corrupted, and so index solved it.

I will leave my original answer and the comments mostly to give people context on what was going on, as I suspect other people would jump to the same initial conclusions I had based on the problem description.

------

Most likely nothing's wrong with the diff. Instead look at the target git repository. While you are doing git reset --hard HEAD, nothing guarantees you that the HEAD on that other repository is the same as the HEAD on your.

Do git log on the target repo and look at the commit at the top. Is it the same as the one you produced the diff from? Most likely it is not. Look down the history and check if the commit you need is there. If it is, then the target repo is ahead of yours, and you have to go back, do git pull (or git rebase) and produce a new diff. If it isn't, then the target repo is behind yours, and you need to do git pull (or git rebase) on the target repo to bring it up to speed.

Keep in mind that if you have other people committing to your "master" repo (the one where bot yours and the target repositories are pulling from), you might have to git pull both repositories, to get them to a reasonably recent common commit.

Molybdenous

In the procedure I described above the current HEAD is the same for both diff creation and diff application. It's the last commit in that branch. I do the reset exactly to eliminate any issues from different HEADs. so that can't be it. Updating when necessary is already being handled automatically, but if the apply doesn't work in the above test situation, it will fail there, too, so that has to be solved before. – Squiffy

It's not clear from your description if both HEADs point to the same commit. My guess is that they aren't, but if you checked them and they are, then indeed something's wrong with the diff. – Molybdenous

Also, try git apply -v, or playing with the --check and -index options. – Molybdenous

really? how can the HEADs be different if all I do is git-checkout, change some files and then git-reset --hard HEAD, without committing in between? I've already tried those options, but all I get is some stuff about whitespace. will do again and post some more info. – Squiffy

git checkout and git reset are both local repository commands. Unless the target repo is the upstream repo for the source repo (in which case I'd expect you to do git push, not do a diff), you have to do git pull or git rebase before checkout. – Molybdenous

Or are you doing git checkout origin/pr-branch? – Molybdenous

Um, I'm testing this on the local repo, pull or rebase doesn't come into the problem? The patch is intended to facilitate users observing the code style, therefore the patch which they can apply quickly&easily (but this is really tangential to the problem) – Squiffy

let us continue this discussion in chat – Squiffy

OK, @Franci, the --index option seems to have solved the problem, could you make another answer I can then accept? – Squiffy

Sorry, I was in a meeting and couldn't do chat. :-( – Molybdenous

@FranciPenov "Do git log on the target repo and look at the commit at the top" - do you mean patches can only be applied if the HEADs are "the same"? Can't I pick a patch from whatever repo with a same file name (e.g.) and apply it anywhere? – Atween

Answer 2

Try to check against your patch file - example :

git apply --reject mypatch.patch

this will show you differences if any - here is an example of how it could look like :

error: patch failed: <filename>:<linenumber>
error: while searching for :
    cout << "}" << endl; // example of a line in your patch

Bethesde

@YasinOkumuş You are right, it's not resolve the problem but it skip the error and apply the rest and explain why, so basically it's all you can ask for :) – Public

Answer 3

In such cases, if you already tried all options listed below:

1. Investigating potential non-printable/invisible characters in file
2. Using dos2unix for conversion
3. changing the git config to adjust the way of handling line endings for eg:

git config --global core.autocrlf input

Another root of evil in such cases could be your IDE.

For example, some of the JetBrains IDEs like PHPStorm has following option:

After spending more than 4 bloody hours today on a weird patching issue ending up with following error:

git apply < my.git.patch --verbose
...
error: patch failed: plugins/some/file.php:74

I realized that I was victim of my editor (I am not sure if this was the default behavior).

I was using PHPStorm to check the patch file and it was silently removing a single whitespace from the patch file which literally exist on the target file that needs to be patched.

This was super sneaky and I ended up with setting Strip trailing spaces on Save for option to None to never hit such issue again.

Also keep in mind that platform related differences like case-sensitivity of the underlying filesystem (MacOS/Linux for eg) may also be the suspect.

Dairyman