Edit, Jun 2022
I'm just boosting the signal from Rufus' comment below:
https://github.com/emilio/clang-format-merge contains code that provides a merge driver, rather than clean and smudge filters. It looks likely to be useful though, especially for repositories that have never had standard formatting enforced.
Recipe with assumptions
(note: I have not tested any of this)
We'll assume the reformatter is in ~/Downloads/android-studio/bin/format.sh and [note: apparently this is a bad assumption!] that it reads stdin and writes stdout, and works on one file at a time. (It's possible, but very difficult, to make this work with something that needs more than one file at a time. You cannot use this recipe for this case, though. Git's basic filtering mechanism requires that each filter simply read stdin and write stdout. By default Git assumes the filter works, even if it exits with a failure status.)
Choose where to run the filter as well; here I've set it up as the "clean" filter only.
In ~/.gitconfig or .git/config, add the definition for the filter:
[filter "my-xyz-language-formatter"]
clean = ~/Downloads/android-studio/bin/format.sh
smudge = cat
(this assumes that running cat runs a filter that writes, to its stdout, its unchanged input; this is true on any Unix-like system).
Then, create a .gitattributes file if needed. It will apply to the directory you create it in, and all sub-directories, unless overridden in those sub-directories, so place it in the highest sensible location, usually the root of the repository, but sometimes underneath a source/ or src/ or whatever directory. Add line(s) to direct file(s) matching some pattern(s) through your formatter. We'll assume here that all files named *.xyz should be formatted:
*.xyz filter=my-xyz-language-formatter
This filter will now apply to all extractions and insertions of *.xyz files. The gitattributes documentation talks about these being applied at check-out and check-in time, but that's not quite precisely correct. Instead, a clean filter is applied whenever Git copies from work-tree to index (essentially, git add—well before git commit unless you use git commit -a or similar flags). A smudge filter is applied whenever Git copies from index to work-tree (essentially, git checkout, but also some additional cases, such as git reset --hard).
Note that spinning up one filter for each file can be quite slow. There's a "long running filter process" protocol you can use if you have a lot of control over the filter, which can speed this up (especially on Windows). That's beyond the scope of this answer, though.
Running git merge normally does not use the filters (it works on the copies that are already in the index, which is outside the filtering step). However, adding -X renormalize to a standard merge will make git merge do the "virtual check-in and check-out" described below, so that it will apply the filters. This happens for all three commits involved in the merge (and in both directions—clean and smudge—so it's roughly 6x slower than for just one commit).
Description (see below)
Git itself is only partially helpful here.
Fundamentally, the problem is that Git is stupid and line-oriented: it runs git diff from the merge base commit to each tip commit. If one or both of these git diffs sees a lot of formatting changes, it considers those significant and worthy of applying to the base. It has no semantic knowledge of the input code.
(Since you can take over the entire merge process, you could write a smarter merge that does use semantic analysis. This is pretty difficult, though. The only system I know of that does this, or something approaching this, is Ira Baxter's commercial software, and I've never actually used that; I just understand the theory behind it.)
There is a solution that does not depend on making Git smarter. If you have a semantic analyzer that outputs consistently formatted code, regardless of the input form, you can feed all three versions—B for base, L for left or local or --ours, and R for right or remote or other or --theirs—into this formatter:
reformat < B > B.formatted
reformat < L > L.formatted
reformat < R > R.formatted
Now you can have Git merge all three formatted versions, rather than merging the original possibly-not-yet-formatted (but maybe formatted) versions.
The result of this merge will, of course, be re-formatted. But presumably this is what you'd like anyway.
The way to achieve this with Git's built-in tools is to use what it calls smudge and clean filters. A smudge filter is applied to files as they are extracted from the repository into the work-tree. A clean filter is applied to files whenever they go from the work-tree into the repository.
In this case, the smudge filter can be "do nothing to the data", preserving exactly what was committed. The clean filter can be the reformatter. Or, if you prefer, the smudge filter can be the reformatter, and the clean filter can be the reformatter again, or a no-op filter. Once you have this in place—this is something you set up in .gitattributes, by defining a filter for particular files by path names, and the filter-driver in .git/config or your main (user or system wide) .gitconfig.
Once you have all that set up, you can run git merge -X renormalize. Git will extract the B, L, and R versions as usual, but then run them through a "virtual check-out and check-in" step, making three temporary commits,1 B.formatted and so on. It then does the merge using the three temporary commits, rather than from the original three commits.
The hard part is finding a reformatter that does just what you want / need. Some modern systems have them, e.g., gofmt or clang-format. If there's one that does what you need, it just becomes a matter of plugging all this together—and getting buy-in from the rest of your group, that this reformatting is a good idea.
1Technically it just makes tree objects; there's no need for actual commits.
