black as pre-commit hook always fails my commits

Question

I'm trying to use pre-commit to manage Black as a Git pre-commit hook, but I must be doing it wrong.

In my pre-commit config file I have:

-   repo: https://github.com/psf/black
    rev: 19.3b0
    hooks:
    -   id: black

What I'm expecting to happen is for Black to just modify the staged file, and for the commit to succeed. Because the whole point of Black is that it auto-enforces Python code style rules, no questions asked.

What actually happens when I stage a (non-black-compliant) file and try to commit: Black goes ahead and modifies the file to make it compliant, as intended... But the problem is that it returns a "fail". So the commit fails. And then I have to unstage the file, then restage it before committing again... and only then does the commit succeed.

This is a huge annoyance and can't possibly be the intended workflow?

What am I doing wrong?

Moldy

I also think that for a tool like black, which is intended specifically for modifying files without the need of human supervision, should commit the files even if it has changed them. I've opened an issue on the black repository: github.com/psf/black/issues/1857 – Stickler

A pre-commit hook shouldn't modify staged files without the opportunity for inspection as a matter of principle. It sounds like what you want would be better achieved by aliasing/overriding git add to run black before staging; leaving the pre-commit hook as a validity check, not a formatting tool. – Pya

@Pya I want this for other team members, where I can't modify their shell aliases or GUI workflows. – Jurat

Answer 1

(author of pre-commit here)

the framework intentionally does not provide a way to auto-commit modifications. Here's a few issues which have asked for such:

• pre-commit/pre-commit#806
• pre-commit/pre-commit#747

A comment from one of those issues:

pre-commit itself will never touch the staging area. These are good ways to silently break commits. In my mind this is one of the worst things that [other frameworks do and suggest] -- hooks are very frequently not perfect and magically changing what's being committed should not be taken lightly.

That said, if you would like to foot gun, your hook can call git add -u and pre-commit won't know any better :) a sketch of that (untested, discouraged)

  - id: yapf
    entry: bash -c 'yapf "$@"; git add -u' --

(note: using bash will potentially reduce portability)

Another comment notes

Fortunately, git add -u && !! is pretty easy to run if you're ok firing from the hip :)

Consubstantiation

Sounds reasonable enough. I'm left struggling to understand what the black pre-commit hook is supposed to do / how it's supposed to be used. Because the whole point of black is that it modifies files, no questions asked. I guess that's a questions for the authors of that hook. – Expunction

where do you add this entry? both black and pre-commit-hook say: yapf is not present in repository <either black or pre-commit-hooks>. Typo? Perhaps it is introduced in a newer version? Often pre-commit autoupdate fixes this. – Pathe

@Pathe yapf is just an example for yapf (copied from the ticket), you want to make sure you're configuring the hook you want to configure (id: black for instance) – Consubstantiation

I don't understand how this answer is accepted - I'm left confused just like @Jean-FrançoisCorbett. Whats the use of a pre-commit hook that formats code if it doesn't modify the file in staging? – Giesser

The purpose of the pre-commit hook is to ensure that the formatting is done correctly. It's on the individual developers to ensure that they do that using the same code formatters with same settings on their editors. – Darryl

github.com/psf/black/issues/285 – Misanthropy

Answer 2

One of my developers came with a good tip, in case you have a fail by commit due to black (for example due to the single/double quotes) that's resolved with the pre-commit-hook (like with double-quote-string-fixer). You get in kind of 'nobodies land git situation'. There is a changed file in the staged files, but cannot be committed by the pre-commit-hook, git status won't see any changes, but commit fails (a really black hole in my opinion). You only get a fail on commit, but nothing can be done (excecpt a reset head on this file). Once you are in this situation and run: use commit -m 'Resolving pre-commit-hook changes' --no-verify ..... tada! It's resolved.

Pathe

In this situation, your staged files still have the wrong single-quotes, as black only modifies the working files. In order for git commit to succeed, first stage the changes black made with git add otherwise you'll commit the wrong single-quotes. – Wattmeter

@CarlWalsh Your comment is a little ambiguous. "In order for git commit to succeed" - git commit will succeed because the answer suggests using --no-verify flag. Maybe would be better to rephrase it to something like "In order to avoid committing unformatted code". – Pullen

Answer 3

Looking at black's README, you probably want to use the --check option, which simply exits successfully or unsuccessfully depending on whether the file meets standards. That will cause the commit to fail without modifying the file.

Pantile

I see what you mean, but that's the opposite of what I'm trying to do. I'd like black to just modify the file, and for the commit to succeed. Now clarified in the question. – Expunction

Answer 4

In my experience there is no need to unstage any files, either just git add the files that black, or other pre-commit hooks, as modified again which will overwrite them in the stage area and commit with the same message as before unless the error was with a commit message check.

If your commit includes all of your changed files then simply changing the commit command from git commit -m "Your commit message" to git commit -am "Your commit message" will do the job nicely.

Worshipful

Thanks, in my case the file was both in the staged and modified files area. – Dariusdarjeeling

Answer 5

I'm the same boat as you. From what I've been researching, commits can't be amended by pre-commit hooks.

As far as I can figure out, the best next thing is what bk2204 outlined. We ask black to prevent any commits that include python files that haven't been formatted properly by black. It still makes sure that any commits are formatted, but it is indeed annoying that it won't just automatically format the files for us.

It makes sense. Any changes in a commit have to be staged. If we could do that from the git hook, then our problem would be solved. You got halfway there by modifying the file directly from the git hook. The next half would be staging all the changes of the modified files. But evidently... "You cannot amend a commit in a pre commit hook" which means no staging. https://mcmap.net/q/297707/-git-hook-modify-commit-files

I would have commented on bk2204's answer, but I don't have 50 rep yet.

Scratch all that, this answer (https://mcmap.net/q/297708/-can-you-change-a-file-content-during-git-commit) claims that commits can be changed in a pre commit hook. In that case files are being added, so I bet in our case files can be restaged/amended.

Upholstery

Interesting, basically "it's a bad idea to change what's committed during pre-commit hook." It seems like a better solution to the problem is to have a wrapper script that runs black before you run git >.> – Wattmeter

Answer 6

I spent an hour or so scratching my head with this one in VS Code where I had pre-commit already installed and it was running 'black' for me as part of the pre-commit hook.

As explained here running black as part of the pre-commit hook will generate an updated file which is at this point unstaged - not added to your commit.

Therefore you can get yourself into a commit cycle which is going nowhere in VS Code at least. The way to get out of this cycle is to open up your terminal in your git repo and:

   % git commit
   % git add .
   % git commit

The first commit will run black and generate your updated files with new indentation etc. Then you can add and commit this as per usual. Then you're back in the game and can push your changes.

Parhelion

Answer 7

How about you simply modify your editor[s] to always produce black-compliant code ?

See https://black.readthedocs.io/en/stable/integrations/editors.html .

E.g. for emacs, there's three modes, https://github.com/wbolster/emacs-python-black with python-black-on-save-mode works nicely (except for huuuge files).

For IntelliJ, there's even a built-in integration. Just install black, and enable:

Go to Preferences or Settings -> Tools -> Black and configure Black to your liking.

Basement