Git pre-commit hooks keeps modifying files even after I have staged previously modified files

Question

I'm running git pre-commit and running black as one of the hooks.

Now when I run commit, black fails and says:

All done! ✨ 🍰 ✨
15 files reformatted, 1 file left unchanged.

I reviewed the reformatted files and I'm fine with them. So I stage those files and try running commit again, but I keep getting the same message as above. I have tried the following commands with no success.

git add .
git add -A
git add -u

This is my .pre-commit-config.yaml file:

repos:
    -   repo: https://github.com/psf/black
        rev: 19.10b0
        hooks:
            - id: black
              language_version: python3.6
    -   repo: https://github.com/pre-commit/pre-commit-hooks
        rev: v2.5.0
        hooks:
            -   id: check-merge-conflict
            -   id: check-docstring-first
            -   id: check-json
            -   id: check-yaml
            -   id: debug-statements
            -   id: double-quote-string-fixer
            -   id: end-of-file-fixer
            -   id: name-tests-test
                args: [--django]
            -   id: requirements-txt-fixer
            -   id: trailing-whitespace
    -   repo: https://gitlab.com/pycqa/flake8
        rev: 3.7.9
        hooks:
            -   id: flake8
                additional_dependencies: [flake8-typing-imports==1.6.0]
    - repo: https://github.com/asottile/reorder_python_imports
      rev: v1.4.0
      hooks:
            -   id: reorder-python-imports
                args: [--py3-plus]
    -   repo: https://github.com/Lucas-C/pre-commit-hooks-bandit
        rev: v1.0.4
        hooks:
            -   id: python-bandit-vulnerability-check
                args: [-l, --recursive, -x, tests]
                files: .py$
    -   repo: local
        hooks:
            -   id: tests
                name: run tests
                entry: venv/bin/pytest -v -m fast
                language: python
                additional_dependencies: [pre-commit, pytest]
                always_run: true
                pass_filenames: false
                types: [python]
                stages: [commit]
    -   repo: local
        hooks:
            -   id: tests
                name: run tests
                entry: venv/bin/pytest -x
                language: system
                types: [python]
                stages: [push]

When I do git status --short, I get this:

M  .pre-commit-config.yaml
M  pytest.ini
M  setup.cfg
RM tests/tests_report.html -> tests/commit_pytest_report.html
R  report.html -> tests/commit_tests_report.html
AM tests/coverage/index.html
A  tests/coverage/file_1.png

When I run git commit -m "test", after running git add ., git add -A, or git add -u; I get this:

black....................................................................Failed
    - hook id: black
    - files were modified by this hook

reformatted <filename>
...
All done! ✨ 🍰 ✨
15 files reformatted, 1 file left unchanged.

Check for merge conflicts................................................Passed
Check docstring is first.................................................Passed
Check JSON...............................................................Passed
Check Yaml...............................................................Passed
Debug Statements (Python)................................................Passed
Fix double quoted strings................................................Failed
- hook id: double-quote-string-fixer
- exit code: 1
- files were modified by this hook

Fixing strings in <file_name>
...

Fix End of Files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing <file_name>
...

Tests should end in _test.py.............................................Passed
Fix requirements.txt.................................(no files to check)Skipped
Trim Trailing Whitespace.................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

<file_name>: <some flake8 error>
...

Reorder python imports...................................................Passed
bandit...................................................................Passed
run tests................................................................Failed
- hook id: tests
- files were modified by this hook

============================= test session starts ==============================
platform darwin -- Python 3.6.9, pytest-5.4.1, py-1.8.1, pluggy-0.13.1

<test details>

(0.00 durations hidden.  Use -vv to show these durations.)
====================== 2 passed, 113 deselected in 2.51s =======================

I'm not sure what I'm doing wrong; git doesn't seem to have updated my commits with blacks formatting. I couldn't find anything through my Google research. Thanks!

Unhallowed

Hmm well you have to add files before commiting them. What does git status --short say? – Sejant

It's generally unwise to let any pre-commit hook try to modify what's to be committed. The pre-commit package program you're using has special code in it to try to handle this, but there are situations where it's too hard (git commit --only) and it will go wrong. It's better to just have a pre-commit hook verify that a commit is OK, or not, and if not, tell you what you should do to fix it, so as to avoid these corner cases. – Dressy

That said, it's not clear why you're seeing what you are seeing—but I haven't used this pre-commit package myself. Perhaps the reformatting simply hasn't actually occurred in your work-tree. (The package carefully extracts staged files to a temporary directory, which is the only sane way to deal with the fact that Git commits from its index, not from your work-tree.) – Dressy

could you provide more information and the output of the commit? I'm guessing you have two hooks which disagree on how something is formatted (isort and black maybe?) – Coupling

Hi @MCI, I have added requested details to the question. – Unhallowed

Hi @torek, I have added requested details to the question. – Unhallowed

Hi @AnthonySottile, I have added requested details to the question. – Unhallowed

Answer 1

It appears you're using black and double-quote-strings-fixer together

• the former likes double quoted strings in python (you can disable this by configuring black to skip-string-normalization in pyproject.toml)
• the latter likes single quoted strings in python (you can remove it if you'd like double quoted strings)

If two formatters fight, the end result will be a failure as pre-commit checks to make sure everything resolves

---

disclaimer: I'm the author of pre-commit and pre-commit-hooks

Coupling

Pre-commit (the program, not the Git hook :-) —the name is confusing this way!) is pretty clever about this, auto-detecting files that changed. I'd suggest perhaps making it more verbose in this particular case: make it say something along the lines of "formatters X and Y both tried to change <file> and we can't decide what to do about that", for instance. – Dressy

Beautiful! Yes that was the issue, I misconstrued "double-quote-strings-fixer" as enforcing double-quotes since that's what I try to use mostly. I appreciate your response @ AnthonySottile – Unhallowed

@Dressy yeah that would be nice, but unfortunately it's difficult to know what changed, and that two things disagree -- I've been looking at improving this but currently it just treats the differences opaquely – Coupling