Pre-commit flake8 with setup.cfg in subfolder

Question

I use flake8 with a bunch of plugins (flake8-docstrings, flake8-isort, flake8-black). I have them all pre-installed into a venv.

My repo to be checked with pre-commit:

• Root folder has two packages
• Each has its own
  • pyproject.toml (configures black and isort)
  • setup.cfg (configures flake8 and pydocstyle)

├── foo
│   ├── pyproject.toml
│   ├── setup.cfg
│   └── (the package)
├── bar
│   ├── pyproject.toml
│   ├── setup.cfg
│   └── (the package)
└── venv

I want to invoke flake8 via pre-commit for the two packages.

Here's how I do it currently:

---
repos:
  - repo: local
    hooks:
      - id: flake8-foo
        name: Run flake8 in foo package
        entry: bash -c "cd foo && flake8"
        language: python
      - id: flake8-bar
        name: Run flake8 in bar package
        entry: bash -c "cd bar && flake8"
        language: python

When I run pre-commit run --all-files and there's an error in foo, it prints the same output many times:

./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood
./path/in/foo/to/file.py:49:1: D401 First line should be in imperative mood

1. Is there a better way to go about this?
   • No, I am not open to splitting up the packages into their own repos
2. How can I have the error message only print once?

Cassiterite

Answer 1

pre-commit by design operates on files, it also is optimized to batch runs of linters against files into multiple processes

what's happening here is your configuration is running several invocations (~1 per processor) of bash -c "cd bar && flake8" file1 file2 file3 etc. etc.

fortunately there's a setting you can use to fix this for you:

• pass_filenames: false

with that:

---
repos:
  - repo: local
    hooks:
      - id: flake8-foo
        name: Run flake8 in foo package
        entry: bash -c "cd foo && flake8"
        language: python
        pass_filenames: false
        files: ^foo/
        types: [python]
      - id: flake8-bar
        name: Run flake8 in bar package
        entry: bash -c "cd bar && flake8"
        language: python
        pass_filenames: false
        files: ^bar/
        types: [python]

that said, you're losing most of the benefits of the framework by going to a repo: local hook:

• pre-commit isn't managing the installation of the tools (each of your developers has to install the tool separately and at a particular version)
• any filename-based optimizations aren't happening
  • if you only change one file, you're currently linting your whole repository twice
  • during merge conflicts, pre-commit optimizes which files to run (not the whole repo)
  • and more

what I'd suggest instead for your monorepo setup is to still call flake8 in the normal way but utilize --config such that it works against your sub-repos:

repos:
-   repo: https://gitlab.com/pycqa/flake8
    rev: 3.8.4
    hooks:
    -   id: flake8
        name: flake8 ./foo/
        alias: flake8-foo
        files: ^foo/
        args: [--config, foo/setup.cfg]
    -   id: flake8
        name: flake8 ./bar/
        alias: flake8-bar
        files: ^bar/
        args: [--config, bar/setup.cfg]

---

disclaimer: I'm the author of pre-commit and the current maintainer of flake8

Atrophy

Thank you for both directly answering my question and providing a better overall solution @AnthonySottile! You're doing great things for the software community – Cassiterite