How to have a single source of truth for poetry and pre-commit package version?

Question

I'm looking into this Python project template. They use poetry to define dev dependencies

[tool.poetry.dev-dependencies]
black = {version = "*", allow-prereleases = true}
flake8 = "*"
isort = "^5.6"
mypy = ">0.900,<1"
...

They use also pre-commit to check housekeeping things (e.g., formatting, linting, issorting, ...), both for git and for CI workflows:

minimum_pre_commit_version: 2.8.0
default_stages: [commit, push, manual]
repos:
  - repo: https://github.com/psf/black
    rev: 21.11b1
    hooks:
      - id: black
  - repo: https://github.com/pycqa/flake8
    rev: 4.0.1
    hooks:
      - id: flake8
        args: [--max-line-length=88]
  - repo: https://github.com/pycqa/isort
    rev: 5.10.1
    hooks:
      - id: isort
        args: [--filter-files]
  - ...

In my case, I definitely want a local version of dev packages managed by poetry for my IDE, and I'd like also to harness the pre-commit framework "as is", without switching to language: system. Working this way, I need to manage each package version in two different places.

Is there a non-manual way to keep dev packages versions (i.e,. black, flake8, isort, mypy, ...) aligned to a single source of truth? A Coockiecutter template could be an option, but it looks overkilling.

Bilbrey

at some point you'll need the information in two places -- the template has versions also in poetry.lock so you'd need some code to synchronize the two – Cronus

Yep, true. An option could be using e metatemplate to sync them all. Thanks. – Bilbrey

Finally, I created a pre-commit hook to do the job. See my answer here below – Bilbrey

Answer 1

Finally, I created a pre-commit hook to do the job: https://github.com/floatingpurr/sync_with_poetry

Edit: If you use PDM, you can refer to this one: https://github.com/floatingpurr/sync_with_pdm

This hook just keeps in sync the repos rev in .pre-commit-config.yaml with the packages version locked into poetry.lock.

If you use:

• Poetry for dependency management
• local dev packages for your IDE (via Poetry)
• pre-commit hooks

this meta-hook can be useful to (un-)bump repos rev for you.

Bilbrey

I made a tool to do the same job, but with a different approach. Instead of being a pre-commit hook, it's distributed as a plugin for Poetry or PDM. It's heavily inspired by your work, thanks for the inspiration @floatingpurr! github.com/GabDug/sync-pre-commit-lock – Spector

Hi @DuGabiche, that's cool! Thank you for sharing this approach – Bilbrey

Answer 2

I would recommend keeping the linter stuff only in the config of pre-commit.

pre-commit doesn't necessarily run as a pre-commit hook. You can run the checks every time by pre-commit run --all-files or if you want to run it only on given files with pre-commit run --files path/to/file.

You can even say which which check should run, e.g. pre-commit run black --all-files

Freeload

I see, but I need another copy of them for live interaction provided by the IDE (e.g., VS Code) – Bilbrey

Not sure how it works in VS Code. Because you can run a specific pre-commit check on a specific file (pre-commit run <check> --files <filepath>) it is no problem in PyCharm to run it as in "External Tool". I guess VS Code has something similar. – Freeload

Edited comment: thanx for your suggestion. Your guess is right. VS Code needs to point at each linter / formatter path that is then passed to the the specified python interpreter. E.g., "python.formatting.blackPath":"black". I've not found a way to pass in the config pre-commit run black --files <filepath>. This seems something that does not play nicely with the way VS Code works. – Bilbrey

Thanks for the suggestion @finswimmer! – Bilbrey

Answer 3

I am not sure if this is considered a good practice, solved it by just calling the local, poetry managed version of (e.g.) black like this:

repos:
 - repo: local
   hooks:
    - id: black
      name: black
      language: system
      entry: poetry
      args: [ "run", "black", "--config", "black.toml", "--check" ]
      files: [src/]
      types: [python]

Edit: I learned that the config is much more complex than just calling the script because pre-commit hook passes the files itself to the program. This could be made easier by using pass_filenames: false but I think that will cost you in terms of parallelity and, hence, speed.

Tartu