Black not respecting extend-exclude in pyproject.toml

Question

In VSCode, with Python 3.9 and black==22.6.0, I have a project structure like:

--- root
------src
---------module0.py
---------module1.py
------tests
---------test_folder0
------------test_file0.py
------------test_file1.py
---------test_folder1
---------etc.

In pyproject.toml I can't get the extend-exlude part to actually exclude my test files. I've tried multiple different ways, both for the entire tests folder as well as for the test_whatever.py files but nothing seems to work, even though my various attempts have been validated by https://regex101.com/. The simplest example:

[tool.black]
line-length = 200
target-version = ['py39']
include = '\.pyi?$'
extend-exclude = '''.*test_.*'''

Either my regex is wrong (or black requires some modifications) or VSCode is ignoring my project's configuration or idk.

Barbarbarbara

How are you asking VSCode to format your codebase? If you're using the "Format Document" hotkey (<Ctrl-Shift-I> IIRC) then Black will ignore all file collection options like --extend-exclude because VSCode will call Black with the filepath given directly. If you take a look at the "Output" tab and then run "Format Document", you'll notice this, for example here's what happens when I do this: ./venv/bin/python -m black --safe --diff --quiet ./tests/data/nothing-changed.py – Lofty

Hi @ichard26, I have VSCode set to "Edit: Format on Save" so I'm not actually running "Format Document". And of course "Python > Formatting: Provider" is set to "black". – Barbarbarbara

ah alright. It's the same underlying issue though with "Format on Save", VSCode passes the file to Black directly an as argument so --extend-exclude won't ever affect it. Your only option is to use --force-exclude which is like --extend-exclude but is always applied. Your only way of formatting a file that's force excluded is to pipe in its contents to black - and save STDOUT to the same file. – Lofty

Ok so from what I'm understanding then I can't just use force-exclude in my pyproject.toml and be done with it? I'm not sure if I should create a little script that pipes the content and saves to STDOUT, but this just kind of feels like a shortcoming of black no? Anyway thanks @ichard26, I feel like you should copy your comment as an answer so I can consider this question "resolved". – Barbarbarbara

Answer 1

Maintainer of Black here :wave:

OK, so I actually missed a few points in my comments. To address the main question, this is 100% expected behaviour. Your regex is fine.

The thing is that when you ask VSCode to format your file on save, it calls Black passing the filepath to your current file (you just saved) directly. If you open the "Output" tab in the bottom panel and then save a Python file, you'll notice something like this:

./venv/bin/python -m black --safe --diff --quiet ./tests/data/nothing-changed.py

--include, --exclude, and --extend-exclude only affect files and directories that are discovered by Black itself. You might be wondering, huh, Black can look for files to format? Yes, it does when you run black . or give Black any other directory. The flipside of this is that these options do nothing to files given as an argument to Black.

If you want to keep Format on Save enabled, your only recourse is to use --force-exclude which is similar to --extend-exclude but it's always enforced. You can either configure --force-exclude in pyproject.toml or via VSCode's Black arguments setting (preferably for the current workspace only).

The difference between putting it in pyproject.toml and configuring VSCode to pass extra options to Black is well, when it's applied. If it's in pyproject.toml it will always be enforced, even when you're not using VSCode and instead are using a bash shell or whatever. This can be useful if you're using pre-commit (which passes files to Black directly just like VSCode) or similar, but can be annoying otherwise.

(if you do choose to force exclude project-wide via pyproject.toml, you can format force-excluded files by either piping it in or temporarily clearing --force-exclude on the CLI, e.g. black --force-exclude='' file_you_want_to_format_even_though_it_is_force_excluded.py)

Lofty

Thanks for the help and for maintaining Black. I can confirm that with my above pyproject.toml snippet but changing extend-exclude to force-exclude, my project's code gets formatted but my tests remain unformatted, as hoped. (Honestly thought I tried it as soon as you mentioned it but I guess I was doing something different). Thanks again! – Barbarbarbara