Python3.12 SyntaxWarning on triplequoted string `\d` must be `\\d`

Question

After updating to Python 3.12, I get warnings about invalid escape sequence on some triple-quotes comments.

Is this a new restriction? I have the habit of documenting code using triple-quoted string, but this has never been a problem prior to Python 3.12.

python3 --version
Python 3.12.0
$ ./some_script.py
/some_script.py:123: SyntaxWarning: invalid escape sequence '\d'
  """

I tried replacing all lines with \d:

20230808122708.445|INFO|C:\dist\work\trk-fullstack-test\namespaces.py

with \\d:

20230808122708.445|INFO|C:\\dist\work\trk-fullstack-test\namespaces.py

The warning disappears.

Suppressing the warning do not seem to work:

import warnings
warnings.filterwarnings('ignore', category=SyntaxWarning)

Any pointers on how to do this correctly? I hope I do not have to escape all Windows paths documented in triplequotes in our code.

Bettyannbettye

Answer 1

Back in Python 3.6, using invalid escape sequences in string literals was deprecated (bpo-27364). Since then, attempting to use an invalid escape sequence has emitted a DeprecationWarning. This can often go unnoticed if you don't run Python with warnings enabled. DeprecationWarnings are silenced by default.

Python 3.12 upgraded the DeprecationWarning to a SyntaxWarning. SyntaxWarnings are emitted by the compiler when the code is parsed, not when it's being run, so they cannot be ignored using a runtime warning filter. Unlike DeprecationWarnings, SyntaxWarnings are displayed by default, which is why you're seeing it now. This increase in visibility was intentional. In a future version of Python, using invalid escape sequence in string literals is planned to eventually become a hard SyntaxError.

The simplest solution would be to use comments for comments instead of string literals. Unlike string literals, comments aren't required to be syntactically valid. See also the discussion in Python comments: # vs. strings for more on the drawbacks of using string literals as comments.

To address this warning in general, you can make the string literal a raw string literal r"...". Raw string literals do not process escape sequences. For example, the string "\n" contains a single newline character, whereas the string r"\n" contains the two characters \ and n.

Small

It would appear no longer valid then to put raw-string regular expressions into docstrings, which is a bummer. For sure I can replace re.compile(r"\d+") with re.compile("\\d+") but certainly the problem isn't so much for trivial cases. – Madrid

@Madrid I'm not fully sure what you mean. If you're worried escape sequences being expanded inside a docstring, you can use a raw string literal for the docstring. – Small

Horror. Now I'm getting a bunch of messages because of used modules that use such sequences. What am I supposed to do, manually send them corrections? For each module? – Orsa

Answer 2

You can encode your regex string under r'' in Python 3.12.3. It works for me.

In 3.11.4 before : '(?:\d{4}(-)?\d{3}(-)?[8]\d{2}(-)?\d{2})'

In 3.12.3 after : r'(?:\d{4}(-)?\d{3}(-)?[8]\d{2}(-)?\d{2})'

Heterolecithal