werkzeug: disable bash colors when logging to file

Question

In a Flask application, I use a RotatingFileLogger to log werkzeug access logs to a file like shown in this question:

file_handler_access_log = RotatingFileHandler("access.log",
                                              backupCount=5,
                                              encoding='utf-8')
formatter = logging.Formatter('%(asctime)s %(module)s %(levelname)s: %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
file_handler_access_log.setFormatter(formatter)
werkzeug_logger.addHandler(file_handler_access_log)
werkzeug_logger.setLevel(logging.DEBUG)

In the access.log file, the request looks like this:

2020-10-07 09:43:51 _internal INFO: 127.0.0.1 - - [07/Oct/2020 09:43:51] "[37mGET /api/foo HTTP/1.1[0m" 200 -

I want to get rid of the color codes like [37m in the log file.

The werkzeug documentation states:

The development server can optionally highlight the request logs in different colors based on the status code. Install Click to enable this feature.

Click is a Flask dependency, so I cannot uninstall it. How can I disable the colored logging?

Euthanasia

Can you please share formatter definition? – Rellia

@DušanMaďar I added it in the code. The color codes appear in the %(message)s part. – Euthanasia

Answer 1

OK, so what you are hitting is

if click:
    color = click.style

    if code[0] == "1":  # 1xx - Informational
        msg = color(msg, bold=True)
    ...
self.log("info", '"%s" %s %s', msg, code, size)

Source: https://github.com/pallets/werkzeug/blob/ef545f0d0bf28cbad02066b4cb7471bea50a93ee/src/werkzeug/serving.py

Not easy to prevent the behavior. The second option is to remove color codes from messages. I would try to use log Filter to update the message, something like

import logging

import click
    

class RemoveColorFilter(logging.Filter):
    def filter(self, record):
        if record and record.msg and isinstance(record.msg, str):
            record.msg = click.unstyle(record.msg) 
        return True

remove_color_filter = RemoveColorFilter()
file_handler_access_log.addFilter(remove_color_filter)

The above suggestion was inspired by the following answer https://mcmap.net/q/1328482/-add-information-to-every-log-message-in-python-logging.

I didn't test the proposed solution.

Josiejosler

Some other libraries will log error objects instead of strings and therefore cause type errors. I added a type check to your answer – Euthanasia

Instead of the type check, you can also exclude other loggers with if record and record.name == 'werkzeug' – Euthanasia

Answer 2

This is a similar idea to @jhodges, but even simpler - it just removes all of the "click" module's styling altogether.

import click
click.style = lambda text, *args, **kwargs: text

EDIT: In the end, I wound up going with a filter to remove escape sequences from log messages headed for a file handler. Because werkzeug puts escape sequences in the args list for the log message, those need to be stripped as well. This is the basic outline:

class NoEscape(logging.Filter):
    def __init__(self):
        self.regex = re.compile(r'(\x9B|\x1B\[)[0-?]*[ -\/]*[@-~]')
    def strip_esc(self, s):
        try: # string-like 
            return self.regex.sub('',s)
        except: # non-string-like
            return s
    def filter(self, record: logging.LogRecord) -> int:
        record.msg = self.strip_esc(record.msg)
        if type(record.args) is tuple:
            record.args = tuple(map(self.strip_esc, record.args))
        return 1

Hope this helps!

Erymanthus

Actually I do not want to remove all of click's styling because I also log to console and I want to keep the colored output there and only remove the color codes from the logger that logs to a file. – Euthanasia

Sorry to revive a month-old thread, but in the end I would up in the same position as you, and just had to implement a filter to strip the escapes out. If you're still on this, see above. Hope this helps! – Erymanthus

Answer 3

I went with what i think is a more straightforward solution, which is to simply throw away all the styling arguments - something like this in your application startup

    old_color = click.style

    def new_color(text, fg=None, bg=None, bold=None, dim=None, underline=None, blink=None, reverse=None, reset=True):
        return old_color(text)

    # replace flask styling with non-colorized styling
    click.style = new_color

Rhyme

Thanks for sharing this approach. It worked for me with a minor fix, will post in a separate answer. – Shatter

Answer 4

Finally, I searched through the werkzeug source and came to this solution:

import werkzeug
werkzeug.serving._log_add_style = False

Put this after your imports in your app.py file. All the other suggestions either didn't work for me, or looked too hacky for my taste.

Facies

Agree, adding the control codes in then removing them seems completely the wrong way around. – Prosody

Answer 5

I came across this question myself, but the filter solutions presented in other answers modified the record for all cases it was emitted, whereas I wanted to only remove styling for file handlers and leave it for console handlers.

I ended up subclassing Formatter instead of Filter:

import logging
import click

class AntiColorFormatter(logging.Formatter):
    def format(self, record: logging.LogRecord) -> str:
        return click.unstyle(super().format(record))

which then lends itself to something like:

logger = logging.getLogger()
FMT = '{asctime} {levelname} {message}'

handler = logging.StreamHandler()
# regular console output gets normal styling
handler.setFormatter(logging.Formatter(FMT, style='{'))
logger.addHandler(handler)

handler = logging.FileHandler('log.log', encoding='utf8')
# file handler gets styling stripped
handler.setFormatter(AntiColorFormatter(FMT, style='{'))
logger.addHandler(handler)

Rett

Answer 6

Thanks to @jhodges answer, that worked for me with a minor change:

# workaround to suppress color logging in werkzeug
try:
    import click
    old_color = click.style

    def _color(text, fg=None, bg=None, bold=None, dim=None, underline=None, blink=None, reverse=None, reset=True):
        return click.unstyle(text)

    click.style = _color
except:
    pass

Calling click.style(text) still produces escape sequences in some cases which I would like to avoid completely, so I used unstyle that works perfectly here.

Shatter