How to log uncaught exceptions in Flask routes with logging?
Asked Answered
S

4

5

What is the standard way to log uncaught expressions in Flask routes with logging?

This nearly works:

import logging, sys, flask
logging.basicConfig(filename='test.log', filemode='a', format='%(asctime)s %(levelname)s %(message)s')
sys.excepthook = lambda exctype, value, tb: logging.error("", exc_info=(exctype, value, tb))
logging.warning("hello")
app = flask.Flask('hello')
@app.route('/')
def index():
    sjkfq          # uncaught expresion
    return "hello"
app.run()

but there are some ANSI escape characters in the log: [31m[1m etc. (probably for console colors)

See here in the produced test.log file:

2022-10-21 16:23:06,817 WARNING hello
2022-10-21 16:23:07,096 INFO [31m[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.[0m
 * Running on http://127.0.0.1:5000
2022-10-21 16:23:07,097 INFO [33mPress CTRL+C to quit[0m
2022-10-21 16:23:07,691 ERROR Exception on / [GET]
Traceback (most recent call last):
  File "C:\Python38\lib\site-packages\flask\app.py", line 2525, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Python38\lib\site-packages\flask\app.py", line 1822, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Python38\lib\site-packages\flask\app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Python38\lib\site-packages\flask\app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "D:\test.py", line 10, in index
    sjkfq
NameError: name 'sjkfq' is not defined
2022-10-21 16:23:07,694 INFO 127.0.0.1 - - [21/Oct/2022 16:23:07] "[35m[1mGET / HTTP/1.1[0m" 500 -

Note that this is totally reproducible if you run the same code.

Is there a documented way to do proper logging of uncaught exceptions in Flask routes, with Python logging? I didn't exactly find this in https://flask.palletsprojects.com/en/2.2.x/logging/.

(Note: this code shows it's not the right way to do logging. We shouldn't intercept a logging string, do some reverse engineering to clean the escape characters, and then log to a file. There surely is a a standard logging way.)

Seiler answered 21/10, 2022 at 14:27 Comment(5)
Those special characters are for print colors and other font tricksHertzfeld
I don't understand the "Not reproducible or was caused by a typo" flag. This MCVE is totally reproducible, not caused by a typo. The logging happens into a file test.log, and it's totally possible to see that it is not logged as expected (there is surely a better way with Flask).Seiler
@Hertzfeld Yes but in my reproducible MCVE, there is no stdin/stdout/stderr happening, and this should not happen. What is the right way to log uncaught exceptions into a .log file with Flask + logging module?Seiler
Not sure what exactly you're asking because the error is already being logged? Are you asking how to prevent the escape characters from being printed?Egression
@DMalan This code shows it's not the right way to do logging. We shouldn't intercept a logging string, do some reverse engineering to clean the escape characters, and then log to a file. There surely is a better logging way. What is the standard Flask way to log to a file the uncaught exceptions?Seiler
E
6

How to log uncaught exceptions in Flask routes with logging?

Flask is a popular web framework for Python that allows you to create web applications easily and quickly. However, sometimes your Flask routes may encounter uncaught exceptions that cause your application to crash or return an error response. To debug and fix these errors, you need to log them using the logging module.

Logging uncaught exceptions in Flask routes

The logging module is a standard library module that provides a flexible and powerful way to handle and record different levels of events, errors, and messages in your Python programs. You can use logging to configure different handlers, formatters, and levels for your log messages, and send them to different destinations, such as files, consoles, emails, or web services.

To log uncaught exceptions in Flask routes, you need to do two things:

  • Configure a logger object with a handler and a formatter that suit your needs
  • Register an error handler function that logs the exception information using the logger object

Configuring a logger object

To configure a logger object, you can use the logging.basicConfig() function, which sets up a default handler and formatter for the root logger. The root logger is the parent of all other loggers and it handles all the log messages that are not handled by any other logger. You can pass different parameters to the basicConfig() function, such as:

  • level: the minimum level of severity that the logger will handle, such as logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR, or logging.CRITICAL
  • filename: the name of the file where the log messages will be written
  • filemode: the mode of opening the file, such as 'a' for append or 'w' for write
  • format: the format string that specifies how the log messages will be displayed, such as '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
  • datefmt: the format string that specifies how the date and time will be displayed, such as '%Y-%m-%d %H:%M:%S'

For example, you can configure a logger object with a file handler and a simple formatter like this:

import logging

logging.basicConfig(level=logging.ERROR,
                    filename='app.log',
                    filemode='a',
                    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
                    datefmt='%Y-%m-%d %H:%M:%S')

This will create a file named app.log in the same directory as your Flask app, and write all the log messages with level ERROR or higher to it, using the specified format and date format.

Registering an error handler function

To register an error handler function, you can use the app.errorhandler() decorator, which takes an HTTP status code or an exception class as an argument, and wraps a function that handles the error. The function should take an exception object as a parameter, and return a response object or a tuple of (response, status code).

Inside the error handler function, you can use the logging.exception() method, which logs a message with level ERROR and adds the exception information to the log message. You can pass a custom message as an argument, or use the default message 'Exception occurred'.

For example, you can register an error handler function for the generic Exception class like this:

from flask import Flask, render_template

app = Flask(__name__)

@app.errorhandler(Exception)
def handle_exception(e):
    # log the exception
    logging.exception('Exception occurred')
    # return a custom error page or message
    return render_template('error.html'), 500

This will log any uncaught exception that occurs in your Flask routes, and return a custom error page with status code 500.

Example of logging uncaught exceptions in Flask routes

To demonstrate how logging uncaught exceptions in Flask routes works, let's create a simple Flask app that has two routes: one that returns a normal response, and one that raises a ZeroDivisionError. We will use the same logger configuration and error handler function as above.

from flask import Flask, render_template

app = Flask(__name__)

# configure the logger
logging.basicConfig(level=logging.ERROR,
                    filename='app.log',
                    filemode='a',
                    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
                    datefmt='%Y-%m-%d %H:%M:%S')

# register the error handler
@app.errorhandler(Exception)
def handle_exception(e):
    # log the exception
    logging.exception('Exception occurred')
    # return a custom error page or message
    return render_template('error.html'), 500

# define the normal route
@app.route('/')
def index():
    return 'Hello, world!'

# define the route that raises an exception
@app.route('/error')
def error():
    # this will cause a ZeroDivisionError
    x = 1 / 0
    return 'This will never be returned'

# run the app
if __name__ == '__main__':
    app.run(debug=True)

If we run this app and visit the / route, we will see the normal response 'Hello, world!'. However, if we visit the /error route, we will see the custom error page that says 'Something went wrong'. We can also check the app.log file and see the log message that contains the exception information, such as:

2021-07-07 12:34:56 - werkzeug - ERROR - Exception occurred
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/user/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/user/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/user/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/user/flask_app.py", line 31, in error
    x = 1 / 0
ZeroDivisionError: division by zero

This way, we can easily identify and debug the source of the error, and improve our Flask app accordingly. Logging uncaught exceptions in Flask routes is a good practice that can help you maintain and troubleshoot your web applications.

See also:

Flask - Handling Application Errors and Unhandled exceptions.

Engle answered 29/10, 2022 at 10:35 Comment(1)
Thanks @AhmedMohamed! Your answer + this answer to remove the ANSI escape characters [31m[1m etc., and it's perfect :)Seiler
R
2

I use a decorator to catch exceptions, this can also log the error:

from flask import abort
from functools import wraps


def catch_uncaught(function):
"""
catches uncaught exceptions and logs them
aborts the request
"""

    @wraps(function)
    def wrapper(*args, **kwargs):
        try:
            res = function(*args, **kwargs)
            return res  # no errors occurred
        except Exception as e:
            # log stack trace
            logging.error("\n" + traceback.format_exc() + "\n")
            # abort the request instead of crashing
            abort(500, "Internal Server error, do not retry")
    return wrapper

(This will also abort the request instead of crashing the server - but this is "optional". if you prefer to crash, return function(*args, **kwargs) after logging instead of abort)

you can then put this decorator on top of the routes you want to catch exceptions from to log them:

@app.route("/")
@catch_uncaught
def index():
    sjkfq  # uncaught exception
    return "hello"

This probably isn't the standard way, but it works quite well

Reggy answered 24/10, 2022 at 13:14 Comment(0)
R
1

So the problem is that those ANSII characters [31m[1m specify the colour of the message in terminal.

To my understanding Flask takes logs from werkzeug and adds some colouring.

As you can see from this issue, there is no way to disable colourings in flask logging by any simple flag in a set up (unfortunately)

What you could do is to add formatter that will get message and remove colour stylings before logging.

import logging, sys, flask
import click


class NoColorFormatter(logging.Formatter):
    def format(self, record: logging.LogRecord) -> str:
        return click.unstyle(super().format(record))


werkzeug_logger = logging.getLogger("werkzeug")
root_logger = logging.getLogger()

handler = logging.FileHandler("test.log")
handler.setFormatter(NoColorFormatter("%(asctime)s %(levelname)s %(message)s"))
werkzeug_logger.addHandler(handler)
root_logger.addHandler(handler)

sys.excepthook = lambda exctype, value, tb: logging.error(
    "", exc_info=(exctype, value, tb)
)

logging.warning("hello")

app = flask.Flask("hello")


@app.route("/")
def index():
    sjkfq  # uncaught expresion
    return "hello"


if __name__ == "__main__":
    app.run()

Repressive answered 24/10, 2022 at 9:45 Comment(0)
C
1

The problem is hidden in the werkzeug server under the flask hood. If we patch the variable werkzeug.serving._log_add_style then function _ansi_style will not change the color of any log entry.

import flask
import logging
import werkzeug.serving
werkzeug.serving._log_add_style = False
logging.basicConfig(filename='test.log', filemode='a', format='%(asctime)s %(levelname)s %(message)s')
logging.warning("hello")
app = flask.Flask("hello")
@app.route("/")
def index():
    sjkfq  # uncaught expression
    return "hello"
app.run()

and the log will look like this

2022-10-27 18:31:56,193 WARNING hello
2022-10-27 18:31:56,199 INFO WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
2022-10-27 18:31:56,199 INFO Press CTRL+C to quit
2022-10-27 18:32:01,920 ERROR Exception on / [GET]
Traceback (most recent call last):
  File "/Users/me/.env/test/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
  File "/Users/me/.env/test/lib/python3.8/site-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
  File "/Users/me/.env/test/lib/python3.8/site-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
  File "/Users/me/.env/test/lib/python3.8/site-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "flask_logging.py", line 10, in index
sjkfq  # uncaught expression
NameError: name 'sjkfq' is not defined
2022-10-27 18:32:01,922 INFO 127.0.0.1 - - [27/Oct/2022 18:32:01] "GET / HTTP/1.1" 500 -
Cause answered 27/10, 2022 at 14:38 Comment(1)
Great answer! Your answer + the accepted answer together (for the @app.errorhandler part) = perfect solution!Seiler

© 2022 - 2024 — McMap. All rights reserved.