How to call a function when Apache terminates a Flask process?

Question

I have a Flask app running behind Apache HTTPD. Apache is configured to have multiple child processes.

The Flask app creates a file on the server with the file's name equal to its process ID. The code looks something like this:

import os

@app.before_first_request
def before_first_request():
    filename = os.getpid()
    with open(filename, 'w') as file:
        file.write('Hello')

When the child process is killed/ended/terminated I would like the Flask app to remove this file.

It's not terribly important that removal of a file happens, as these files won't take up much space, so if bizarre errors occur I don't need to handle them. But for normal workflow, I would like to have some cleanup when Apache shuts down the Flask process.

Any idea on the best way to do this?

Stagger

Answer 1

The best way to add cleanup functionality before graceful termination of a server-controlled Python process (such as a Flask app running in Apache WSGI context, or even better, in Gunicorn behind Apache) is using the atexit exit handler.

Elaborating on your original example, here's the addition of the exit handler doing the cleanup of .pid file:

import atexit
import os

filename = '{}.pid'.format(os.getpid())

@app.before_first_request
def before_first_request():
    with open(filename, 'w') as file:
        file.write('Hello')

def cleanup():
    try:
        os.remove(filename)
    except Exception:
        pass

atexit.register(cleanup)

Loadstone

Apache/mod_wsgi certainly always tries to ensure that atexit callbacks are invoked, even in cases where the process is having to be shutdown due to conditions such as request timeouts etc. In gunicorn there aren't as good a guarantees and you would have to be much more careful about using atexit under gunicorn as various cases it wouldn't be called. In either case you shouldn't rely on it and an application should be resilient to a file existing on a subsequent run if it shouldn't exist. A separate task to remove that might required as other answer describes. – Katowice

For some other WSGI servers atexit callbacks may not even be called at all. I believe this is still the case with uWSGI unless you force it to use a single interpreter, as it doesn't take steps to ensure atexit callbacks are called in sub interpreters. It would have to take special steps as Python doesn't call atexit callbacks in sub interpreters and Apache/mod_wsgi is using special tricks to ensure they are. – Katowice

great info @GrahamDumpleton! Thanks for that, and the exception guard around os.remove call. – Loadstone

Answer 2

The simplest way would be to handle this outside of the Apache process. There's no guaranteed way that the process will always remove the files (for example, if you restart the apache server mid-request).

The approach I've taken in the past is to use cron. Write a small script somewhere in your repository and have it executed on a schedule (daily usually works). This script can just clear out all files in the directorty that are older than 24 hours, so you'll always have a rolling window of 1 days worth of files.

This has two benefits:

1. You're in total control of the script, and can use any language you want.
2. You can do fancy stuff with these files. You can compress them and store them elsewhere, delete them, or perform analytics on them. Entirely up to you!

Most scripting languages have a small wrapper class that can be used to make cron more friendly. A popular one for Ruby is whenever.

Sabec