Why does using threading.Event result in SIGTERM not being caught?

Question

I have a threaded Python daemon. Like any good daemon, it wants to launch all of its worker threads, then wait around until it's told to terminate. The normal signal for termination is SIGTERM, and in most languages I'd hold to terminate by waiting on an event or mutex, so using threading.Event made sense to me. The problem is that Python's Event object and Unix signals don't appear to be playing well together.

This works as expected, terminating on SIGTERM:

import signal
import time

RUN = True

def handle(a, b):
    global RUN
    print "handled"
    RUN = False

signal.signal(signal.SIGTERM, handle)
while RUN:
    time.sleep(0.250)
print "Stopping"

but this results in no SIGTERM being delivered (i.e., quite apart from quitting, "handled" never gets printed):

import signal
import threading

RUN_EVENT = threading.Event()

def handle(a, b):
    print "handled"
    RUN_EVENT.set()

signal.signal(signal.SIGTERM, handle)
RUN_EVENT.wait()
print "Stopping"

So my question is:

1. Am I misusing threading.Event in some way?
2. If I am not, is there an alternative other than the poll-and-sleep mechanism from the first example?
3. Also if I am not, why does using threading.Event kill the signal handler?

Cryogen

Answer 1

From Python documentation on signals:

Although Python signal handlers are called asynchronously as far as the Python user is concerned, they can only occur between the “atomic” instructions of the Python interpreter. This means that signals arriving during long calculations implemented purely in C (such as regular expression matches on large bodies of text) may be delayed for an arbitrary amount of time.

I tested various threading and thread classes and none of them work the way you want it -- this is probably because of how Python handles signals.

In signal, however, there is a pause() function that sleeps until a signal is received by the process. Your modified example would look like this:

import signal

RUN = True

def handle(a, b):
    global RUN
    print "handled"
    RUN = False

signal.signal(signal.SIGTERM, handle)
signal.signal(signal.SIGINT, handle)
signal.signal(signal.SIGHUP, handle)

while RUN:
    signal.pause()

print "Stopping"

I checked it on Linux, it works. I don't think it classifies as poll-and-sleep anymore if your application doesn't use a lot of other signals.

It's good to handle SIGINT and SIGHUP too, to properly handle user interruption (usually by pressing Ctrl+C) and user disconnection (closing the parent terminal) respectively.

Also, note that signal.pause() is not available on Windows systems.

Stephan

Annoying feature of Python, but perfect solution. Thank you. I'll confess it didn't occur to me to look for additional restrictions on signal use, since I knew that the equivalent C would function just fine. – Cryogen

Does this mean the only way to handle signals is to dedicate the main thread (parent) to catching signals (by blocking on 'signal.pause()')? What that implies is that the main thread can't do anything useful anymore. In other words, you can't have a master/worker model (where the 2 threads talk to each other) but you need a master/worker+worker model (where the 2 workers talk to each other and master does nothing. – Sketchbook

Answer 2

In Python 3, this works:

from threading import Event
from os import getpid
from signal import SIGTERM, SIGINT, SIGHUP, signal

stop_event = Event()

def handler(signum, frame):
    stop_event.set()

def main():
    signal(SIGTERM, handler)
    signal(SIGINT, handler)
    signal(SIGHUP, handler)
    print('start, pid:', getpid())
    stop_event.wait()
    print('done')

if __name__ == '__main__':
    main()

It looks that in Python 2.7 it works only if you specify wait interval: stop_event.wait(number_of_seconds)

Imperfect