Ensuring subprocesses are dead on exiting Python program

Question

Is there a way to ensure all created subprocess are dead at exit time of a Python program? By subprocess I mean those created with subprocess.Popen().

If not, should I iterate over all of the issuing kills and then kills -9? anything cleaner?

Bruell

related: How to terminate a python subprocess launched with shell=True – Pentecostal

related: Python: how to kill child process(es) when parent dies? – Pentecostal

Answer 1

You can use atexit for this, and register any clean up tasks to be run when your program exits.

atexit.register(func[, *args[, **kargs]])

In your cleanup process, you can also implement your own wait, and kill it when a your desired timeout occurs.

>>> import atexit
>>> import sys
>>> import time
>>> 
>>> 
>>>
>>> def cleanup():
...     timeout_sec = 5
...     for p in all_processes: # list of your processes
...         p_sec = 0
...         for second in range(timeout_sec):
...             if p.poll() == None:
...                 time.sleep(1)
...                 p_sec += 1
...         if p_sec >= timeout_sec:
...             p.kill() # supported from python 2.6
...     print 'cleaned up!'
...
>>>
>>> atexit.register(cleanup)
>>>
>>> sys.exit()
cleaned up!

Note -- Registered functions won't be run if this process (parent process) is killed.

The following windows method is no longer needed for python >= 2.6

Here's a way to kill a process in windows. Your Popen object has a pid attribute, so you can just call it by success = win_kill(p.pid) (Needs pywin32 installed):

    def win_kill(pid):
        '''kill a process by specified PID in windows'''
        import win32api
        import win32con

        hProc = None
        try:
            hProc = win32api.OpenProcess(win32con.PROCESS_TERMINATE, 0, pid)
            win32api.TerminateProcess(hProc, 0)
        except Exception:
            return False
        finally:
            if hProc != None:
                hProc.Close()

        return True

Premonish

Can you explain a little what you are doing in windows code. – Windhoek

Why is the 'win_kill' needed, given that p.kill() exists? Is it for pre-2.6 python users? – Klara

Yes, I believe at the time, 2.5 was still in wide usage, and p.kill() was not available in windows. – Premonish

This answer could be simplified for Python 3.3+, where Popen.wait() accepts a timeout parameter. First loop through a call p.teminate() on the subprocesses (requesting graceful shutdown). Then call wait() on each process with the timeout set, and call p.kill() if the timeout expiry exception is thrown. – Hagen

Answer 2

On *nix's, maybe using process groups can help you out - you can catch subprocesses spawned by your subprocesses as well.

if __name__ == "__main__":
  os.setpgrp() # create new process group, become its leader
  try:
    # some code
  finally:
    os.killpg(0, signal.SIGKILL) # kill all processes in my group

Another consideration is to escalate the signals: from SIGTERM (default signal for kill) to SIGKILL (a.k.a kill -9). Wait a short while between the signals to give the process a chance to exit cleanly before you kill -9 it.

Repudiate

It looks like this strategy includes the parent process in the process group. Therefore when you send SIGTERM to the process group the parent also gets it. This may be undesirable (as it was in my flow). – Deadline

Answer 3

The subprocess.Popen.wait() is the only way to assure that they're dead. Indeed, POSIX OS's require that you wait on your children. Many *nix's will create a "zombie" process: a dead child for which the parent didn't wait.

If the child is reasonably well-written, it terminates. Often, children read from PIPE's. Closing the input is a big hint to the child that it should close up shop and exit.

If the child has bugs and doesn't terminate, you may have to kill it. You should fix this bug.

If the child is a "serve-forever" loop, and is not designed to terminate, you should either kill it or provide some input or message which will force it to terminate.

---

Edit.

In standard OS's, you have os.kill( PID, 9 ). Kill -9 is harsh, BTW. If you can kill them with SIGABRT (6?) or SIGTERM (15) that's more polite.

In Windows OS, you don't have an os.kill that works. Look at this ActiveState Recipe for terminating a process in Windows.

We have child processes that are WSGI servers. To terminate them we do a GET on a special URL; this causes the child to clean up and exit.

Mcnally

Answer 4

Find out a solution for linux (without installing prctl):

def _set_pdeathsig(sig=signal.SIGTERM):
    """help function to ensure once parent process exits, its childrent processes will automatically die
    """
    def callable():
        libc = ctypes.CDLL("libc.so.6")
        return libc.prctl(1, sig)
    return callable


subprocess.Popen(your_command, preexec_fn=_set_pdeathsig(signal.SIGTERM)) 

Saberio

see also pyprctl and Add support for making Linux prctl(...) calls to subprocess. limitation: "PDEATHSIG has an infamous footgun where the the signal is sent on exit of the forking thread, which is not necessarily the exit of the invoking process." – Acie

Answer 5

Warning: Linux-only! You can make your child receive a signal when its parent dies.

First install python-prctl==1.5.0 then change your parent code to launch your child processes as follows

subprocess.Popen(["sleep", "100"], preexec_fn=lambda: prctl.set_pdeathsig(signal.SIGKILL))

What this says is:

• launch subprocess: sleep 100
• after forking and before exec of the subprocess, the child registers for "send me a SIGKILL when my parent terminates".

Doradorado

Another option is pyprctl, which doesn't compile a C extension (don't need compiler and libcap-dev) and is more openly licensed. Works the same way. – Everywhere

Answer 6

orip's answer is helpful but has the downside that it kills your process and returns an error code your parent. I avoided that like this:

class CleanChildProcesses:
  def __enter__(self):
    os.setpgrp() # create new process group, become its leader
  def __exit__(self, type, value, traceback):
    try:
      os.killpg(0, signal.SIGINT) # kill all processes in my group
    except KeyboardInterrupt:
      # SIGINT is delievered to this process as well as the child processes.
      # Ignore it so that the existing exception, if any, is returned. This
      # leaves us with a clean exit code if there was no exception.
      pass

And then:

  with CleanChildProcesses():
    # Do your work here

Of course you can do this with try/except/finally but you have to handle the exceptional and non-exceptional cases separately.

Filariasis

Doesn't this disable control-c? – Isthmian

Answer 7

I needed a small variation of this problem (cleaning up subprocesses, but without exiting the Python program itself), and since it's not mentioned here among the other answers:

p=subprocess.Popen(your_command, preexec_fn=os.setsid)
os.killpg(os.getpgid(p.pid), 15)

setsid will run the program in a new session, thus assigning a new process group to it and its children. calling os.killpg on it thus won't bring down your own python process also.

Jessie

No, you cannot... that will change the session of the process itself, if what you're after is killing only the children it isn't what you want – Jessie

I've read the question, have you read my answer? I explicitly wrote that I needed a small variation of this problem... if you google for " subprocess exit children" this will be the first result you'll find. In fact, being able to kill the children without exiting is a more general problem than simply doing it when exiting, thus it will be useful to other people stumbling upon the same problem. – Jessie

The stackoverflow faq says: What, specifically, is the question asking for? Make sure your answer provides that – or a viable alternative. I've provided an alternative and an useful answer, in the meanwhile you just downvoted both the answers I posted. If you'd stop harassing people who try to be helpful to stackoverflow users, I'm sure that the whole community would welcome that. – Jessie

Answer 8

poll( )

Check if child process has terminated. Returns returncode attribute.

Oversight

Answer 9

A solution for windows may be to use the win32 job api e.g. How do I automatically destroy child processes in Windows?

Here's an existing python implementation

https://gist.github.com/ubershmekel/119697afba2eaecc6330

Bad

Answer 10

Is there a way to ensure all created subprocess are dead at exit time of a Python program? By subprocess I mean those created with subprocess.Popen().

You could violate encapsulation and test that all Popen processes have terminated by doing

subprocess._cleanup()
print subprocess._active == []

If not, should I iterate over all of the issuing kills and then kills -9? anything cleaner?

You cannot ensure that all subprocesses are dead without going out and killing every survivor. But if you have this problem, it is probably because you have a deeper design problem.

Airscrew

Answer 11

I actually needed to do this, but it involved running remote commands. We wanted to be able to stop the processes by closing the connection to the server. Also, if, for example, you are running in the python repl, you can select to run as foreground if you want to be able to use Ctrl-C to exit.

import os, signal, time

class CleanChildProcesses:
    """
    with CleanChildProcesses():
        Do work here
    """
    def __init__(self, time_to_die=5, foreground=False):
        self.time_to_die = time_to_die  # how long to give children to die before SIGKILL
        self.foreground = foreground  # If user wants to receive Ctrl-C
        self.is_foreground = False
        self.SIGNALS = (signal.SIGHUP, signal.SIGTERM, signal.SIGABRT, signal.SIGALRM, signal.SIGPIPE)
        self.is_stopped = True  # only call stop once (catch signal xor exiting 'with')

    def _run_as_foreground(self):
        if not self.foreground:
            return False
        try:
            fd = os.open(os.ctermid(), os.O_RDWR)
        except OSError:
            # Happens if process not run from terminal (tty, pty)
            return False

        os.close(fd)
        return True

    def _signal_hdlr(self, sig, framte):
        self.__exit__(None, None, None)

    def start(self):
        self.is_stopped = False
        """
        When running out of remote shell, SIGHUP is only sent to the session
        leader normally, the remote shell, so we need to make sure we are sent 
        SIGHUP. This also allows us not to kill ourselves with SIGKILL.
        - A process group is called orphaned when the parent of every member is 
            either in the process group or outside the session. In particular, 
            the process group of the session leader is always orphaned.
        - If termination of a process causes a process group to become orphaned, 
            and some member is stopped, then all are sent first SIGHUP and then 
            SIGCONT.
        consider: prctl.set_pdeathsig(signal.SIGTERM)
        """
        self.childpid = os.fork()  # return 0 in the child branch, and the childpid in the parent branch
        if self.childpid == 0:
            try:
                os.setpgrp()  # create new process group, become its leader
                os.kill(os.getpid(), signal.SIGSTOP)  # child fork stops itself
            finally:
                os._exit(0)  # shut down without going to __exit__

        os.waitpid(self.childpid, os.WUNTRACED)  # wait until child stopped after it created the process group
        os.setpgid(0, self.childpid)  # join child's group

        if self._run_as_foreground():
            hdlr = signal.signal(signal.SIGTTOU, signal.SIG_IGN)  # ignore since would cause this process to stop
            self.controlling_terminal = os.open(os.ctermid(), os.O_RDWR)
            self.orig_fore_pg = os.tcgetpgrp(self.controlling_terminal)  # sends SIGTTOU to this process
            os.tcsetpgrp(self.controlling_terminal, self.childpid)
            signal.signal(signal.SIGTTOU, hdlr)
            self.is_foreground = True

        self.exit_signals = dict((s, signal.signal(s, self._signal_hdlr))
                                 for s in self.SIGNALS)                                     

    def stop(self):
        try:
            for s in self.SIGNALS:
                #don't get interrupted while cleaning everything up
                signal.signal(s, signal.SIG_IGN)

            self.is_stopped = True

            if self.is_foreground:
                os.tcsetpgrp(self.controlling_terminal, self.orig_fore_pg)
                os.close(self.controlling_terminal)
                self.is_foreground = False

            try:
                os.kill(self.childpid, signal.SIGCONT)
            except OSError:
                """
                can occur if process finished and one of:
                - was reaped by another process
                - if parent explicitly ignored SIGCHLD
                    signal.signal(signal.SIGCHLD, signal.SIG_IGN)
                - parent has the SA_NOCLDWAIT flag set 
                """
                pass

            os.setpgrp()  # leave the child's process group so I won't get signals
            try:
                os.killpg(self.childpid, signal.SIGINT)
                time.sleep(self.time_to_die)  # let processes end gracefully
                os.killpg(self.childpid, signal.SIGKILL)  # In case process gets stuck while dying
                os.waitpid(self.childpid, 0)  # reap Zombie child process
            except OSError as e:
                pass
        finally:
            for s, hdlr in self.exit_signals.iteritems():
                signal.signal(s, hdlr)  # reset default handlers

    def __enter__(self):
        if self.is_stopped:
            self.start()

    def __exit__(self, exit_type, value, traceback):
        if not self.is_stopped:
            self.stop()

Thanks to Malcolm Handley for the initial design. Done with python2.7 on linux.

Raiment

Answer 12

You can try subalive, a package I wrote for similar problem. It uses periodic alive ping via RPC, and the slave process automatically terminates when the master stops alive pings for some reason.

https://github.com/waszil/subalive

Example for master:

from subalive import SubAliveMaster

# start subprocess with alive keeping
SubAliveMaster(<path to your slave script>)

# do your stuff
# ...

Example for slave subprocess:

from subalive import SubAliveSlave

# start alive checking
SubAliveSlave()

# do your stuff
# ...

Bahaism

Answer 13

It's possible to get some more guarantees on windows by spawning a separate process to oversee the destruction.

import subprocess
import sys
import os

def terminate_process_on_exit(process):
    if sys.platform == "win32":
        try:
            # Or provide this script normally. 
            # Here just to make it somewhat self-contained.
            # see https://mcmap.net/q/25581/-wait-for-process-to-end-in-windows-batch-file
            # see https://superuser.com/a/1299350/388191
            with open('.process_watchdog_helper.bat', 'x') as file:
                file.write(""":waitforpid
tasklist /nh /fi "pid eq %1" 2>nul | find "%1" >nul
if %ERRORLEVEL%==0 (
    timeout /t 5 /nobreak >nul
    goto :waitforpid
) else (
    wmic process where processid="%2" call terminate >nul
)""")
        except:
            pass
        
        # After this spawns we're pretty safe. There is a race, but we do what we can.
        subprocess.Popen(
            ['.process_watchdog_helper.bat', str(os.getpid()), str(process.pid)],
            stdout=subprocess.DEVNULL,
            stderr=subprocess.DEVNULL
        )

# example
class DummyProcess:
    def __init__(self, pid):
        self.pid = pid
set_terminate_when_this_process_dies(DummyProcess(7516))

Gallnut

Answer 14

This is what I did for my posix app:

When your app exists call the kill() method of this class: http://www.pixelbeat.org/libs/subProcess.py

Example use here: http://code.google.com/p/fslint/source/browse/trunk/fslint-gui#608

Manx

Answer 15

help for python code: http://docs.python.org/dev/library/subprocess.html#subprocess.Popen.wait

Rabideau

Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference. – Daffie