python how to run process in detached mode

Question

here is a example:

from multiprocessing import Process
import time


def func():
    print('sub process is running')
    time.sleep(5)
    print('sub process finished')


if __name__ == '__main__':
    p = Process(target=func)
    p.start()
    print('done')

what I expect is that the main process will terminate right after it start a subprocess. But after printing out 'done', the terminal is still waiting....Is there any way to do this so that the main process will exit right after printing out 'done', instead of waiting for subprocess? I'm confused here because I'm not calling p.join()

Parsimony

Answer 1

Python will not end if there exists a non-daemon process.

By setting, daemon attribute before start() call, you can make the process daemonic.

p = Process(target=func)
p.daemon = True  # <-----
p.start()
print('done')

NOTE: There will be no sub process finished message printed; because the main process will terminate sub-process at exit. This may not be what you want.

You should do double-fork:

import os
import time
from multiprocessing import Process


def func():
    if os.fork() != 0:  # <--
        return          # <--
    print('sub process is running')
    time.sleep(5)
    print('sub process finished')


if __name__ == '__main__':
    p = Process(target=func)
    p.start()
    p.join()
    print('done')

Selwin

I love this approach! But i have heard that there are some concerns with using os.fork. Does this use case pose any risks? – Sequin

@NolanConaway, man 2 fork :) Fork early if possible to avoid unnecessary resource copy. If there's a lock hold before fork, you should carefully handle it. .... – Selwin

Also relevant, new in 3.7 docs.python.org/3/library/os.html#os.register_at_fork – Sequin

Answer 2

Following the excellent answer from @falsetru, I wrote out a quick generalization in the form of a decorator.

import os
from multiprocessing import Process


def detachify(func):
    """Decorate a function so that its calls are async in a detached process.

    Usage
    -----

    .. code::
            import time

            @detachify
            def f(message):
                time.sleep(5)
                print(message)

            f('Async and detached!!!')

    """
    # create a process fork and run the function
    def forkify(*args, **kwargs):
        if os.fork() != 0:
            return
        func(*args, **kwargs)

    # wrapper to run the forkified function
    def wrapper(*args, **kwargs):
        proc = Process(target=lambda: forkify(*args, **kwargs))
        proc.start()
        proc.join()
        return

    return wrapper

Usage (copied from docstring):

import time

@detachify
def f(message):
    time.sleep(5)
    print(message)

f('Async and detached!!!')

Or if you like,

def f(message):
    time.sleep(5)
    print(message)


detachify(f)('Async and detached!!!')

Sequin

Update: Tested and working on python 2.7, 3.5, 3.6, 3.7 – Sequin

This looks useful. I haven't tested though. Would this create a new parent process or a child process? I am trying to find a way to create a completely new process so I can close the terminal from which I have summoned this process. – Magnanimity

you can indeed create a process and exit the terminal! – Sequin