Launch a completely independent process

Question

I want to initiate a process from my python script main.py. Specifically, I want to run the below command:

`nohup python ./myfile.py &`

and the file myfile.py should continue running, even after the main.py script exits.

I also wish to get the pid of the new process.

I tried:

• os.spawnl*
• os.exec*
• subprocess.Popen

and all are terminating the myfile.py when the main.py script exits.

---

Update: Can I use os.startfile with xdg-open? Is it the right approach?

---

Example

a = subprocess.Popen([sys.executable, "nohup /usr/bin/python25 /long_process.py &"],\
     stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
print a.pid

If I check ps aux | grep long_process, there is no process running.

long_process.py which keeps on printing some text: no exit.

Am I doing anything wrong here?

Orb

Can you post a minimal example that fails to work? After a trivial python -c 'import subprocess; subprocess.Popen(["sleep", "60"])' the output of ps shows that the sleep keeps running just fine after Python has exited. – Sleety

You are running Python with a single argument, "nohup /usr/bin/python25 ...", which cannot work because the python executable will look for a script in a file named exactly nohup /usr/bin/..., which doesn't exist. And since you specify stderr as PIPE without ever reading the pipe's content, you never get to see the error message. Lose the nohup and &, and simply run subprocess.Popen([sys.executable, "/.../long_process.py"]). Also, don't specify stdin and stderr as pipes unless you mean it. – Sleety

nohup makes most sense when you start a process from a shell. I don't see shell=True in your Popen call. – Ostracon

Very related question: subprocess - Run a program from python, and have it continue to run after the script is killed - Stack Overflow – Georgiana

Answer 1

You open your long-running process and keep a pipe to it. So you expect to talk to it. When yor launcher script exits, you can no longer talk to it. The long-running process receives a SIGPIPE and exits.

The following just worked for me (Linux, Python 2.7).

Create a long-running executable:

$ echo "sleep 100" > ~/tmp/sleeper.sh

Run Python REPL:

$ python
>>>

import subprocess
import os
p = subprocess.Popen(['/bin/sh', os.path.expanduser('~/tmp/sleeper.sh')])
# look ma, no pipes!
print p.pid
# prints 29893

Exit the REPL and see the process still running:

>>> ^D
$ ps ax | grep sleeper
29893 pts/0    S      0:00 /bin/sh .../tmp/sleeper.sh
29917 pts/0    S+     0:00 grep --color=auto sleeper

If you want to first communicate to the started process and then leave it alone to run further, you have a few options:

• Handle SIGPIPE in your long-running process, do not die on it. Live without stdin after the launcher process exits.
• Pass whatever you wanted using arguments, environment, or a temporary file.
• If you want bidirectional communication, consider using a named pipe (man mkfifo) or a socket, or writing a proper server.
• Make the long-running process fork after the initial bi-direcional communication phase is done.

Ostracon

Great answer, it's a good point that the process dies due to SIGPIPE. (In the code from the question the process never got started, but the OP might have also tried other variants that did start and died due to SIGPIPE.) – Sleety

It does not create "a completely independent process" (that what python-daemon package does). Also, a child process won't get SIGPIPE in Python 2. Though in simple cases, your solution is enough (you should redirect to os.devnull child's stdin/stdout/stderr to avoid waiting for input and/or a spurious output to the terminal). – Pawl

@user4815162342: It won't die due to SIGPIPE in Python 2 (the question has python2.7 tag). – Pawl

here's an example of why the redirection is important: it may allow a parent process to exit before its child finished writing its output: Python subprocess .check_call vs .check_output – Pawl

Answer 2

You can use os.fork().

import os
pid=os.fork()
if pid==0: # new process
    os.system("nohup python ./myfile.py &")
    exit()
# parent process continues

Carn

Note that os.fork() does not exist for Windows – Justly

Using nohup here is pointless. It does exactly two things: (1) disabling HUP propagation, which is already turned off for noninteractive shells and so doesn't need to be further disabled; and (2) redirecting stdin, stdout and stderr if they're attached to the terminal (which you can do yourself trivially; </dev/null >nohup.out 2>&1 and you just did the exact same redirections nohup would do). Much more efficient to just leave all the shells out of it entirely, as the answer by @Pawl does. – Manoeuvre

Answer 3

I could not see any process running.

You don't see any process running because the child python process exits immediately. The Popen arguments are incorrect as user4815162342 says in the comment.

To launch a completely independent process, you could use python-daemon package or use systemd/supervisord/etc:

#!/usr/bin/python25
import daemon
from long_process import main

with daemon.DaemonContext():
    main()

Though it might be enough in your case, to start the child with correct Popen arguments:

with open(os.devnull, 'r+b', 0) as DEVNULL:
    p = Popen(['/usr/bin/python25', '/path/to/long_process.py'],
              stdin=DEVNULL, stdout=DEVNULL, stderr=STDOUT, close_fds=True)
time.sleep(1) # give it a second to launch
if p.poll(): # the process already finished and it has nonzero exit code
    sys.exit(p.returncode)

If the child process doesn't require python2.5 then you could use sys.executable instead (to use the same Python version as the parent).

Note: the code closes DEVNULL in the parent without waiting for the child process to finish (it has no effect on the child).

Pawl