Python - Launch a Long Running Process from a Web App

Question

I have a python web application that needs to launch a long running process. The catch is I don't want it to wait around for the process to finish. Just launch and finish.

I'm running on windows XP, and the web app is running under IIS (if that matters).

So far I tried popen but that didn't seem to work. It waited until the child process finished.

Whack

instead of writing "that didn't seem to work", try writing what exactly happened. Did it raise an exception? Did it do something other than what you expected? (if so, what did it do?). This will usually make it easier for other people to help you. – Amboina

Here are a few windows API commands that look promising, I'll try them out if we don't come up with anything else: timgolden.me.uk/pywin32-docs/win32api__ShellExecute_meth.html timgolden.me.uk/pywin32-docs/… timgolden.me.uk/pywin32-docs/… – Whack

Answer 1

Ok, I finally figured this out! This seems to work:

from subprocess import Popen
from win32process import DETACHED_PROCESS

pid = Popen(["C:\python24\python.exe", "long_run.py"],creationflags=DETACHED_PROCESS,shell=True).pid
print pid
print 'done' 
#I can now close the console or anything I want and long_run.py continues!

Note: I added shell=True. Otherwise calling print in the child process gave me the error "IOError: [Errno 9] Bad file descriptor"

DETACHED_PROCESS is a Process Creation Flag that is passed to the underlying WINAPI CreateProcess function.

Whack

Any idea on how to do this on Unix? – Pendleton

This flag does not seem needed on Unix – Evolute

I don't need shell=True for this to work. The bad file descriptor error could be unrelated. – Evolute

Answer 2

Instead of directly starting processes from your webapp, you could write jobs into a message queue. A separate service reads from the message queue and runs the jobs. Have a look at Celery, a Distributed Task Queue written in Python.

Cathycathyleen

Answer 3

This almost works (from here):

from subprocess import Popen

pid = Popen(["C:\python24\python.exe", "long_run.py"]).pid
print pid
print 'done'

'done' will get printed right away. The problem is that the process above keeps running until long_run.py returns and if I close the process it kills long_run.py's process.

Surely there is some way to make a process completely independent of the parent process.

Whack

Answer 4

subprocess.Popen does that.

Amboina

Do you know how I would call it? When I tried it, it waited until the long running process finished. Maybe I was calling it wrong. – Whack

You must not wait for the application output or termination :) – Professoriate