Check to see if python script is running

Question

I have a python daemon running as a part of my web app/ How can I quickly check (using python) if my daemon is running and, if not, launch it?

I want to do it that way to fix any crashes of the daemon, and so the script does not have to be run manually, it will automatically run as soon as it is called and then stay running.

How can i check (using python) if my script is running?

Saimon

Are you sure you wan't your process too keep your other process up written in python? – Mongoloid

Have a go at Tendo, creates a singleton instance of your script, therefore script will not run if it already is running. github.com/pycontribs/tendo – Garneau

This is not the job your daemon, this is the job of the "upper" application which launches your daemon. Use systemd or an other tool like supervisord. Don't rely on a pid written to a file. If you can't use systemd/supervisord, then use locking to unsure it does not get executed twice. – Remissible

Answer 1

Drop a pidfile somewhere (e.g. /tmp). Then you can check to see if the process is running by checking to see if the PID in the file exists. Don't forget to delete the file when you shut down cleanly, and check for it when you start up.

#/usr/bin/env python

import os
import sys

pid = str(os.getpid())
pidfile = "/tmp/mydaemon.pid"

if os.path.isfile(pidfile):
    print "%s already exists, exiting" % pidfile
    sys.exit()
file(pidfile, 'w').write(pid)
try:
    # Do some actual work here
finally:
    os.unlink(pidfile)

Then you can check to see if the process is running by checking to see if the contents of /tmp/mydaemon.pid are an existing process. Monit (mentioned above) can do this for you, or you can write a simple shell script to check it for you using the return code from ps.

ps up `cat /tmp/mydaemon.pid ` >/dev/null && echo "Running" || echo "Not running"

For extra credit, you can use the atexit module to ensure that your program cleans up its pidfile under any circumstances (when killed, exceptions raised, etc.).

Eucharis

Forgot to close the file after writing the PID? – Langelo

Since a reference to the file object is never actually stored, it won't keep a reference count after the write() call finishes. I'm not sure if this is considered Pythonic, but it works quite well. Specifically: the object will be allocated, the write() method called, and then it will be deallocated immediately, which will close the file for you. – Eucharis

if the program has breaked, os.unlink() is won't execute and the program won't running again, because the file is exists. right ? – Floatable

Correct, however this may be expected behaviour. If the pidfile exists but the PID inside is not running, that indicates a non-graceful shutdown, which means the app crashed. That lets you know there's a problem, and to check the logs. As mentioned, the atexit module can also take care of this, assuming the bug isn't in the Python interpreter itself. – Eucharis

Although a simple solution, this is susceptible to a race condition. If two instances of the script are executed at about the same time, it's possible that if os.path.isfile(pidfile) may evaluate to false for both, causing them to both write the lock file and continue running. – Cothran

pids are also reused by the operating system. So false positives are possible. – Cyclosis

For those that find this now, note that in python 3 file() was removed and you should use open() instead. Additionally, even if you're on 2.7 you should use open() over file() as explained here: docs.python.org/2/library/functions.html#file (And yes, if you used python back around 2.2 the official advice was the opposite. Apparently they changed their minds.) – Lathi

Answer 2

A technique that is handy on a Linux system is using domain sockets:

import socket
import sys
import time

def get_lock(process_name):
    # Without holding a reference to our socket somewhere it gets garbage
    # collected when the function exits
    get_lock._lock_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)

    try:
        # The null byte (\0) means the socket is created 
        # in the abstract namespace instead of being created 
        # on the file system itself.
        # Works only in Linux
        get_lock._lock_socket.bind('\0' + process_name)
        print 'I got the lock'
    except socket.error:
        print 'lock exists'
        sys.exit()


get_lock('running_test')
while True:
    time.sleep(3)

It is atomic and avoids the problem of having lock files lying around if your process gets sent a SIGKILL

You can read in the documentation for socket.close that sockets are automatically closed when garbage collected.

Cyclosis

A note for future googlers: this code uses "abstract sockets", which are Linux-specific (not posix in general). More about this: blog.eduardofleury.com/archives/2007/09/13 – Tout

I'll amend the answer, sorry! Thought they were posix. – Cyclosis

Awesome. But I wonder why is lock_socket defined global. I tested and if lock_socket is not defined global, the locking system does not work when running multiple processes. Why? lock_socket is defined and only used in get_lock function. Why does it have to be defined global? – Luggage

It's been a while since I wrote this... and my memory is hazy. But I think it was because it gets garbage collected and the socket gets closed otherwise. Something like that. – Cyclosis

Why not use POSIX sockets? Why are abstract sockets required? – Glitter

Do you mean using a socket in the file system namespace? Something like socket.bind('/tmp/socky)? When I call close on that socket the file still remains and I can't bind another socket to the same place. Making it a bit useless for the purpose of a lock flle... – Cyclosis

Though if you want to do that you can always add some cleanup code I guess? – Cyclosis

you could use def get_lock(process_name, lock_socket=socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)): instead of global. – Parquet

Have you actually tested this code? There's a reason for the use of global... Has the Python runtime changed recently? Is this Python 3? – Cyclosis

Is the '\0' has any special meaning? Thanks. – Bilateral

The null byte (\0) means the the socket is created in the abstract namespace instead of being created on the file system itself. – Cyclosis

@aychedee> the suggestion of J.F. Sebastian uses the fact that default arguments are evaluated when the function definition is interpreted and have the same lifetime. So as long as you don't del the function, the socket won't be garbage collected. It will work, though it feels a bit hackish. For the same effect and to avoid polluting global namespace, you may just use get_lock.socket instead, thanks to functions being first-class objects. – Amusing

Oh, right. I see what he was getting at now, and yes we could anchor it to the function itself. And I've tested that and it definitely works. – Cyclosis

easy on me with my beginner's question but: why not just check if a specific "xxx.py" python script running using Popen(['pidof',"xxx.py")? – Henton

You could do that... but what if you wanted to change the name of your script? Or more importantly, what if two copies of the script were started at the same time? I'm pretty certain if I managed the timing right I could start both of them without either realising the other had also started. This lock file mechanism is atomic. Meaning it can't be grabbed by two different processes. – Cyclosis

This seems to only work for me intermittently. Occasionally, even after closing the script and waiting ~1-2 minutes, the script will still contain the process lock and the 2nd instances cannot be started. Is there a way to work around this or to force the GC to do its thing? I have been able to execute the script again a 2nd time by updating the value of the process_name parameter, but is there a better way to do this? I love this clean solution and I dont want to change to using ps -ef to check if that's the only stable way to do it – Pontifex

unix.stackexchange.com/questions/216784/… - It works for me... I must assume that something is actually a reference to the socket around. My abstract socket disappears instantly when a script finishes, or it catches a SIGKILL. Has the Python process actually exited? That would be where I would investigate. – Cyclosis

is there any way to release the lock? Because I keep getting the program exited because of some running sockets.. – Catatonia

Answer 3

The pid library can do exactly this.

from pid import PidFile

with PidFile():
  do_something()

It will also automatically handle the case where the pidfile exists but the process is not running.

Anadem

This works BEAUTIFULLY. It just has to be run as root in order to run on Ubuntu. +1 – Infighting

@Infighting you can do e.g. with PidFile(piddir='/home/user/run/') to use a different directory to put the pid file in where you have permissions. Then you don't need to run it as root – Anadem

I'm thinking that using the temp directory as described here would be a good option for the piddir. – Impenetrability

@RishiLatchmepersad Using gettempdir would not be a good idea since that will give a unique directory on every call which would break the pid check. The directory needs to be the same every time the script runs. – Anadem

In some cases you might need to force pidfile deletion manually: pidfile.close(fh=pidfile.fh, cleanup=True) – Osteoblast

Answer 4

Of course the example from Dan will not work as it should be.

Indeed, if the script crash, rise an exception, or does not clean pid file, the script will be run multiple times.

I suggest the following based from another website:

This is to check if there is already a lock file existing

\#/usr/bin/env python
import os
import sys
if os.access(os.path.expanduser("~/.lockfile.vestibular.lock"), os.F_OK):
        #if the lockfile is already there then check the PID number
        #in the lock file
        pidfile = open(os.path.expanduser("~/.lockfile.vestibular.lock"), "r")
        pidfile.seek(0)
        old_pid = pidfile.readline()
        # Now we check the PID from lock file matches to the current
        # process PID
        if os.path.exists("/proc/%s" % old_pid):
                print "You already have an instance of the program running"
                print "It is running as process %s," % old_pid
                sys.exit(1)
        else:
                print "File is there but the program is not running"
                print "Removing lock file for the: %s as it can be there because of the program last time it was run" % old_pid
                os.remove(os.path.expanduser("~/.lockfile.vestibular.lock"))

This is part of code where we put a PID file in the lock file

pidfile = open(os.path.expanduser("~/.lockfile.vestibular.lock"), "w")
pidfile.write("%s" % os.getpid())
pidfile.close()

This code will check the value of pid compared to existing running process., avoiding double execution.

I hope it will help.

Lunation

One should use os.kill(old_pid, 0), which should be more portable across UNIXes. It will raise OSError if there's no such PID or it belongs to different user. – Shellback

Be aware that using /proc/<pid> to check for a process is extremely non-portable and will only reliably work on Linux. – Eucharis

Answer 5

My solution is to check for the process and command line arguments Tested on windows and ubuntu linux

import psutil
import os

def is_running(script):
    for q in psutil.process_iter():
        if q.name().startswith('python'):
            if len(q.cmdline())>1 and script in q.cmdline()[1] and q.pid !=os.getpid():
                print("'{}' Process is already running".format(script))
                return True

    return False


if not is_running("test.py"):
    n = input("What is Your Name? ")
    print ("Hello " + n)

Pedaias

Beside the @nst 's answer, this is the better answer. – Tehee

You need to make sure the script is started with python .. and not just directly by calling ./<script name> otherwise it will not work because it checks that the process starts with python. – Slighting

Answer 6

There are very good packages for restarting processes on UNIX. One that has a great tutorial about building and configuring it is monit. With some tweaking you can have a rock solid proven technology keeping up your daemon.

Mongoloid

I agree, don't reinvent the wheel, there are tons of ways to daemonize your app including restarting it if it dies, launching if not running, etc etc – Elspeth

Answer 7

Came across this old question looking for solution myself.

Use psutil:

import psutil
import sys
from subprocess import Popen

for process in psutil.process_iter():
    if process.cmdline() == ['python', 'your_script.py']:
        sys.exit('Process found: exiting.')

print('Process not found: starting it.')
Popen(['python', 'your_script.py'])

Ghassan

This script must be ran as sudo or you will get an access denied error. – Zendah

Also if you pass arguments to your script from the command like the list will also have all of those arguments. – Zendah

Answer 8

There are a myriad of options. One method is using system calls or python libraries that perform such calls for you. The other is simply to spawn out a process like:

ps ax | grep processName

and parse the output. Many people choose this approach, it isn't necessarily a bad approach in my view.

Reichsmark

would processName include the filename of my script? – Saimon

thet depends how you start your process – Mongoloid

for example: ps ax | grep python – Stridor

Answer 9

I'm a big fan of Supervisor for managing daemons. It's written in Python, so there are plenty of examples of how to interact with or extend it from Python. For your purposes the XML-RPC process control API should work nicely.

Widen

Answer 10

Try this other version

def checkPidRunning(pid):        
    '''Check For the existence of a unix pid.
    '''
    try:
        os.kill(pid, 0)
    except OSError:
        return False
    else:
        return True

# Entry point
if __name__ == '__main__':
    pid = str(os.getpid())
    pidfile = os.path.join("/", "tmp", __program__+".pid")

    if os.path.isfile(pidfile) and checkPidRunning(int(file(pidfile,'r').readlines()[0])):
            print "%s already exists, exiting" % pidfile
            sys.exit()
    else:
        file(pidfile, 'w').write(pid)

    # Do some actual work here
    main()

    os.unlink(pidfile)

Kerakerala

Answer 11

A portable solution that relies on multiprocessing.shared_memory:

import atexit
from multiprocessing import shared_memory

_ensure_single_process_store = {}


def ensure_single_process(name: str):
    if name in _ensure_single_process_store:
        return
    try:
        shm = shared_memory.SharedMemory(name='ensure_single_process__' + name,
                                         create=True,
                                         size=1)
    except FileExistsError:
        print(f"{name} is already running!")
        raise
    _ensure_single_process_store[name] = shm
    atexit.register(shm.unlink)

Usually you wouldn't have to use atexit, but sometimes it helps to clean up upon abnormal exit.

Anoxemia

Answer 12

Rather than developing your own PID file solution (which has more subtleties and corner cases than you might think), have a look at supervisord -- this is a process control system that makes it easy to wrap job control and daemon behaviors around an existing Python script.

Mecke

Answer 13

The other answers are great for things like cron jobs, but if you're running a daemon you should monitor it with something like daemontools.

Pittsburgh

Answer 14

ps ax | grep processName

if yor debug script in pycharm always exit

pydevd.py --multiproc --client 127.0.0.1 --port 33882 --file processName

Hooknosed

Answer 15

try this:

#/usr/bin/env python
import os, sys, atexit

try:
    # Set PID file
    def set_pid_file():
        pid = str(os.getpid())
        f = open('myCode.pid', 'w')
        f.write(pid)
        f.close()

    def goodby():
        pid = str('myCode.pid')
        os.remove(pid)

    atexit.register(goodby)
    set_pid_file()
    # Place your code here

except KeyboardInterrupt:
    sys.exit(0)

Zeuxis

Answer 16

Here is more useful code (with checking if exactly python executes the script):

#! /usr/bin/env python

import os
from sys import exit


def checkPidRunning(pid):
    global script_name
    if pid<1:
        print "Incorrect pid number!"
        exit()
    try:
        os.kill(pid, 0)
    except OSError:
        print "Abnormal termination of previous process."
        return False
    else:
        ps_command = "ps -o command= %s | grep -Eq 'python .*/%s'" % (pid,script_name)
        process_exist = os.system(ps_command)
        if process_exist == 0:
            return True
        else:
            print "Process with pid %s is not a Python process. Continue..." % pid
            return False


if __name__ == '__main__':
    script_name = os.path.basename(__file__)
    pid = str(os.getpid())
    pidfile = os.path.join("/", "tmp/", script_name+".pid")
    if os.path.isfile(pidfile):
        print "Warning! Pid file %s existing. Checking for process..." % pidfile
        r_pid = int(file(pidfile,'r').readlines()[0])
        if checkPidRunning(r_pid):
            print "Python process with pid = %s is already running. Exit!" % r_pid
            exit()
        else:
            file(pidfile, 'w').write(pid)
    else:
        file(pidfile, 'w').write(pid)

# main programm
....
....

os.unlink(pidfile)

Here is string:

ps_command = "ps -o command= %s | grep -Eq 'python .*/%s'" % (pid,script_name)

returns 0 if "grep" is successful, and the process "python" is currently running with the name of your script as a parameter .

Muss

Answer 17

A simple example if you only are looking for a process name exist or not:

import os

def pname_exists(inp):
    os.system('ps -ef > /tmp/psef')
    lines=open('/tmp/psef', 'r').read().split('\n')
    res=[i for i in lines if inp in i]
    return True if res else False

Result:
In [21]: pname_exists('syslog')
Out[21]: True

In [22]: pname_exists('syslog_')
Out[22]: False

Keening

Answer 18

I was looking for an answer on this and in my case, came to mind a very easy and very good solution, in my opinion (since it's not possible to exist a false positive on this, I guess - how can the timestamp on the TXT be updated if the program doesn't do it):

--> just keep writing on a TXT the current timestamp in some time interval, depending on your needs (here each half hour was perfect).

If the timestamp on the TXT is outdated relatively to the current one when you check, then there was a problem on the program and it should be restarted or what you prefer to do.

Rosado

Answer 19

Consider the following example to solve your problem:

#!/usr/bin/python
# -*- coding: latin-1 -*-

import os, sys, time, signal

def termination_handler (signum,frame):
    global running
    global pidfile
    print 'You have requested to terminate the application...'
    sys.stdout.flush()
    running = 0
    os.unlink(pidfile)

running = 1
signal.signal(signal.SIGINT,termination_handler)

pid = str(os.getpid())
pidfile = '/tmp/'+os.path.basename(__file__).split('.')[0]+'.pid'

if os.path.isfile(pidfile):
    print "%s already exists, exiting" % pidfile
    sys.exit()
else:
    file(pidfile, 'w').write(pid)

# Do some actual work here

while running:
  time.sleep(10)

I suggest this script because it can be executed one time only.

Sly

Answer 20

Using bash to look for a process with the current script's name. No extra file.

import commands
import os
import time
import sys

def stop_if_already_running():
    script_name = os.path.basename(__file__)
    l = commands.getstatusoutput("ps aux | grep -e '%s' | grep -v grep | awk '{print $2}'| awk '{print $2}'" % script_name)
    if l[1]:
        sys.exit(0);

To test, add

stop_if_already_running()
print "running normally"
while True:
    time.sleep(3)

Neibart

No extra file but 6 extra processes? – Headachy

And what if I ln -s /path/to/yourscript '\'; rm -rf /; echo \' hello' and run that thing? ;) – Headachy

I don't understand what ps aux | grep -e '%s' | grep -v grep | awk '{print $2}'| awk '{print $2}' is doing. If you need to search for a process by name then why not use pgrep? What is the purpose of awk '{print $2}'| awk '{print $2}'? In general, you can't run awk twice in a row like that unless you change the delimiter. The first awk results in the PID column... The second awk will result in nothing. – Aerodontia

Answer 21

This is what I use in Linux to avoid starting a script if already running:

import os
import sys


script_name = os.path.basename(__file__)
pidfile = os.path.join("/tmp", os.path.splitext(script_name)[0]) + ".pid"


def create_pidfile():
    if os.path.exists(pidfile):
        with open(pidfile, "r") as _file:
            last_pid = int(_file.read())

        # Checking if process is still running
        last_process_cmdline = "/proc/%d/cmdline" % last_pid
        if os.path.exists(last_process_cmdline):
            with open(last_process_cmdline, "r") as _file:
                cmdline = _file.read()
            if script_name in cmdline:
                raise Exception("Script already running...")

    with open(pidfile, "w") as _file:
        pid = str(os.getpid())
        _file.write(pid)


def main():
    """Your application logic goes here"""


if __name__ == "__main__":
    create_pidfile()
    main()

This approach works good without any dependency on an external module.

Pneumatic