Ensure a single instance of an application in Linux

Question

I'm working on a GUI application in WxPython, and I am not sure how I can ensure that only one copy of my application is running at any given time on the machine. Due to the nature of the application, running more than once doesn't make any sense, and will fail quickly. Under Win32, I can simply make a named mutex and check that at startup. Unfortunately, I don't know of any facilities in Linux that can do this.

I'm looking for something that will automatically be released should the application crash unexpectedly. I don't want to have to burden my users with having to manually delete lock files because I crashed.

Longinus

Answer 1

There are several common techniques including using semaphores. The one I see used most often is to create a "pid lock file" on startup that contains the pid of the running process. If the file already exists when the program starts up, open it up and grab the pid inside, check to see if a process with that pid is running, if it is check the cmdline value in /proc/pid to see if it is an instance of your program, if it is then quit, otherwise overwrite the file with your pid. The usual name for the pid file is application_name.pid.

Cushitic

Out of curiosity, doesn't simply opening the file for exclusive access do the work of a mutex? – Harless

Menkboy, if the file is properly closed in the event of a crash, then I think that will work perfectly and simplify things further. Thank you. – Longinus

This is the first description of pidfiles on SO I've seen that explains all the steps needed to make sure they work properly. Also, the last two steps can be done at once - checking /proc/pid/cmdline will fail if 'pid' isn't a running process, since the directory won't exist. – Tamah

This appears to have a race condition? First, suppose an instance of my program dies, leaving behind the stale PID file. Then I start up two new instances of my program at the same time. The two new instances might both decide that the PID file is stale, and then one after another they will overwrite the PID file. This is not a problem for traditional UNIX daemons, which are only started at boot and by explicit administrator command-line choice. But it might be a problem for other programs. – Phenothiazine

Note that procfs is quite specifically a Linux thing -- this code will not work under MacOS X, the BSDs, or traditional Unixen. To be sure, this question is tagged linux, but it's still worth taking portability concerns into account. – Campanology

@Phenothiazine This has to be implemented by means of a complete removal of the stale PID file, and an atomic creation via O_CREAT and O_EXCL. The condition for being the single instance has to be "I successfully created the file, with O_EXCL." Then you're good --- unless /var/run is NFS mounted, oops. Due to NFS, this has to be a directory: as a rule of thumb, mkdir is atomic on NFS; file creation may not be. – Unswear

@Unswear I think that just moves the race to the delete. Suppose: Process A creates the file then crashes, leaving the file behind. Processes B and C then start, and both check the file and see it contains the stale PID, so they both decide to delete-and-recreate. Process B deletes the file and atomically recreates it. Then process C deletes the file and atomically recreates it. Processes B and C both think they have the exclusive lock. Fundamentally, handling stale locks without kernel support is hard - that's why you should use flock() instead. – Phenothiazine

I think this works because PID's are usually assigned sequentially - superuser.com/questions/135007/how-are-pids-generated - but if they were assigned randomly, one day it might fail. https://mcmap.net/q/25638/-linux-pid-recycling-closed – Resigned

Answer 2

The Right Thing is advisory locking using flock(LOCK_EX); in Python, this is found in the fcntl module.

Unlike pidfiles, these locks are always automatically released when your process dies for any reason, have no race conditions exist relating to file deletion (as the file doesn't need to be deleted to release the lock), and there's no chance of a different process inheriting the PID and thus appearing to validate a stale lock.

If you want unclean shutdown detection, you can write a marker (such as your PID, for traditionalists) into the file after grabbing the lock, and then truncate the file to 0-byte status before a clean shutdown (while the lock is being held); thus, if the lock is not held and the file is non-empty, an unclean shutdown is indicated.

Campanology

This is the correct method to lock, however its also good to write a PID file that is cleaned up on normal exit, as well as to create an entry in /var/lock/subsys (if it exists). This allows your program to realize if its re-starting from a crash, among other things. So, doing both helps. – Osana

@tinkertim - Not a bad suggestion, though it makes sense to flock() the pidfile rather than having more than one. – Campanology

are the locks released on system crashes / someone turning off the power to your machine? – Bonsai

@Bonsai POSIX advisory locks only last as long as the file descriptors do. Yes, they're correctly released on hard shutdown and the like. – Campanology

@Kaz, eh? These patterns solve completely different problems. pthread_mutex_t is appropriate when you're protecting in-memory content, sure -- but if you're protecting content on a filesystem, the other processes holding the lock may not even live on the same host. And if there are recovery operations to be taken when taking ownership of on-disk data in an unknown state, it's wise to always perform those when granted the lock -- for all you know, the system may have suffered a hard shutdown (ie. power loss) prior. – Campanology

Answer 3

Complete locking solution using the fcntl module:

import fcntl
pid_file = 'program.pid'
fp = open(pid_file, 'w')
try:
    fcntl.lockf(fp, fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
    # another instance is running
    sys.exit(1)

Guttery

Assuming the lock file is the same for all users, as it ought to be for the lock to be useful, this can create a write permissions problem. I have described and addressed this problem in an answer. – Ostia

I don't know why but this does not work in Python 3.4.1. Two instances run without any errors. – Jadwiga

@boreq, could you be a little more explicit about exactly how you're testing (filesystem in use, expected behavior, actual behavior, etc)? – Campanology

@boreq, ...for instance, if one were deleting the pidfiles on exit, that would definitely break it in the manner described. – Campanology

Answer 4

wxWidgets offers a wxSingleInstanceChecker class for this purpose: wxPython doc, or wxWidgets doc. The wxWidgets doc has sample code in C++, but the python equivalent should be something like this (untested):

  name = "MyApp-%s" % wx.GetUserId()
  checker = wx.SingleInstanceChecker(name)
  if checker.IsAnotherRunning():
      return False

Bixby

Answer 5

This builds upon the answer by user zgoda. It mainly addresses a tricky concern having to do with write access to the lock file. In particular, if the lock file was first created by root, another user foo can then no successfully longer attempt to rewrite this file due to an absence of write permissions for user foo. The obvious solution seems to be to create the file with write permissions for everyone. This solution also builds upon a different answer by me, having to do creating a file with such custom permissions. This concern is important in the real world where your program may be run by any user including root.

import fcntl, os, stat, tempfile

app_name = 'myapp'  # <-- Customize this value

# Establish lock file settings
lf_name = '.{}.lock'.format(app_name)
lf_path = os.path.join(tempfile.gettempdir(), lf_name)
lf_flags = os.O_WRONLY | os.O_CREAT
lf_mode = stat.S_IWUSR | stat.S_IWGRP | stat.S_IWOTH  # This is 0o222, i.e. 146

# Create lock file
# Regarding umask, see https://mcmap.net/q/152332/-write-file-with-specific-permissions-in-python
umask_original = os.umask(0)
try:
    lf_fd = os.open(lf_path, lf_flags, lf_mode)
finally:
    os.umask(umask_original)

# Try locking the file
try:
    fcntl.lockf(lf_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
    msg = ('Error: {} may already be running. Only one instance of it '
           'can run at a time.'
           ).format('appname')
    exit(msg)

A limitation of the above code is that if the lock file already existed with unexpected permissions, those permissions will not be corrected.

I would've liked to use /var/run/<appname>/ as the directory for the lock file, but creating this directory requires root permissions. You can make your own decision for which directory to use.

Note that there is no need to open a file handle to the lock file.

Ostia

Answer 6

Here's the TCP port-based solution:

# Use a listening socket as a mutex against multiple invocations
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('127.0.0.1', 5080))
s.listen(1)

Zebulon

Answer 7

The set of functions defined in semaphore.h -- sem_open(), sem_trywait(), etc -- are the POSIX equivalent, I believe.

Harless

Answer 8

Look for a python module that interfaces to SYSV semaphores on unix. The semaphores have a SEM_UNDO flag which will cause the resources held by the a process to be released if the process crashes.

Otherwise as Bernard suggested, you can use

import os
os.getpid()

And write it to /var/run/application_name.pid. When the process starts, it should check if the pid in /var/run/application_name.pid is listed in the ps table and quit if it is, otherwise write its own pid into /var/run/application_name.pid. In the following var_run_pid is the pid you read from /var/run/application_name.pid

cmd = "ps -p %s -o comm=" % var_run_pid
app_name = os.popen(cmd).read().strip()
if len(app_name) > 0:
    Already running

Anaesthesia

+1 for suggesting SYSV semaphores, -1 for suggesting calling ps rather than something more efficient (say, kill -0 -- via the signal call rather than the command if one prefers to avoid an extra fork/exec) and suggesting pidfiles without advisory locking (ie. flock) to avoid race conditions and PID collisions. – Campanology

Answer 9

If you create a lock file and put the pid in it, you can check your process id against it and tell if you crashed, no?

I haven't done this personally, so take with appropriate amounts of salt. :p

Potation

Using the PID file is common, but is not without issues. For one, there can be race conditions. Second, it may not get cleaned up if the app is killed. – Ostia

Answer 10

Can you use the 'pidof' utility? If your app is running, pidof will write the Process ID of your app to stdout. If not, it will print a newline (LF) and return an error code.

Example (from bash, for simplicity):

linux# pidof myapp
8947
linux# pidof nonexistent_app

linux#

Packston

This is neat, less code required than the lockfile approach. Downside is it isn't exactly atomic, but for single instance detection, that may not be necessary. – Longinus

This won't work if you're running "pidof app" from inside the app – Anaesthesia

It also won't work if there is a different running program with the same name, or if another user is also running the program. – Webbing

Answer 11

By far the most common method is to drop a file into /var/run/ called [application].pid which contains only the PID of the running process, or parent process. As an alternative, you can create a named pipe in the same directory to be able to send messages to the active process, e.g. to open a new file.

Mccurdy

Using the PID file is common, but is not without issues. For one, there can be race conditions. Second, it may not get cleaned up if the app is killed. – Ostia

Answer 12

I've made a basic framework for running these kinds of applications when you want to be able to pass the command line arguments of subsequent attempted instances to the first one. An instance will start listening on a predefined port if it does not find an instance already listening there. If an instance already exists, it sends its command line arguments over the socket and exits.

code w/ explanation

Pena