Shell scripts and how to avoid running the same script at the same time on a Linux machine

Question

I have Linux centralize server – Linux 5.X.

In some cases on my Linux server the get_hosts.ksh script could be run from some other different hosts.

For example get_hosts.ksh could run on my Linux machine three or more times at the same time.

My question:

• How to avoid running multiple instances of process/script?

Varicocele

Make a file at start of the script, check if file exists before that, if it does exit.Delete it when script terminates – Deleterious

Maybe helpful: What are pid and lock files for? – Inerrable

The standard trick is to have the script create a lock file somewhere when it starts, and to remove it when it finishes. If the script starts and detects the lock file, it exits. Maybe you record the PID of the process in the lock file so you can detect when the process died without removing the lock file. – Fond

right but the big problem is that -> what in case the script fail and still you have the file so the second time that you run it then it will not run because that file -:( – Varicocele

You record the PID in the lock file; you check whether the process is still running. – Fond

can I use this command pidof -x get_hosts.ksh in order to find the proccess PID number ? – Varicocele

use flock - manage locks from shell scripts. man flock – Opportina

Answer 1

A common solution for your problem on *nix systems is to check for a lock file existence.
Usually lock file contains current process PID.
This is an example ksh script:

#!/bin/ksh

pid="/var/run/get_hosts.pid"
trap "rm -f $pid" SIGSEGV
trap "rm -f $pid" SIGINT

if [ -e $pid ]; then
    exit # pid file exists, another instance is running, so now we politely exit
else
    echo $$ > $pid # pid file doesn't exit, create one and go on
fi

# your normal workflow here...

rm -f $pid # remove pid file just before exiting
exit

UPDATE: Answering to OP comment, I add handling program interruptions and segfaults with trap command.

Catto

can I use this command pidof -x get_hosts.ksh in order to find the proccess PID number ? – Varicocele

Yes. But, inside your shell script, you have the PID of the running process in $$ variable... Why do you need to access get_hosts.ksh PID from command line? – Catto

so I need to insert the command echo $$ > $pid in the second line in your code – Varicocele

No, just do echo $$ > $pid in the else part of the test for the $pid file existence (my code should work as-is...). – Catto

Answer 2

The normal way of doing this is to write the process id into a file. The first thing the script does is check for the existence of the file, read the pid, check if a process with that pid exists, and for extra paranoia points, if that process actually runs the script. If yes, the script exits.

Here's a simple example. The process in question is a binary, and this script makes sure the binary runs only once. This is not exactly what you need, but you should be able to adapt this:

RUNNING=0
PIDFILE=$PATH_TO/var/run/example.pid
if [ -f $PIDFILE ]
then
  PID=`cat $PIDFILE`
  ps -eo pid | grep $PID >/dev/null 2>&1
  if [ $? -eq 0 ]
  then
      RUNNING=1
  fi
fi
if [ $RUNNING -ne 1 ]
then
    run_binary
    PID=$!
    echo $PID > $PIDFILE
fi

This is not very elaborate but should get you on the right track.

Antipode

please do you have real example for that? – Varicocele

Be careful with grep $PID. Suppose the PID is 300; there are quite a few process IDs that contain 300 (1300, 3001, 30010, etc). Also, you should just write if ps -eo pid | grep "^$PID$" >/dev/null 2>&1; then ...; no need to run the command and then test $? separately. – Fond

Yes, the grep should probably need to match the whole line that is output by ps -eo pid – Antipode

Correct me if I am wrong but this will not prevent running more binary instances at all. As far as I know shell is waiting for binary to finish the job. It means it would prevent it if you spawn the binary in the background and detached from current shell but than binary itself has to remove the PIDFILE by the end of the execution. – Killoran

You're right, run_binary needs to be run_binary &. I wouldn't rely on the binary to remove the PID file, though, because it may crash, or fail to remove the file for other reasons. It's better to not assign state to the pidfile's content itself, but rather to treat it as a pointer to grep the output of ps, or even to /proc/$PID. – Antipode

Answer 3

You can use a pid file to keep track of when the process is running. At the top of the script, check for the existence of the pid file and if it doesn't exist, create it and run the script, otherwise return.

Some sample code can be seen in this answer to a similar question.

Gillian

Answer 4

You might consider using the (optional) lockfile(1) command (provided by procmail package on Debian).

Bubal

Answer 5

I have a lot of scripts, and using this below code for prevent multiple/simulate run:

PID="/var/scripts/PID.txt" # Temp file
if [ ! -f "$PID" ]; then
    echo $$ > "$PID" # Print actual PID into a file
else
    ps -p $(cat "$PID") > /dev/null && exit || echo $$ > "$PID"
fi

Horseplay

Answer 6

Building on wallenborn's answer I also added a "staleness" check just in case the PID lock file is beyond a certain expected age in seconds.

# prevent simultaneous executions within an hourish
pid_file="$HOME/.harness.pid"
max_stale_seconds=3600
if [ -f $pid_file ]; then
    pid="$(cat "$pid_file")"
    let age_in_seconds="$(date +%s) - $(date -r "$pid_file" +%s)"
    if ps $pid >/dev/null && [ $age_in_seconds -lt $max_stale_seconds ]; then
        exit 1
    fi
fi
echo $$>"$pid_file"
trap "rm -f \"$pid_file\"" SIGSEGV
trap "rm -f \"$pid_file\"" SIGINT

This could be made "smarter" to kill off the other executions should the PID be valid but this would be dangerous. Consider a sudden power failure and reset situation where the PID file contains a number that may now reference a completely different process.

Blouin