PHP - Preventing collision in Cron - File lock safe?

Question

I'm trying to find a safe way to prevent a cron job collision (ie. prevent it from running if another instance is already running).

Some options I've found recommend using a lock on a file.

Is that really a safe option? What would happen if the script dies for example? Will the lock remain?

Are there other ways of doing this?

Dayle

If you open the file for writing isn't it locked to one process already? – Mclaren

@zerkms: I guess I really need to review that stuff thanks. – Mclaren

no, that solution is not good. It is affected endless lock if process died and race condition. The better solution would be to use flock – Legault

if script dies then lock acquired by flock will be released. – Legault

Answer 1

This sample was taken at http://php.net/flock and changed a little and this is a correct way to do what you want:

$fp = fopen("/path/to/lock/file", "w+");
if (flock($fp, LOCK_EX | LOCK_NB)) { // do an exclusive lock
  // do the work
  flock($fp, LOCK_UN); // release the lock
} else {
  echo "Couldn't get the lock!";
}
fclose($fp);

Do not use locations such as /tmp or /var/tmp as they could be cleaned up at any time by your system, thus messing with your lock as per the docs:

Programs must not assume that any files or directories in /tmp are preserved between invocations of the program.

https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s18.html https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s15.html

Do use a location that is under your control.

Credits:

• Michaël Perrin - for proposing to use w+ instead of r+

Legault

Sorry, I was meant to include that ref too (net.tutsplus.com/tutorials/other/…). My question remains I think, even with flock, what happens if the script dies? – Dayle

Ie. if the script dies between flock($fp, LOCK_EX) and flock($fp, LOCK_UN) ? – Dayle

@Ben: if script dies then lock acquired by flock will be released. – Legault

Works fine. However I use w+ mode for fopen in case the lock file wouldn't exist yet. – Broomrape

According to this Q&A, using fopen in w+ mode could cause problems using flock: #13039565 – Ganesa

@Nate: not sure I can get your exact point. w+ doesn't have any issues when is used with flock and behaves exactly as expected. – Legault

With this specific question, using a lock file to prevent multiple scripts running at the same time, w+ works, but if someone has an application where there is content in the lock file (such as the time that last file finished running), w+ will overwrite the contents. This means that since fopen is used before checking to see if the file is locked, the contents of the lock file will be erased before having the opportunity to read it. A better option is to use c+. I only commented on this because I'm kind of a noob and this problem took me a while to debug. – Ganesa

@Nate: well, it's obvious that w+ would overwrite the contents. If one just copies the code without understanding how it works - it's their first issue :-) – Legault

What may not immediately be obvious, though, is that it overwrites the contents even if a lock cannot be established, and my point is that using c+ produces the same results as w+ except that it doesn't create this problem. Hopefully if anyone has this issue they will read my comments. – Ganesa

Answer 2

In Symfony Framework you could use the lock component symfony/lock

https://symfony.com/doc/current/console/lockable_trait.html

Fatuous

Answer 3

I've extended the concept from zerkms to create a function that can be called from the start of a cron.

Using the Cronlocker you specify a lock name, then the name of a callback function to be called if the cron is OFF. Optionally you may give an array of parameters to pass to the callback function. There's also an optional callback function if you need to do something different if the lock is ON.

In some cases I got a few exceptions and wanted to be able to trap them, and I added a function for handling fatal exceptions, which should be added. I wanted to be able to hit the file from a browser and bypass the cronlock, so that's built in.

I found as I used this a lot there were cases where I wanted to block other crons from running while this cron is running, so I added an optional array of lockblocks, which are other lock names to block.

Then there were cases where I wanted this cron to run after other crons had finished, so there's an optional array of lockwaits, which are other lock names to wait until none of which are running.

simple example:

Cronlocker::CronLock('cron1', 'RunThis');
function RunThis() {
    echo('I ran!');
}

callback parameters and failure functions:

Cronlocker::CronLock('cron2', 'RunThat', ['ran'], 'ImLocked');
function RunThat($x) {
    echo('I also ran! ' . $x);
}
function ImLocked($x) {
    echo('I am locked :-( ' . $x);
}

blocking and waiting:

Cronlocker::CronLock('cron3', 'RunAgain', null, null, ['cron1'], ['cron2']);
function RunAgain() {
    echo('I ran.<br />');
    echo('I block cron1 while I am running.<br />')
    echo('I wait for cron2 to finish if it is running.');
}

class:

class Cronlocker {

    private static $LockFile = null;
    private static $LockFileBlocks = [];
    private static $LockFileWait = null;

    private static function GetLockfileName($lockname) {
        return "/tmp/lock-" . $lockname . ".txt";
    }

    /**
     * Locks a PHP script from being executed more than once at a time
     * @param string $lockname          Use a unique lock name for each lock that needs to be applied.
     * @param string $callback          The name of the function to call if the lock is OFF
     * @param array $callbackParams Optional array of parameters to apply to the callback function when called
     * @param string $callbackFail      Optional name of the function to call if the lock is ON
     * @param string[] $lockblocks      Optional array of locknames for other crons to also block while this cron is running
     * @param string[] $lockwaits       Optional array of locknames for other crons to wait until they finish running before this cron will run
     * @see https://mcmap.net/q/102564/-php-preventing-collision-in-cron-file-lock-safe
     */
    public static function CronLock($lockname, $callback, $callbackParams = null, $callbackFail = null, $lockblocks = [], $lockwaits = []) {

        // check all the crons we are waiting for to finish running
        if (!empty($lockwaits)) {
            $waitingOnCron = true;
            while ($waitingOnCron) {
                $waitingOnCron = false;
                foreach ($lockwaits as $lockwait) {
                    self::$LockFileWait = null;
                    $tempfile = self::GetLockfileName($lockwait);
                    try {
                        self::$LockFileWait = fopen($tempfile, "w+");
                    } catch (Exception $e) {
                        //ignore error
                    }
                    if (flock(self::$LockFileWait, LOCK_EX | LOCK_NB)) { // do an exclusive lock
                        // cron we're waiting on isn't running
                        flock(self::$LockFileWait, LOCK_UN); // release the lock
                    } else {
                        // we're wating on a cron
                        $waitingOnCron = true;
                    }
                    if (is_resource(self::$LockFileWait))
                        fclose(self::$LockFileWait);
                    if ($waitingOnCron) break;      // no need to check any more
                }
                if ($waitingOnCron) sleep(15);      // wait a few seconds
            }
        }

        // block any additional crons from starting
        if (!empty($lockblocks)) {
            self::$LockFileBlocks = [];
            foreach ($lockblocks as $lockblock) {
                $tempfile = self::GetLockfileName($lockblock);
                try {
                    $block = fopen($tempfile, "w+");
                } catch (Exception $e) {
                    //ignore error
                }
                if (flock($block, LOCK_EX | LOCK_NB)) { // do an exclusive lock
                    // lock made
                    self::$LockFileBlocks[] = $block;
                } else {
                    // couldn't lock it, we ignore and move on
                }
            }
        }

        // set the cronlock
        self::$LockFile = null;
        $tempfile = self::GetLockfileName($lockname);
        $return = null;
        try {
            if (file_exists($tempfile) && !is_writable($tempfile)) {
                //assume we're hitting this from a browser and execute it regardless of the cronlock
                if (empty($callbackParams))
                    $return = $callback();
                else
                    $return = call_user_func_array($callback, $callbackParams);
            } else {
                self::$LockFile = fopen($tempfile, "w+");
            }
        } catch (Exception $e) {
            //ignore error
        }
        if (!empty(self::$LockFile)) {
            if (flock(self::$LockFile, LOCK_EX | LOCK_NB)) { // do an exclusive lock
                // do the work
                if (empty($callbackParams))
                    $return = $callback();
                else
                    $return = call_user_func_array($callback, $callbackParams);
                flock(self::$LockFile, LOCK_UN); // release the lock
            } else {
                // call the failed function
                if (!empty($callbackFail)) {
                    if (empty($callbackParams))
                        $return = $callbackFail();
                    else
                        $return = call_user_func_array($callbackFail, $callbackParams);
                }
            }
            if (is_resource(self::$LockFile))
                fclose(self::$LockFile);
        }

        // remove any lockblocks
        if (!empty($lockblocks)) {
            foreach (self::$LockFileBlocks as $LockFileBlock) {
                flock($LockFileBlock, LOCK_UN); // release the lock
                if (is_resource($LockFileBlock))
                    fclose($LockFileBlock);
            }
        }

        return $return;
    }

    /**
     * Releases the Cron Lock locking file, useful to specify on fatal errors
     */
    public static function ReleaseCronLock() {
        // release the cronlock
        if (!empty(self::$LockFile) && is_resource(self::$LockFile)) {
            var_dump('Cronlock released after error encountered: ' . self::$LockFile);
            flock(self::$LockFile, LOCK_UN);
            fclose(self::$LockFile);
        }
        // release any lockblocks too
        foreach (self::$LockFileBlocks as $LockFileBlock) {
            if (!empty($LockFileBlock) && is_resource($LockFileBlock)) {
                flock($LockFileBlock, LOCK_UN);
                fclose($LockFileBlock);
            }
        }
    }
}

Should also be implemented on a common page, or built into your existing fatal error handler:

function fatal_handler() {
    // For cleaning up crons that fail
    Cronlocker::ReleaseCronLock();
}
register_shutdown_function("fatal_handler");

Cauliflower