How to check if directory contents has changed with PHP?

Question

I'm writing a photo gallery script in PHP and have a single directory where the user will store their pictures. I'm attempting to set up page caching and have the cache refresh only if the contents of the directory has changed. I thought I could do this by caching the last modified time of the directory using the filemtime() function and compare it to the current modified time of the directory. However, as I've come to realize, the directory modified time does not change as files are added or removed from that directory (at least on Windows, not sure about Linux machines yet).

So my questions is, what is the simplest way to check if the contents of a directory have been modified?

Elegize

Thanks for the responses, but most of your answers aren't valid for a script that many end users on many different systems will use. For a better understanding of what I'm implementing this for, check out the gallery page at code.web-geek.net/ck-gallery – Elegize

I would personally use an event-driven approach. Have a cache object subscribe to a fileChanged($event,$fileName) event & have the submission script do SomeEventsApi::fileChanged($fileName). If not this, I'd recursively call filemtime and use a singular cache file & only re-run the recursive filemtime if it's been 5 minutes since the last search... or something. – Visionary

Answer 1

Uh. I'd simply store the md5 of a directory listing. If the contents change, the md5(directory-listing) will change. You might get the very occasional md5 clash, but I think that chance is tiny enough..
Alternatively, you could store a little file in that directory that contains the "last modified" date. But I'd go with md5.

---

PS. on second thought, seeing as how you're looking at performance (caching) requesting and hashing the directory listing might not be entirely optimal..

Sublimate

Answer 2

As already mentioned by others, a better way to solve this would be to trigger a function when particular events happen, that changes the folder. However, if your server is a unix, you can use inotifywait to watch the directory, and then invoke a PHP script.

Here's a simple example:

#!/bin/sh
inotifywait --recursive --monitor --quiet --event modify,create,delete,move --format '%f' /path/to/directory/to/watch |
  while read FILE ; do
    php /path/to/trigger.php $FILE
  done

See also: http://linux.die.net/man/1/inotifywait

Vetter

I think the solution should be kept at the PHP level. This solution entangles the PHP app with the underlying OS and lose portability. – Farreaching

Some tasks are much better dealt with at the OS level. For example, setting up a cronjob is inherently OS-specific. There is no way to do that in PHP. – Vetter

This is about caching a HTML page with PHP, it has to do with PHP. Involving the OS in this will complicate matters for nothing. Not to mention the hidden magic this solution is hidding might confuse programmers. – Farreaching

So you say. There may well be cases for this solution. For example, if the PHP application isn't the only way images can be uploaded. I agree that it's probably better to solve in PHP, but without the context, I can't know for sure, can I? – Vetter

I go according to the poster's question. we could play the 'what if' game forever... – Farreaching

The poster asked how to check if a directory has changed. I'm not guessing here - you are. – Vetter

@Vetter It is possible to create a cronjob in PHP only: programmierer-forum.de/… The trick is that the script sleeps 15 seconds before calling itself through fopen again. By that you have up to 4 requests per minute and every request is able to restart the cron intervall if some of them break. By that it needs only sometimes resurrection (eg through 1x1 images). – Desertion

Answer 3

What about touching the directory after a user has submitted his image? Changelog says: Requires php 5.3 for windows to work, but I think it should work on all other environments

Swett

Answer 4

with inotifywait inside php

$watchedDir = 'watch';

$in = popen("inotifywait --monitor --quiet --format '%e %f' --event create,moved_to '$watchedDir'", 'r');
if ($in === false)
    throw new Exception ('fail start notify');

while (($line = fgets($in)) !== false) 
{
    list($event, $file) = explode(' ', rtrim($line, PHP_EOL), 2);
    echo "$event $file\n";
}

Indigested

Answer 5

IMO edubem's answer is the way to go, however you can do something like this:

if (sha1(serialize(Map('/path/to/directory/', true))) != /* previous stored hash */)
{
    // directory contents has changed
}

Or a more weak / faster version:

if (Size('/path/to/directory/', true) != /* previous stored size */)
{
    // directory contents has changed
}

Here are the functions used:

function Map($path, $recursive = false)
{
    $result = array();

    if (is_dir($path) === true)
    {
        $path = Path($path);
        $files = array_diff(scandir($path), array('.', '..'));

        foreach ($files as $file)
        {
            if (is_dir($path . $file) === true)
            {
                $result[$file] = ($recursive === true) ? Map($path . $file, $recursive) : $this->Size($path . $file, true);
            }

            else if (is_file($path . $file) === true)
            {
                $result[$file] = Size($path . $file);
            }
        }
    }

    else if (is_file($path) === true)
    {
        $result[basename($path)] = Size($path);
    }

    return $result;
}

function Size($path, $recursive = true)
{
    $result = 0;

    if (is_dir($path) === true)
    {
        $path = Path($path);
        $files = array_diff(scandir($path), array('.', '..'));

        foreach ($files as $file)
        {
            if (is_dir($path . $file) === true)
            {
                $result += ($recursive === true) ? Size($path . $file, $recursive) : 0;
            }

            else if (is_file() === true)
            {
                $result += sprintf('%u', filesize($path . $file));
            }
        }
    }

    else if (is_file($path) === true)
    {
        $result += sprintf('%u', filesize($path));
    }

    return $result;
}

function Path($path)
{
    if (file_exists($path) === true)
    {
        $path = rtrim(str_replace('\\', '/', realpath($path)), '/');

        if (is_dir($path) === true)
        {
            $path .= '/';
        }

        return $path;
    }

    return false;
}

Feodor

Will be using array_diff(scandir($path), array('.', '..'));. I always do if ($file=='.'||$file=='..')continue;, which I've always hated and never tried to make better lol. – Visionary

Answer 6

Here's what you may try. Store all pictures in a single directory (or in /username subdirectories inside it to speed things up and to lessen the stress on the FS) and set up Apache (or whaterver you're using) to serve them as static content with "expires-on" set to 100 years in the future. File names should contain some unique prefix or suffix (timestamp, SHA1 hash of file content, etc), so whenever uses changes the file its name gets changed and Apache will serve a new version, which will get cached along the way.

Cybele

This also circumvents browsers not caching things due to query string parameters and the like (Safari, iirc). – Currycomb

Where does the page caching come in all this? – Farreaching

You usually bookmark not to images themselves, but pages the images contained in. – Cybele

Tell that to your average user. – Farreaching

Answer 7

You're thinking the wrong way.

You should execute your directory indexer script as soon as someone's uploaded a new file and it's moved to the target location.

Player

Seems like a good solution at first, but there's a GOTCHA. If he ever wants to change the template of his page, your solution fails as you cannot delete the cached items for them to refresh. It would require another script to rebuild the whole thing. – Farreaching

Why should you not be able to delete the cached items? Have you not heard of chmod? – Player

All I can suggest is this to reread carefully his question, your answer, my response on bottom and my comment here. Caching upfront is faulty in this situation. – Farreaching

I really think you are not getting the question. He wants to have the cache to be refreshed if there is a new file uploaded into the directory where they place their photos. My solution is to wait for that trigger, then delete the cache. Whats the problem? – Player

As i re-read your response, you mean exactly the same solution as I do. – Player

"You should execute your directory indexer script..." sounds vaguely like generating an index page whenever an upload is done. – Farreaching

Unfortunately this method will not work. The script is only executed whenever the page is requested, and in order for this to work on anyone's server, there's no way I can implement a directory listener. – Elegize

Answer 8

Try deleting the cached version when a user uploads a file to his directory.

When someone tries to view the gallery, look if there's a cached version first. If there's a cached version, load it, otherwise, generate the page, cache it, done.

Farreaching

This makes sense, but how do I monitor for an uploaded file? My script only executes when someone visits the page, I can't actively monitor a directory, that's why I tried to check the modified date of the dir. – Elegize

Answer 9

I was looking for something similar and I just found this:

http://www.franzone.com/2008/06/05/php-script-to-monitor-ftp-directory-changes/

For me looks like a great solution since I'll have a lot of control (I'll be doing an AJAX call to see if anything changed).

Hope that this helps.

Kunstlied

I forgot to mention... my plan is to change it to read a local directory instead of using FTP. But the basics are still in there. – Kunstlied

Answer 10

Here is a code sample, that would return 0 if the directory was changed. I use it in backups.

The changed status is determined by presence of files and their filesizes. You could easily change this, to compare file contents by replacing

$longString .= filesize($file);

with

$longString .= crc32(file_get_contents($file));

but it will affect execution speed.

#!/usr/bin/php
<?php

$dirName = $argv[1];
$basePath = '/var/www/vhosts/majestichorseporn.com/web/';
$dataFile = './backup_dir_if_changed.dat';

# startup checks
if (!is_writable($dataFile))
    die($dataFile . ' is not writable!');

if (!is_dir($basePath . $dirName))
    die($basePath . $dirName . ' is not a directory');

$dataFileContent = file_get_contents($dataFile);
$data = @unserialize($dataFileContent);
if ($data === false)
    $data = array();

# find all files ang concatenate their sizes to calculate crc32
$files = glob($basePath . $dirName . '/*', GLOB_BRACE);

$longString = '';
foreach ($files as $file) {
    $longString .= filesize($file);
}
$longStringHash = crc32($longString);

# do changed check
if (isset ($data[$dirName]) && $data[$dirName] == $longStringHash)
    die('Directory did not change.');

# save hash do DB
$data[$dirName] = $longStringHash;

file_put_contents($dataFile, serialize($data));
die('0');

Eskil

Answer 11

very simple. when user uploads or alters something via form submissions then create a small file (i.e. newDetected.txt) in somewhere. then on page load check if file_exists(newDetected.txt) unlink(newDetected.txt) and run recreate function to rebuild dir tree.

Savior