How to watch a folder and subfolders for changes

Question

I´m trying to watch a specific folder for changes, and then if any addition/edition/removal happens inside of it, I need to get the change type of all files in that folder and its subfolders. I'm using WatchService for this but it only watches a single path, it doesn't handle subfolders.

Here's my approach:

try {
        WatchService watchService = pathToWatch.getFileSystem().newWatchService();
        pathToWatch.register(watchService, StandardWatchEventKinds.ENTRY_CREATE,
                StandardWatchEventKinds.ENTRY_MODIFY, StandardWatchEventKinds.ENTRY_DELETE);

        // loop forever to watch directory
        while (true) {
            WatchKey watchKey;
            watchKey = watchService.take(); // This call is blocking until events are present

            // Create the list of path files
            ArrayList<String> filesLog = new ArrayList<String>();
            if(pathToWatch.toFile().exists()) {
                File fList[] = pathToWatch.toFile().listFiles();
                for (int i = 0; i < fList.length; i++) { 
                    filesLog.add(fList[i].getName());
                }
            }

            // Poll for file system events on the WatchKey
            for (final WatchEvent<?> event : watchKey.pollEvents()) {
                printEvent(event);
            }

            // Save the log
            saveLog(filesLog);

            if(!watchKey.reset()) {
                System.out.println("Path deleted");
                watchKey.cancel();
                watchService.close();
                break;
            }
        }

    } catch (InterruptedException ex) {
        System.out.println("Directory Watcher Thread interrupted");
        return;
    } catch (IOException ex) {
        ex.printStackTrace();  // Loggin framework
        return;
    }

Like I said before, I'm getting the log only for the files in the selected path, and I want to watch all folders and subfolders files, something like:

Example 1:

FileA (Created)
FileB
FileC
FolderA FileE
FolderA FolderB FileF

Example 2:

FileA
FileB (Modified)
FileC
FolderA FileE
FolderA FolderB FileF

Is there any better solution?

Implicit

This seems to be how Oracle would implement this: docs.oracle.com/javase/tutorial/essential/io/examples/… – Jaenicke

Answer 1

A WatchService only watches the Paths you register. It does not go through those paths recursively.

Given /Root as a registered path

/Root
    /Folder1
    /Folder2
        /Folder3

If there is a change in Folder3, it won't catch it.

You can register the directory paths recursively yourself with

private void registerRecursive(final Path root) throws IOException {
    // register all subfolders
    Files.walkFileTree(root, new SimpleFileVisitor<Path>() {
        @Override
        public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
            dir.register(watchService, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
            return FileVisitResult.CONTINUE;
        }
    });
}

Now the WatchService will notify all changes in all subfolders of Path root, ie. the Path argument you pass.

Rouge

How badly is performance degraded if we implement watch service for the folder and all its subfolders? Will it be cheaper to hook onto the system's "every single file change" watcher instead? – Anderton

@Anderton I do not know the underlying implementation, but I really don't think it's a big deal. Which system watcher are you referring to? – Rouge

A listener that gets pinged everytime a file in a particular drive is updated. If each subfolder needs to have its own listener, I think it will severely degrade system performance just to watch a folder with 7 subfolders, each with 7 subfolders etc, up to 7 levels giving us 7^7 = 823543 watchers. – Anderton

@Anderton Wait, no. Don't create that many watchers. Have a single Watcher register for a number of Paths. – Rouge

I'm not talking about Java. Yes we can register a Java Watcher with 823543 Paths but doesn't each register use a "system watcher" internally? – Anderton

@Anderton The OS handles the notification as far as I know. There must an OS file system process that does that. I don't know the details. – Rouge

How do you watch the SUB-directory that does not exist ? In other words if I have folder I am watching and if I drop a folder in it, does that count as modify or create ? – Herniotomy

@java_developer I'd have to try it, but I think there will be an ENTRY_CREATE for the new folder added to the one you are watching and an ENTRY_UPDATE for the last modified date of the folder you are watching. – Rouge

@SotiriosDelimanolis I'm trying to accomplish this now (2018) and find that there are corner cases where events are lost. Specifically, when you do a multi-directory create, i.e. mkdir -p a/b/c (or on Windows mkdir a\b\c with command extensions on), the Java code does not see the second and third subdirectory creations because they happen too fast. By the time you get the event for the first directory and manage to register it, the other subdirs already exist and are not seen. – Casemate

It gets even worse. When deleting a tree (as with rm -rf) on Windows 10 you can get the delete event for a parent directory before the delete for the child. Crazy. Seems like the API doesn't really work on Windows at least. – Casemate

Answer 2

Registering recursively will work as Sotirios has indicated. This effectively registers each directory/sub-directory that currently exists.

You can alternatively import and use *com.sun.nio.file.ExtendedWatchEventModifier.FILE_TREE* as in:

dir.register(watcher, standardEventsArray, ExtendedWatchEventModifier.FILE_TREE);

This will watch the entire sub-tree for change AND account for added directories and sub-directories.

Otherwise you will have to monitor for any new directories/sub-directories and register them also. There can also be an issue with deleting parts of the directory hierarchy since each registered directory has a handle watching it so the (lowest) sub-directories need to be removed first when deleting parts of the structure.

Duumvir

From the doc: "Note that this modifier is only available on the Windows platform. If specified on other platforms, Path.register() will throw an UnsupportedOperationException" – Booth

This does not work on Linux as previous comment has already said. – Vary

Answer 3

I have implemented something like this using Java 8 streams and lambdas.

The recursive folder discovery is implemented as a Consumer @FunctionalInterface:

    final Map<WatchKey, Path> keys = new HashMap<>();

    Consumer<Path> register = p -> {
        if (!p.toFile().exists() || !p.toFile().isDirectory()) {
            throw new RuntimeException("folder " + p + " does not exist or is not a directory");
        }
        try {
            Files.walkFileTree(p, new SimpleFileVisitor<Path>() {
                @Override
                public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
                    LOG.info("registering " + dir + " in watcher service");
                    WatchKey watchKey = dir.register(watcher, new WatchEvent.Kind[]{ENTRY_CREATE}, SensitivityWatchEventModifier.HIGH);
                    keys.put(watchKey, dir);
                    return FileVisitResult.CONTINUE;
                }
            });
        } catch (IOException e) {
            throw new RuntimeException("Error registering path " + p);
        }
    };

The above code is called every time a new folder is created to dynamically add folders at later stages. Full solution and more details here.

Commutate

If I had to guess I'd say the downvote relates to this old revision: stackoverflow.com/revisions/37658255/1, but who voted and why can't be determined ever – Handsel