Why is Java 7 Files.walkFileTree throwing exception on encountering a tar file on remote drive
Asked Answered
S

1

18

Im using Files.WalkFileTree() to navigate folder and counting audio files, but there is a problem when it encounters a tar file, it seems to be treating it as an actual folder I was expecting it to just skip over it.

I cannot see any options that let me control this behaviour

Code:

package com.jthink.songkong.fileloader;


import com.jthink.songkong.cmdline.SongKong;
import com.jthink.songkong.ui.MainWindow;

import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.concurrent.Callable;
import java.util.logging.Level;

/**
 * Count the number of files that can be loaded, for information purposes only
 */
public class CountFilesinFolder implements Callable<Boolean> {
    public static class CountFiles
            extends SimpleFileVisitor<Path> {
        private int fileCount = 0;
        private final PathMatcher matcher;

        CountFiles(String pattern) {
            matcher =
                    FileSystems.getDefault()
                            .getPathMatcher("regex:" + pattern);
        }

        /**
         * Find Music file
         *
         * @param file
         * @param attr
         * @return
         */
        @Override
        public FileVisitResult visitFile(Path file,
                                         BasicFileAttributes attr) {
            Path name = file.getFileName();
            if (name != null && matcher.matches(name)) {
                fileCount++;
            }
            return FileVisitResult.CONTINUE;
        }

        public int getFileCount() {
            return fileCount;
        }
    }


    private Path scanDir;
    public CountFilesinFolder(Path scanDir) {
        this.scanDir = scanDir;
    }

    public Boolean call() {
        CountFiles countFiles = null;
        try {
            countFiles = new CountFiles("^(?!._).*[.](?:mp3|mp4|m4p|m4b|m4a|ogg|flac|wma)$");
            Files.walkFileTree(scanDir, countFiles);
        }
        catch (Exception e) {
            MainWindow.logger.log(Level.SEVERE, "Unable to find file for deriving base folder", e);
        }
        MainWindow.logger.severe("Music File Count:"+countFiles.getFileCount());
        SongKong.setMaxProgress(countFiles.getFileCount());
        return true;
    }
}

gives this stacktrace

java.nio.file.NoSuchFileException: Z:\Scratch\fred.tar
    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
    at sun.nio.fs.WindowsDirectoryStream.<init>(WindowsDirectoryStream.java:86)
    at sun.nio.fs.WindowsFileSystemProvider.newDirectoryStream(WindowsFileSystemProvider.java:526)
    at java.nio.file.Files.newDirectoryStream(Files.java:411)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:179)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:69)
    at java.nio.file.Files.walkFileTree(Files.java:2591)
    at java.nio.file.Files.walkFileTree(Files.java:2624)
    at com.jthink.songkong.fileloader.CountFilesinFolder.call(CountFilesinFolder.java:68)
    at com.jthink.songkong.fileloader.CountFilesinFolder.call(CountFilesinFolder.java:15)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

but that is a a remote drive (nas drive), I get no such error on local drive

EDIT Implemented the following based o the answer below I thought worked

    @Override
            public FileVisitResult preVisitDirectory(Path dir, 
BasicFileAttributes attrs)
                    throws IOException {
                if(dir.endsWith(".tar"))
                {
                    return FileVisitResult.SKIP_SUBTREE;
                }
                return super.preVisitDirectory(dir, attrs);
            }

but my testing was amiss, it doesnt in fact work because the code in FileTreeWalker that fails is called before the previsit method

try {
            DirectoryStream<Path> stream = null;
            FileVisitResult result;

            // open the directory
            try {
                stream = Files.newDirectoryStream(file);
            } catch (IOException x) {
                return visitor.visitFileFailed(file, x);
            } catch (SecurityException x) {
                // ignore, as per spec
                return FileVisitResult.CONTINUE;
            }

            // the exception notified to the postVisitDirectory method
            IOException ioe = null;

            // invoke preVisitDirectory and then visit each entry
            try {
                result = visitor.preVisitDirectory(file, attrs);
                if (result != FileVisitResult.CONTINUE) {
                    return result;
                }
Swell answered 21/1, 2013 at 9:50 Comment(20)
error loading libastral.so, show your code, pleaseFerdelance
Could you post your code? It seems like an interesting issue but without a running example it is hard to help.Zoubek
code added, just realized error only occurs if the files are on a remote driveSwell
How do you mount the remote drive - SMB? And what FS on the remote drive? Could this be a permissions issue?Lethbridge
Windows properties says its a NTFS fileystem, its mounted from a Netgear ReadyNASDuo using CIFS, Not permissions issue as I can see/open folder in WIndows Explorer. I Just realized the tar is actually empty,maybe not even a tar but that doesnt justify it throwing said exception.Swell
and there is no such error remotely accessing said file from OSXSwell
I tried to reproduce the issue by putting an empty test.tar file on a network drive and running your code but I was not able to do so. The exception is not thrown and the call() method ends without any problems.Zoubek
@Zoubek same for me. I put two files with "mp3" extension and one empty fred.tar file into the directory, and the code properly returns a count of 2 with no exception.Tulle
I do not know whether this is any important but I ran my test on Windows 7 and JRE 1.7.0_11.Zoubek
Hm, Im using Windows 7 and Java 1.7.0_10Swell
@PaulTaylor can you quickly cross-check on _11? (You should upgrade anyways due to security issues with _10 ;) )Tulle
Can you do a quick test to see if it actually consideres the .tar to be a directory: public static void main(String[] argv) { Path p = Paths.get("Z:/Scratch/fred.tar"); System.out.println(Files.isDirectory(p)); }Melonymelos
Hi, yes tried and it considers to be a directory on Z:\drive, but not on c:\driveSwell
I think it makes sense to limit the question to a smaller example then. Or does site-rules prevents doing this?Melonymelos
@PaulTaylor Does it behave the same when you have a .zip file instead of .tar?Tulle
Implementation if something is a directory or not is depending on the state of the values returned by docs.oracle.com/javase/7/docs/api/java/nio/file/spi/…, java.lang.Class, java.nio.file.LinkOption...) try to check which filesystem is in play: Path p1 = Paths.get("z:/"); Path p2 = Paths.get("c:/"); FileSystem fs = p1.getFileSystem(); FileSystem fs2 = p2.getFileSystem(); System.out.println(fs); System.out.println(fs2);Melonymelos
In both cases it returns sun.nio.fs.WindowsFileSystem. I wonder if that is correct as I assume ReadyNAS is running Linux.Swell
If the drive is seen as a letter, then I guess it is correct. You might have found a deeper problem here, I do not think the Java is working here wrongly, it is rather an underlying operating system API.Zoubek
A shot in the dark: I guess the TAR has a directory attribute on the native Linux system, but doesn't have it after copying it to your local drive. Best would be to check on the native file system if you can access it somehow. Maybe this is of any help: https://mcmap.net/q/742543/-how-to-change-a-file-that-used-to-be-a-directory-back-to-directory-on-linux/44522Handwriting
I think you are right, but I get no such error accessing the same faile from my OSX (unix based) system. So assume it also sees it as an archive, yet doesnt complain about it.Swell
M
2

Workaround for the problem at hand:

But implement a visitFileFailed and you should be ok.

public class MyFileVisitor extends SimpleFileVisitor<Path> {
    @Override
    public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
        if (file.toString().endsWith(".tar")) {
            return FileVisitResult.CONTINUE;
        }
        return super.visitFileFailed(file, exc);
    }
}

Update: If we look closer we can see that walkFileTree uses the Files.readAttributes which turns to the current provider in play: WindowsFileSystemProvider.readAttributes to determine if a path is a directory.

As someone mentioned in the comments I also dont think the fault is in the Java-implementation but the OS-native-call that returns the wrong attribute

If you wanted to do a workaround for this, one option would be to implement your own FileSystem that wraps the WindowsFileSystem implementation transparently, except readAttributes returns .tar-paths as file instead of dir.

Melonymelos answered 21/1, 2013 at 20:50 Comment(4)
Thankyou that works but ONLY if you change (file.endsWith(".tar")) to (file.toString().endsWith(".tar")) otherwise it never matches, i.e only works if you simple string match rather than file specific endsWith match. I think we have bug with Java somewhere in here.Swell
But if the OS-native-call is returning incorrectly then don't you think WindowsFileSystemProvider shoud handle this issue rather than me (and eveyone else thatmay encounter this issue). Even if the error is not with java code I think its still something that needs to be fixed/workarounds in the system java code.Swell
yes ofc :), but it was more as a sugestion for workaround, fixes to jre tends to take time.Melonymelos
oh okay, well the visitFileFailed workaround is working for this quite obscure issue so I think I'll stick with that and move on.Swell

© 2022 - 2024 — McMap. All rights reserved.