Recursive stream

Question

I want to list all the files on my computer recursively using Java 8.

Java 8 provides a listFiles method that returns all the files and directories but without recursion. How can I use it to get a full recursive list of files (without using a mutating collection)?

I've tried the code below but it only goes one level deep:

static Function<Path, Stream<Path>> listFiles = p -> {
    if (p.toFile().isDirectory()) {
        try { return Files.list(p); }
        catch (Exception e) { return Stream.empty(); }
    } else {
        return Stream.of(p);
    }
};

public static void main(String[] args) throws IOException {
    Path root = Paths.get("C:/temp/");
    Files.list(root).flatMap(listFiles).forEach(System.out::println);
}

And using return Files.list(p).flatMap(listFiles); does not compile (not sure why)...

Note: I am not interested in solutions involving FileVisitors or external libraries.

Discomfort

Files.walkFileTree ? Or, do you really really just want to use a recursive stream? :-) – Prerecord

@StuartMarks Yes I just want to use a recursive stream! And walkFileTree is quite verbose so I was trying to find a "one-liner". – Discomfort

Whoops, I meant Files.walk. It takes a Path and returns a Stream<Path>. – Prerecord

Aha - I had completely missed that new method... – Discomfort

Answer 1

A new API to generate a stream of Paths by walking the filesystem recursively is Files.walk.

If you really want to generate a stream recursively (not necessarily walking the file tree, but I'll continue using that as an example), it might be a bit more straightforward to accomplish the recursion using method references:

class RecursiveStream {
    static Stream<Path> listFiles(Path path) {
        if (Files.isDirectory(path)) {
            try { return Files.list(path).flatMap(RecursiveStream::listFiles); }
            catch (Exception e) { return Stream.empty(); }
        } else {
            return Stream.of(path);
        }
    }

    public static void main(String[] args) {
        listFiles(Paths.get(".")).forEach(System.out::println);
    }
}

Method references turn out to be quite useful for adapting a named method that has the same "shape" (arguments and return type) as a functional interface to that functional interface. This also avoids the potential initialization circularity with storing a lambda in an instance or static variable and calling itself recursively.

Prerecord

Could there be a resource leak, if Files.list throws? – Contrapositive

Answer 2

It is apparently not possible to refer to a function within that function definition through a method reference but it works with a lambda.

So in the function, return Files.list(p).flatMap(listFiles); does not compile but return Files.list(p).flatMap(q -> listFiles.apply(q)); does.

This prints all the files in the given folder recursively:

static final Function<Path, Stream<Path>> listFiles = p -> {
    if (p.toFile().isDirectory()) {
        try { return Files.list(p).flatMap(q -> listFiles.apply(q)); }
        catch (Exception e) { return Stream.empty(); }
    } else {
        return Stream.of(p);
    }
};

public static void main(String[] args) throws IOException {
    Path root = Paths.get("C:/temp/");
    Files.list(root).flatMap(listFiles).forEach(System.out::println);
}

---

but as pointed out, this is unnecessary:

Files.walk(root).forEach(System.out::println);

does the same thing...

Discomfort

It does not compile on JDK 8 b127 either: Cannot reference a field before it is defined. – Trainman

Just to note: this relies on listFiles being a static field as shown in the example, and wouldn't work in other cases. Even so, with the latest JDK, I couldn't get it to compile unless I first initialized the field to null and then turned around and re-initialized it with the lambda. Of course, this all seems very smelly. – Sylvia