How to traverse all the files in a directory; if it has subdirectories, I want to traverse files in subdirectories too

Question

opendir(DIR,"$pwd") or die "Cannot open $pwd\n";
    my @files = readdir(DIR);
    closedir(DIR);
    foreach my $file (@files) {
        next if ($file !~ /\.txt$/i);
        my $mtime = (stat($file))[9];
        print $mtime;
        print "\n";
    }

Basically I want to note the timestamp of all the txt files in a directory. If there is a subdirectory I want to include files in that subdirectory too.

Can someone help me in modifying the above code so that it includes subdirectories too.

if i am using the code below in windows iam getting timestamps of all files which are in folders even outside my folder

 my @dirs = ("C:\\Users\\peter\\Desktop\\folder");
    my %seen;
    while (my $pwd = shift @dirs) {
            opendir(DIR,"$pwd") or die "Cannot open $pwd\n";
            my @files = readdir(DIR);
            closedir(DIR);
            #print @files;
            foreach my $file (@files) {
                    if (-d $file and !$seen{$file}) {
                            $seen{$file} = 1;
                            push @dirs, "$pwd/$file";
                    }
                    next if ($file !~ /\.txt$/i);
                    my $mtime = (stat("$pwd\$file"))[9];
                    print "$pwd $file $mtime";
                    print "\n";
            }
    }

Cockaigne

Answer 1

use warnings;
use strict;

my @dirs = (".");
my %seen;
while (my $pwd = shift @dirs) {
        opendir(DIR,"$pwd") or die "Cannot open $pwd\n";
        my @files = readdir(DIR);
        closedir(DIR);
        foreach my $file (@files) {
                next if $file =~ /^\.\.?$/;
                my $path = "$pwd/$file";
                if (-d $path) {
                        next if $seen{$path};
                        $seen{$path} = 1;
                        push @dirs, $path;
                }
                next if ($path !~ /\.txt$/i);
                my $mtime = (stat($path))[9];
                print "$path $mtime\n";
        }
}

Fu

$file !~ /^\.*$/ is $file =~ /[^.]/. But I have been severely reprimanded in the past for excluding names that are just three dots or longer, as they are valid names for Linux files. So the test should be $file !~ /^\.\.?$/ – Ravid

@Fu if i want to open files and search soome particular string in the file and seperate such files, I am thinking to do that with OPEN INPUT, $file , then $lines = <INPUT> then seraching on it , is it fine ? or there is some other easy way – Cockaigne

You can use grep for that. If you don't want to use any tools then it is fine to do what you suggest. – Fu

Hi can you all see the edited question and help me out .. i tried the code in windows , but i am getting timestamps of files which are in folders outside my specified folder too, i have pasted the code in question – Cockaigne

Can you try these changes? push @dirs, "$pwd\\$file"; my $mtime = (stat("$pwd\\$file"))[9]; – Fu

I'd add a bit to check for symbolic links too. I've made lots of dumb mistakes going through parts of the directory structure many times more than I needed because they linked to other parts of the filesystem. – Poll

The -d test and ! $seen{$file} both need to work with "$pwd/$file" or this will not do what you intended. Also, the second half of the for loop needs to be enclosed in an else block (or, better, an elsif ( -f "$pwd/$file" ) to reject symlinks) as it doesn't apply to directories – Ravid

Can't believe I'm missing it, but didn't OP want to visit subdirectories? The solution given here doesn't descend into subdirectories for me. (perl 5.22.1) – Strontium

Yeah this doesn't work and suffers from a number of bugs. $file is not relative to the dir so -d $file won't do what you want and directories within directories won't work. – Plume

I think the hash "seen" is not needed at all. namely "... if $seen{$path};" will be always false – Deplume

Answer 2

File::Find is best for this. It is a core module so doesn't need installing. This code does the equivalent of what you seem to have in mind

use strict;
use warnings;

use File::Find;

find(sub {
  if (-f and /\.txt$/) {
    my $mtime = (stat _)[9];
    print "$mtime\n";
  }
}, '.');

where '.' is the root of the directory tree to be scanned; you could use $pwd here if you wish. Within the subroutine, Perl has done a chdir to the directory where it found the file, $_ is set to the filename, and $File::Find::name is set to the full-qualified filename including the path.

Ravid

Not everybody will agree with you on File::Find being best. – Burro

File::Find is slow, and the API is frustrating, but I think it is ideal for something trivial like this. I would like to hear what anyone has against it. – Ravid

@Burro Very old comment I know, but the link now points to a login page. – Julian

Answer 3

use File::Find::Rule

File::Find::Rule is a friendlier interface to File::Find. It allows you to build rules which specify the desired files and directories.

Burro

Answer 4

You can use recursion: define a function that goes through files and calls itself on directories. Then call the function on the top directory.

See also File::Find.

Indeterminate

how to differentiate between a file and a directory inside a directory – Cockaigne

@Peter: use the -d and -f operators, documented here – Ravid

Answer 5

if i am using the code below in windows iam getting timestamps of all files which are in folders even outside my folder

I suspect that the problem may have been an issue with the . and .. directories which if you tried to follow them would have taken you up the directory tree. What you were missing was a:

foreach my $file (@files) {
    # skip . and .. which cause recursion and moving up the tree
    next if $file =~ /^\.\.?$/;
    ...

Your script also suffers from a couple of bugs. $file is not relative to the $dir so -d $file would only work in the current directory and not below.

Here's my fixed version:

use warnings;
use strict;

# if unix, change to "/";
my $FILE_PATH_SLASH = "\\";    
my @dirs = (".");
my %seen;
while (my $dir = shift @dirs) {
        opendir(DIR, $dir) or die "Cannot open $dir\n";
        my @files = readdir(DIR);
        closedir(DIR);
        foreach my $file (@files) {
                # skip . and ..
                next if $file =~ /^\.\.?$/;
                my $path = "$dir$FILE_PATH_SLASH$file";
                if (-d $path) {
                        next if $seen{$path};
                        $seen{$path} = 1;
                        push @dirs, $path;
                }
                next unless $path ~= /\.txt$/i;
                my $mtime = (stat($path))[9];
                print "$path $mtime\n";
        }
}

Plume

Answer 6

Here's my answer to read all files in folder and subfolders and capture that content into a file

create a perl script called amazing_perl.pl and add this to it

Then run it like this > perl.exe amazing_perl.pl c:\temp

#!/usr/bin/perl
use strict;
use warnings;
use File::Find qw(finddepth);

my ($TOP_LEVEL_DIR) = @ARGV;

print "Perl Starting ...\n\n";

#create output file
open(my $output_file_handle, '>', "ALL_FILES_AND_FOLDERS.txt");


print "Processing ...\n";
finddepth(sub {
    return if($_ eq '.' || $_ eq '..');
      
    print "Processing ... $_ \n";
  
    my $filePath = $File::Find::name;
    #print "filePath ... $filePath \n";

    print $output_file_handle "$filePath\n";    
      
 }, "$TOP_LEVEL_DIR");

 
print "\n\nPerl End ...\n\n";

sub trim {
(my $s = $_[0]) =~ s/^\s+|\s+$//g;
return $s;
}
1
;

Alabaster