Perform an action in every sub-directory using Bash
Asked Answered
D

10

232

I am working on a script that needs to perform an action in every sub-directory of a specific folder.

What is the most efficient way to write that?

Disused answered 22/10, 2010 at 20:15 Comment(2)
Please consider coming back through and reevaluating answers for correctness -- you've got an accepted answer getting a lot of views despite major bugs (f/e, running it over a directory where someone previously ran mkdir 'foo * bar' will cause foo and bar to be iterated over even if they don't exist, and the * will be replaced with a list of all filenames, even non-directory ones).Ewe
...even worse is if someone ran mkdir -p '/tmp/ /etc/passwd /' -- if someone runs a script following this practice on /tmp to, say, find directories to delete, they could end up deleting /etc/passwd.Ewe
F
189
for D in $(find . -mindepth 1 -maxdepth 1 -type d); do
    //Do whatever you need with D
done
Friedman answered 22/10, 2010 at 20:29 Comment(10)
this will break on white spacesCadaverous
Also, note you need to tweak the find params if you want recursive or non-recursive behavior.Coaptation
The above answer gave me the self directory as well, so the following worked a bit better for me: find . -mindepth 1 -type dUriniferous
@JoshC, both variants will break the directory created by mkdir 'directory name with spaces' into four separate words.Ewe
I needed to add -mindepth 1 -maxdepth 1 or it went too deep.Butcherbird
The same in a single line (can be run at command-line): for D in `find . -type d`; do //Do whatever you need with D; doneLody
@Butcherbird just replace both with -depth 1Iaea
@Iaea -depth Process each directory's contents before the directory itself. I think it isn't a replace for mindepth and maxdepth.Eskilstuna
@Eskilstuna You are correct that it would traverse in a different order. I might have misunderstood the desired outcome, thanks for bringing this up :)Iaea
If using find I think one might as well get rid of the for loop and use find with its exec argument instead.Displode
M
338

A version that avoids creating a sub-process:

for D in *; do
    if [ -d "${D}" ]; then
        echo "${D}"   # your processing here
    fi
done

Or, if your action is a single command, this is more concise:

for D in *; do [ -d "${D}" ] && my_command; done

Or an even more concise version (thanks @enzotib). Note that in this version each value of D will have a trailing slash:

for D in */; do my_command; done
Metalinguistic answered 22/10, 2010 at 20:33 Comment(8)
You can avoid the if or [ with: for D in */; doFoamy
+1 because directory names don't begin with ./ as opposed to accepted answerBiopsy
This one is correct even up to spaces in the directory names +1Werewolf
There is one problem with the last command: if you are in a directory without subdirectories; it will return "*/". So better use the second command for D in *; do [ -d "${D}" ] && my_command; done or a combination of the two latest: for D in */; do [ -d $D ] && my_command; doneFrigate
Note that this answer ignores hidden directories. To include hidden directories use for D in .* *; do instead for D in *; do.Travers
note for noobs like me that if you're wanting to do this on a directory that isn't the current, the syntax is for D in ./directory/*; doTowhee
use ${D%/*} instead of $D to trim the trailing slash.Whitewing
@Whitewing ${D%/} is probably safer so extra characters after the / aren't accidentally removed.Metalinguistic
F
189
for D in $(find . -mindepth 1 -maxdepth 1 -type d); do
    //Do whatever you need with D
done
Friedman answered 22/10, 2010 at 20:29 Comment(10)
this will break on white spacesCadaverous
Also, note you need to tweak the find params if you want recursive or non-recursive behavior.Coaptation
The above answer gave me the self directory as well, so the following worked a bit better for me: find . -mindepth 1 -type dUriniferous
@JoshC, both variants will break the directory created by mkdir 'directory name with spaces' into four separate words.Ewe
I needed to add -mindepth 1 -maxdepth 1 or it went too deep.Butcherbird
The same in a single line (can be run at command-line): for D in `find . -type d`; do //Do whatever you need with D; doneLody
@Butcherbird just replace both with -depth 1Iaea
@Iaea -depth Process each directory's contents before the directory itself. I think it isn't a replace for mindepth and maxdepth.Eskilstuna
@Eskilstuna You are correct that it would traverse in a different order. I might have misunderstood the desired outcome, thanks for bringing this up :)Iaea
If using find I think one might as well get rid of the for loop and use find with its exec argument instead.Displode
R
133

The simplest non recursive way is:

for d in */; do
    echo "$d"
done

The / at the end tells, use directories only.

There is no need for

  • find
  • awk
  • ...
Raposa answered 14/11, 2016 at 8:35 Comment(5)
Note: this will not include dot dirs (which can be a good thing, but it important to know).Littell
Useful to note that you can use shopt -s dotglob to include dotfiles/dotdirs when expanding wildcards. See also: gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.htmlSnifter
I think you meant /* instead of */ with / representing the path you want to use.Rootstock
@Shule /* would be for absolute path whereas */ would include the subdirectories from the current locationKelbee
helpful hint: if you need to trim the trailing '/' from $d, use ${d%/*}Whitewing
C
24

Use find command.

In GNU find, you can use -execdir parameter:

find . -type d -execdir realpath "{}" ';'

or by using -exec parameter:

find . -type d -exec sh -c 'cd -P "$0" && pwd -P' {} \;

or with xargs command:

find . -type d -print0 | xargs -0 -L1 sh -c 'cd "$0" && pwd && echo Do stuff'

Or using for loop:

for d in */; { echo "$d"; }

For recursivity try extended globbing (**/) instead (enable by: shopt -s extglob).


For more examples, see: How to go to each directory and execute a command? at SO

Culinary answered 11/9, 2015 at 22:39 Comment(1)
-exec {} + is POSIX-specified, -exec sh -c 'owd=$PWD; for arg; do cd -- "$arg" && pwd -P; cd -- "$owd"; done' _ {} + is another legal option, and invokes fewer shells than -exec sh -c '...' {} \;.Ewe
M
15

Handy one-liners

for D in *; do echo "$D"; done
for D in *; do find "$D" -type d; done ### Option A

find * -type d ### Option B

Option A is correct for folders with spaces in between. Also, generally faster since it doesn't print each word in a folder name as a separate entity.

# Option A
$ time for D in ./big_dir/*; do find "$D" -type d > /dev/null; done
real    0m0.327s
user    0m0.084s
sys     0m0.236s

# Option B
$ time for D in `find ./big_dir/* -type d`; do echo "$D" > /dev/null; done
real    0m0.787s
user    0m0.484s
sys     0m0.308s
Meteoric answered 20/3, 2014 at 18:3 Comment(0)
H
10

find . -type d -print0 | xargs -0 -n 1 my_command

Hydrolyze answered 22/10, 2010 at 20:18 Comment(2)
can I feed the return value of that find into a for loop? This is part of a larger script...Disused
@Mike, unlikely. $? will probably get you the status of the find or the xargs command, rather than my_command.Hydrolyze
H
7

This will create a subshell (which means that variable values will be lost when the while loop exits):

find . -type d | while read -r dir
do
    something
done

This won't:

while read -r dir
do
    something
done < <(find . -type d)

Either one will work if there are spaces in directory names.

Hematite answered 23/10, 2010 at 0:26 Comment(2)
For even better handling of weird filenames (including names that end with whitespace and/or include linefeeds), use find ... -print0 and while IFS="" read -r -d $'\000' dirDisaffection
@GordonDavisson, ...indeed, I'd even argue that -d '' is less misleading about bash syntax and capabilities, since -d $'\000' implies (falsely) that $'\000' is in some way different from '' -- indeed, one could readily (and again, falsely) infer from it that bash supports Pascal-style strings (length-specified, able to contain NUL literals) rather than C strings (NUL delimited, unable to contain NULs).Ewe
C
6

the accepted answer will break on white spaces if the directory names have them, and the preferred syntax is $() for bash/ksh. Use GNU find -exec option with +; eg

find .... -exec mycommand +; #this is same as passing to xargs

or use a while loop

find .... | while read -r D
do
    # use variable `D` or whatever variable name you defined instead here
done 
Cadaverous answered 22/10, 2010 at 23:19 Comment(3)
what param will hold the directory name? eg, chmod +x $DIR_NAME (yes, i know there is a chmod option for only directories)Sypher
There is one subtle difference between find -exec and passing to xargs: find will ignore the exit value of the command being executed, while xargs will fail on a nonzero exit. Either might be correct, depending on your needs.Romie
find ... -print0 | while IFS= read -r d is safer -- supports names that begin or end in whitespace, and names that contain newline literals.Ewe
B
6

You could try:

#!/bin/bash
### $1 == the first args to this script
### usage: script.sh /path/to/dir/

for f in `find . -maxdepth 1 -mindepth 1 -type d`; do
  cd "$f"
  <your job here>
done

or similar...

Explanation:

find . -maxdepth 1 -mindepth 1 -type d : Only find directories with a maximum recursive depth of 1 (only the subdirectories of $1) and minimum depth of 1 (excludes current folder .)

Brokenhearted answered 20/4, 2016 at 11:57 Comment(7)
This is buggy -- try with a directory name with spaces. See BashPitfalls #1, and DontReadLinesWithFor.Ewe
Directory name with spaces is enclosed in quotes and therefore works and OP is not trying to read lines from file.Brokenhearted
it works in the cd "$f". It doesn't work when the output from find is string-split, so you'll have the separate pieces of the name as separate values in $f, making how well you do or don't quote $f's expansion moot.Ewe
I didn't say they were trying to read lines from a file. find's output is line-oriented (one name to a line, in theory -- but see below) with the default -print action.Ewe
Line-oriented output, as from find -print is not a safe way to pass arbitrary filenames, since one can run something mkdir -p $'foo\n/etc/passwd\nbar' and get a directory that has /etc/passwd as a separate line in its name. Handling names from files in /upload or /tmp directories without care is a great way to get privilege escalation attacks.Ewe
In general, not speaking specifically to UNIX or to filenames, to unambiguously distinguish where one string ends and the next one begins in a list, one must either pass length out-of-band or use a sigil as separator that can't be present in one of those strings. The only character that can't be present in a filename on UNIX systems is NUL, so the only way to safely pass a list of filenames is a NUL-delimited list. (Since arguments on UNIX are C strings, and thus NUL-delimited, this is why find ... -exec is safe: Arguments passed on an argv list are individually terminated by NULs).Ewe
...anyhow, you can easily test this yourself. ( tempdir=$(mktemp -d "$PWD"/tempdir.XXXXXX) && cd "$tempdir" && { mkdir 'directory with spaces' $'directory\nwith\nnewlines'; for d in $(find . -type d -print); do echo "Found: $d"; done; }; rm -rf "$tempdir" ) and you'll see each word on its own separate line in the output, with its own Found: prefix.Ewe
I
1

if you want to perform an action INSIDE the folder and not ON folder.

Explanation: You have many pdfs and you would like to concetrate them inside a single folder. my folders

   AV 001/
   AV 002/
  • for D in *; do cd "$D"; # VERY DANGEROUS COMMAND - DONT USE #-- missing "", it will list files too. It can go up too.
  • for d in */; do cd "$d"; echo $d; cd ..; done; # works succesfully
  • for D in "$(ls -d */)"; do cd "$D"; done; # bash: cd: $'Athens Voice 001/\nAthens Voice 002/' - there is no such folder
  • for D in "$(*/)"; do cd "$D"; done; # bash: Athens Voice 001/: is folder
  • for D in "$(`find . -type d`)"; do cd $D; done; # bash: ./Athens: there is no such folder or file
  • for D in *; do if [ -d "${D}" ] then cd ${D}; done; # many arguments
Innocence answered 19/8, 2022 at 3:10 Comment(1)
missing "", it will list files too. It can go up too.Innocence

© 2022 - 2024 — McMap. All rights reserved.