'find -exec' a shell function in Linux

Question

Is there a way to get find to execute a function I define in the shell?

For example:

dosomething () {
  echo "Doing something with $1"
}
find . -exec dosomething {} \;

The result of that is:

find: dosomething: No such file or directory

Is there a way to get find's -exec to see dosomething?

Triny

Answer 1

Since only the shell knows how to run shell functions, you have to run a shell to run a function. You also need to mark your function for export with export -f, otherwise the subshell won't inherit them:

export -f dosomething
find . -exec bash -c 'dosomething "$0"' {} \;

Imperfection

You beat me. By the way you can put the braces inside the quotes instead of using $0. – Forewent

@Dennis: Ah, good call, I thought that {} argument to find had to be a separate word, but apparently not. – Imperfection

@alxndr: You can get monospaced text in comments by putting backticks (`) around your text. – Imperfection

@alxndr: that'll fail on filenames with double-quotes, backquotes, dollar-signs, some escape combos, etc... – Heliocentric

I'm trying to make it fail with weird filenames. I'm using this command : find . -exec ls -d "{}" \; I've got a a file named ./file"; rm -rf . ; echo " hi nothing bad seems to be happening – Th

None of the your global variables will be set in the function you'll either need to export everything or use @Jac's method. – Pachysandra

Note also that any functions your function might be calling will not be available unless you export -f those as well. – Tracheitis

The export -f will work only in some versions of bash. It's not posix, not crossplatforn, /bin/sh will have an error with it – Temptation

I think this could break if the filename has special meaning to the shell. Also it's inconsistent with arguments starting at $1. If the mini-script becomes a little more complicated, this could be very confusing. I propose to use export -f dosomething; find . -exec bash -c 'dosomething "$1"' _ {} \; instead. – Olden

Out of curiosity, what is the role of "$0" in bash -c 'dosomething "$0"' {} or "$1" in bash -c 'dosomething "$1"' _ {}? Is there a name for that feature? What should I search for if I wanted more information? – Superfamily

@Superfamily Those are called the positional parameters: gnu.org/software/bash/manual/bash.html#Positional-Parameters . They evaluate to the nth parameter that was passed to the current script or function, with the 0th parameter having the special meaning of the name of the shell. Unfortunately, variables such as these are difficult to search for if you don't already know the name of them; but as a general rule, looking through the Bash manual for these is a good strategy. – Imperfection

@AdamRosenfield I just found that in this particular case (using bash -c command-string ...), $0 is not the shell name, but the first argument following the command-string (and $1 is the 2nd. such argument, etc.) Hence the use of a throwaway parameter (just an underscore) in the example bash -c 'dosomething "$1"' _ {} – Mouflon

export -f didn't work for me. which shell is this for? – Hemipode

@DennisWilliamson it is a bad practice, see comments in another answer. – Reduplicative

Answer 2

find . | while read file; do dosomething "$file"; done

Hopeh

Nice solution. Doesn't require exporting the function or messing around escaping arguments and is presumably more efficient since it's not spawning subshells to execute each function. – Zigzagger

Keep in mind, though, that it will break on filenames containing newlines. – Dieselelectric

This is more "shell'ish" as your global variables and functions will be available, without creating an entirely new shell/environment each time. Learned this the hard way after trying Adam's method and running into all sorts of environment problems. This method also does not corrupt your current user's shell with all the exports and requires less dicipline. – Pachysandra

I keep having problems where, when multiple files are found, some are randomly skipped when doing this. No weird characters in the filenames, either. Using bash 4.2.46, GNU find 4.5.11, RHEL 7 – Hannus

looks like while read has issues when running commands with ssh; #13800725 – Hannus

also I fixed my issue by changing while read for a for loop; for item in $(find . ); do some_function "${item}"; done – Hannus

user5359531, that won't work with evil filenames since the output of find is expanded onto the command line and thus subject to word splitting. It's basically only reliable to expand "$@" (or array elements or subscripts) after keyword 'in', and the double quotes are essential. – Herbarium

+1, this is probaby the best approach. See pajamian's answer for a solution that won't break on newline in filenames. – Shortly

Answer 3

Jac's answer is great, but it has a couple of pitfalls that are easily overcome:

find . -print0 | while IFS= read -r -d '' file; do dosomething "$file"; done

This uses null as a delimiter instead of a linefeed, so filenames with line feeds will work. It also uses the -r flag which disables backslash escaping, and without it backslashes in filenames won't work. It also clears IFS so that potential trailing white spaces in names are not discarded.

Harebrained

It's good for /bin/bash but will not work in /bin/sh. What's a pity. – Temptation

@РоманКоптев How fortunate that at least it works in /bin/bash. – Mouflon

This makes it impossible to interrupt the script. – Bathroom

Answer 4

Add quotes in {} as shown below:

export -f dosomething
find . -exec bash -c 'dosomething "{}"' \;

This corrects any error due to special characters returned by find, for example files with parentheses in their name.

Tailwind

This is not the correct way to use {}. This will break for a filename containing double quotes. touch '"; rm -rf .; echo "I deleted all you files, haha. Oops. – Muscolo

Yes, this is very bad. It can be exploited by injections. Very unsafe. Do not use this! – Olden

@kdubs: Use $0 (unquoted) within the command-string and pass the filename as the first argument: -exec bash -c 'echo $0' '{}' \; Note that when using bash -c, $0 is the first argument, not the script name. – Mouflon

@Mouflon You should double quote $0 to avoid word splitting. But in Bash it does not seem to be neccessary to quote {}. I guess it is necessary for some shell since they tell you to quote it in manual page of find. – Reduplicative

Answer 5

Processing results in bulk

For increased efficiency, many people use xargs to process results in bulk, but it is very dangerous. Because of that there was an alternate method introduced into find that executes results in bulk.

Note though that this method might come with some caveats like for example a requirement in POSIX-find to have {} at the end of the command.

export -f dosomething
find . -exec bash -c 'for f; do dosomething "$f"; done' _ {} +

find will pass many results as arguments to a single call of bash and the for-loop iterates through those arguments, executing the function dosomething on each one of those.

The above solution starts arguments at $1, which is why there is a _ (which represents $0).

Processing results one by one

In the same way, I think that the accepted top answer should be corrected to be

export -f dosomething
find . -exec bash -c 'dosomething "$1"' _ {} \;

This is not only more sane, because arguments should always start at $1, but also using $0 could lead to unexpected behavior if the filename returned by find has special meaning to the shell.

Olden

Answer 6

Have the script call itself, passing each item found as an argument:

#!/bin/bash

if [ ! $1 == "" ] ; then
   echo "doing something with $1"
   exit 0
fi

find . -exec $0 {} \;

exit 0

When you run the script by itself, it finds what you are looking for and calls itself passing each find result as the argument. When the script is run with an argument, it executes the commands on the argument and then exits.

Darkling

cool idea but bad style: uses same script for two purposes. if you want to reduce the number of files in your bin/ then you could merge all your scripts into a single one that has a big case clause at the start. very clean solution, isn't it? – Cysto

not to mention this will fail with find: ‘myscript.sh’: No such file or directory if started as bash myscript.sh... – Emmons

Answer 7

Just a warning regaring the accepted answer that is using a shell, despite it well answer the question, it might not be the most efficient way to exec some code on find results:

Here is a benchmark under bash of all kind of solutions, including a simple for loop case: (1465 directories, on a standard hard drive, armv7l GNU/Linux synology_armada38x_ds218j)

dosomething() { echo $1; }

export -f dosomething
time find . -type d -exec bash -c 'dosomething "$0"' {} \; 
real    0m16.102s

time while read -d '' filename; do   dosomething "${filename}" </dev/null; done < <(find . -type d -print0) 
real    0m0.364s

time find . -type d | while read file; do dosomething "$file"; done 
real    0m0.340s

time for dir in $(find . -type d); do dosomething $dir; done 
real    0m0.337s

"find | while" and "for loop" seems best and similar in speed.

Fado

Answer 8

For those of you looking for a Bash function that will execute a given command on all files in current directory, I have compiled one from the above answers:

toall(){
    find . -type f | while read file; do "$1" "$file"; done
}

Note that it breaks with file names containing spaces (see below).

As an example, take this function:

world(){
    sed -i 's_hello_world_g' "$1"
}

Say I wanted to change all instances of "hello" to "world" in all files in the current directory. I would do:

toall world

To be safe with any symbols in filenames, use:

toall(){
    find . -type f -print0 | while IFS= read -r -d '' file; do "$1" "$file"; done
}

(but you need a find that handles -print0 e.g., GNU find).

Coddle

Answer 9

It is not possible to executable a function that way.

To overcome this you can place your function in a shell script and call that from find

# dosomething.sh
dosomething () {
  echo "doing something with $1"
}
dosomething $1

Now use it in find as:

find . -exec dosomething.sh {} \;

Accusatory

Was trying to avoid more files in my ~/bin. Thanks though! – Triny

I considered downvoting but the solution in itself is not bad. Please just use correct quoting: dosomething $1 => dosomething "$1" and start your file correctly with find . -exec bash dosomething.sh {} \; – Emmons

This is the correct approach. There's reallly no concern about additional files in ~/bin; presumably you already have a definition of dosomething in a startup file somewhere, and proper maintenance of your startup files will have you splitting them into distinct files anyway, so you might as well put that definition in an executable script. – Gunflint

Answer 10

To provide additions and clarifications to some of the other answers, if you are using the bulk option for exec or execdir (-exec command {} +), and want to retrieve all the positional arguments, you need to consider the handling of $0 with bash -c.

More concretely, consider the command below, which uses bash -c as suggested above, and simply echoes out file paths ending with '.wav' from each directory it finds:

find "$1" -name '*.wav' -execdir bash -c 'echo "$@"' _ {} +

The Bash manual says:

If the -c option is present, then commands are read from the first non-option argument command_string. If there are arguments after the command_string, they are assigned to positional parameters, starting with $0.

Here, 'echo "$@"' is the command string, and _ {} are the arguments after the command string. Note that $@ is a special positional parameter in Bash that expands to all the positional parameters starting from 1. Also note that with the -c option, the first argument is assigned to positional parameter $0.

This means that if you try to access all of the positional parameters with $@, you will only get parameters starting from $1 and up. That is the reason why Dominik's answer has the _, which is a dummy argument to fill parameter $0, so all of the arguments we want are available later if we use $@ parameter expansion for instance, or the for loop as in that answer.

Of course, similar to the accepted answer, bash -c 'shell_function "$0" "$@"' would also work by explicitly passing $0, but again, you would have to keep in mind that $@ won't work as expected.

Hutchings

Answer 11

Put the function in a separate file and get find to execute that.

Shell functions are internal to the shell they're defined in; find will never be able to see them.

Eichhorn

Gotcha; makes sense. Was trying to avoid more files in my ~/bin though. – Triny

Answer 12

I find the easiest way is as follows, repeating two commands in a single do:

func_one () {
  echo "The first thing with $1"
}

func_two () {
  echo "The second thing with $1"
}

find . -type f | while read file; do func_one $file; func_two $file; done

Inauspicious

This has all the usual beginner mistakes, and will break on many different types of unusual filenames (filenames with newlines, filenames with backslashes, filenames with irregular whitespace, filenames with wildcard characters in them, etc). – Sheepskin

Answer 13

Not directly, no. Find is executing in a separate process, not in your shell.

Create a shell script that does the same job as your function and find can -exec that.

Louielouis

Was trying to avoid more files in my ~/bin. Thanks though! – Triny

Answer 14

I would avoid using -exec altogether. Use xargs:

find . -name <script/command you're searching for> | xargs bash -c

Repute

At the time, IIRC it was in an attempt to reduce the amount of resources used. Think finding millions of empty files and deleting them. – Triny