How to replace spaces in file names using a bash script

Question

Can anyone recommend a safe solution to recursively replace spaces with underscores in file and directory names starting from a given root directory? For example:

$ tree
.
|-- a dir
|   `-- file with spaces.txt
`-- b dir
    |-- another file with spaces.txt
    `-- yet another file with spaces.pdf

becomes:

$ tree
.
|-- a_dir
|   `-- file_with_spaces.txt
`-- b_dir
    |-- another_file_with_spaces.txt
    `-- yet_another_file_with_spaces.pdf

Conclusion

What do you want to happen if there is a file called foo bar and another file called foo_bar in the same directory? – Beefcake

Good question. I wouldn't want to overwrite existing files or lose any data. It should leave it unchanged.. ideally printing a warning but that's probably asking too much. – Conclusion

Answer 1

Use rename (aka prename) which is a Perl script which may be on your system already. Do it in two steps:

find . -name "* *" -type d | rename 's/ /_/g'    # do the directories first
find . -name "* *" -type f | rename 's/ /_/g'

Based on Jürgen's answer and able to handle multiple layers of files and directories in a single bound using the "Revision 1.5 1998/12/18 16:16:31 rmb1" version of /usr/bin/rename (a Perl script):

find . -depth -name "* *" -execdir rename 's/ /_/g' "{}" \;

Metrology

No need for two steps: Use Depth-first search: find dir -depth – Willock

hmm.. Dennis, what would happen if you have "a a", "a a/b b" directories? Wouldn't it try to rename "a a" to "a_a" and then "a a/b b" (which doesn't exist anymore) to "a_a/b_b"? – Jahveh

Oh, I've just read the rename manpage (I didn't know the tool) and I think you can optimize your code by changing s/ /_/g to y/ /_/ ;-) – Jahveh

Micro-optimization. In my tests there was negligible difference in speed. time for i in {1..2000}; do echo "a b c d [repeated to a length of 320 characters]" | perl -pe 'y/ abcdefghi/_ABCDEFGHI/' >/dev/null; done compared to 's/ /_/g; s/a/A/g; s/b/B/g; s/c/C/g; s/d/D/g; s/e/E/g; s/f/F/g; s/g/G/g; s/h/H/g; s/i/I/g'. Of course, the transliteration command has its advantages, but so does the substitute command. – Metrology

Of course you're not going to get a performance boost from it. It's more about using the right tool. And this whole question is about micro-optimizing more or less. Isn't it fun, after all? ;-) – Jahveh

If you're running this on OS X, you'll need to brew install rename – Saltzman

Note that the command has a trailing ; issue when aliased in a bash profile. But it is totally fine in a bash profile function. – Haplo

@PhysicalChemist: Can you be more specific? – Metrology

Sure. Bash Profile Alias: alias space_to_under='find . -depth -name "* *" -execdir rename 's/ /_/g' "{}" \;' and the Terminal Output after sourcing -bash: alias: /_/g "{}" \;: not found – Haplo

@PhysicalChemist: You have single quotes inside your alias. Try alias space_to_under='find . -depth -name "* *" -execdir rename "s/ /_/g" "{}" \;' or alias space_to_under='find . -depth -name "* *" -execdir rename '\''s/ /_/g'\'' "{}" \;' – Metrology

You forgot to specify . in the find command so there is no directory root to search. – Incertitude

@FredConcklin: In GNU find . (the current directory) is the default search root. – Metrology

This doesn't work on Centos 7, as the rename command is completely different (it's a binary, not a perl script), and it doesn't accept data from stdin. – Nernst

@CpnCrunch: The Perl script is very short and various versions are available on CPAN and elsewhere. – Metrology

@Nernst Same in RHEL 6.2 and Cygwin (rename --version says rename from util-linux 2.x.x, but a good tool for mass renaming anyway – Magel

rename seems to be the most incompatible script (fedora don't work, custom ubuntu half does) - could someone please include a list of working rename versions.... – Stanza

On OS X, first do this: https://mcmap.net/q/94031/-how-can-i-make-the-quot-find-quot-command-on-os-x-default-to-the-current-directory $brew install findutils rename $ alias find=gfind – Jones

you cannot use rename on macOS unless you explicitly install it – Indevout

The question is tagged linux and the issue of OS X installation has been addressed in previous comments. – Metrology

@CpnCrunch, as the answer says, the utility is sometimes named prename. I found "prename" using Yum and it works as advertised. Now, writing a portable script, that is a whole different matter... – Shirline

Answer 2

I use:

for f in *\ *; do mv "$f" "${f// /_}"; done

Though it's not recursive, it's quite fast and simple. I'm sure someone here could update it to be recursive.

The ${f// /_} part utilizes bash's parameter expansion mechanism to replace a pattern within a parameter with supplied string. The relevant syntax is ${parameter/pattern/string}. See: https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html or http://wiki.bash-hackers.org/syntax/pe .

Manion

Simple and work in mac. (mac doesnt have rename, and its too hard to install this with brew..) – Stipitate

awesome answer. i used for d in *\ *; do mv "$d" "${d// /}"; done non under score. – Microelectronics

Unlike the 'find -name' answer, this one worked on my OS X! Thank you sir! – Grose

For reference, this can easily become recursive in bash for using shopt -s globstar and for f in **/*\ *; do .... The globstar option is internal to bash, whereas the rename command is a common Linux tool and not part of bash. – Tyrr

worked for me... plus one... Can anyone answer me how "${f// /_}" does this part work? – Duggins

${f// /_} is a Bash variable expansion for search and replace. - The f is the variable from the for loop for each file that contains a space. - The first // means "replace all" (don't stop at first occurrence). - Then the ` /_` means "replace space with underscore" – Amphicoelous

@Amphicoelous , thank you so much for explaning all three slashes! The original answer only explains two slashes, which I find very inconvenient. – Bench

based on your solution, I made it recursive: while read line ; do mv "$line" "${line// /}" ; done < <(find /path/ -iname "* *") – Oppilate

I wanted to remove spaces (as opposed to replacing with _). Removing the underscore in the statement worked: for f in *\ *; do mv "$f" "${f// /}"; done – Cholecystitis

What does '*\ *' mean? – Amidase

Any files with a space. * are wildcards and \ escapes the space – Manion

Answer 3

find . -depth -name '* *' \
| while IFS= read -r f ; do mv -i "$f" "$(dirname "$f")/$(basename "$f"|tr ' ' _)" ; done

failed to get it right at first, because I didn't think of directories.

Jahveh

Dennis, good catch, easily fixed by putting IFS='' in front of read. Also, for what I can tell by other comments, sort step can be dropped in favor of -depth option to find. – Jahveh

Does no't work if a filename contain a \ (backslash). Can be fixed by adding a -r option to read. – Alfreda

This must be the 50th time I visit this page to copy and use your solution. Thank you very much. I prefer your answer, as I am on a Mac and do not have the rename command suggested by Dennis. – Maya

@AlexConstantin, don't macports have the rename? I have never bothered to find out because I don't think the task justifies utility. And if you don't have macports, you should consider installing them ;) – Jahveh

For this particular task, installing anything extra seemed like cheating :) Also, I prefer homebrew – Maya

@AlexConstantin, agree on the first part and as for the brew — in this context it's synonymous to macports ;-) – Jahveh

That't kind of solution i like the most - works eyerywhere without instaling anything :) That's really important for me, because i'm using 3 different OS(Mac OS X, Windows with cygwin, Debian) and i don't like to think before each operation "did i installed this util here?" :) Thank you very much! :) – Otherworld

it works perfectly, I have only had to replace 'mv -i' by 'mv -f' to avoid the overwrite question. Many thanks. – Mauri

What does '* *' mean? – Amidase

@mtk, it's a glob pattern, basically "anything-space-anything". Could also be reworded as "anything containing space". – Jahveh

Answer 4

you can use detox by Doug Harple

detox -r <folder>

Banded

Answer 5

A find/rename solution. rename is part of util-linux.

You need to descend depth first, because a whitespace filename can be part of a whitespace directory:

find /tmp/ -depth -name "* *" -execdir rename " " "_" "{}" ";"

Orban

I get no change at all when I run yours. – Metrology

Check util-linux setup: $ rename --version rename (util-linux-ng 2.17.2) – Willock

Grepping /usr/bin/rename (a Perl script) reveals "Revision 1.5 1998/12/18 16:16:31 rmb1" – Metrology

Hmm... where is your util-linux binary gone? This file path should be owned by util-linux. You don't us a GNU-Linux system? – Willock

It's named rename.ul on my system. – Metrology

which only changes one space in my run, so "go tell fire on the mountain" becomes "go_tell fire on the mountain". – Mabel

Answer 6

you can use this:

find . -depth -name '* *' | while read fname 

do
        new_fname=`echo $fname | tr " " "_"`

        if [ -e $new_fname ]
        then
                echo "File $new_fname already exists. Not replacing $fname"
        else
                echo "Creating new file $new_fname to replace $fname"
                mv "$fname" $new_fname
        fi
done

Damaging

See the other answers using find, you should include the -depth flag to find. Otherwise you may rename directories before the files in the directories. Same issue with dirname and basename so you don't try to rename dir one/file two in one step. – Inequitable

Answer 7

bash 4.0

#!/bin/bash
shopt -s globstar
for file in **/*\ *
do 
    mv "$file" "${file// /_}"       
done

Inveigh

Looks like this will do a mv to itself if a file or directory name has no space in it (mv: cannot move a' to a subdirectory of itself, a/a') – Conclusion

don't matter. just remove the error message by redirecting to /dev/null. – Inveigh

ghostdog, spawning mv fifty five thousands times only to rename four files may be a bit of overhead even if you don't flood user with messages. – Jahveh

krelin, even find will go through those 55000 files you mentioned to find those with spaces and then do the rename. At the back end, its still going through all. If you want, an initial check for spaces before rename will do it . – Inveigh

I was talking about spawning mv, not going through. Wouldn't for file in *' '* or some such do a better job? – Jahveh

that syntax doesn't recurse directory – Inveigh

Your edit is exactly the "somesuch" I was talking about. ;-) I didn't know what ** is, anyway. Now that you explained that your code expands into the list of all files I think it may turn out to be way too resource consuming for otherwise bearable directory tree ;-) – Jahveh

its the same as using find to recurse. – Inveigh

no, the memory consumption differs dramatically. With find you stream the results and process them one by one, with expansion you hold the whole thing in memory. – Jahveh

if OP has to start searching from /, then maybe not this method. Otherwise, its alright. And its definitely faster than your find+basename+tr solution on a given directory. lastly If you find this method not to your taste, then don't use it. Its just an alternative for OP. – Inveigh

What does it have to do with /? My openembedded build directory, for instance, has 898199 files (just counted). My solution may be slower, right, I needed that to handle the situation described in my comment to Dennis' answer. Note, that your solution suffers the same problem. Otherwise, yes, your solution would be valid even if I don't like it. – Jahveh

when i say "/", i mean if OP is searching the whole file system recursively..ie /etc/, /var/, /tmp, /usr/, /opt...and any other directories under "/". Then it might not be advisable to use globstar. – Inveigh

The bash string manipulation way ${file// /_} shown here is still something that makes this answer worthwhile. – Kraul

Answer 8

In macOS

Just like the chosen answer.

brew install rename

# 
cd <your dir>
find . -name "* *" -type d | rename 's/ /_/g'    # do the directories first
find . -name "* *" -type f | rename 's/ /_/g'

Lakitalaks

Answer 9

Recursive version of Naidim's Answers.

find . -name "* *" | awk '{ print length, $0 }' | sort -nr -s | cut -d" " -f2- | while read f; do base=$(basename "$f"); newbase="${base// /_}"; mv "$(dirname "$f")/$(basename "$f")" "$(dirname "$f")/$newbase"; done

Lheureux

Answer 10

For those struggling through this using macOS, first install all the tools:

 brew install tree findutils rename

Then when needed to rename, make an alias for GNU find (gfind) as find. Then run the code of @Michel Krelin:

alias find=gfind 
find . -depth -name '* *' \
| while IFS= read -r f ; do mv -i "$f" "$(dirname "$f")/$(basename "$f"|tr ' ' _)" ; done   

Jones

find . -depth -name '* *' \ | while IFS= read -r f ; do mv -i "$f" "$(dirname "$f")/$(basename "$f"|tr ' ' _)" ; done was the only solution that worked for me on Alpine Linux – Prolactin

Answer 11

An easy alternative to recursive version is to increase the range of for loop step by step(n times for n sub-levels irrespective of number of sub-directories at each level). i.e from the outermost directory run these.

for f in *; do mv "$f" "${f// /_}"; done 

for f in */*; do mv "$f" "${f// /_}"; done 

for f in */*/*; do mv "$f" "${f// /_}"; done 

To check/understand what's being done, run the following before and after the above steps.

for f in *;do echo $f;done 

for f in */*;do echo $f;done 

for f in */*/*;do echo $f;done 

Generalize

Answer 12

Here's a (quite verbose) find -exec solution which writes "file already exists" warnings to stderr:

function trspace() {
   declare dir name bname dname newname replace_char
   [ $# -lt 1 -o $# -gt 2 ] && { echo "usage: trspace dir char"; return 1; }
   dir="${1}"
   replace_char="${2:-_}"
   find "${dir}" -xdev -depth -name $'*[ \t\r\n\v\f]*' -exec bash -c '
      for ((i=1; i<=$#; i++)); do
         name="${@:i:1}"
         dname="${name%/*}"
         bname="${name##*/}"
         newname="${dname}/${bname//[[:space:]]/${0}}"
         if [[ -e "${newname}" ]]; then
            echo "Warning: file already exists: ${newname}" 1>&2
         else
            mv "${name}" "${newname}"
         fi
      done
  ' "${replace_char}" '{}' +
}

trspace rootdir _

Firecrest

Answer 13

This one does a little bit more. I use it to rename my downloaded torrents (no special characters (non-ASCII), spaces, multiple dots, etc.).

#!/usr/bin/perl

&rena(`find . -type d`);
&rena(`find . -type f`);

sub rena
{
    ($elems)=@_;
    @t=split /\n/,$elems;

    for $e (@t)
    {
    $_=$e;
    # remove ./ of find
    s/^\.\///;
    # non ascii transliterate
    tr [\200-\377][_];
    tr [\000-\40][_];
    # special characters we do not want in paths
    s/[ \-\,\;\?\+\'\"\!\[\]\(\)\@\#]/_/g;
    # multiple dots except for extension
    while (/\..*\./)
    {
        s/\./_/;
    }
    # only one _ consecutive
    s/_+/_/g;
    next if ($_ eq $e ) or ("./$_" eq $e);
    print "$e -> $_\n";
    rename ($e,$_);
    }
}

Longstanding

Answer 14

I found around this script, it may be interesting :)

 IFS=$'\n';for f in `find .`; do file=$(echo $f | tr [:blank:] '_'); [ -e $f ] && [ ! -e $file ] && mv "$f" $file;done;unset IFS

Soothsay

Fails on files with newlines in their name. – Tyrr

Answer 15

Here's a reasonably sized bash script solution

#!/bin/bash
(
IFS=$'\n'
    for y in $(ls $1)
      do
         mv $1/`echo $y | sed 's/ /\\ /g'` $1/`echo "$y" | sed 's/ /_/g'`
      done
)

Neigh

Never parse the output of ls. – Tyrr

Answer 16

This only finds files inside the current directory and renames them. I have this aliased.

find ./ -name "* *" -type f -d 1 | perl -ple '$file = $_; $file =~ s/\s+/_/g; rename($_, $file);

Loan

Answer 17

I just make one for my own purpose. You may can use it as reference.

#!/bin/bash
cd /vzwhome/c0cheh1/dev_source/UB_14_8
for file in *
do
    echo $file
    cd "/vzwhome/c0cheh1/dev_source/UB_14_8/$file/Configuration/$file"
    echo "==> `pwd`"
    for subfile in *\ *; do [ -d "$subfile" ] && ( mv "$subfile" "$(echo $subfile | sed -e 's/ /_/g')" ); done
    ls
    cd /vzwhome/c0cheh1/dev_source/UB_14_8
done

Inwardly

Answer 18

For files in folder named /files

for i in `IFS="";find /files -name *\ *`
do
   echo $i
done > /tmp/list


while read line
do
   mv "$line" `echo $line | sed 's/ /_/g'`
done < /tmp/list

rm /tmp/list

Sigler

Answer 19

My solution to the problem is a bash script:

#!/bin/bash
directory=$1
cd "$directory"
while [ "$(find ./ -regex '.* .*' | wc -l)" -gt 0 ];
do filename="$(find ./ -regex '.* .*' | head -n 1)"
mv "$filename" "$(echo "$filename" | sed 's|'" "'|_|g')"
done

just put the directory name, on which you want to apply the script, as an argument after executing the script.

Donica

Answer 20

Use below command to replace space with underscore in filename as well as directory name.

find -name "* *" -print0 | sort -rz | \
  while read -d $'\0' f; do mv -v "$f" "$(dirname "$f")/$(basename "${f// /_}")"; done

Unwind

Answer 21

If you need to rename only files in one directory by replacing all spaces. Then you can use this command with rename.ul:

for i in *' '*; do rename.ul ' ' '_' *; done

Medieval

Answer 22

Actually, there's no need to use rename script in perl:

find . -depth -name "* *" -execdir bash -c 'mv "$1" `echo $1 | sed s/ /_/g`' -- {} \;

Vassili

Answer 23

use fd and rename fd find all files recursively, then use rename to replace space with _

fd -X rename 's/ /_/g' {}

Tacitus