How to Use Find - exec and Tr to process a large number of files

Question

I'm having a little trouble with find and exec in bash:

Suppose I have a bunch of files that I need to replace '\r' characters in. (previous question: Joining Columns on Command Line with Paste or PR Not Working) For each file, I want to read it, and replace all the '\r', and then write it back to the same file name:

The command I'm using is find . -exec cat {} | tr -d "\r" > {} \;, but I'm getting two errors:

tr: extra operand `;'
Only one string may be given when deleting without squeezing repeats.
Try `tr --help' for more information.
find: missing argument to `-exec'

it seems like tr is interpreting ';' to be an argument, while -exec is not recognizing it. Is there a way to change this? I'm also creating {} as a file in the directory, instead of having {} being substituted for the file name.

I've also tried:

find . -exec cat {} | tr -d "\r" > "new_{}"; \;

but "new_{}" does not turn into "new_filename", bash just takes it literally and creates a file called "new{}".

Thanks!

Chisholm

Answer 1

If you want to redirect the output of a command used with -exec, you need to execute a shell command. Moreover you'd need to redirect the output to a new file and move it back to the original filename.

Say:

find . -type f -exec sh -c 'tr -d "\r" < "{}" > "{}".new && mv "{}".new "{}"' -- {} \;

An alternate would be to use sed:

find . -type f -exec sed -i 's/\r//' {} \;

or dos2unix (as pointed out by kojiro):

find . -type f -exec dos2unix {} \;

Studdard

This seems to work, but when running file on the new file, it still says: ASCII text, with CRLF line terminators, so it seems like it didn't replace the CR – Chisholm

@MaxSong The quoting in the first option was slightly wrong. Should be fixed now. – Studdard

Wonderful! Both commands work perfectly. Thanks for the reply! – Chisholm

Probably worth pointing out that dos2unix will do an in-place overwrite for you, vastly simplifying the commands needed. – Ocher

@Ocher Thanks, added that option. I actually started by telling the OP of the problem and the most obvious failed to occur! – Studdard

If you're anal-retentive like me, you'd probably do something like find . -type f -exec grep -qF $'\r' {} \; -exec dos2unix {} + – this would avoid touching files that don't have to be touched. – Ocher

Don't inject the filenames directly into the script. Pass them as argument(s). Instead of -exec sh -c '..."{}"...' -- {} \;, do: -exec sh -c '..."$1"...' sh {} \; – Hyperkinesia

@Studdard why this difference with respect to executing the command in a shell between tr and sed? – Nurseryman