Convert all EOL (dos->unix) of all files in a directory and sub-directories recursively without dos2unix

Question

How do I convert all EOL (dos->unix) of all files in a directory and sub-directories recursively without dos2unix? (I do not have it and cannot install it.)

Is there a way to do it using tr -d '\r' and pipes? If so, how?

Marcmarcano

Answer 1

For all files in current directory you can do it with a Perl one-liner: perl -pi -e 's/\r\n/\n/g' * (stolen from here)

EDIT: And with a small modification you can do subdirectory recursion:

find | xargs perl -pi -e 's/\r\n/\n/g'

Lambrecht

If your find and xargs support it: find -print0 | xargs -0 ... will handle filenames with whitespace. – Middleaged

Watch out if you use this command on Git repository. Exclude .git subdirectory from find. – Woods

Answer 2

You can use sed's -i flag to change the files in-place:

find . -type f -exec sed -i 's/\x0d//g' {} \+

If I were you, I would keep the files around to make sure the operation went okay. Then you can delete the temporary files when you get done. This can be done like so:

find . -type f -exec sed -i'.OLD' 's/\x0d//g' {} \+
find . -type f -name '*.OLD' -delete

Lustre

This should handle filenames which contain whitespace. Won't adding a newline at the end of each line cause the files to become double-spaced - or even triple spaced (since you're replacing the carriage returns with newlines)? Yes, I just tested it. – Middleaged

@DennisWilliamson I just yanked that from a script I have at the office without even thinking twice. Now I'm wondering how those scripts work at all. (It might have to do with the fact that they were meant to convert files created in Microsoft Office for Mac.) The new version should work properly. – Lustre

Answer 3

Do you have sane file names and directory names without spaces, etc in them?

If so, it is not too hard. If you've got to deal with arbitrary names containing newlines and spaces, etc, then you have to work harder than this.

tmp=${TMPDIR:-/tmp}/crlf.$$
trap "rm -f $tmp.?; exit 1" 0 1 2 3 13 15

find . -type f -print |
while read name
do
    tr -d '\015' < $name > $tmp.1
    mv $tmp.1 $name
done

rm -f $tmp.?
trap 0
exit 0

The trap stuff ensures you don't get temporary files left around. There other tricks you can pull, with more random names for your temporary file names. You don't normally need them unless you work in a hostile environment.

Mel

Note that tr actually recognizes \r (both BSD and GNU variants), so you can use tr -d '\r'. – Frydman

Answer 4

You can also use the editor in batch mode.

find . -type f -exec bash -c 'echo -ne "%s/\\\r//\nx\n" | ex "{}" ' \;

Zibet

This solution was the only one that worked for me on Mac OS X. The others got hung up on various issues like Can't do inplace edit: ./lang is not a regular file, <> line 22167. – Maunsell

Answer 5

If \r isn't followed by \n (maybe the case in files of Tim Pote):

• deleting \r (using tr -d) may remove newlines
• replacing \r with \n may not cause double / triple newlines

Maybe Tim Pote could verify the points above for the files he mentioned.

Lujan

This looks to me as a good comment on Tim Pote's answer, but it's not very clear on how it actually answers the question. – Pernik

Answer 6

This removes carriage returns from all files in the current directory and all subdirectories, and should work on most Unix-like OSs:

grep -lIUre '\r' | xargs sed -i 's/\r//'

Polyamide

Answer 7

If its done in widows:

try to run the command in git bash:

$ find | xargs perl -pi -e 's/\r\n/\n/g'

It can show some Can't do inplace edit: type a message so ignore it

Sepsis