How to automate dos2unix using shell script?
Asked Answered
B

4

8

I have a bunch of xml files in a directory that need to have the dos2unix command performed on them and new files will be added every so often. I Instead of manually performing dos2unix command on each files everytime I would like to automate it all with a script. I have never even looked at a shell script in my life but so far I have this from what I have read on a few tutorials:

FILES=/tmp/testFiles/*
for f in $FILES
do
  fname=`basename $f`
  dos2unix *.xml $f $fname
done

However I keep getting the 'usage' output showing up. I think the problem is that I am not assigning the name of the new file correctly (fname).

Buchalter answered 2/7, 2012 at 10:39 Comment(0)
N
10

The reason you're getting a usage message is that dos2unix doesn't take the extra arguments you're supplying. It will, however, accept multiple filenames (also via globs). You don't need a loop unless you're processing more files than can be accepted on the command line.

dos2unix /tmp/testFiles/*.xml

Should be all you need, unless you need recursion:

find /tmp/testFiles -name '*.xml' -exec dos2unix {} +

(for GNU find)

Nobie answered 2/7, 2012 at 11:31 Comment(0)
A
2

If all files are in one directory (no recursion needed) then you're almost there.

for file in /tmp/testFiles/*.xml ; do
    dos2unix "$file"
done

By default dos2unix should convert in place and overwrite the original.

If recursion is needed you'll have to use find as well:

find /tmp/testFiles -name '*.xml' -print0 | while IFS= read -d '' file ; do
    dos2unix "$file"
done

Which will work on all files ending with .xml in /tmp/testFiles/ and all of its sub-directories.

If no other step are required you can skip the shell loop entirely:

Non-recursive:

find /tmp/testFiles -maxdepth 1 -name '*.xml' -exec dos2unix {} +

And for recursive:

find /tmp/testFiles -name '*.xml' -exec dos2unix {} +

In your original command I see you finding the base name of each file name and trying to pass that to dos2unix, but your intent is not clear. Later, in a comment, you say you just want to overwrite the files. My solution performs the conversion, creates no backups and overwrites the original with the converted version. I hope this was your intent.

Anywhere answered 2/7, 2012 at 11:22 Comment(2)
He's not storing the results of an expanded glob. The glob is not expanded on assignment.Nobie
@DennisWilliamson: I see that you are right. I have removed this comment, thank you for the correction.Anywhere
A
1
mkdir /tmp/testFiles/converted/
for f in /tmp/testFiles/*.xml
do
  fname=`basename $f`
  dos2unix $f ${f/testFiles\//testFiles\/converted\/}
  # or for pure sh:
  # dos2unix $f $(echo $f | sed s@testFiles/@testFiles/converted/@)
done

The result will be saved in the converted/ subdirectory.

The construction ${f/testFiles\//testFiles\/converted\/} (thanks to Rush) or sed is used here to add converted/ before the name of the file:

$ echo /tmp/testFiles/1.xml | sed s@testFiles/@testFiles/converted/@
/tmp/testFiles/converted/1.xml
Ardath answered 2/7, 2012 at 10:43 Comment(8)
Thank you but can you explain what this does: "$(echo $f | sed s@testFiles/@testFiles/converted/)" Edit: this command does not work. Error: sed: command garbled: s@testFiles/@testFiles/converted/, removing the sed command gives error: COuldn't open input file FILESBuchalter
I just add converted/ before that name of the file.Ardath
You don't need sed for that. You can do it with ${f/testFiles\//testFiles\/converted\/}Gertiegertrud
@Rush: that works only in bash not in Bourne shell (/bin/sh)Ardath
Guys, I want to overwrite the files so moving the new files to another directory isn't an issue. The above command doesn't work. Any other suggestions? Thanks.Buchalter
What does it mean "doesn't work"? Which error do you get? Can please show the error message?Ardath
@AlanSmith: How about find /tmp/testFiles/ -maxdepth 1 -type f -exec dos2unix {} + (if you dont want to have a backup)?Devonna
@IgorChubin: This question is tagged bash and is not tagged sh or shell or posix-sh, so I think bashisms are acceptable.Anywhere
C
-1

It is not clear which implementation of dos2unix you are using. Different implementations require different arguments. There are many different implementations around.

On RedHat/Fedora/Suse Linux you could just type

    dos2unix /tmp/testFiles/*.xml

On SunOS you are required to give an input and output file name, and the above command would destroy several of your files.

Conjunctive answered 23/12, 2012 at 9:31 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.