dos2unix doesn't convert ^M

Question

I exported results in a text file from a program running on Windows 7, and copied the file on Xubuntu 14.04. In a terminal, I ran dos2unix file.txt, which tells me converting file out_mapqtl.txt to Unix format. However, when I look at the file with less, I still see the Windows end-of-line as ^M, and wc -l returns me "0".

I tried several things described here, but none works. I then opened the file in Vim and did :%s/\r/\r/g as explained there, which worked fine. So any idea why dos2unix didn't work? Would there be a way to avoid opening Vim every time?

Pash

A bit late here... But I wrote a small program that makes life easier than dos2unix when you're not sure about the input format, or when input formats are intermingled : github.com/mdolidon/endlines – Evaporation

@MathiasDolidon Thank you, Mathias! – Proverbial

Your question can be rephrased as: Why doesn't dos2unix modify a text file? ~ * ~ Here is a question asking the opposite: Why does dos2unix modify a binary file? ~ * ~ A simple answer to both questions is: because dos2unix isn't foolproof. – Unipolar

Answer 1

\r denotes a carriage return, and on MAC it is used without \n to denote a line break. Are you sure the file is in DOS (\r\n) format and not MAC (\r)?

If VIM really turns out to be the only thing that'll repair your files, you can also invoke it as:

vim somefile.txt +"%s/\r/\r/g" +wq

This will open the file, perform the operation, save it, then quit.

Can you give us an example of the file, so that we can investigate further?

Avictor

sorry for the delay of my answer. The file was produced on Windows 7, thus I assume it is in the DOS format. I'm not even sure the program runs on Mac. By the way, the program is called MapQTL. Anyway, my problem is now solved, thanks. – Pash

Thanks for letting us know your problem has been solved, and accepting my solution as the answer. – Avictor

hi, what does " %s/\r/\r/g" do ? it looks like sed expression . replace '\r' with "\r"? – Earthlight

yup, that's what it does. The % in front applies the expression to all lines in the file. Due to the line-ending-handling magic of VIM, this seemingly useless expression apprarently does something. – Avictor

Answer 2

I know you have gotten this resolved, but I wanted to add a note for reference, based on some testing I've done.

If less is showing ^M, then like Sybren I suspect it is a MAC style ending (\r), not DOS (\r\n). You can determine that easily using cat:

$ cat -e filename

• Unix endings (\n) show as $
• MAC endings (\r) show as ^M (less shows these)
• DOS\Windows endings (\r\n) show as ^M$ (less does not appear to show these)

Use dos2unix to get rid of the DOS (^M$) endings

Use mac2unix to get rid of the MAC (^M) endings - dos2unix won't get rid of these.

I had a file where I had to use dos2unix and mac2unix to get rid of all the non-Unix endings.

Racialism

thanks for this answer, I was stumped running dos2unix, didn't realise the file was using the old mac line endings somehow. – Sequela

IMO, should be actually the answer. – Rondi

Answer 3

Try this:

tr -d '\r' < file

Searles

Answer 4

I have used Notepad++ feature: Edit>EOL Conversions>Unix(LF).

Now export this file to the Unix machine using pscp.exe.

Let me know if that worked for you.

Munday