sed replace multiple blank lines with single blank line

Question

I have written a sed script, which replaces multiple blank lines with a single one, but it is not working as supposed to. I will be thankful to everybody, who can explain me why. Please do not refer me to working examples, I am familiar with Google. I just want to understand how sed works.

The code is

sed ':a;/^\n*$/{N;ba};s/^\n\n*/\n/' input_file

so the logic is simple: when sed reads the line and it is either blank or has several newline symbols (this is /^*\n$ condition), I tell sed to append the next line to pattern space. as soon as a non-blank line is found, the substitution s/^\n\n*/\n/ is done.

Everything works fine except the cases when I have blank lines in the end of the file. These blanks are not replaced with a single blank and I do not understand why.

Any ideas?

Illustrational

Answer 1

The problem is sed getting EOF while reading next line.

Your command is getting completed at the time of reading next line using N. Because, While reading the next line, sed getting EOF . So, It won't process s/^\n\n*/\n/ substitution. That is why, You are not able to remove sequence of empty lines which has appeared in end of file.

My solution is:

sed ':a; /^\n*$/{ s/\n//; N;  ba};' yourfile

Pulvinus

can you explain the syntax – Zooplankton

also i found another answer sed -i '/^$/d;G' DjangoDocumentaion.md which Delete blank lines and insert one blank line after each line – Zooplankton

Answer 2

Instead of \n\n*, you could use \n+

By the way, this is what it would look like as a Perl one-liner:

perl -0777 -pe 's/\n+/\n/g' yourfile

Or, for the same result but fewer substitutions:

perl -0777 -pe 's/\n\n+/\n/g' yourfile

Or, even more magical, try this compact solution suggested by @hwnd:

perl -00 -pe ''

Keil

op wants to replace all the consecutive blanklines with a single blank line. You command removes all the blank lines. – Zettazeugma

this does not address the issue, newlines at the end of the file will still be ignored if you use \n+. Plus, OP specified he wanted to work with sed – Fruin

@Keil If you want to remove consecutive blank lines, use perl -00 -pe '' – Raskind