Explain how Windows batch newline variable hack works

Question

Can someone please explain how this works?

@echo off
REM Creating a Newline variable (the two blank lines are required!)
set NLM=^


set NL=^^^%NLM%%NLM%^%NLM%%NLM%
REM Example Usage:
echo There should be a newline%NL%inserted here.

emits:

There should be a newline
inserted here.

from How can you echo a newline in batch files?

Vagary

Is there a reason to do this other than to create a new blank line? – Cheung

There are many, I added some explanations to my answer – Revelry

Answer 1

The trick uses the behaviour of the caret.
Also explained at Long commands split over multiple lines in Windows Vista batch (.bat) file

The caret is an escape character for the next character, or at the line end it is used as multiline character, but this is nearly the same.

At the line end it simply escapes the next character, in this case the <Linefeed>, but there is a hidden feature, so if the escaped character is a <LF> it is ignored and the next character is read and escaped, but this charater will be always escaped, even if it is also a <LF>.

Now you can understand

set NLM=^


rem Two empty lines are required here

The NLM-Variable contains exactly one <LF> character. But if you try to use it with echo Line1%NLM%Line2 it fails, as the parser stops parsing at a single <LF>.
But this works

echo Line1^

Line2

So you need to add an escaped linefeed into the line and that is the NL-Variable. The NL-Variable consists of only three characters.
NL=^<LF><LF> And if this is expanded, it creates only one escaped <LF> as the first <LF> after the caret will be ignored.

Btw. In my opinion, it is much easier to use linefeeds with delayed expansion, as there is no need to escape anything.
In this example I use %=EMPTY=% instead of an empty line (for self commenting), but as the variable =EMPTY= can't exists it will be expanded to an empty line.

setlocal EnableDelayedExpansion
(set NLM=^
%=EMPTY=%
)
echo Line1!NLM!Line2

EDIT: Append some hints for useful using the <LF>

1) Use it as newline in an echo

setlocal EnableDelayedExpansion
(set LF=^
%=EMPTY=%
)
echo Line1!LF!Line2

2) Use it to split commands in a parenthesis block

setlocal EnableDelayedExpansion
(set LF=^
%=EMPTY=%
)
(
    echo Line1%LF%set x=4%LF%echo !x!%LF%
)

3) Create a (nearly) empty EOL-chararcter in a FOR /F loop, as <LF> is the line delimiter an EOL of <LF> is the same than an empty one.

FOR /F ^"eol^=^

delims^=^" %%a in (myFile.php) do echo %%a

4) Use LF for splitting text in a FOR /F loop

setlocal EnableDelayedExpansion
(set LF=^
%=EMPTY=%
)
set "var=Content1;Content2"
FOR /F "delims=" %%a in ("%var:;=!LF!%") do (
  echo %%a
)

Revelry

Great topic! How do I insert a newline character into a string that I will echo after I get the multi-line string built? Thanks – Nipa

@Nipa I'm not sure that I got your question, but perhaps you mean set multiStr=Line1!LF!Line2 & echo !multiStr! – Revelry

Ahhh yes! Got it. Thanks jeb – Nipa

Just one more puzzle: you say the next character is <LF> but in fact it's the <CR> that is in front of the <LF>, isn't it? – Gulick

@Gulick Yes, but carriage returns are removed by the parser in the previous phase – Revelry

Answer 2

There seems a way that also works with pipe:

(echo 1st line^
&echo 2nd line) | sort

Heliotrope

But this also sorts each line. So this trick does not work for concatenating multi-line files – Aloha