Why doesn't Rebol new-line? treat the newline keyword and the newline character similarly?

Question

I would think that the following Rebol 3 code:

x: [newline 1 2]
y: [
1 2]

print x
print new-line? x
print y
print new-line? y

should output:

<empty line>
1 2
true
<empty line>
1 2
true

but what is output is:

<empty line>
1 2
false
1 2
true

Both blocks, when reduced, should yield a newline character followed by '1' and '2' and so, IMO, should print identically. Less clear is whether new-line? on the two blocks should also give the same result since the newline keyword should be equivalent to the literal newline for this kind of test.

Colourable

Answer 1

The flag which is checked by new-line? and set by new-line is used only by LOAD and MOLD. For all other semantic purposes in the program, it might as well not be there.

Therefore your x and y are completely different. Note that:

x: [newline 1 2]
y: [
1 2]

3 = length? x
2 = length? y

It's a quirk of Rebol that it singles out this one bit of whitespace information to stow in a hidden place. But arguably the choice to break a line represents something that is often significant in source, that if you reflect it back out into text you'd like to preserve more than the rest of the whitespace.

Sixgun

Answer 2

Let's start with NEWLINE: newline is a word bound to a char! value:

>> ? newline
NEWLINE is a char of value: #"^/"

That's Rebol's escape sequence for Unicode codepoint U+000A, commonly used as line feed ("LF") control code.

So your first example code [newline 1 2] has nothing to do with the NEW-LINE function. It simply describes a block containing three values: newline (a word!), 2 (an integer!), and 3 (another integer!). If you REDUCE the block from your first example, you'll get another block of three values: char!, integer!, and integer!:

>> reduce [newline 1 2]
== [#"^/" 1 2]

Now PRINT does not only REDUCE, it does REFORM (first REDUCE, then FORM) a block argument. FORM of a block converts the elements to a string representation and then joins those with spaces in between:

>> form [1 2 3]
== "1 2 3"

Putting those pieces together we finally know how to arrive at the output you are seeing for your first example:

>> basis: [newline 1 2 3]
== [newline 1 2 3]
>> step1: reduce basis
== [#"^/" 1 2 3]
>> step2: form step1
== "^/1 2 3"
>> print step2

1 2 3

So the question remains, why does the second example not print identically?

That's because FORM (used by PRINT, as described above) does not respect the NEW-LINE flag when converting from a block to a string.

This flag is "meta-data", not unlike e.g. the index position of an element within a block. So just as you don't have elements at index positions 8 and 6 just because you write a block like [8 6], you don't set the new-line flag for a position just because you happen to put an element there which is a character that on some systems represents a line break: [1 newline 2].

And this finally brings us to the last part of the puzzle: NEW-LINE? does not check if a given string represents a line break. It checks if a block (at its current position) has the new-line flag set.

Terryn