How do I handle newlines in JSON?

Question

I've generated some JSON and I'm trying to pull it into an object in JavaScript. I keep getting errors. Here's what I have:

var data = '{"count" : 1, "stack" : "sometext\n\n"}';
var dataObj = eval('('+data+')');

This gives me an error:

unterminated string literal

With JSON.parse(data), I see similar error messages: "Unexpected token ↵" in Chrome, and "unterminated string literal" in Firefox and IE.

When I take out the \n after sometext the error goes away in both cases. I can't seem to figure out why the \n makes eval and JSON.parse fail.

Sever

Try using a real json parser instead of eval. – Accommodation

Would like to mention that, at least for c#.net that escaping the characters on the server side was not working for us (they'd get un-escaped by the serialization layer I guess). The trick that did it for us was to perform the escapes on the CLIENT. – Westnorthwest

Answer 1

This is what you want:

var data = '{"count" : 1, "stack" : "sometext\\n\\n"}';

You need to escape the \ in your string (turning it into a double-\), otherwise it will become a newline in the JSON source, not the JSON data.

Sail

This is of course correct, but I'd like to add the reason for having to do this: the JSON spec at ietf.org/rfc/rfc4627.txt contains this sentence in section 2.5: "All Unicode characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)." Since a newline is a control character, it must be escaped. – Adey

+ 1. I was having trouble understanding JSON-encoding but "will become a newline in the JSON source, not the JSON data" made it clear for me. – Shaniceshanie

People may think escaping \n is escaping \ (don't know how to quote \ in line...) as \\ plus n. Actually, \n is a control character, denoted as \u000A as a unicode point. "\n" is exactly same as "\u000A". So escaping \n is escaping \u000A. Its escaped form is \n, and we should write \ as \\ in Javascript string, which is why "\\n" is the right answer. For powershell, ` instead of \ is the escape character. New line character is denoted as `n. ConvertTo-Json "`n" will get "\n", and ConvertFrom-Json ‘"Line 1\nLine 2"’ will get Line 1 and Line 2 in two lines. – Fatshan

Answer 2

You will need to have a function which replaces \n to \\n in case data is not a string literal.

function jsonEscape(str)  {
    return str.replace(/\n/g, "\\\\n").replace(/\r/g, "\\\\r").replace(/\t/g, "\\\\t");
}

var data = '{"count" : 1, "stack" : "sometext\n\n"}';
var dataObj = JSON.parse(jsonEscape(data));

Resulting dataObj will be

Object {count: 1, stack: "sometext\n\n"}

Maggs

you need to escape your escape characters (i.e. .replace("\\n", "\\\\n")) and I would also suggest using regex to allow replacing multiple instances (i.e. .replace(/\n/g, "\\\\n")) – Genip

why do you need to escape escape characters? I mean something like .replace("\n", "\\n") should do the job fine!! For example, var test = [{"description":"Some description about the product. This can be multi-line text."}]; console.log(JSON.parse(test.replace(/\n/g, "\\n"))); will output the object perfectly fine to browser console as [{"description":"Some description about the product.\nThis can be multi-line text."}] – Greenberg

BTW, in above comment, original JSON string has a new line, which is removed by stackoverflow's comment formatter.. You can see that the final output after replace should insert a new-line char \n in the value. – Greenberg

Because in the console, it should print \\n and not \n – Maggs

-1 This answer first constructs a string of invalid JSON (since newline is a control character), then tries to fix it with a series of incomplete replacements (there are more than 3 control characters). Then to top it off, it also manages to use the eval function. 17 upvotes??? – Malacca

Last 2 lines are just test data for calling the function and testing it. You might be actually getting that json from, for example a textarea in HTML page. – Maggs

@manish_s - in which case it's untrusted input, and therefore really shouldn't be passed to eval, ever. eval has its use cases, but should never be used on data that isn't prevetted and implicitly trusted. Replace the last line with var dataObj = JSON.parse(jsonEscape(data));. – Abscissa

Edited to use JSON.parse() – Maggs

What about quotation marks that need to be escaped too? – Pantechnicon

Answer 3

TLDR: A solution to the author's problem.

The json that you have generated is wrong because it contains newlines in one of its values. To generate a json that represents (not contains) newlines in one of its values you can use:

Use String.raw literal:

var data = String.raw`{"count" : 1, "stack" : "sometext\n\n"}`;

or apply JSON.stringify to a javascript object (which is json-like in the first place).

var data = JSON.stringify({"count" : 1, "stack" : "sometext\n\n"});

---

For some reason all answers inside here focus on how to parse a JSON string representation in JavaScript, which may cause confusion regarding how to represent newlines on actual JSON. The latter is not language-dependent.

Strictly based on the question title :

How do I handle newlines in JSON?

Let's say you parse a JSON file using the following code in node (it could be any language though):

let obj = JSON.parse(fs.readFileSync('file.json'));
console.log(obj.mykey)

Below is the output for each of the possible contents of file.json:

Input 1:

{
  "mykey": "my multiline
   value"
}

Output 1:

SyntaxError: Unexpected token

Input 2:

{
  "mykey": "my multiline\nvalue"
}

Output 2:

my multiline
value

Input 3:

{
  "mykey": "my multiline\\nvalue"
}

Output 3:

my multiline\nvalue

Conclusion 1:

To represent a newline inside a json file we should use the \n character. To represent the \n we should use \\n.

---

How would we define each of the above inputs using JavaScript (instead of input file):

When we need to define a string containing JSON in JavaScript, things change a bit because of the special meaning that \n has also for JavaScript. But also notice how String.raw literal fixes this.

Input1:

let input1 = '{"mykey": "my multiline\nvalue"}'

//OR
let input1 = `{
  "mykey": "my multiline
   value"
}`;
//(or even)
let input1 = `{
  "mykey": "my multiline\nvalue"
}`;

//OR
let input1 = String.raw`{
  "mykey": "my multiline
   value"
}`;

console.log(JSON.parse(input1).mykey);

//SyntaxError: Unexpected token
//in JSON at position [..]

Input 2:

let input2 = '{"mykey": "my multiline\\nvalue"}'

//OR
let input2 = `{
  "mykey": "my multiline\\nvalue"
}`;

//OR (Notice the difference from default literal)
let input2 = String.raw`{
  "mykey": "my multiline\nvalue"
}`;

console.log(JSON.parse(input2).mykey);

//my multiline
//value

Input 3:

let input3 = '{"mykey": "my multiline\\\\nvalue"}'

//OR
let input3 = `{
  "mykey": "my multiline\\\\nvalue"
}`;

//OR (Notice the difference from default literal)
let input3 = String.raw`{
  "mykey": "my multiline\\nvalue"
}`;

console.log(JSON.parse(input3).mykey);

//my multiline\nvalue

Conclusion 2:

To define a json string in javascript the easiest way would be to use String.raw, because it does not require any escaping (Well apart from backtick which is escaped like this String.raw`abc${"`"}def` ).

Of course, the easiest way to create json in javascript, in general, is to convert a javascript object to json (using JSON.stringify).

---

Important note:

Don't get confused by how chrome/firefox devtools preview variables containing newlines (e.g. when you just write the variable name and press Enter). When previewing objects they always represent newlines of string values like \n. But, when previewing a string (e.g. dataObj.stack) firefox will show a newline as an empty line but chrome will still show it as \n. So, to be safe, you should always preview with console.log(dataObj.stack) to better understand what's actually contained inside the specific variable.

Varanasi

This is true, however as @jerryurenaa pointed out you need to handle the whitespace wherever you display the json. – Anglophobe

@Anglophobe The author question is about parsing. Rendering is a different thing. Parsing json may be in a context where no html view is available. What I wanted to point out in my answer is that in pure json newlines are represented like this: \n (by spec), while in a javascript string definition, json newlines need to be escaped: \\n. Finally a newline byte is not a valid character inside a json value of type string. – Varanasi

This answer explains how to handle new lines in .json files. although it is not the answer for the OP's question, a lot of people working with .json files are landing here – Triadelphous

Answer 4

According to the specification, http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf:

A string is a sequence of Unicode code points wrapped with quotation marks (U+0022). All characters may be placed within the quotation marks except for the characters that must be escaped: quotation mark (U+0022), reverse solidus (U+005C), and the control characters U+0000 to U+001F. There are two-character escape sequence representations of some characters.

So you can't pass 0x0A or 0x0C codes directly. It is forbidden! The specification suggests to use escape sequences for some well-defined codes from U+0000 to U+001F:

• \f represents the form feed character (U+000C).
• \n represents the line feed character (U+000A).

As most of programming languages uses \ for quoting, you should escape the escape syntax (double-escape - once for language/platform, once for JSON itself):

jsonStr = "{ \"name\": \"Multi\\nline.\" }";

Ovariotomy

In the JSON file itself it seems you can/should use \n, not \\n, since at ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf it says: "There are two-character escape sequence representations of some characters ... \r represents the carriage return character (U+000D) ..." – Ambulant

Answer 5

well, it is not really necessary to create a function for this when it can be done simply with 1 CSS class.

just wrap your text around this class and see the magic :D

 <p style={{whiteSpace: 'pre-line'}}>my json text goes here \n\n</p>

note: because you will always present your text in frontend with HTML you can add the style={{whiteSpace: 'pre-line'}} to any tag, not just the p tag.

Demolish

the correct Syntex is : white-space: pre-line; – Tyrontyrone

@ShurvirMori: in CSS it is. But in React, which seems to be in use here, it's camelCase. – Acadian

@SinustheTentacular, oh, Sorry I don't have any idea about that. – Tyrontyrone

Answer 6

You could just escape your string on the server when writing the value of the JSON field and unescape it when retrieving the value in the client browser, for instance.

The JavaScript implementation of all major browsers have the unescape command.

Example:

On the server:

response.write "{""field1"":""" & escape(RS_Temp("textField")) & """}"

In the browser:

document.getElementById("text1").value = unescape(jsonObject.field1)

Shenyang

Answer 7

You might want to look into this C# function to escape the string:

http://www.aspcode.net/C-encode-a-string-for-JSON-JavaScript.aspx

public static string Enquote(string s)  
{ 
    if (s == null || s.Length == 0)  
    { 
        return "\"\""; 
    } 
    char         c; 
    int          i; 
    int          len = s.Length; 
    StringBuilder sb = new StringBuilder(len + 4); 
    string       t; 

    sb.Append('"'); 
    for (i = 0; i < len; i += 1)  
    { 
        c = s[i]; 
        if ((c == '\\') || (c == '"') || (c == '>')) 
        { 
            sb.Append('\\'); 
            sb.Append(c); 
        } 
        else if (c == '\b') 
            sb.Append("\\b"); 
        else if (c == '\t') 
            sb.Append("\\t"); 
        else if (c == '\n') 
            sb.Append("\\n"); 
        else if (c == '\f') 
            sb.Append("\\f"); 
        else if (c == '\r') 
            sb.Append("\\r"); 
        else 
        { 
            if (c < ' ')  
            { 
                //t = "000" + Integer.toHexString(c); 
                string t = new string(c,1); 
                t = "000" + int.Parse(tmp,System.Globalization.NumberStyles.HexNumber); 
                sb.Append("\\u" + t.Substring(t.Length - 4)); 
            }  
            else  
            { 
                sb.Append(c); 
            } 
        } 
    } 
    sb.Append('"'); 
    return sb.ToString(); 
} 

Haden

Why does this escape > ? – Veranda

Answer 8

I used this function to strip newline or other characters in data to parse JSON data:

function normalize_str($str) {

    $invalid = array(
        'Š'=>'S', 'š'=>'s',  'Đ'=>'Dj', 'đ'=>'dj', 'Ž'=>'Z', 'ž'=>'z',
        'Č'=>'C', 'č'=>'c',  'Ć'=>'C',  'ć'=>'c',  'À'=>'A', 'Á'=>'A', 'Â'=>'A', 'Ã'=>'A',
        'Ä'=>'A', 'Å'=>'A',  'Æ'=>'A',  'Ç'=>'C',  'È'=>'E', 'É'=>'E', 'Ê'=>'E', 'Ë'=>'E',
        'Ì'=>'I', 'Í'=>'I',  'Î'=>'I',  'Ï'=>'I',  'Ñ'=>'N', 'Ò'=>'O', 'Ó'=>'O', 'Ô'=>'O',
        'Õ'=>'O', 'Ö'=>'O',  'Ø'=>'O',  'Ù'=>'U',  'Ú'=>'U', 'Û'=>'U', 'Ü'=>'U', 'Ý'=>'Y',
        'Þ'=>'B', 'ß'=>'Ss', 'à'=>'a',  'á'=>'a',  'â'=>'a', 'ã'=>'a', 'ä'=>'a', 'å'=>'a',
        'æ'=>'a', 'ç'=>'c',  'è'=>'e',  'é'=>'e',  'ê'=>'e', 'ë'=>'e', 'ì'=>'i', 'í'=>'i',
        'î'=>'i', 'ï'=>'i',  'ð'=>'o',  'ñ'=>'n',  'ò'=>'o', 'ó'=>'o', 'ô'=>'o', 'õ'=>'o',
        'ö'=>'o', 'ø'=>'o',  'ù'=>'u',  'ú'=>'u',  'û'=>'u', 'ý'=>'y', 'ý'=>'y', 'þ'=>'b',
        'ÿ'=>'y', 'Ŕ'=>'R',  'ŕ'=>'r',
        "`" => "'", "´" => "'",  '"' => ',',  '`' => "'",
        '´' => "'", '"' => '\"', '"' => "\"", '´' => "'",
        "’" => "'",
        "{" => "",
        "~" => "",  "–" => "-",  "'" => "'",  "     " => " ");

    $str = str_replace(array_keys($invalid), array_values($invalid), $str);

    $remove = array("\n", "\r\n", "\r");
    $str = str_replace($remove, "\\n", trim($str));

    //$str = htmlentities($str, ENT_QUOTES);

    return htmlspecialchars($str);
}

echo normalize_str($lst['address']);

Stenson

In most languages you have better ways to strip accents from unicode strings than writing down your own mapping function. See this question for an example in python: #518423 – Cymophane

ya we have many ways to control the special chars in diff languages. – Stenson

That's all kind of bad to strip them in general. Better encode them as XML numeric character reference and then decode on receiving end. – Codding

Answer 9

I encountered that problem while making a class in PHP 4 to emulate json_encode (available in PHP 5). Here's what I came up with:

class jsonResponse {
    var $response;

    function jsonResponse() {
        $this->response = array('isOK'=>'KO', 'msg'=>'Undefined');
    }

    function set($isOK, $msg) {
        $this->response['isOK'] = ($isOK) ? 'OK' : 'KO';
        $this->response['msg'] = htmlentities($msg);
    }

    function setData($data=null) {
        if(!is_null($data))
            $this->response['data'] = $data;
        elseif(isset($this->response['data']))
            unset($this->response['data']);
    }

    function send() {
        header('Content-type: application/json');
        echo '{"isOK":"' . $this->response['isOK'] . '","msg":' . $this->parseString($this->response['msg']);
        if(isset($this->response['data']))
            echo ',"data":' . $this->parseData($this->response['data']);
        echo '}';
    }

    function parseData($data) {
        if(is_array($data)) {
            $parsed = array();
            foreach ($data as $key=>$value)
                array_push($parsed, $this->parseString($key) . ':' . $this->parseData($value));
            return '{' . implode(',', $parsed) . '}';
        }
        else
            return $this->parseString($data);
    }

    function parseString($string) {
            $string = str_replace("\\", "\\\\", $string);
            $string = str_replace('/', "\\/", $string);
            $string = str_replace('"', "\\".'"', $string);
            $string = str_replace("\b", "\\b", $string);
            $string = str_replace("\t", "\\t", $string);
            $string = str_replace("\n", "\\n", $string);
            $string = str_replace("\f", "\\f", $string);
            $string = str_replace("\r", "\\r", $string);
            $string = str_replace("\u", "\\u", $string);
            return '"'.$string.'"';
    }
}

I followed the rules mentioned here. I only used what I needed, but I figure that you can adapt it to your needs in the language your are using. The problem in my case wasn't about newlines as I originally thought, but about the / not being escaped. I hope this prevent someone else from the little headache I had figuring out what I did wrong.

Needlefish

The 6 shorthands for control characters specified on json.org is not an exhaustive list of all control characters. As a result, this function could generate invalid JSON. – Malacca

Answer 10

As I understand you question, it is not about parsing JSON because you can copy-paste your JSON into your code directly - so if this is the case then just copy your JSON direct to dataObj variable without wrapping it with single quotes (tip: eval==evil)

var dataObj = {"count" : 1, "stack" : "sometext\n\n"};

console.log(dataObj);

Yunyunfei