jQuery.parseJSON throws “Invalid JSON” error due to escaped single quote in JSON

Question

I’m making requests to my server using jQuery.post() and my server is returning JSON objects (like { "var": "value", ... }). However, if any of the values contains a single quote (properly escaped like \'), jQuery fails to parse an otherwise valid JSON string. Here’s an example of what I mean (done in Chrome’s console):

data = "{ \"status\": \"success\", \"newHtml\": \"Hello \\\'x\" }";
eval("x = " + data); // { newHtml: "Hello 'x", status: "success" }

$.parseJSON(data); // Invalid JSON: { "status": "success", "newHtml": "Hello \'x" }

Is this normal? Is there no way to properly pass a single quote via JSON?

Tormoria

Answer 1

According to the state machine diagram on the JSON website, only escaped double-quote characters are allowed, not single-quotes. Single quote characters do not need to be escaped:

Update - More information for those that are interested:

---

Douglas Crockford does not specifically say why the JSON specification does not allow escaped single quotes within strings. However, during his discussion of JSON in Appendix E of JavaScript: The Good Parts, he writes:

JSON's design goals were to be minimal, portable, textual, and a subset of JavaScript. The less we need to agree on in order to interoperate, the more easily we can interoperate.

So perhaps he decided to only allow strings to be defined using double-quotes since this is one less rule that all JSON implementations must agree on. As a result, it is impossible for a single quote character within a string to accidentally terminate the string, because by definition a string can only be terminated by a double-quote character. Hence there is no need to allow escaping of a single quote character in the formal specification.

---

Digging a little bit deeper, Crockford's org.json implementation of JSON for Java is more permissible and does allow single quote characters:

The texts produced by the toString methods strictly conform to the JSON syntax rules. The constructors are more forgiving in the texts they will accept:

...

• Strings may be quoted with ' (single quote).

This is confirmed by the JSONTokener source code. The nextString method accepts escaped single quote characters and treats them just like double-quote characters:

public String nextString(char quote) throws JSONException {
    char c;
    StringBuffer sb = new StringBuffer();
    for (;;) {
        c = next();
        switch (c) {

        ...

        case '\\':
            c = this.next();
            switch (c) {

            ...

            case '"':
            case '\'':
            case '\\':
            case '/':
                sb.append(c);
                break;
        ...

At the top of the method is an informative comment:

The formal JSON format does not allow strings in single quotes, but an implementation is allowed to accept them.

So some implementations will accept single quotes - but you should not rely on this. Many popular implementations are quite restrictive in this regard and will reject JSON that contains single quoted strings and/or escaped single quotes.

---

Finally to tie this back to the original question, jQuery.parseJSON first attempts to use the browser's native JSON parser or a loaded library such as json2.js where applicable (which on a side note is the library the jQuery logic is based on if JSON is not defined). Thus jQuery can only be as permissive as that underlying implementation:

parseJSON: function( data ) {
    ...

    // Attempt to parse using the native JSON parser first
    if ( window.JSON && window.JSON.parse ) {
        return window.JSON.parse( data );
    }

    ...

    jQuery.error( "Invalid JSON: " + data );
},

As far as I know these implementations only adhere to the official JSON specification and do not accept single quotes, hence neither does jQuery.

Cabin

I had a similar problem using strus2 because the tag <s:property escapeJavaScript="true"/> when rendered, try to quote the single quote (!) This is WRONG!!! – Inelegancy

UPDATE:: JQuery is very restrictive when pasing JSON. If you try alert($.parseJSON("[\"Ciao\\'\"]")); it does not work because of what Justin reported – Inelegancy

"Crockford's org.json implementation of JSON for Java is more permissible and does allow single quote characters" # That's just good practice: robustness principle – Aboutface

@DuncanJones - This article might provide insight into why none of the browsers seem to follow that principle with regard to JSON: joelonsoftware.com/items/2008/03/17.html – Cabin

JSON.parse('"hi"') vs JSON.parse("'hi'"). Interestingly enough the second one throws a syntax error (in Chrome). >> Edit: ah, now I see why - just needed to read a little more carefully. – Subsoil

@Subsoil - Right, the first one parses fine because "hi" is sent as JSON, and the second one fails since 'hi' is sent (IE, a single-quoted string) which is not valid JSON. – Cabin

I should say JSON.parse('"hi"') vs JSON.parse("'hi'") for readability. – Subsoil

@JustinEthier as pointed out by this answer https://mcmap.net/q/47899/-spring-escapebody-results-in-invalid-json, the JSON spec tools.ietf.org/html/rfc7159 says Any character may be escaped, this may explain why some implementation would allow single quotes to be escaped. – Caryloncaryn

@AdrienBe - Interesting... but did they mean any character may be escaped if it consists of 4 hex digits? According to both the state diagram above and the one in section 7 of the RFC, escape of a single quote as written \' is still not allowed. It would be nice if the RFC was more explicit on this point. – Cabin

@AdrienBe - Also, see my comment regarding "May" in your link - it is to be interpreted per RFC 2119, meaning anything marked as "may" is optional. – Cabin

@JustinEthier the only clear conclusion I reach is that the spec is unclear. Thanks for looking further into this tho. – Caryloncaryn

Answer 2

If you need a single quote inside of a string, since \' is undefined by the spec, use \u0027 see http://www.utf8-chartable.de/ for all of them

edit: please excuse my misuse of the word backticks in the comments. I meant backslash. My point here is that in the event you have nested strings inside other strings, I think it can be more useful and readable to use unicode instead of lots of backslashes to escape a single quote. If you are not nested however it truly is easier to just put a plain old quote in there.

Christie

No. Just use a a plain single quote. – Reduplication

Sometimes, it's just plain easier to use unicode than tons of back-ticks. Particularly when inside alternating back-ticks. – Christie

Why would you need backticks? If you have a string like "foo 'bar'" then you just leave the single quotes unescaped. – Reduplication

exactly what I was looking for. I'm trying to write a json string onto a page as a js string var and enclose it in single quotes and it was terminating early whenever a property value had a single quote in it. Now I just do a json.Replace("'", "\u0027") in code behind before writing it onto the page. – Geaghan

@Geaghan You should not enclose JSON with quotes. If you need a string out of an existing JSON string, just stringify it again. in PHP, that would be var jsonEncodedAsString = <?= json_encode(myEncodedJson) ?> where myEncodedJson is the result of a previous json_encode That will take care of escaping your single quote, actually, it will just output something a big string wrapped in double quotes, so single quotes won't be escaped, but double quotes will. – Macadam

Answer 3

I understand where the problem lies and when I look at the specs its clear that unescaped single quotes should be parsed correctly.

I am using jquery`s jQuery.parseJSON function to parse the JSON string but still getting the parse error when there is a single quote in the data that is prepared with json_encode.

Could it be a mistake in my implementation that looks like this (PHP - server side):

$data = array();

$elem = array();
$elem['name'] = 'Erik';
$elem['position'] = 'PHP Programmer';
$data[] = json_encode($elem);

$elem = array();
$elem['name'] = 'Carl';
$elem['position'] = 'C Programmer';
$data[] = json_encode($elem);

$jsonString = "[" . implode(", ", $data) . "]";

The final step is that I store the JSON encoded string into an JS variable:

<script type="text/javascript">
employees = jQuery.parseJSON('<?=$marker; ?>');
</script>

If I use "" instead of '' it still throws an error.

SOLUTION:

The only thing that worked for me was to use bitmask JSON_HEX_APOS to convert the single quotes like this:

json_encode($tmp, JSON_HEX_APOS);

Is there another way of tackle this issue? Is my code wrong or poorly written?

Thanks

Multiflorous

'<?=$marker; ?>' this is not valid json. The surrounding quotes are "eaten up" by the javascript interpreter, leaving a string that begins with a < ... what you really wanted to try was either of these: jQuery.parseJSON('"<?=$marker; ?>"'); jQuery.parseJSON("\"<?=$marker; ?>\""); According to the spec, json strings must use double quotes, but javascript doesn't care, so, you either have a single-quote javascript string, or a double quote, but if you use the latter, you have to them escape all the uses of double quotes inside the string. – Bonefish

Answer 4

When You are sending a single quote in a query

empid = " T'via"
empid =escape(empid)

When You get the value including a single quote

var xxx  = request.QueryString("empid")
xxx= unscape(xxx)

If you want to search/ insert the value which includes a single quote in a query xxx=Replace(empid,"'","''")

Foray

But there is no need to escape a single quote when passing it as part of a JSON string... – Cabin

Answer 5

Striking a similar issue using CakePHP to output a JavaScript script-block using PHP's native json_encode. $contractorCompanies contains values that have single quotation marks and as explained above and expected json_encode($contractorCompanies) doesn't escape them because its valid JSON.

<?php $this->Html->scriptBlock("var contractorCompanies = jQuery.parseJSON( '".(json_encode($contractorCompanies)."' );"); ?>

By adding addslashes() around the JSON encoded string you then escape the quotation marks allowing Cake / PHP to echo the correct javascript to the browser. JS errors disappear.

<?php $this->Html->scriptBlock("var contractorCompanies = jQuery.parseJSON( '".addslashes(json_encode($contractorCompanies))."' );"); ?>

Lavoie

Answer 6

I was trying to save a JSON object from a XHR request into a HTML5 data-* attribute. I tried many of above solutions with no success.

What I finally end up doing was replacing the single quote ' with it code ' using a regex after the stringify() method call the following way:

var productToString = JSON.stringify(productObject);
var quoteReplaced = productToString.replace(/'/g, "'");
var anchor = '<a data-product=\'' + quoteReplaced + '\' href=\'#\'>' + productObject.name + '</a>';
// Here you can use the "anchor" variable to update your DOM element.

Diagnostician

Answer 7

Interesting. How are you generating your JSON on the server end? Are you using a library function (such as json_encode in PHP), or are you building the JSON string by hand?

The only thing that grabs my attention is the escape apostrophe (\'). Seeing as you're using double quotes, as you indeed should, there is no need to escape single quotes. I can't check if that is indeed the cause for your jQuery error, as I haven't updated to version 1.4.1 myself yet.

Artiste

I use a library in PHP to generate the JSON object - kinda missed to write that down. Thanks for pointing that out. – Tetragram