Safely parsing a JSON string with unquoted keys

Question

json2.js is strict requiring all object keys be double-quoted. However, in Javascript syntax {"foo":"bar"} is equivalent to {foo:"bar"}.

I have a textarea that accepts JSON input from the user and would like to "ease" the restriction on double quoting the keys. I've looked at how json2.js validates a JSON string in four stages before it evals it. I was able to add a 5th stage to allow unquoted keys and would like to know if there are any security implications to this logic.

var data = '{name:"hello", age:"23"}';

// Make sure the incoming data is actual JSON
// Logic borrowed from http://json.org/json2.js
if ( /^[\],:{}\s]*$/.test(data.replace(/\\(?:["\\\/bfnrt]|u[0-9a-fA-F]{4})/g, "@")
     .replace(/"[^"\\\n\r]*"|true|false|null|-?\d+(?:\.\d*)?(?:[eE][+\-]?\d+)?/g, "]")
     .replace(/(?:^|:|,)(?:\s*\[)+/g, ":") // EDITED: allow key:[array] by replacing with safe char ":"
     /** everything up to this point is json2.js **/

     /** this is the 5th stage where it accepts unquoted keys **/         
     .replace(/\w+\s*\:/g, ":")) ) { // EDITED: allow any alphanumeric key

  console.log( (new Function("return " + data))() );
}
else {
  throw( "Invalid JSON: " + data );
}

Haemato

You assume a JavaScript Object Literal is equivalent to JSON, which it is not. – Innate

{name:"Joe"} is valid Javascript, but it's invalid JSON. Also you do not want to hack json2.js because it just mirrors how browsers native JSON support works. In Chrome, for instance, JSON.parse() without json2.js would choke on that as well. But worse is that json2.js will not load anything if the browser does have native JSON support. So you will be in a situation where browsers with native JSON suport never see this hack, because its using native code to parse it instead. – Bespoke

@Innate You are right. Maybe I should rephrase the question as "Safely parsing JavaScript Object Literal and converting to JSON"? – Haemato

Then that would be a valid pursuit. – Innate

@Squeegy I was not proposing to overwrite JSON.parse implemented by the browser or by json2.js. I was simply suggesting a safe "Javascript Object Literal" parser that builds on the precautions of json2.js parser. – Haemato

The old adage "be liberal in what you accept" is, in my book, a bad idea. Rejecting invalid input with an informative error message would educate users who may not realise they are creating invalid JSON. Accepting it without warning may give users the false sense that what they are doing is valid JSON when it isn't, and their code will break with other JSON parsers. – Esterify

Answer 1

data.replace(/(['"])?([a-zA-Z0-9]+)(['"])?:/g, '"$2":');

That will replace any single quotes on the parameter name, and add any that are missing.

Fab

This seems to work. Except you failed to handle the underscore. Here's an updated regex: hash.replace(/(['"])?([a-zA-Z0-9_]+)(['"])?:/g, '"$2":'); – Rori

This answer is far from perfect. Try {a: ['b', 'c']} or {a: "Note: something happened."} – Irretentive

Beware that while this regex might work on some very specific scenarios, it will not work more complex stuff like: { location: 'http://www.google.com' }, you'll end up with invalid JSON: {"location": "http"://www.google.com'} – Betz

Don't forget about the dollar sign either, and spaces between the name and colon. replace(/(['"])?([a-zA-Z0-9_\$]+)(['"])?\s*:/g, '"$2":') – Leucoplast

Answer 2

Use JSON5.parse

JSON5 is a superset of JSON that allows ES5 syntax, including unquoted property keys. The JSON5 reference implementation (json5 npm package) provides a JSON5 object that has the same methods with the same args and semantics as the built-in JSON object.

JSON5 is used by many high profile projects:

JSON5 was started in 2012, and as of 2022, now gets >65M downloads/week, ranks in the top 0.1% of the most depended-upon packages on npm, and has been adopted by major projects like Chromium, Next.js, Babel, Retool, WebStorm, and more. It's also natively supported on Apple platforms like MacOS and iOS.

~ json5.org homepage

Gereron

Answer 3

JSON does not allow unquoted keys. JSON is a subset of JavaScript notation, and that does not include unquoted keys. Passing unquoted keys to just about any JSON parser will likely throw an error or return "unexpected" results.

Hope this helps

Misappropriate

True about JSON of course, but JavaScript does allow unquoted keys in object literals. It's just a bit problematic as you can't use dashes or reserved words without quotes. I think the asker knows this already, however. – Sandiesandifer

Answer 4

Maybe you can use:

Replace Key & Value

str.replace(/([a-zA-Z0-9-]+):([a-zA-Z0-9-]+)/g, "\"$1\":\"$2\"");

Replace Key only

str.replace(/([a-zA-Z0-9-]+):([a-zA-Z0-9-]+)/g, "\"$1\":$2");

Replace Value only

str.replace(/([a-zA-Z0-9-]+):([a-zA-Z0-9-]+)/g, "\$1\:\"$2\"");

Pensive

Answer 5

The below snippet will covert majority of js~like structures into valid JSON syntax. While a little more extended than other answers the payoff is a more concise result. In situations where the parse cannot apply, we will fail gracefully, print the error to console and return the original input.

Example

Flems Playground

Capabilities

  ✓ Omits any trailing commas.
  ✓ Handles array [] entries, converting alphanumeric occurences to strings.
  ✓ Preserves boolean occurrences of true or false.
  ✓ Preserves number or float occurrences.

Code

function toJSON (input) {

  try {

    const json = input
      .replace(/\\'|'/g, (m) => m[0] === '\\' ? m : '"')
      .replace(/\[|[^\s[\]]*|\]/g,  match => /[[\]]/.test(match) 
        ? match 
        : match.split(',').map(
          value => value
          .replace(/^(\w+)$/, '"$1"')
          .replace(/^"([\d.]+)"$/g, '$1')).join(','))
      .replace(/([a-zA-Z0-9_-]+)\s*:/g,  '"$1":')
      .replace(/:\s*([$a-zA-Z_-]+)\s*([,\]}])/g, ':"$1"$2')
      .replace(/,([\]}])/g, '$1')
      .replace(/([a-zA-Z_-]+)\s*,/g,  '"$1",')
      .replace(/([\]},\s]+)?"(true|false)"([\s,{}\]]+)/g, '$1$2$3')

    return JSON.parse(json);

  } catch (e) {
    
    console.error(e, JSON.stringify(input, null, 2));
    return input;
    
  }
}

Breakdown

Here is brief overview of what is happening:

1. We convert any single quotations into double quotations.
2. We handle all array item entries of varying types.
3. We wrap all key property occurrences in double quotations.
4. We wrap all non digit value occurrences in double quotation.
5. We remove any trailing comma occurrences.
6. We remove double quotations from boolean types true and false
7. We pass the replaced string to JSON.parse
8. We return the result.

Disclaimer

One should typically avoid converting string json~like structures with regex and always opt for a tool designed for the job because the structures can be unpredictable and you can never be certain of the result using the "hack" I've provided. However, not everything is black and white, as such in cases where complete accuracy is not demanded, the above will suffice.

Rosinski

Answer 6

"JSON with comments" is actually a valid javascript, therefore in javascript environment the simplest native way to parse it is just evaluate it like this

function evalJs(js) {
    let fn = new Function("return (" + js + ")"),
        res = fn()
    return res;
}

let json5 = "{\n" +
    "//////\n" +
    "key: 5," +
    "}"

let data = evalJs(json5)
console.info(data)

Output is

{ key: 5 }

Especially

This may work. But the very first word of the question is "safely" which usually excludes user-provided code execution! – Kotick

Answer 7

!!! considering security risks -- do not eval blindly

Object.defineProperty(String.prototype, "eval", {
    get: function() {
       let v;
       eval('v =' + this);
       return v;
    }
 });

---

var object = '{A:1,"B":[1,\'alfa\',"beta"]}'.eval 

Topple