How to Manipulate JSON AST in Scala

Question

I am experimenting with the json4s library (based on lift-json). One of the things I would like to do is to parse a JSON string into an AST, and then manipulate it.

For example, I would like to upsert a field (insert the field into the AST if it does not exist, or update its value if it does).

I have not been able to find how to do it in the documentation. Experimenting with the available methods, I have come up with the following, which works, but feels clumsy.

import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._

object TestJson {
  implicit val formats = DefaultFormats

  def main(args: Array[String]): Unit = {
    val json = """{"foo":1, "bar":{"foo":2}}"""
    val ast = parse(json).asInstanceOf[JObject]

    println( upsertField(ast, ("foo" -> "3")) )
    println( upsertField(ast, ("foobar" -> "3")) )
  }

  def upsertField(src:JObject, fld:JField): JValue = {
    if(src \ fld._1 == JNothing){
      src ~ fld
    }
    else{
      src.replace(List(fld._1), fld._2)
    }
  }
}

I dislike it for many reasons:

1. Having to explicitly cast the results of parse(json) to JObject
2. The result of the upsertField function is a JValue, which I will have to recast if I want to manipulate the object further
3. The upsertField function just feels very unelegant
4. It does not work for fields that are not at the top level of the hierarchy

Is there a better way to transform the AST?

EDIT: as a workaround to the problem, I have managed to convert my JSON to Scala regular classes, and manipulate them with lenses (Using Lenses on Scala Regular Classes)

Vang

what does AST stand for? – Irvin

@QuyTang AST stands for "abstract syntax tree" – Vang

Answer 1

There is the merge function which creates or overrides a field. You can also update fields that are not at the root level of the tree.

import org.json4s._
import org.json4s.JsonDSL._
import org.json4s.jackson.JsonMethods._

object mergeJson extends App {

  val json =
    """
      |{
      |  "foo":1,
      |  "bar": {
      |    "foo": 2
      |  }
      |}
      |""".stripMargin

  val ast = parse(json)

  val updated = ast merge (("foo", 3) ~ ("bar", ("fnord", 5)))

  println(pretty(updated))

  //  {
  //    "foo" : 3,
  //    "bar" : {
  //      "foo" : 2,
  //      "fnord" : 5
  //    }
  //  }

}

Planimetry

Answer 2

Let me also give you the SON of JSON version:

import nl.typeset.sonofjson._

val json = parse("""{ "foo" : 1, "bar" : { "foo" : 2 } }""")

// or, perhaps a little easier
val json = obj(foo = 1, bar = obj(foo = 2))

json.foo = "3"
json.foobar = "3"

Kneepad

Thank you. I will give it a try in my next project! – Vang

Answer 3

When I was implementing some very specific json diff using lift json I used a lot of recursive functions to get to the jpath where I need to modify value, and modified json was constructed when recursion "collapsed". LiftJson is immutable after all. You mentioned lenses as another approach, which is very interesting by itself. But my current favourite is play-json library that is working like a charm in a situation when you need to do json-to-json transformation:

from Mandubian Blog:

val gizmo2gremlin = (
  (__ \ 'name).json.put(JsString("gremlin")) and
  (__ \ 'description).json.pickBranch(
      (__ \ 'size).json.update( of[JsNumber].map{ case JsNumber(size) => JsNumber(size * 3) } ) and
      (__ \ 'features).json.put( Json.arr("skinny", "ugly", "evil") ) and
      (__ \ 'danger).json.put(JsString("always"))
      reduce
  ) and
  (__ \ 'hates).json.copyFrom( (__ \ 'loves).json.pick )
) reduce

Yummy Features: all transformers are combinators that can be mixed together, validation, shapeless support, auto marshaling of case classes with implicit overrides, stand-alone library.

Coming