Scala dispatch GET request, fail to parse response to json

Question

im writing a function that :

1) send HTTP GET request (response is a valid JSON)

2) parse the response to a json object

code snippet :

val page = url("http://graph.facebook.com/9098498615")
val response = Http(page OK dispatch.as.String)
Await.result(response , 10 seconds)
val myJson= JSON.parseFull(response .toString)
//this isnt helping -> val myJson= JSON.parseRaw(response .toString)

Problem is after this myJson is None while im expecting it to hold the json data from the response.

Help ?

Canotas

no error .. just that myJson is None and not the json response im expecting – Canotas

Answer 1

It's not a good idea to use Http(page OK as.String) because all responses different from HTTP 200 will result in failed Futures. If you need more fine grained control over error handling/reporting, target specific scenarios instead.

import org.jboss.netty.handler.codec.http.{ HttpRequest, HttpResponse, HttpResponseStatus }
def getFacebookGraphData: Either[Exception, String] = {
  val page = url("http://graph.facebook.com/9098498615")
  val request = Http(page.GET);
  val response = Await.result(request, 10 seconds);
  (response.getStatusCode: @annotation.switch) match {
    case HttpResponseStatus.OK => {
      val body = response.getResponseBody() // dispatch adds this method
      // if it's not available, then:
      val body = new String(response.getContent.array);
      Right(body)
    }
    // If something went wrong, you now have an exception with a message.
    case _ => Left(new Exception(new String(response.getContent.array)));
  }
}

The default Scala JSON library is not a very good idea either, it's very rough compared to others. Try lift-json for instance.

import net.liftweb.json.{ JSONParser, MappingException, ParseException };

case class FacebookGraphResponse(name: String, id: String);// etc
implicit val formats = net.liftweb.DefaultFormats;
val graphResponse = JSONParser.parse(body).extract[FacebookGraphResponse];
// or the better thing, you can catch Mapping and ParseExceptions.

Renner

It's not really true that OK means "neglecting" all non-200 responses—it returns a Future that will fail on those responses, but that's often exactly what you want. You can still process non-200 responses cleanly. – Pich

I hope you don't mind that I've added my own answer using Dispatch's "built-in" (as a sub-project) JSON support. – Pich

Answer 2

Dispatch includes some really nice (and under-advertised) facilities for parsing JSON, which you can use like this (note that you can handle non-200 responses using any of the standard approaches for dealing with failed futures):

import dispatch._
import org.json4s._, org.json4s.native.JsonMethods._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{ Failure, Success }

val page = url("http://graph.facebook.com/9098498615")
val response = Http(page OK dispatch.as.json4s.Json)

response onComplete {
  case Success(json) => println(json \ "likes")
  case Failure(error) => println(error)
}

This example uses the Json4s library, and similar support is provided for Lift JSON (but unfortunately nothing for Argonaut, although it's not too hard to write such a thing yourself).

Pich

I always get object json4s is not a member of package dispatch.as. Edit: Just realized I had to add the dependency to my build.sbt mvnrepository.com/artifact/net.databinder.dispatch/… – Marin

@Marin would you mind being more precise as to what you added to your build.sbt? I had the same problem, added the dependency but still having the issue. – Hbeam

"net.databinder.dispatch" % "dispatch-json4s-native_2.11" % "0.11.1" – Marin

Answer 3

You can also use ur own favorite json-library (e.g. the play-framework-json-lib) like this:

val response = Http(requestUrl OK CarJsonDeserializer)

U have just to extend the (Response => Car) trait by the JsonDeserializer.

object CarJsonDeserializer extends (Response => Car) {
  override def apply(r: Response): Car = {
    (dispatch.as.String andThen (jsonString => parse(jsonString)))(r)
  }
}

and the json-parser:

implicit val carReader: Reads[Car] = (
  (JsPath \ "color").read[String] and
  (JsPath \ "model").read[String]
)(Monitor.apply _)

private def parse(jsonString: String) = {
  val jsonJsValue = Json.parse(jsonString)
  jsonJsValue.as[Car]
}

See this blog post about this: https://habashics.wordpress.com/2014/11/28/parsing-json-play-lib-with-dispatch/

Seminal