How to decode an ADT with circe without disambiguating objects
Asked Answered
G

2

33

Suppose I've got an ADT like this:

sealed trait Event

case class Foo(i: Int) extends Event
case class Bar(s: String) extends Event
case class Baz(c: Char) extends Event
case class Qux(values: List[String]) extends Event

The default generic derivation for a Decoder[Event] instance in circe expects the input JSON to include a wrapper object that indicates which case class is represented:

scala> import io.circe.generic.auto._, io.circe.parser.decode, io.circe.syntax._
import io.circe.generic.auto._
import io.circe.parser.decode
import io.circe.syntax._

scala> decode[Event]("""{ "i": 1000 }""")
res0: Either[io.circe.Error,Event] = Left(DecodingFailure(CNil, List()))

scala> decode[Event]("""{ "Foo": { "i": 1000 }}""")
res1: Either[io.circe.Error,Event] = Right(Foo(1000))

scala> (Foo(100): Event).asJson.noSpaces
res2: String = {"Foo":{"i":100}}

This behavior means that we never have to worry about ambiguities if two or more case classes have the same member names, but it's not always what we want—sometimes we know the unwrapped encoding would be unambiguous, or we want to disambiguate by specifying the order each case class should be tried, or we just don't care.

How can I encode and decodes my Event ADT without the wrapper (preferably without having to write my encoders and decoders from scratch)?

(This question comes up fairly often—see e.g. this discussion with Igor Mazor on Gitter this morning.)

Greggrega answered 10/2, 2017 at 17:38 Comment(0)
G
53

Enumerating the ADT constructors

The most straightforward way to get the representation you want is to use generic derivation for the case classes but explicitly defined instances for the ADT type:

import cats.syntax.functor._
import io.circe.{ Decoder, Encoder }, io.circe.generic.auto._
import io.circe.syntax._

sealed trait Event

case class Foo(i: Int) extends Event
case class Bar(s: String) extends Event
case class Baz(c: Char) extends Event
case class Qux(values: List[String]) extends Event

object Event {
  implicit val encodeEvent: Encoder[Event] = Encoder.instance {
    case foo @ Foo(_) => foo.asJson
    case bar @ Bar(_) => bar.asJson
    case baz @ Baz(_) => baz.asJson
    case qux @ Qux(_) => qux.asJson
  }

  implicit val decodeEvent: Decoder[Event] =
    List[Decoder[Event]](
      Decoder[Foo].widen,
      Decoder[Bar].widen,
      Decoder[Baz].widen,
      Decoder[Qux].widen
    ).reduceLeft(_ or _)
}

Note that we have to call widen (which is provided by Cats's Functor syntax, which we bring into scope with the first import) on the decoders because the Decoder type class is not covariant. The invariance of circe's type classes is a matter of some controversy (Argonaut for example has gone from invariant to covariant and back), but it has enough benefits that it's unlikely to change, which means we need workarounds like this occasionally.

It's also worth noting that our explicit Encoder and Decoder instances will take precedence over the generically-derived instances we'd otherwise get from the io.circe.generic.auto._ import (see my slides here for some discussion of how this prioritization works).

We can use these instances like this:

scala> import io.circe.parser.decode
import io.circe.parser.decode

scala> decode[Event]("""{ "i": 1000 }""")
res0: Either[io.circe.Error,Event] = Right(Foo(1000))

scala> (Foo(100): Event).asJson.noSpaces
res1: String = {"i":100}

This works, and if you need to be able to specify the order that the ADT constructors are tried, it's currently the best solution. Having to enumerate the constructors like this is obviously not ideal, though, even if we get the case class instances for free.

A more generic solution

As I note on Gitter, we can avoid the fuss of writing out all the cases by using the circe-shapes module:

import io.circe.{ Decoder, Encoder }, io.circe.generic.auto._
import io.circe.shapes
import shapeless.{ Coproduct, Generic }

implicit def encodeAdtNoDiscr[A, Repr <: Coproduct](implicit
  gen: Generic.Aux[A, Repr],
  encodeRepr: Encoder[Repr]
): Encoder[A] = encodeRepr.contramap(gen.to)

implicit def decodeAdtNoDiscr[A, Repr <: Coproduct](implicit
  gen: Generic.Aux[A, Repr],
  decodeRepr: Decoder[Repr]
): Decoder[A] = decodeRepr.map(gen.from)

sealed trait Event

case class Foo(i: Int) extends Event
case class Bar(s: String) extends Event
case class Baz(c: Char) extends Event
case class Qux(values: List[String]) extends Event

And then:

scala> import io.circe.parser.decode, io.circe.syntax._
import io.circe.parser.decode
import io.circe.syntax._

scala> decode[Event]("""{ "i": 1000 }""")
res0: Either[io.circe.Error,Event] = Right(Foo(1000))

scala> (Foo(100): Event).asJson.noSpaces
res1: String = {"i":100}

This will work for any ADT anywhere that encodeAdtNoDiscr and decodeAdtNoDiscr are in scope. If we wanted it to be more limited, we could replace the generic A with our ADT types in those definitions, or we could make the definitions non-implicit and define implicit instances explicitly for the ADTs we want encoded this way.

The main drawback of this approach (apart from the extra circe-shapes dependency) is that the constructors will be tried in alphabetical order, which may not be what we want if we have ambiguous case classes (where the member names and types are the same).

The future

The generic-extras module provides a little more configurability in this respect. We can write the following, for example:

import io.circe.generic.extras.auto._
import io.circe.generic.extras.Configuration

implicit val genDevConfig: Configuration =
  Configuration.default.withDiscriminator("what_am_i")

sealed trait Event

case class Foo(i: Int) extends Event
case class Bar(s: String) extends Event
case class Baz(c: Char) extends Event
case class Qux(values: List[String]) extends Event

And then:

scala> import io.circe.parser.decode, io.circe.syntax._
import io.circe.parser.decode
import io.circe.syntax._

scala> (Foo(100): Event).asJson.noSpaces
res0: String = {"i":100,"what_am_i":"Foo"}

scala> decode[Event]("""{ "i": 1000, "what_am_i": "Foo" }""")
res1: Either[io.circe.Error,Event] = Right(Foo(1000))

Instead of a wrapper object in the JSON we have an extra field that indicates the constructor. This isn't the default behavior since it has some weird corner cases (e.g. if one of our case classes had a member named what_am_i), but in many cases it's reasonable and it's been supported in generic-extras since that module was introduced.

This still doesn't get us exactly what we want, but it's closer than the default behavior. I've also been considering changing withDiscriminator to take an Option[String] instead of a String, with None indicating that we don't want an extra field indicating the constructor, giving us the same behavior as our circe-shapes instances in the previous section.

If you're interested in seeing this happen, please open an issue, or (even better) a pull request. :)

Greggrega answered 10/2, 2017 at 17:38 Comment(5)
The last option looks indeed very promising. As always, I'd be interested whether there are any principal caveats concerning performance.Alternately
I came across this problem many times and this solution is exactly what I need. I've made a sample project using this solution for future reference.Gladwin
I can't seem to get the middle example with circe-shapes to work for me. I notice this post has been copied to the docs: circe.github.io/circe/codecs/adt.html I'm assuming the import io.circe.shapes should be import io.circe.shapes._. I put up an example at github.com/LeifW/CirceSumTypes/blob/master/src/main/scala/… But I get: { "Animal" : { "sodium" : 5 } } instead of: {"sodium": 5} I tried 0.12 and 0.13, as well as generic-extras, with the same results.Croom
Oh, it works with auto, just not with semiauto. Hrm.Croom
Turns out it does work with semiauto - sorry for the noise. It's just that shapes._ and encodeAdtNoDiscr need to be in scope when calling deriveEncoder in the containing class - I had assumed they needed to be in scope when calling deriveEncoder in the sum type itself. So for sealed trait SumType = A | B | C; case class ContainingType(things: List[SumType]), those extra things need to be in scope when calling deriveEncoder in the companion object to ContainingType - they have no effect when in scope in SumType.Croom
C
1

I have to deal a lot ADT to JSON lately, so decide to maintain my own extension library, which provides a bit different way to solve it using annotations and a macro:

ADT definitions:

import org.latestbit.circe.adt.codec._


sealed trait TestEvent

@JsonAdt("my-event-1") 
case class MyEvent1(anyYourField : String /*, ...*/) extends TestEvent

@JsonAdt("my-event-2")
case class MyEvent2(anyOtherField : Long /*, ...*/) extends TestEvent


Usage:


import io.circe._
import io.circe.parser._
import io.circe.syntax._

// This example uses auto coding for case classes. 
// You decide here if you need auto/semi/custom coders for your case classes.
import io.circe.generic.auto._ 

// One import for this ADT/JSON codec
import org.latestbit.circe.adt.codec._

// Encoding

implicit val encoder : Encoder[TestEvent] = 
  JsonTaggedAdtCodec.createEncoder[TestEvent]("type")

val testEvent : TestEvent = TestEvent1("test")
val testJsonString : String = testEvent.asJson.dropNullValues.noSpaces

// Decoding
implicit val decoder : Decoder[TestEvent] = 
  JsonTaggedAdtCodec.createDecoder[TestEvent] ("type")

decode[TestEvent] (testJsonString) match {
   case Right(model : TestEvent) => // ...
}

Details: https://github.com/abdolence/circe-tagged-adt-codec

Curren answered 8/4, 2020 at 12:37 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.