Caching the circe implicitly resolved Encoder/Decoder instances

Question

I am using circe to serialize/deserialize some reasonably large models, where each leaf field is a strong type (e.g. case class FirstName(value: String) extends AnyVal).

Implicit resolution/derivation of an Encoder or Decoder is slow.

I have my own codec for which I add some extra Encoder and Decoder instances:

trait JsonCodec extends AutoDerivation {
    // ...
}

With the following method to help with decoding:

package json extends JsonCodec {

  implicit class StringExtensions(val jsonString: String) extends AnyVal {
    def decodeAs[T](implicit decoder: Decoder[T]): T =
      // ...
  }

}

The problem is that every time I call decodeAs, it implicitly derives a Decoder which causes the compilation times to increase massively.

Is there any way I can (generically) cache the implicits such that it will only generate a Decoder once?

Doglike

I'm not sure if it would be possible in your StringExtensions class. But, whenever you call your decodeAs method, you may first do this :implicit val tDecoder: Decoder[T] = derieveDecoder (change T with your own type). That way, all calls to decodeAs[T] it will use the val, instead of derive a new Decoder. Note: If your model consist of many nested types, create decoders for each of them in the reverse order. – Sanmicheli

I don't think that Luis idea as is saves anything because the compiler still needs to generate all those Decoders for every implicit val. The only way I can think of is to put all those decoders for your types as implicit vals in some global known static places like your json package object itself. Then there can be only one such implicit val for each time and the compiler can use those once-derived values every time such Decoder is needed (assuming you import them into your context). – Suit

Answer 1

Why you can't do this generically

This is not possible, since what you are asking boils down to caching a def. Part of the problem is that producing an implicit instance can (although it rarely does) have side effects. Pathological example:

scala> var myVar: Int = 0
myVar: Int = 0

scala> :paste
// Entering paste mode (ctrl-D to finish)

trait DummyTypeclass[T] { val counter: Int }
implicit def dummyInstance[T]: DummyTypeclass[T] = {
  myVar += 1
  new DummyTypeclass[T] {
    val counter = myVar
  }
}

// Exiting paste mode, now interpreting.

defined trait DummyTypeclass
dummyInstance: [T]=> DummyTypeclass[T]

scala> implicitly[DummyTypeclass[Int]].count
res1: Int = 1

scala> implicitly[DummyTypeclass[Boolean]].counter
res2: Int = 2

scala> implicitly[DummyTypeclass[Int]].counter
res3: Int = 3

As you can see, caching the value of DummyTypeclass[Int] would break its "functionality".

The next best thing

The next best thing is to manually cache instances for a bunch of types. In order to avoid boilerplate, I recommend the cachedImplicit macro from Shapeless. For your decoder example, you end up with:

package json extends JsonCodec {

  import shapeless._

  implicit val strDecoder:  Decoder[String]    = cachedImplicit
  implicit val intDecoder:  Decoder[Int]       = cachedImplicit
  implicit val boolDecoder: Decoder[Boolean]   = cachedImplicit
  implicit val unitDecoder: Decoder[Unit]      = cachedImplicit
  implicit val nameDecoder: Decoder[FirstName] = cachedImplicit
  // ...

  implicit class StringExtensions(val jsonString: String) extends AnyVal {
    // ...
  }

}

If you don't like macros, you can do this manually (basically just what the Shapeless macro does), but it might be less fun. This uses a little known trick that implicits can be "hidden" by shadowing their name.

package json extends JsonCodec {

  implicit val strDecoder:  Decoder[String] = {
    def strDecoder = ???
    implicitly[Decoder[String]]
  }
  implicit val intDecoder:  Decoder[Int] = {
    def intDecoder = ???
    implicitly[Decoder[Int]]
  }
  // ...

}

Rebeccarebecka