In scala 2 or 3, is it possible to debug implicit resolution process in runtime?

Question

In scala language, implicit resolution is often done in compile-time and sometimes throws obfuscating error information, one famous example of such error is when shapeless Generic throws error information like:

error: could not find implicit value for parameter encoder: CsvEncoder[Foo]

(see https://books.underscore.io/shapeless-guide/shapeless-guide.html for detail)

A solution to this problem is to run implicit resolution algorithm (should be a graph query algorithm internally) in runtime, this has at least 2 benefits:

• debugging tools can be used to reproduce the resolution process step-by-step, so even error information & documentations are incomplete it would be easy to spot the error.

• in many cases type information can be impossible to be determined in compile-time (e.g. type depending on the control flow). If implicit conversion cannot be delayed to runtime phase, many benefit of defining implicit conversion will be nullified.

So my question is, does this feature exist in Scala 2.x or Dotty? Or is it on the roadmap?

Thanks a lot for your opinion.

Malebranche

Implicit resolution is done by the compiler, there is not compiler in runtime. Second implicit resolution is type driven, there aren't types in compile time. So no, you can't. However for debugging you can use the -Xlog-implicits & -explaintypes compiler flags. - For your second point, as I said, there are not types in runtime, so your proposal doesn't make sense. Additionally you shouldn't need to know about runtime information to refine your types, that is the idea of a strong type system. – Sessions

This is strange idea to debug implicits at runtime since there are ways to debug things at compile time – Legalism

@DmytroMitin why not? in gradle you can even debug the build & dependency resolution process using groovy or kotlin script (and it helps when a build plugin is too new to have documentations or stable behaviour). Why not doing the same for compiler? In addition, implicit is a concept derived from church type system, which already integrated compile-time mirror and runtime mirror in reflection. Calling compiler features in runtime is common in libraries like REPL, ammonite & scalameta – Malebranche

BTW there are types in runtime, they are encoded by TypeTag[_] – Malebranche

@Malebranche right, you have to do some extra efforts to transfer some info from compile time to runtime but at compile time you already have all necessary info. Could you provide some reference regarding "implicit is a concept derived from church type system"? Could you write some example regarding "Calling compiler features in runtime ... in scalameta"? I guess scalameta works not at runtime and not at compile time but at time before compile time. – Legalism

@Malebranche maybe you can be interested in staging in dotty dotty.epfl.ch/docs/reference/metaprogramming/staging.html – Legalism

@Malebranche nothing prevents you from running compiler at runtime and doing whatever you want (resolving implicits, inferring types...) if you actually need that. – Legalism

@Malebranche "in scala language, implicit resolution is often done in compile-time and sometimes throws obfuscating error information... error: could not find implicit..." this usually happens when annotations @implicitNotFound and @implicitAmbiguous are not properly used. – Legalism

Probably, but how could you figure out these options when they are not documented due being a new feature? And there are more edge cases that are hard to explain, how about this constraint from the same book? "If the compiler sees the same type constructor twice and the complexity of the type parameters is increasing, it assumes that branch of search is “diverging”" – Malebranche

@Malebranche What features are new? If you're interested in what "diverging" precisely means you can always read sources github.com/scala/scala/blob/2.13.x/src/compiler/scala/tools/nsc/… github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/… – Legalism

You can debug the compiler. But unless you're familiar with the compiler internals or interested in becoming familiar enough with them in order to change the compiler itself, it'll probably be easier to just use the user-facing compiler flags instead. – Lowestoft

Answer 1

You can debug implicits at compile time:

1. switch on compiler flag -Xlog-implicits

2. try to resolve implicits manually (maybe specifying type parameters as well) and see compile errors

    implicitly[...](...manually...)
   

3. use scala.reflect

    println(reify { implicitly[...] }.tree)
   

   (or switch on compiler flag -Xprint:typer) in order to see how implicits are resolved

4. use IDE functionality to show implicits

   https://github.com/ljwagerfield/debugging-scala-implicits-in-intellij

   https://www.jetbrains.com/help/idea/edit-scala-code.html

5. using macros with compiler internals you can debug implicit resolution

   Is there a type-class that checks for existence of at least one implicit of a type?

   create an ambiguous low priority implicit

   Using the "Prolog in Scala" to find available type class instances

   Finding the second matching implicit

   How to summon a `given` member?

   shapeless/package.scala (def cachedImplicitImpl[T](implicit tTag: WeakTypeTag[T]): Tree = ...)

If you're developing a type class don't forget to use annotations @implicitNotFound and @implicitAmbiguous.

---

You can always postpone compilation of your program till runtime. So instead of program

object App {
  def main(args: Array[String]): Unit = {
    println("test") // test
  }
}

you can have

import scala.reflect.runtime.currentMirror
import scala.reflect.runtime.universe._
import scala.tools.reflect.ToolBox
val toolbox = currentMirror.mkToolBox()

toolbox.eval(q"""
  object App {
    def main(args: Array[String]): Unit = {
      println("test")
    }
  }

  App.main(Array())
""") // test

And instead of

implicitly[Numeric[Int]]

you can have

toolbox.compile(q"""
  implicitly[Numeric[Int]]
""")

or

toolbox.inferImplicitValue(
  toolbox.typecheck(tq"Numeric[Int]", mode = toolbox.TYPEmode).tpe, 
  silent = false
)

But it's too optimistic to think that postponing program compilation till runtime you'll be able to debug implicits easier at runtime rather than at compile time. Actually postponing program compilation till runtime you add one level of indirection more i.e. make things harder to debug.

Legalism