In scala, are there any condition where implicit view won't be able to propagate to other implicit function?

Question

Assuming that A class called 'summoner' was defined, that is capable of summoning implicit views from the scope:

  case class Summoner[R]() {

    def summon[T](v: T)(implicit ev: T => R): R = ev(v)
  }

I found that it works most of the time, but there are cases where it doesn't work, e.g. the following is a (not too) short case which uses the singleton-ops library:

import shapeless.Witness
import singleton.ops.+
import singleton.ops.impl.Op

  trait Operand {

    def +[
        X >: this.type <: Operand,
        Y <: Operand
    ](that: Y): Op2[X, Y] = {

      Op2[X, Y](this, that)
    }
  }

  object Operand {

    abstract class ProvenToBe[O <: Arity]()(implicit val out: O) extends Operand {}

    object ProvenToBe {

      implicit class Trivial[O <: Arity, T <: ProvenToBe[O]](
          val self: T
      ) extends Proof {

        override type Out = O

        override def out: Out = self.out
      }
    }
  }

  trait Proof extends Serializable {

    def self: Operand

    type Out <: Arity

    def out: Out
  }

  object Proof {

    trait Out_=[+O <: Arity] extends Proof {
      type Out <: O
    }

    trait Invar[S] extends Out_=[Arity.Const[S]] {

      type SS = S
    }
  }

  trait Arity extends Operand {}

  object Arity {

    trait Const[S] extends Arity {

      type SS = S
    }

    object Const {

      implicit class Same[S](val self: Const[S]) extends Proof.Invar[S] {
        override type Out = Const[S]

        override def out: Const[S] = self
      }
    }

    class FromOp[S <: Op]() extends Const[S]

    object FromOp {

      implicit def summon[S <: Op](implicit s: S): FromOp[S] = new FromOp[S]()
    }

    class FromLiteral[S <: Int](val w: Witness.Lt[Int]) extends Const[S] {}

    object FromLiteral {

      implicit def summon[S <: Int](implicit w: Witness.Aux[S]): FromLiteral[S] =
        new FromLiteral[S](w)
    }

    def apply(w: Witness.Lt[Int]): FromLiteral[w.T] = {

      FromLiteral.summon[w.T](w) //TODO: IDEA inspection error
    }

  }

  case class Op2[
      +A1 <: Operand,
      +A2 <: Operand
  ](
      a1: A1,
      a2: A2
  ) extends Operand {}

  object Op2 {

    implicit class ProveInvar[
        A1 <: Operand,
        A2 <: Operand,
        S1,
        S2
    ](
        val self: Op2[A1, A2]
    )(
        implicit
        bound1: A1 => Proof.Invar[S1],
        bound2: A2 => Proof.Invar[S2]
    ) extends Proof.Invar[S1 + S2] {

      override type Out = Arity.FromOp[S1 + S2]

      override def out: Out = new Arity.FromOp[S1 + S2]()
    }
  }

When attempting to use the implicit view as-is:

  implicit val a = Arity(3)
  implicit val b = Arity(4)

  val op = a + b

  op: Proof // implicit view works

But when using the summoner:

  val summoner = Summoner[Proof]()

  summoner.summon(op) // oops

[Error] /home/peng/git/shapesafe/spike/src/main/scala/edu/umontreal/kotlingrad/spike/arity/package.scala:141: No implicit view available from edu.umontreal.kotlingrad.spike.arity.package.Op2[edu.umontreal.kotlingrad.spike.arity.package.Arity.FromLiteral[Int(3)],edu.umontreal.kotlingrad.spike.arity.package.Arity.FromLiteral[Int(4)]] => edu.umontreal.kotlingrad.spike.arity.package.Proof.
one error found

FAILURE: Build failed with an exception.

This error message looks quite bland, almost resembles a common implicit type mismatch error, but previous usage has already culled out that possibility. So my questions are:

1. What's the cause of this behaviour?

2. How do you know it?

Strobile

Answer 1

I told you about debugging implicits with reify, -Xlog-implicits and manual resolution of implicits in In scala 2 or 3, is it possible to debug implicit resolution process in runtime?

If you print tree

import scala.reflect.runtime.universe._
println(reify{
  op: Proof
}.tree)

you'll see how implicit conversion is resolved

(App.this.Op2.ProveInvar(App.this.op)(((self) => Arity.this.Const.Same(self)), ((self) => Arity.this.Const.Same(self))): App.this.Proof)

Indeed, manually resolved

summoner.summon[Op2[Arity.FromLiteral[3], Arity.FromLiteral[4]]](op)(t =>
  Op2.ProveInvar(t)(a1 => Arity.Const.Same(a1), a2 => Arity.Const.Same(a2))
)

compiles but compiler itself can't find implicit conversion

summoner.summon[Op2[Arity.FromLiteral[3], Arity.FromLiteral[4]]](op) //doesn't compile

If you switch -Xlog-implicits on you'll see details

Information: $conforms is not a valid implicit value for App.Arity.FromLiteral[3] => App.Proof.Invar[Nothing] because:
hasMatchingSymbol reported error: type mismatch;
 found   : App.Arity.FromLiteral[3] => App.Arity.FromLiteral[3]
 required: App.Arity.FromLiteral[3] => App.Proof.Invar[Nothing]
  summoner.summon[Op2[Arity.FromLiteral[3], Arity.FromLiteral[4]]](op)

Information: Arity.this.Const.Same is not a valid implicit value for App.Arity.FromLiteral[3] => App.Proof.Invar[Nothing] because:
hasMatchingSymbol reported error: type mismatch;
 found   : App.Arity.Const[Nothing] => App.Arity.Const.Same[Nothing]
 required: App.Arity.FromLiteral[3] => App.Proof.Invar[Nothing]
  summoner.summon[Op2[Arity.FromLiteral[3], Arity.FromLiteral[4]]](op)

Information: App.this.Op2.ProveInvar is not a valid implicit value for App.Op2[App.Arity.FromLiteral[3],App.Arity.FromLiteral[4]] => App.Proof because:
hasMatchingSymbol reported error: No implicit view available from App.Arity.FromLiteral[3] => App.Proof.Invar[Nothing].
  summoner.summon[Op2[Arity.FromLiteral[3], Arity.FromLiteral[4]]](op)

As I told you in When calling a scala function with compile-time macro, how to failover smoothly when it causes compilation errors? you can not always check existence of implicit conversion with implicit parameter (implicit ev: T => R). Sometimes existence of implicit instance T => R is not the same as existence of implicit conversion T => R (not all implicit conversions are typeclass-based). Try to replace

val summoner = Summoner[Proof]()
summoner.summon(op) //doesn't compile

with

summonImplicitView[Proof](op) //compiles

def summonImplicitView[B] = new PartiallyAppliedSummonImplicitView[B]

class PartiallyAppliedSummonImplicitView[B] {
  def apply[A](a: A): B = macro summonImplicitViewImpl[A, B]
}

def summonImplicitViewImpl[A: c.WeakTypeTag, B: c.WeakTypeTag](c: whitebox.Context)(a: c.Tree): c.Tree = {
  import c.universe._
  val tpA = weakTypeOf[A]
  val tpB = weakTypeOf[B]
  val view = c.inferImplicitView(tree = a, from = tpA, to = tpB, silent = false)
  q"$view($a)"
}

You can also try type class ImplicitView from question

case class Summoner[R]() {
  def summon[T](v: T)(implicit ev: ImplicitView[T, R]): R = ev.instance(v)
}

val summoner = Summoner[Proof]()
summoner.summon(op) // compiles

but this type class will work not always because it's type-based and not all implicit conversions are type-based, it ignores value of v during implicit resolution.

---

I guess I finally found the issue (such that if we fix it Summoner will work without macros). You again lost type refinement.

case class Summoner[R]() {
  def summon[T](v: T)(implicit ev: T => R): R = ev(v)
}

val summoner = Summoner[Proof {type Out <: Arity.FromOp[3 + 4]}]() 

// or even
//val summoner = Summoner[Proof {type Out <: Arity.FromOp[3 + 4]; type SS = 3 + 4}]()

summoner.summon(op) //compiles

That's why you had Nothings in -Xlog-implicits logs.

---

I guess I fixed your code. While writing your logic you mixed implicit instances with implicit conversions. Implicit conversions are tricky. I would recommend to write your logic only in terms of type classes (MyTransform) and then if you need conversions define them (myConversion) with respect to these type classes.

// doesn't extend T => R intentionally
trait MyTransform[-T, +R] {
  def transform(v: T): R
}
implicit def myConversion[T, R](v: T)(implicit mt: MyTransform[T, R]): R = mt.transform(v)

case class Summoner[R]() {    
  def summon[T](v: T)(implicit ev: MyTransform[T, R]): R = ev.transform(v)
}

trait Operand {  
  def +[
    X >: this.type <: Operand,
    Y <: Operand
  ](that: Y): Op2[X, Y] = {    
    Op2[X, Y](this, that)
  }
}
object Operand {   
  abstract class ProvenToBe[O <: Arity]()(implicit val out: O) extends Operand {}    
  object ProvenToBe {   
    implicit def trivial[O <: Arity, T <: ProvenToBe[O]]: MyTransform[T, Trivial[O, T]] = self => new Trivial(self)

    /*implicit*/ class Trivial[O <: Arity, T <: ProvenToBe[O]](
      val self: T
    ) extends Proof {   
      override type Out = O
      override def out: Out = self.out
    }
  }
}

trait Proof extends Serializable {    
  def self: Operand
  type Out <: Arity
  def out: Out
}
object Proof {
  trait Out_=[+O <: Arity] extends Proof {
    type Out <: O
  }

  trait Invar[S] extends Out_=[Arity.Const[S]] {
    type SS = S
  }
}

trait Arity extends Operand {}
object Arity {
  trait Const[S] extends Arity {
    type SS = S
  }
  object Const {
    implicit def same[S]: MyTransform[Const[S], Same[S]] = self => new Same(self)

    /*implicit*/ class Same[S](val self: Const[S]) extends Proof.Invar[S] {
      override type Out = Const[S]
      override def out: Const[S] = self
    }
  }

  class FromOp[S <: Op]() extends Const[S]
  object FromOp {
    implicit def summon[S <: Op](implicit s: S): FromOp[S] = new FromOp[S]()
  }

  class FromLiteral[S <: Int](val w: Witness.Lt[Int]) extends Const[S] {}
  object FromLiteral {
    implicit def summon[S <: Int](implicit w: Witness.Aux[S]): FromLiteral[S] =
      new FromLiteral[S](w)
  }

  def apply(w: Witness.Lt[Int]): FromLiteral[w.T] = {
    FromLiteral.summon[w.T](w) //TODO: IDEA inspection error
  }
}

case class Op2[
  +A1 <: Operand,
  +A2 <: Operand
](
   a1: A1,
   a2: A2
 ) extends Operand {}
object Op2 {
  implicit def proveInvar[A1 <: Operand, A2 <: Operand, S1, S2](implicit
    bound1: MyTransform[A1, Proof.Invar[S1]],
    bound2: MyTransform[A2, Proof.Invar[S2]]
  ): MyTransform[Op2[A1, A2], ProveInvar[A1, A2, S1, S2]]
  = self => new ProveInvar(self)

  /*implicit*/ class ProveInvar[
    A1 <: Operand,
    A2 <: Operand,
    S1,
    S2
  ](
     val self: Op2[A1, A2]
   )/*(
     implicit
     bound1: A1 => Proof.Invar[S1],
     bound2: A2 => Proof.Invar[S2]
   )*/ extends Proof.Invar[S1 + S2] {
    override type Out = Arity.FromOp[S1 + S2]
    override def out: Out = new Arity.FromOp[S1 + S2]()
  }
}

implicit val a = Arity(3)
implicit val b = Arity(4)

val op = a + b

op: Proof // compiles

val summoner = Summoner[Proof]()
summoner.summon(op) // compiles

Fortyniner

Thanks a lot for the answer, actually the type of [3+4] is deliberately missing, as a strong enough type inference should be able to deduce it (it is the only view in the scope). The [Nothing] in the search log is indeed a sharp observation, but it is also incorrect, as Invar[S] is non-variant or covariant. If it is contravariant it actually make sense to search from Invar[Nothing] as it would be the superset of all kinds of Invar[_], but the compiler only choose it when op: Proof is used – Strobile

I tried the implementation in which Invar[+S] is defined and val summoner = Summoner[Proof.Invar[Any]](), the search still starts from Nothing tho. Kind of weird – Strobile

@Strobile I guess I fixed your code (without macros). I replaced implicit conversions with a type class. See update. – Fortyniner

@Strobile "The [Nothing] in the search log is indeed a sharp observation, but it is also incorrect, as Invar[S] is non-variant or covariant." Normally Nothing signals not about variance but that some type is not inferred. – Fortyniner

thanks a lot! I saw the update and it seems to delay the problem a bit: try a + b + b or a + b + a + b them all hell break loose :-< Further more, in both of our examples, the IntelliJ IDE successfully show implicit hints: github.com/tribbloid/shapesafe/issues/1. Is EPFL being outsmarted by St. Peterburg? :-> – Strobile

@Strobile it seems to compile both in 2.13.2 scastie.scala-lang.org/pzJ6KHu5QUKIwyHtYoO50w and 2.12.11 scastie.scala-lang.org/k3aacN98T7OV5lHHiFdW3A – Fortyniner

Yeah you are right, I made a mistake and used the old summoner definition. Wow this is blowing my mind. Turns out that view bound doesn't have any special treatment. I'll see if I can attract some attention on scalac's issue tracker – Strobile

@Strobile Here Martin Odersky explains why presence of implicit conversions makes type inference worse contributors.scala-lang.org/t/… – Fortyniner