Getting a structural type with an anonymous class's methods from a macro

Question

Suppose we want to write a macro that defines an anonymous class with some type members or methods, and then creates an instance of that class that's statically typed as a structural type with those methods, etc. This is possible with the macro system in 2.10.0, and the type member part is extremely easy:

object MacroExample extends ReflectionUtils {
  import scala.language.experimental.macros
  import scala.reflect.macros.Context

  def foo(name: String): Any = macro foo_impl
  def foo_impl(c: Context)(name: c.Expr[String]) = {
    import c.universe._

    val Literal(Constant(lit: String)) = name.tree
    val anon = newTypeName(c.fresh)

    c.Expr(Block(
      ClassDef(
        Modifiers(Flag.FINAL), anon, Nil, Template(
          Nil, emptyValDef, List(
            constructor(c.universe),
            TypeDef(Modifiers(), newTypeName(lit), Nil, TypeTree(typeOf[Int]))
          )
        )
      ),
      Apply(Select(New(Ident(anon)), nme.CONSTRUCTOR), Nil)
    ))
  }
}

(Where ReflectionUtils is a convenience trait that provides my constructor method.)

This macro lets us specify the name of the anonymous class's type member as a string literal:

scala> MacroExample.foo("T")
res0: AnyRef{type T = Int} = $1$$1@7da533f6

Note that it's appropriately typed. We can confirm that everything's working as expected:

scala> implicitly[res0.T =:= Int]
res1: =:=[res0.T,Int] = <function1>

Now suppose that we try to do the same thing with a method:

def bar(name: String): Any = macro bar_impl
def bar_impl(c: Context)(name: c.Expr[String]) = {
  import c.universe._

  val Literal(Constant(lit: String)) = name.tree
  val anon = newTypeName(c.fresh)

  c.Expr(Block(
    ClassDef(
      Modifiers(Flag.FINAL), anon, Nil, Template(
        Nil, emptyValDef, List(
          constructor(c.universe),
          DefDef(
            Modifiers(), newTermName(lit), Nil, Nil, TypeTree(),
            c.literal(42).tree
          )
        )
      )
    ),
    Apply(Select(New(Ident(anon)), nme.CONSTRUCTOR), Nil)
  ))
}

But when we try it out, we don't get a structural type:

scala> MacroExample.bar("test")
res1: AnyRef = $1$$1@da12492

But if we stick an extra anonymous class in there:

def baz(name: String): Any = macro baz_impl
def baz_impl(c: Context)(name: c.Expr[String]) = {
  import c.universe._

  val Literal(Constant(lit: String)) = name.tree
  val anon = newTypeName(c.fresh)
  val wrapper = newTypeName(c.fresh)

  c.Expr(Block(
    ClassDef(
      Modifiers(), anon, Nil, Template(
        Nil, emptyValDef, List(
          constructor(c.universe),
          DefDef(
            Modifiers(), newTermName(lit), Nil, Nil, TypeTree(),
            c.literal(42).tree
          )
        )
      )
    ),
    ClassDef(
      Modifiers(Flag.FINAL), wrapper, Nil,
      Template(Ident(anon) :: Nil, emptyValDef, constructor(c.universe) :: Nil)
    ),
    Apply(Select(New(Ident(wrapper)), nme.CONSTRUCTOR), Nil)
  ))
}

It works:

scala> MacroExample.baz("test")
res0: AnyRef{def test: Int} = $2$$1@6663f834

scala> res0.test
res1: Int = 42

This is extremely handy—it lets you do things like this, for example—but I don't understand why it works, and the type member version works, but not bar. I know this may not be defined behavior, but does it make any sense? Is there an cleaner way to get a structural type (with the methods on it) from a macro?

Nonresistant

Interestingly enough, if you write the same code in REPL instead of generating it in a macro, it works: scala> { final class anon { def x = 2 }; new anon } res1: AnyRef{def x: Int} = anon$1@5295c398. Thanks for the report! I'll take a look this week. – Translucid

Note that I've filed an issue here. – Nonresistant

Nope, not a blocker, thanks—the extra anonymous class trick has worked for me whenever I needed it. I just noticed a couple of upvotes on the question and was curious about the status. – Nonresistant

Eugene, for posterity's sake, if this is already listed as a bug somewhere can you please link it? – Gur

@TomerGabel: It's SI-6992 (linked in a comment above). – Nonresistant

type member part is extremely easy--> wTF? you are extremely crack!in the good way of course :) – Ubiquitous

There are 153 upvotes here, and only 1 for the issue on scala-lang.org. More upvotes there might get it resolved faster? – Ideality

For the record, SI-6992 was fixed last month (thanks, Eugene!) and has just been closed. – Nonresistant

Scala 3 #74549977 – Weisman

Answer 1

This question is answered in duplicate by Travis here. There are links to the issue in the tracker and to Eugene's discussion (in the comments and mailing list).

In the famous "Skylla and Charybdis" section of the type checker, our hero decides what shall escape dark anonymity and see the light as a member of the structural type.

There are a couple of ways to trick the type checker (which do not entail Odysseus's ploy of hugging a sheep). The simplest is to insert a dummy statement so that the block doesn't look like an anonymous class followed by its instantiation.

If the typer notices that you're a public term that isn't referenced by the outside, it will make you private.

object Mac {
  import scala.language.experimental.macros
  import scala.reflect.macros.Context

  /* Make an instance of a structural type with the named member. */
  def bar(name: String): Any = macro bar_impl

  def bar_impl(c: Context)(name: c.Expr[String]) = {
    import c.universe._
    val anon = TypeName(c.freshName)
    // next week, val q"${s: String}" = name.tree
    val Literal(Constant(s: String)) = name.tree
    val A    = TermName(s)
    val dmmy = TermName(c.freshName)
    val tree = q"""
      class $anon {
        def $A(i: Int): Int = 2 * i
      }
      val $dmmy = 0
      new $anon
    """
      // other ploys
      //(new $anon).asInstanceOf[{ def $A(i: Int): Int }]
      // reference the member
      //val res = new $anon
      //val $dmmy = res.$A _
      //res
      // the canonical ploy
      //new $anon { }  // braces required
    c.Expr(tree)
  }
}

Pt

I'll just note that I actually provide the first workaround in this question itself (it's just un-quasiquoted here). I'm happy to have this answer wrap up the question—I think I'd been vaguely waiting for the bug to get fixed. – Nonresistant

@TravisBrown I bet you have other tools in your Bat Belt, too. Thx for the heads up: I assumed your AST was "the old extra braces trick", but I see now that the ClassDef/Apply are not wrapped in their own Block, as happens with new $anon {}. My other take-away is that in future I won't use anon in macros with quasiquotes, or similar special names. – Pt

q"${s: String}" syntax gets delayed a bit, especially if you are using paradise. So more like next month rather than next week. – Bortman

@Pt @denys-shabalin, is there a special kind of trickery for structural types a-la shapeless.Generic? In spite of my best intentions to force Aux pattern return types the compiler refuses to see through the structural type. – Grigg