Why does a for comprehension expand to a `withFilter`

Question

I'm working on a DSL for relational (SQL-like) operators. I have a Rep[Table] type with an .apply: ((Symbol, ...)) => Obj method that returns an object Obj which defines .flatMap: T1 => T2 and .map: T1 => T3 functions. As the type Rep[Table] does not know anything about the underlying table's schema, the apply method acts like a projection - projecting only the fields specified in the argument tuple (a lot like the untyped scalding api). Now type T1 is a "tuple-like", its length constrained to the projection tuple's length by some shapeless magic, but otherwise the types of the tuple elements are decided by the api user, so code like

val a = loadTable(...)
val res = a(('x, 'y)).map { (t: Row2[Int, Int]) =>
  (1, t(0))
}

or

val res = a(('x, 'y)).map { (t: Row2[String, String]) =>
  (1, t(0))
}

works fine. Note that the type of the argument for the map/flatMap function must be specified explicitly. However, when I try to use it with a for comprehension

val a = loadTable(...)
val b = loadTable(...)
val c = loadTable(...)

val res = for {
  as: Row2[Int, Int] <- a(('ax, 'ay))
  bs: Row2[Int, Int] <- b(('bx, 'by))
  cs: Row2[Int, Int] <- c(('cx, 'cy))
} yield (as(0), as(1), bs(0), bs(1), cs(0), cs(1))

it complains about the lack of a withFilter operator. Adding an .withFilter: T1 => Boolean does not cut it - it then complains about "missing parameter type for expanded function", as T1 is parameterised by some type. Only adding .withFilter: Row[Int, Int] => Boolean makes it work, but obviously is not what I want at all.

My questions are: why does the withFilter get called in the first place and how can I use it with my parameterised tuple-like type T1?

---

Edit In the end I went with a .withFilter: NothingLike => BoolLike which is a noop for simple checks like _.isInstanceOf[T1] and a more restricted .filter: T1 => BoolLike to be used in general case.

Blowbyblow

I was always under the impression that withFilter is used on a monadic type in a for comprehension when you included a guard condition (like an if in the for comprehension) but I could be wrong. – Feral

That makes two of us – Blowbyblow

I believe your type annotation on the LHS is making it call withFilter to make sure the result is the right type – Broadsword

@Broadsword Yes, that is exactly what is happening. Unfortunately, the OP wanted this type to propagate into the map method so it does the right thing. – Nonmetal

Answer 1

Unfortunately, you cannot use for-comprehensions when you expect type inference based on the argument type of your lambda.

Actually, in your example, as: Row2[Int, Int] is interpreted as a pattern match:

val res = for {
  as: Row2[Int, Int] <- a(('ax, 'ay))
} yield (...)

Translates to something like:

a(('ax, 'ay)).withFilter(_.isInstanceOf[Row2[Int, Int]]).map(...)

Pattern matching in for comprehensions can be very useful:

val names = for {
  ("name", name) <- keyValPairs
} yield name

But the trade-off is, that you cannot explicitly specify the argument type of the lambda.

Nonmetal

Okay, thanks, that's been revealing. So I could probably work around it with something like withFilter(t: Any => Boolean), but then I get the "unchecked, type erasure" warnings, presumably from the isInstanceOf call? withFilter(t: Nothing => Boolean) works without a complain, but it seems like I could run into trouble when trying to extend it to actually do some useful stuff later... – Blowbyblow

Ahem... withFilter(t: Nothing => Boolean) is almost certainly not what you want. While you can call withFilter with any Function1 that returns a Boolean, you cannot call that function inside the method (since there is no value of type Nothing). – Nonmetal

To clarify a bit with what I ended up with: in my case the function is something which returns an abstract syntax tree instead of a concrete value, the signature is something more like Rep[Nothing] => Rep[Boolean] with covariant Rep type and the call works fine. – Blowbyblow

Answer 2

I bumped into this issue also. Thanks to gzm0 explaining the Scala compilers behaviour I came up with this workaround:

import cats._
import cats.data._
import cats.implicits._

object CatsNEL extends App {
  val nss: NonEmptyList[(Int, String)] = NonEmptyList.of((1,"a"), (2, "b"), (3, "c"))
  val ss: NonEmptyList[String] = for {
    tuple <- nss
    (n, s) = tuple
  } yield s
}

Ministration