Scala: implementing method with return type of concrete instance

Question

I need a way to enforce a method in an abstract class to have a return type of the concrete class of the object it is called on. The most common example is a copy() method, and I'm currently using an approach based on abstract types:

abstract class A(id: Int) {
  type Self <: A
  def copy(newId: Int): Self
}

class B(id: Int, x: String) extends A(id) {
  type Self = B
  def copy(newId: Int) = new B(newId, x)
}

class C(id: Int, y: String, z: String) extends A(id) {
  type Self = C
  def copy(newId: Int) = new C(newId, y, z)
}

I already saw many approaches, including the ones in this great answer. However, none of them really forces a implementation to return its own type. For example, the following classes would be valid:

class D(id: Int, w: String) extends A(id) {
  type Self = A
  def copy(newId: Int) = new D(newId, w) // returns an A
}

class E(id: Int, v: String) extends A(id) {
  type Self = B
  def copy(newId: Int) = new B(newId, "")
}

The fact that I can do that causes that, if I am doing copies of objects of which the only information I have is that they are of a given subclass of A's:

// type error: Seq[A] is not a Seq[CA]!
def createCopies[CA <: A](seq: Seq[CA]): Seq[CA] = seq.map(_.copy(genNewId()))

Is there a better, type-safe way I can do that?

EDIT: If possible, I would like to keep the ability to create arbitrarily deep hierarchies of abstract classes. That is, in the previous example, I'm expecting to be able to create an abstract class A2 that extends A, and then proceed to create A2's concrete subclasses. However, if that simplifies the problem (as it's the case with abstract types), I do not need to further extend already concrete classes.

Homily

you have pointed to this.type case. Why it is not suited for you? – Mays

this.type is an instance-specific type (i.e. the this.type's of two different A instances are not the same), and therefore is not suitable for my copy() method. – Tiffin

I'd like to suggest similar question: how to make contract that any subclass of a defining class should delcare some methods as final – Mays

Newcomers can also read tpolecat's blogpost about subject. – Predicament

out of curiosity, why not make it a constructor? – Demisemiquaver

Answer 1

The only solution I could think of was this one:

trait CanCopy[T <: CanCopy[T]] { self: T =>
  type Self >: self.type <: T
  def copy(newId: Int): Self
}

abstract class A(id: Int) { self:CanCopy[_] =>
  def copy(newId: Int): Self
}

The following would compile:

class B(id: Int, x: String) extends A(id) with CanCopy[B] {
  type Self = B
  def copy(newId: Int) = new B(newId, x)
}

class C(id: Int, y: String, z: String) extends A(id) with CanCopy[C] {
  type Self = C
  def copy(newId: Int) = new C(newId, y, z)
}

The following would not compile:

class D(id: Int, w: String) extends A(id) with CanCopy[D] {
  type Self = A
  def copy(newId: Int) = new D(newId, w) // returns an A
}

class E(id: Int, v: String) extends A(id) with CanCopy[E] {
  type Self = B
  def copy(newId: Int) = new B(newId, "")
}

Edit

I actually forgot to remove the copy method. This might be a bit more generic:

trait StrictSelf[T <: StrictSelf[T]] { self: T =>
  type Self >: self.type <: T
}

abstract class A(id: Int) { self:StrictSelf[_] =>
  def copy(newId:Int):Self
}

Tibetan

I've never seen any construction like your StrictSelf, it seems it constraints Self in a pretty strong way! Are you aware of any flaws in this design? I mean, is there any way to create a subclass A1 which is an A but either isn't a StrictSelf[A1] or doesn't have Self = A1? – Tiffin

I think there are only a few use cases for this pattern. I can think of only one, and that is to help other programmers remember that they need to do something with the class they are writing. The limitation on this pattern is similar to the one of case classes: you can not extend a StrictSelf with another StrictSelf. The mandatory override of the Self type would be incompatible. – Tibetan

Right, I already faced that limitation with other abstract type based approaches and that is the reason why I said I needn't extend a concrete class. This is a great pattern, although I probably won't use it as it introduces too much boilerplate and complexity in an otherwise very simple class. Thank you! – Tiffin

@EECOLOR: This is great. Could you please explain(or any link is appreciated) what 'type Self >: self.type <: T' means? Are you saying that Self is a supertype of self and self is a subtype of T? or Self is a supertype of self and Self is a subtype of T. I am learning via stackoverflow :) – Audrit

@Audrit Self is a super type of the current type (self.type) and a subtype of T. Since T is passed in as type and specified as self type using: self:T the contract can only be solved one way. – Tibetan

One downside with this Curiously Recurring Template Pattern approach is that you cannot extent a concrete instance of CanCopy. Once type Self = B is defined then it is effectively final and you cannot extend it. – Biscay

Answer 2

Do not force the type bound on the declaration side, unless you need that bound within the definition of A itelf. The following is sufficient:

abstract class A(id: Int) {
  type Self
  def copy(newId: Int): Self
}

Now force the Self type on the use site:

def genNewId(): Int = ???
def createCopies[A1 <: A { type Self = A1 }](seq: Seq[A1]): Seq[A1] = 
  seq.map(_.copy(genNewId()))

Overline

I like this approach and I may really have to resort to something like this, although it would be better if I could force all implementations to be correct, as I asked in the question. Forcing this restriction in the use site only, couldn't I avoid using existential types and structural types at all and simply put a context bound? Something in the lines of createCopies[A1 <: A: Copyable](seq: Seq[A1]): Seq[A1], where Copyable would be trait Copyable[A] { def copy(a: A, newId: Int): A }? – Tiffin

Answer 3

I don't think it's possible in scala to do what you want.

If I were to:

class Base { type A }
class Other extends Base
class Sub extends Other

Now... we want type A to refer to "the type of the subclass."

You can see that from the context of Base, it's not particularly clear (from the compiler's perspective) what the specific "type of the subclass" means, nevermind what the syntax would be to refer to it in the parent. In Other it would mean an instance of Other, but in Sub it might mean an instance of Sub? Would it be OK to define an implementation of your method returning an Other in Other but not in Sub? If there are two methods returning A's, and one is implemented in Other and the other in Sub, does that mean the type defined in Base has two different meanings/bounds/restrictions at the same time? Then what happens if A is referred to outside of these classes?

The closest thing we have is this.type. I'm not sure if it would be theoretically possible to relax the meaning of this.type (or provide a more relaxed version), but as implemented it means a very specific type, so specific that the only return value satisfying def foo:this.type is this itself.

I'd like to be able to do what you suggest, but I'm not sure how it would work. Let's imagine that this.type meant... something more general. What would it be? We can't just say "any of the defined types of this," because you wouldn't want class Subclass with MyTrait{type A=MyTrait} to be valid. We could say "a type satisfying all of the types of this," but it gets confusing when someone writes val a = new Foo with SomeOtherMixin... and I'm still not sure it could be defined in a way that would enable an implementation of both Other and Sub defined above.

We're sort-of trying to mix static and dynamically defined types.

In Scala, when you say class B { type T <: B }, T is specific to the instance, and B is static (I'm using that word in the sense of static methods in java). You could say class Foo(o:Object){type T = o.type}, and T would be different for every instance.... but when you write type T=Foo, Foo is the statically specified type of the class. You could just as well have had an object Bar, and had referred to some Bar.AnotherType. The AnotherType, since it's essentially "static," (though not really called "static" in Scala), doesn't participate in inheritance in Foo.

Athiste

Reading your answer, I may have passed the wrong impression that I absolutely needed an abstract type Self in my implementation, but that is not the case. My only concern is with defining a method whose static return type is the same as the concrete instance's type. I believe this requirement that can easily be verified statically (i.e. at compile time) and does not violate Liskov substitution principle - I could still use any instance of a subclass as an instance of its parent. – Tiffin

Of course, a method like this implies that every subclass must override it in order to obey the contract (even when mixing in), and I understand that that is kind of strange... It may be the case that it is indeed not possible or against an OO principle. What I was looking for here was precisely something like the more general this.type you mentioned (as curiosity, a this.type type has actually two valid values: the object itself and null). – Tiffin

Yeah, the return type matching the instance's type is tricky though. The instance isn't static, nor is its type. Classes become "concrete" when implemented... I guess you probably mean to say that the static return type is the same as the concrete class's static type. The compiler will need to know what that type is though, so like, val f = foo.copy();, what's the compile time type of f now? A virtual function lookup has to happen to know which copy method is called at runtime, and there could be multiple mixins or multiple levels of inheritance providing implementations. – Athiste

You're right! I forgot about null... You could probably also call something that returned Nothing (threw an Exception). I wasn't thinking carefully about it. – Athiste

I think to get what you're asking for, you would need syntax enforcing not only that the method be implemented as-such in any subtype, but for the compiler to get something certain you need it to enforce that it be implemented by every subtype in the class hierarchy. Basically, there would need to be syntax to require every concrete subclass to be final, or for every concrete subclass's sub-classes to also re-implement the method. Even then, getting the return types (the static type of val f above), might be tricky. – Athiste

Hmm.... of course the compiler will just need to enforce that the compile time type of f will be the same as the compile time type of foo... but... somehow that doesn't seem sufficient... hmm... – Athiste

Right! OK, if we have classes: Parent, Sub, Sub2, and copy is only implemented in Sub, val f = (new Sub2).copy would be ambiguous. A runtime virtual function lookup would need to happen to know the type (it would need to be Sub and not Sub2) unless we could enforce implementation in Sub2 or that Sub be final. – Athiste

I don't think special syntax is needed; it may be the case that, due to the type constraints you define, you simply can't create a valid subclass of a certain class. One way of doing that is using self types, such as with @Tibetan solution (simpler cases exist, I already implemented some of them). Note also that I'm not looking to return the runtime type of an object. I mean, if I did val a: A = new B(); val ac = a.copy(), ac would be typed as an A (regardless of it's concrete type, which would be B). However, if a was typed as B, ac would also be a B. – Tiffin

hmm... yeah it does look like his self type prevents inheritance from the subclass. Interesting. – Athiste

Answer 4

However, none of them really forces a implementation to return its own type. For example, the following classes would be valid.

But isn't it normal? Otherwise it would mean that you could not merely extend A to add a new method by example, as it would automatically break the contract that you are trying to create (that is, the new class's copy would not return an instance of this class, but of A). The very fact of being able to have a perfectly fine class A that breaks as soon as you extend it as class B feels wrong to me. But to be honest I have trouble putting words on the problems it causes.

UPDATE: After thinking a bit more about this, I think this could be sound if the type check ("return type == most-derived class") was made only in concrete classes and never on abstract classes or traits. I am not aware of any way to encode that in the scala type system though.

The fact that I can do that causes that, if I am doing copies of objects of which the only information I have is that they are of a given subclass of A's

Why can't you just return a Seq[Ca#Self] ? By example, with this change passing a list of B to createCopies will as expected return a Seq[B] (and not just a Seq[A]:

scala> def createCopies[CA <: A](seq: Seq[CA]): Seq[CA#Self] = seq.map(_.copy(123))
createCopies: [CA <: A](seq: Seq[CA])Seq[CA#Self]

scala> val bs = List[B]( new B(1, "one"), new B(2, "two"))
bs: List[B] = List(B@29b9ab6c, B@5ca554da)

scala> val bs2: Seq[B] = createCopies(bs)
bs2: Seq[B] = List(B@92334e4, B@6665696b)

Polka

Each subclass would have to provide their own copy method (i.e. it would be an abstract method until a concrete class implemented it), as only concrete subclasses know what is the return type of a "copy" of an object. That does not seem to me like a forced requirement; in fact, Java does that in Cloneable (docs.oracle.com/javase/1.4.2/docs/api/java/lang/Cloneable.html), but in a non-typesafe way and, if the method is not overridden, uses reflection. CA#Self is not necessarily a CA; therefore, I could not access CA-specific methods and fields in the copied instances. – Tiffin

I don't see how the example of Cloneable is relevant as it precisely does not (and cannot) force the sub-classes to override Clone and return an instance of the most derived class. That aside, I think I can imagine how it could be sound, if the corresponding type check ("return type == most-derived class") was made only in concrete classes and never on abstract classes or traits. I think this can make make some sense (but I don't think you can encode that in scala). BTW, what about my second point (having createCopies return Seq[Ca#Self]) ? – Bathe

You're right, it does not and cannot but it should, as it is in its "semantic" contract that cloning an object should return a value-level copy of it and should therefore be of the same type (because of that and of other restrictions, many say that interface is broken). What I'm trying to do is a type-safe way of doing that, basically :) Regarding the second point, I responded in my first comment. – Tiffin