Why does a generic method inside a trait require trait object to be sized?

Question

I have this code (playground):

use std::sync::Arc;

pub trait Messenger : Sync + Send {
    fn send_embed<F: FnOnce(String) -> String>(&self, u64, &str, f: F)
        -> Option<u64> where Self: Sync + Send;
}

struct MyMessenger {
    prefix: String,
}
impl MyMessenger {
    fn new(s: &str) -> MyMessenger {
        MyMessenger { prefix: s.to_owned(), }
    }
}
impl Messenger for MyMessenger {
    fn send_embed<F: FnOnce(String) -> String>(&self, channel_id: u64, text: &str, f: F) -> Option<u64> {
        println!("Trying to send embed: chid={}, text=\"{}\"", channel_id, text);
        None
    }

}

struct Bot {
    messenger: Arc<Messenger>,
}
impl Bot {
    fn new() -> Bot {
        Bot {
            messenger: Arc::new(MyMessenger::new("HELLO")),
        }
    }
}

fn main() {
    let b = Bot::new();
}

I wanted to make a polymorphic object (trait Messenger and one of polymorphic implementations is MyMessenger). But when I try to compile it I have an error:

error[E0038]: the trait `Messenger` cannot be made into an object
  --> <anon>:25:5
   |
25 |     messenger: Arc<Messenger>,
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Messenger` cannot be made into an object
   |
   = note: method `send_embed` has generic type parameters

I have found that I must require Sized in this case, but this does not solve it. If I change my send_embed method to the following:

fn send_embed<F: FnOnce(String) -> String>(&self, u64, &str, f: F)
    -> Option<u64> where Self: Sized + Sync + Send;

Then it compiles successfully but:

1. Why do we need Sized here? This violates polymorphism if we can not use this method from a trait object.

2. We actually can't use this method from Arc<Messenger> then:

   fn main() {
       let b = Bot::new();
       b.messenger.send_embed(0u64, "ABRACADABRA", |s| s);
   }
   

   Gives:

   error[E0277]: the trait bound `Messenger + 'static: std::marker::Sized` is not satisfied
     --> <anon>:37:17
      |
   37 |     b.messenger.send_embed(0u64, "ABRACADABRA", |s| s);
      |                 ^^^^^^^^^^ the trait `std::marker::Sized` is not implemented for `Messenger + 'static`
      |
      = note: `Messenger + 'static` does not have a constant size known at compile-time
   

I am totally stuck here. No idea how to use polymorphism with generic method in a trait. Is there a way?

Magnetite

This has to do with the notion of trait object and "object safety" which have been discussed quite a bit... but I can't find a good duplicate which explains the issue in details. – Romanfleuve

The answers to this question might be helpful. – Summit

@AleksanderFular Did you try to correct the code on the playground? Because I have tried this before you have said and this did not help me unfortunately. And after your comment I have tried it again, without any success. – Magnetite

Answer 1

Dynamic dispatch (i.e. calling methods through trait objects) works by calling through a vtable, (i.e. using a function pointer), since you don't know at compile time which function it will be.

But if your function is generic, it needs to be compiled differently (monomorphised) for every instance of F which is actually used. Which means you'll have a different copy of send_embed for every different closure type it's called with. Every closure is a different type.

These two models are incompatible: you can't have a function pointer which works with different types.

However, you can change the method to use a trait object as well instead of being compile-time generic:

pub trait Messenger : Sync + Send {
    fn send_embed(&self, u64, &str, f: &dyn Fn(String) -> String)
        -> Option<u64> where Self: Sync + Send;
}

(Playground)

Instead of a different send_embed for every type which can be Fn(String) -> String, it now accepts a trait object reference. (You could also use a Box<Fn()> or similar). You do have to use Fn or FnMut and not FnOnce, since the latter takes self by value, i.e. it's also not object safe (the caller doesn't know what size to pass in as the closure's self parameter).

You can still call send_embed with a closure/lambda function, but it just needs to be by reference, like this:

self.messenger.send_embed(0, "abc", &|x| x);

I've updated the playground to include an example of calling send_embed directly with a referenced closure, as well as the indirect route through a generic wrapper on Bot.

Biamonte

Is not there a way to use lambda then? And what if I use it as b.messenger.send_embed(0u64, "ABRACADABRA", &|s| s); ? Is this okay? – Magnetite

@VictorPolevoy You can - I've updated the answer to show how. – Biamonte

Thanks for this. The last question - should I always do that if I want to have a polymorphic object? Is this a normal practice or just a one-time fix for this specific problem? – Magnetite

If you need runtime polymorphism, then trait objects (optionally via Box, Rc, etc.) are how you do that in Rust. – Biamonte

Here is another question, what if I have send_file<R: std::io::Read>(..) method? If I try to do the same thing, I have another error: the trait std::io::Read is not implemented for &std::io::Read – Magnetite

@VictorPolevoy It should be analogous, so not sure without seeing the code. Sounds like another question (if there isn't an existing question which covers it). – Biamonte

okay, I have found a workaround for that, so, nevermind. But just letting you know, the same method did not work. – Magnetite

Would a similar solution be possible if the Fn parameter references Self? – Rosenberger

Answer 2

Traits and Traits

In Rust, you can use trait to define an interface comprised of:

• associated types,
• associated constants,
• associated functions.

and you can use traits either:

• as compile-time bounds for generic parameters
• as types, behind references or pointers.

However... only some traits can be used directly as types. Those traits that do are labeled Object Safe.

It is now considered unfortunate that a single trait keyword exists to define both full-featured and object-safe traits.

---

Interlude: How does run-time dispatch work?

When using a trait as a type: &Trait, Box<Trait>, Rc<Trait>, ... the run-time implementation uses a fat pointer composed of:

• the data pointer,
• the virtual pointer.

Method calls are dispatched through the virtual pointer to a virtual table.

For a trait like:

trait A {
    fn one(&self) -> usize;
    fn two(&self, other: usize) -> usize;
}

implemented for type X, the virtual table will look like (<X as A>::one, <X as A>::two).

The run-time dispatch is thus performed by:

• picking the right member of the table,
• calling it with the data pointer and arguments.

This means that <X as A>::two looks like:

fn x_as_a_two(this: *const (), other: usize) -> usize {
    let x = unsafe { this as *const X as &X };
    x.two(other)
}

---

Why cannot I use any trait as a type? What's Object Safe?

It's a technical limitation.

There are a number of traits capabilities that cannot be implemented for run-time dispatches:

• associated types,
• associated constants,
• associated generic functions,
• associated functions with Self in the signature.
• ... maybe others ....

There are two ways to signal this issue:

• early: refuse to use a trait as a type if it has any of the above,
• late: refuse to use any of the above on a trait as a type.

For now, Rust chooses to signal the issue early on: traits that do not use any of the above features are call Object Safe and can be used as types.

Traits that are not Object Safe cannot be used as types, and an error is immediately triggered.

---

Now what?

In your case, simply switch from compile-time polymorphism to run-time polymorphism for the method:

pub trait Messenger : Sync + Send {
    fn send_embed(&self, u64, &str, f: &FnOnce(String) -> String)
        -> Option<u64>;
}

There is a little wrinkle: FnOnce requires moving out of the f and it's only borrowed here, so instead you need to use FnMut or Fn. FnMut is next more generic method, so:

pub trait Messenger : Sync + Send {
    fn send_embed(&self, u64, &str, f: &FnMut(String) -> String)
        -> Option<u64>;
}

This makes the Messenger trait Object Safe and therefore allows you to use a &Messenger, Box<Messenger>, ...

Romanfleuve

I don't think &FnOnce() can be called, so you still need to change to Fn or FnMut. – Biamonte

@ChrisEmerson: Good point indeed. It would be impossible to move from a reference. – Romanfleuve

Answer 3

A generic method cannot be made object-safe, because you can't implement a vtable with it. @ChrisEmerson's answer explained in detail why.

In your case, you could make send_embed object-trait, by making f take a trait-object instead of generic parameter. If your function accepts an f: F where F: Fn(X) -> Y, you could make it accept f: &Fn(X) -> Y, similarly for FnMut f: &mut FnMut(X) -> Y. FnOnce is more tricky since Rust doesn't support moving unsized types, but you could try to Box it:

//           ↓ no generic          ↓~~~~~~~~~~~~~~~~~~~~~~~~~~~~ box the closure
fn send_embed(&self, u64, &str, f: Box<FnOnce(String) -> String>) -> Option<u64> 
    where Self: Sync + Send
{
    f("hello".to_string());
    None
}

b.messenger.send_embed(1, "234", Box::new(|a| a));
// note: does not work.

However, as of Rust 1.17.0 you cannot box an FnOnce and call it, you have to use FnBox:

#![feature(fnbox)]
use std::boxed::FnBox;

//                                     ↓~~~~
fn send_embed(&self, u64, &str, f: Box<FnBox(String) -> String>) -> Option<u64> 
    where Self: Sync + Send 
{
    f("hello".to_string());
    None
}

b.messenger.send_embed(1, "234", Box::new(|a| a));

If you don't want to use unstable feature, you could use the crate boxfnonce as a workaround:

extern crate boxfnonce;
use boxfnonce::BoxFnOnce;

fn send_embed(&self, u64, &str, f: BoxFnOnce<(String,), String>) -> Option<u64> 
    where Self: Sync + Send 
{
    f.call("hello".to_string());
    None
}

b.messenger.send_embed(1, "234", BoxFnOnce::from(|a| a));

Matelot

I'd forgotten about FnBox! – Biamonte