Specify Rust closures lifetime

Question

I was making the executor/reactor while discovered this a lifetime problem. It is not related to async/Future and can be reproduced without async sugar.

use std::future::Future;

struct Runtime;

fn start_with_runtime<C, F>(closure: C)
where
    C: for<'a> FnOnce(&'a Runtime) -> F,
    F: Future
{
    let rt = Runtime;
    let _future = closure(&rt);
    // block_on(future); 
}

async fn async_main(_rt: &Runtime) {
    // I can use _rt to do async stuff here
}

fn main() {
    start_with_runtime(|rt| { async_main(rt) });
}

I would like to the start_with_runtime() to run the future and provide the async Runtime reference as parameter.

It does not compile:

error: lifetime may not live long enough
  --> src/main.rs:17:31
   |
17 |     start_with_runtime(|rt| { async_main(rt) });
   |                         ---   ^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                         | |
   |                         | return type of closure is impl std::future::Future
   |                         has type `&'1 Runtime`

I think that this problem seems to be because how rust infers lifetimes for closures:

https://github.com/rust-lang/rust/issues/58052 :

fn main() {
    let f = |x: &i32| x;
    let i = &3;
    let j = f(i);
}

Does not compiles either:

error: lifetime may not live long enough
 --> src/main.rs:2:23
  |
2 |     let f = |x: &i32| x;
  |                 -   - ^ returning this value requires that `'1` must outlive `'2`
  |                 |   |
  |                 |   return type of closure is &'2 i32
  |                 let's call the lifetime of this reference `'1`

Looks like my closure signature inferred as |&'a Runtime| -> impl Future + 'b and thus the lifetime error. I feel that given correct expected signature for closure would help, but how do I provide the correct signature in start_with_runtime?

fn start_with_runtime<C>(closure: C)
where
    C: for<'a> FnOnce(&'a Runtime) -> (impl Future + 'a),

Does not work because impl Trait is not allowed here.

fn start_with_runtime<C,F>(closure: C)
where
    C: for<'a> FnOnce(&'a Runtime) -> F,
    F: Future + 'a

Does not work as well because 'a is not known outside of HRTB expression.

It works if I know the type:

struct MyType<'a> {
    _rt: &'a Runtime
} 
fn start_with_runtime<C>(closure: C)
where
    C: for<'a> FnOnce(&'a Runtime) -> MyType<'a>,

This is kind of sad when you've thought through all the lifetimes but language does not provide a way to express this. Perhaps there is a trick in rust to make this work?

Eurasian

I take it you cant just extract start_with_runtime to main? Because that should work, without any explicit lifetimes. – Tautog

The start_with_runtime supposed to be in a crate and used by the apps (e.g. hiding Runtime construction from apps). This is kind of backup plan that app can let rt = Runtime::new(); rt.run(|rt| my_async_fn(rt)); – Eurasian

The return type of an async function indeed captures all argument lifetimes. It has to, since the arguments need to be stored in the Future whenever the async function awaits some other future. – Darb

Does receiving a shared pointer like Rc<Runtime> instead of a reference to the runtime work for you? – Darb

I suppose Rc<> would work, but this is overhead and does not like a right ownership model to me. – Eurasian

Potentially relevant: github.com/rust-lang/rfcs/pull/3216 – Lombardo

Answer 1

There seem to be two different questions in this one: can the required relationship be expressed in Rust syntax and will it work with closure type inference or not.

Let's start with the first one. You're right that this cannot be expressed with just where clauses. To express this one needs to add a helper trait

trait BorrowingFn<'a> {
    type Fut: std::future::Future<Output = Something> + 'a;
    fn call(self, arg: &'a Runtime) -> Self::Fut;
}

that allows the bound we need to be written as

    C: for<'a> BorrowingFn<'a>,

and provide a blanket implementation of this trait for all applicable functions

impl<'a, Fu: 'a, F> BorrowingFn<'a> for F
where
    F: FnOnce(&'a Runtime) -> Fu,
    Fu: std::future::Future<Output = ()> + 'a,
{
    type Fut = Fu;
    fn call(self, rt: &'a Runtime) -> Fu {
        self(rt)
    }
}

(playground)

Ok, so it works with an async function, but does it work with a closure that needs type inference? Unfortunately, the answer is "no"

error: implementation of `BorrowingFn` is not general enough
  --> src/main.rs:33:5
   |
5  | / trait BorrowingFn<'a> {
6  | |     type Fut: std::future::Future<Output = ()> + 'a;
7  | |     fn call(self, arg: &'a Runtime) -> Self::Fut;
8  | | }
   | |_- trait `BorrowingFn` defined here
...
33 |       start_with_runtime(|rt| async_main(rt)); // however, it does not work with closure type inference :-(
   |       ^^^^^^^^^^^^^^^^^^ implementation of `BorrowingFn` is not general enough
   |
   = note: `[closure@src/main.rs:33:24: 33:43]` must implement `BorrowingFn<'0>`, for any lifetime `'0`...
   = note: ...but `[closure@src/main.rs:33:24: 33:43]` actually implements `BorrowingFn<'1>`, for some specific lifetime `'1`

This is being tracked in rust-lang/rust#70263. The compiler is not smart enough yet to notice that this closure needs a higher-rank type. For fun I tried compiling with -Z chalk on Nightly, but it's not yet ready for this (internal compiler error).

Theodoratheodore

Looks like very clever approach! I think that this is good trade-off for me (it does not involve heap but require the FnOnce to be a function, not the closure, which is kind of ok). – Eurasian

Answer 2

Sorry, this is a limitation in the language. You can only specify the lifetime on concrete types. One workaround is to use the trait object type.

fn start_with_runtime<C, F, T>(closure: C)
where
    C: for<'a> FnOnce(&'a Runtime) -> Pin<Box<dyn Future<Item = T> + Send + 'a>>,
{
    let rt = Runtime;
    let _future = closure(&rt);
    // block_on(future); 
}

Areopagus

With some fixes I was able to compile this version. In this case the close looks like start_with_runtime(|rt| { Box::pin(async_main(rt)) }); so this is a possible workaround. However I found the solution Tanriol looks preferable because it does not require heap and runtime polymorphism. – Eurasian