Why do I need to pin a future before I can await a reference to it?

Question

The tokio tutorial for select! states:

The thing to note is that, to .await a reference, the value being referenced must be pinned or implement Unpin.

Indeed, the following code fails to compile:

let fut = example(); // example is an async fn
(&mut fut).await;

With the following error message:

error[E0277]: `from_generator::GenFuture<[static generator@src/main.rs:15:27: 17:2]>` cannot be unpinned
... snip ...
within `impl futures::Future<Output = i32>`, the trait `Unpin` is not implemented for `from_generator::GenFuture<[static generator@src/main.rs:15:27: 17:2]>
... snip ...
note: consider using `Box::pin`

Pinning the future solves the problem:

let fut = example(); // example is an async fn
tokio::pin!(fut);
(&mut fut).await;

Why is it necessary to pin the future in order to await a reference to it?

Missie

Answer 1

.await generates code that invokes Future::poll. The first parameter has type Pin<&mut Self>. The main characteristic of Pin<P> is that it represents a pointer to a value that is guaranteed to never move after the Pin<P> is created, unless the pointee implements Unpin (which means it doesn't care if it's moved).

Let's first answer: Why is it not necessary to pin a future in order to await it by value? That is, why does this work:

pub async fn foo() {
    let fut = async {};
    fut.await;
}

It's simple: fut.await consumes fut.

pub async fn foo() {
    let fut = async {};
    fut.await;
    drop(fut); // error[E0382]: use of moved value: `fut`
}

There is no opportunity for us to move fut after fut.await, so there is no way we could violate Pin's rules on fut.

However, if we could .await a reference, then we would be able to move the original value after the .await, and that would be disastrous if fut was self-referential.

The error comes from the compiler failing to find an implementation of Future for the expression &mut fut. There is impl<'_, F> Future for &'_ mut F where F: Future + Unpin + ?Sized in the standard library; however, fut doesn't implement Unpin, so that's what the compiler reports in its error message.

tokio::pin!(fut) first moves fut (this is how the macro can ensure it has ownership of the future), then declares a new variable fut of type Pin<&mut F> (where F is the original fut's type).

Thus, the code following tokio::pin!(fut) that uses fut manipulates a pinned reference to the future, rather than the future value directly. If we try to move fut, then we just move the pinned reference. And if we take a mutable reference to fut, then we end up with a reference to a pinned reference (&mut Pin<&mut F>). This type does implement Future because:

• &mut F implements Unpin even if F doesn't implement Unpin
• Pin<&mut F> implements Unpin because &mut F implements Unpin
• Pin<&mut F> implements Future because F implements Future
• &mut Pin<&mut F> implements Future because Pin<&mut F> implements both Future and Unpin

Agitator

Answer 2

Edit: this answer is not entirely correct. See the accepted answer.

Surprising requirements about futures can typically be explained by thinking about what the generated impl Future state machine will look like.

In this case, the problem is the following: the state machine generated by the example code will deduce that it needs to hold onto a reference to fut in order to await it. Because the state machine also captures the stack, it ends up trying to hold both fut and a reference to fut. This is a self-referential struct, and it's problematic because if you were to move the generated future, it would invalidate the self-reference. Solving this problem is the exact purpose of pinning. That's why using tokio::pin! fixes the compiler error.

Missie

It's rare to see someone answering their own questions. good find! – Chaechaeronea