Differences between Futures in Python3 and Promises in ES6

Question

Since Python 3.5, the keywords await and async are introduced to the language. Now, I'm more of a Python 2.7 person and I have been avoiding Python 3 for quite some time so asyncio is pretty new to me. From my understanding it seems like await/async works very similar to how they work in ES6 (or JavaScript, ES2015, however you want to call it.)

Here are two scripts I made to compare them.

import asyncio

async def countdown(n):
    while n > 0:
        print(n)
        n -= 1
        await asyncio.sleep(1)

async def main():
    """Main, executed in an event loop"""

    # Creates two countdowns
    futures = asyncio.gather(
        countdown(3), 
        countdown(2)
    )
    # Wait for all of them to finish
    await futures
    # Exit the app
    loop.stop()

loop = asyncio.get_event_loop()
asyncio.ensure_future(main())
loop.run_forever()

function sleep(n){
    // ES6 does not provide native sleep method with promise support
    return new Promise(res => setTimeout(res, n * 1000));
}

async function countdown(n){
    while(n > 0){
        console.log(n);
        n -= 1;
        await sleep(1);
    }
}

async function main(){
    // Creates two promises
    var promises = Promise.all([
        countdown(3),
        countdown(2)
    ]);
    // Wait for all of them to finish
    await promises;
    // Cannot stop interpreter's event loop
}

main();

One thing to notice is that the codes are very similar and they work pretty much the same.

Here are the questions:

1. In both Python and ES6, await/async are based on generators. Is it a correct to think Futures are the same as Promises?

2. I have seen the terms Task, Future and Coroutine used in the asyncio documentation. What are the differences between them?

3. Should I start writing Python code that always has an event loop running?

Pavilion

Answer 1

1. In both Python and ES6, await/async are based on generators. Is it a correct to think Futures are the same as Promises?

Not Future, but Python's Task is roughly equivalent to Javascript's Promise. See more details below.

2. I have seen the terms Task, Future and Coroutine used in the asyncio documentation. What are the differences between them?

They're quite different concepts. Mainly, Task consists of Future and Coroutine. Let's describe these primitives briefly (I am going to simplify lots of things to describe only main principles):

Future

Future is simply an abstraction of value that may be not computed yet and will be available eventually. It's a simple container that only does one thing - whenever the value is set, fire all registered callbacks.

If you want to obtain that value, you register a callback via add_done_callback() method.

But unlike in Promise, the actual computation is done externally - and that external code has to call set_result() method to resolve the future.

Coroutine

Coroutine is the object very similar to Generator.

A generator is typically iterated within for loop. It yields values and, starting from PEP342 acceptance, it receives values.

A coroutine is typically iterated within the event loop in depths of asyncio library. A coroutine yields Future instances. When you are iterating over a coroutine and it yields a future, you shall wait until this future is resolved. After that you shall send the value of future into the coroutine, then you receive another future, and so on.

An await expression is practically identical to yield from expression, so by awaiting other coroutine, you stop until that coroutine has all its futures resolved, and get coroutine's return value. The Future is one-tick iterable and its iterator returns actual Future - that roughly means that await future equals yield from future equals yield future.

Task

Task is Future which has been actually started to compute and is attached to event loop. So it's special kind of Future (class Task is derived from class Future), which is associated with some event loop, and it has some coroutine, which serves as Task executor.

Task is usually created by event loop object: you give a coroutine to the loop, it creates Task object and starts to iterate over that coroutine in manner described above. Once the coroutine is finished, Task's Future is resolved by coroutine's return value.

You see, the task is quite similar to JS Promise - it encapsulates background job and its result.

Coroutine Function and Async Function

Coroutine func is a factory of coroutines, like generator function to generators. Notice the difference between Python's coroutine function and Javascript's async function - JS async function, when called, creates a Promise and its internal generator immediately starts being iterated, while Python's coroutine does nothing, until Task is created upon it.

3. Should I start writing Python code that always has an event loop running?

If you need any asyncio feature, then you should. As it turns out it's quite hard to mix synchronous and asynchronous code - your whole program had better be asynchronous (but you can launch synchronous code chunks in separate threads via asyncio threadpool API)

Ulani

It was hard to grasp, but it is the best explanation I had come up with so far. – Switcheroo

Answer 2

I see the main difference downstream.

const promise = new Promise((resolve, reject) => sendRequest(resolve, reject));
await promise;

In JavaScript, the two resolve and reject functions are created by the JS engine and they have to be passed around for you to keep track of them. In the end, you're still using two callback functions most of the time and the Promise won't really do more than setTimeout(() => doMoreStuff()) after doStuff calls resolve. There's no way to retrieve an old result or the status of a Promise once the callbacks were called. The Promise is mostly just the glue code between regular calls and async/await (so you can await the promise somewhere else) and a bit of error callback forwarding for chaining .thens.

future = asyncio.Future()
sendRequest(future)
await future

In Python, the Future itself becomes the interface with which a result is returned and it keeps track of the result.

Since Andril has given the closest Python equivalent to JavaScript's Promise (which is Task; you give it a callback and wait for it to complete), I'd like to go the other way.

class Future {
  constructor() {
    this.result = undefined;
    this.exception = undefined;
    this.done = false;
    this.success = () => {};
    this.fail = () => {};
  }
  result() {
    if (!this.done) {
      throw Error("still pending");
    }
    return this.result();
  }
  exception() {
    if (!this.done) {
      throw Error("still pending");
    }
    return this.exception();
  }
  setResult(result) {
    if (this.done) {
      throw Error("Already done");
    }
    this.result = result;
    this.done = true;
    
    this.success(this.result);
  }
  setException(exception) {
    if (this.done) {
      throw Error("Already done");
    }
    this.exception = exception;
    this.done = true;

    this.fail(this.exception);
  }
  then(success, fail) {
    this.success = success;
    this.fail = fail;
  }
}

The JS await basically generates two callbacks that are passed to .then, where in a JS Promise the actual logic is supposed to happen. In many examples, this is where you'll find a setTimeout(resolve, 10000) to demonstrate the jump out of the event loop, but if you instead keep track of those two callbacks you can do with them whatever you want..

function doStuff(f) {
  // keep for some network traffic or so
  setTimeout(() => f.setResult(true), 3000);
}

const future = new Future();
doStuff(future);
console.log('still here');
console.log(await future);

The above example demonstrates that; three seconds after 'still here' you get 'true'.

As you can see, the difference is Promise receives a work function and deals with resolve and reject internally, while Future doesn't internalize the work and only cares about the callbacks. Personally, I prefer the Future because it's one layer less callback hell - which was one of the reasons for Promises in the first place: callback chaining.

Ostensive