Why does C++'s async/await not need an event loop?

Question

For someone who has some async experience from other languages (Python/JavaScript), when talking about async/await there is always an assumption that there is an event loop somewhere. But for C++ I have looked through the documentation and have not found anywhere talking about the event loop. Why is this the case?

For Node, it only has one default event loop. For Python you can create multiple ones if you want. But for C++ is this event loop assumed like Node? Or for some reason, do we not need it at all?

Schauer

The only thing you really need for async/await is a way to schedule work. Whether that's done by posting things to an event loop or simply picking an arbitrary thread pool thread is just an implementation detail. You really only want an event loop based implementation for things that have thread affinity (the common example being GUI programs). For everything else there are better ways to schedule work. – Revet

"there is always an assumption that there is an event loop somewhere" I don't know much about Python, but I was under the impression that it had actual threads and thus you could probably await on actual threads. – Wattenberg

@NicolBolas yeah but still I think most of the time you only run async task in the default event loop. docs.python.org/3/library/asyncio-task.html#asyncio.run – Schauer

Consider waiting for something like an IO completion. The thread which performs the IO will resume and continue the coroutine which is waiting for the IO. Every co_await posts "a continuation" into some completion handler list. – Kovar

@Kovar yeah but if you have more than one await happening you need an event loop to keep track of all your events and see which one comes back first right? Otherwise node.js wont need an event loop. – Schauer

@dorafmon: "you need an event loop to keep track of all your events and see which one comes back first right" Do you know what a "thread" is? – Wattenberg

More than one await happening... where? Every coroutine is either executing or waiting on a single awaitable. – Kovar

@NicolBolas I guess there is no need to be condescending here, I know what a thread is, but the thing is if I have two or more coroutines, there must be some central loop to wait on all of them and check if which of them came back from blocking operations and proceeds with them right? I guess what you and dyp is implying is that it is the OS who does the scheduling here instead of a "green" event loop like node/python for C++, "green" as in "green thread" – Schauer

@Kovar see above comment. – Schauer

@dorafmon: "I know what a thread is" And yet, everything you say after this assumes that everything is running on a single thread, so the different "awaits" have to figure out an order to execute in. They don't, because different awaits are executing potentially in different threads. "the OS who does the scheduling here" That is literally what a thread is. – Wattenberg

@NicolBolas What if you only have a single core and can execute one thread at a time? – Schauer

@NicolBolas I know the OS will emulate threads then it is the OS that functions as the event loop here. That's the point I am trying to get at. – Schauer

Well cppcoro is using dedicated IO threads -> some form of event loop. github.com/lewissbaker/cppcoro#io_service-and-io_work_scope But that doesn't mean that every co_wait pushes into that loop. – Kovar

@NicolBolas but still this confuses me because I believe not all C++ code runs on an environment where there is an OS. – Schauer

@dorafmon: "not all C++ code runs on an environment where there is an OS." ... so what? As I said in my answer, if you want to invent some co-operative multi-processing mechanism, you can totally do that and hook it into co_await with appropriate awaitable types. But this is ultimately a function of the asynchronous process you're waiting on, not of being an asynchronous process. That is, C++ doesn't have a default answer to what happens when you co_await. Threads are the most common answer, but not the default. – Wattenberg

@NicolBolas I think your last comment illustrates the crux very well for me: the event loop in C++ is not a property of coroutines, nor the coroutine who is being suspended, but (if any) of the thing you're waiting for. And since that's specific to the problem, it's not generalized/standardized. – Kovar

Answer 1

Python and JavaScript don't do CPU threads (well, Python can do threads, but that's not relevant here, because Python's thread stuff isn't inherently await-able). So they fake threading with "event loops".

C++ is a low-level language that knows what CPU threads are and expects to make use of them. As such, when you co_await on some expression, you are typically awaiting on a process that is (potentially) happening in another thread.

You certainly can fake threading with an event loop, and create some awaitable type that uses event loop processing. But that's not typically what C++ programmers do with asynchronous processing.

In C++ coroutines, the way the resumption of execution of a coroutine works depends entirely on the awaitable type you're using. It governs the scheduling of the function's resumption. Most awaitables in C++ will use some form of CPU threading (the most common method being to invoke the coroutine on the thread performing the async process it is waiting on). Others may have some kind of event loop or whatever. But the point is that this is a function of the thing which produces the value you're waiting on, not the coroutine itself.

Wattenberg

I would have guessed that C++ coroutines still work with a single thread, provided an async IO mechanism. Maybe the crux is to understand how that async IO mechanism resumes coroutines... – Kovar

@dyp: That can't be answered because you have to provide that mechanism. Or more to the point, the awaitable type provides that mechanism, so it can be whatever it wants. – Wattenberg

@NicolBolas "awaitable type provides that mechanism" can you give a specific example? I think I don't fully understand this statement. – Schauer

@NicolBolas and thanks for spending time on this, I am just totally confused at this moment. – Schauer

ok reading a bit more I think I understand your statements a bit better, you mean we can override what happens when co_await happen. – Schauer

@dorafmon: "override" is incorrect, as it presumes there is some default behavior that you are able to supplant. It would be better to say that an awaitable object defines "what happens when co_await happen". – Wattenberg

I think I would disagree with "Most awaitables in C++ will use some form of CPU threading": Certainly that's one possible use case, but others include Python-style generators where there's little to no asynchronousness at all, only typically short-term suspension. (And also, I thought by tying the return type of the coroutine to what promise object gets produced, it would be pretty much inevitable that the resumption of execution would be fairly tightly coupled with the coroutine implementation and what awaitables make sense within it.) – Herdsman

@DanielSchepler: "others include Python-style generators where there's little to no asynchronousness at all" When I said "awaitable", I meant the things you co_await on. Generators use co_yield, not co_await. And the things generators return are also not something you co_await on. – Wattenberg

co_yield val is equivalent to co_await promise.yield_value(val) so I treat co_yield as being a special case of co_await. – Herdsman

FWIW Node.js uses threads both internally (in its event loop) and explicitly (from 'worker_threads' import ...). In any case for both these languages threading is a part of the platform and not the language. In JavaScript multiple environments (Nashhorn with Java threads, Node.js with worker_threads, browsers with web workers, napajs with "regular" threads etc) do this differently. – Faraday

Answer 2

Coroutines do not need an event loop.

When the computer reads co_await what happens is that it will jump to the function which called the coroutine and save all of its frame (the local variables values and so on).

The magic here is that the next time you call the coroutine you come back to this state. As you can see there is no need for event loop, only a place to store this frame.

Goatsbeard

Actually, depending on what the awaitable object returns from its await_ready or await_suspend the control can remain in the current coroutine, transfer to another coroutine, or as you say it can return control to the caller. – Herdsman

Answer 3

There is no event loop in C++ and threads have very little to do with coroutine either.

When you co_await in C++, the execution of the function is suspended and the code continue to execute the caller, just as if the function had returned. In fact, it is implemented this way. co_await will change the internal state machine of the coroutine and will return.

The execution is resumed when the code explicitly resumes the function.

This is cooperative multitasking. Control of the execution is explicit and predictable.

Now, using most libraries you won't necessarily have to call back the coroutine to be resumed. This is where executors comes in. They are a bit like event loops but inside a library instead of baked in the language. User code can implement them as well, and you can have different one for different use cases. They will usually schedule the execution of coroutines and can also manage multiple threads to execute many of them at once.

For example, you could totally implement an executor on top of a thread pool. Large operation that wait on io won't need to block the thread for themselves, it will start the io operation and give back the thread to other tasks. Internally, the io operation will schedule the coroutine back into the thread pool to be resumed.

Another example would be io_uring on linux, which is the new async io api. One could wrap the facility with an executor and run io operation as coroutines. Technically, you don't need threads to do this one. Calls to co_await will simply schedule the io operation and the coroutine will resume once the kernel has enqueued a result.

Precedency

Answer 4

I'm trying to provide a meta-analysis of the implementation of async/await syntax amongst various of languages.

TL;DR: The behavior of async/await syntax itself does not mandate a eventloop driving the scheduling process at all. It can be also achieved by threads, generators (continuable closures) and state machines.

The misunderstanding of "async/await is backed by eventloop scheduling" starts with GUI applications (e.g. Android, Winform, Dartlang or even JavaScript that runs in browsers, you might even say Unity3D's monobehaviour lifecycle loop is driving the IEnumerator Unity coroutines).

There are tons of counterexamples: Kotlin is just better Java, it runs on JVM after all, and there is no builtin eventloop; Golang uses goroutine and channel which is a "logical blocking" waiting pattern; CSharp's async/await is merely a syntax sugar based on existing conventional thread model (see How Async/Await Really Works in C#), under the hood it is SynchronizationContext and ThreadPool, still no eventloop is found! Let's first talk about builtin eventloops found in languages like JavaScript and Dart. I am not sure if they are in the spec, or just by implementation, but look at their environments: they were built for GUIs which is inherently single threaded, imagine in Android, everytime you want to edit some GUI element you have to Post them back to the main thread to avoid racing condition. One single main thread + builtin default eventloop can simplify the whole architecture design and implementation. The eventloop is like an engine, a driver of everything that's happeing in your program, where threading is basically non-existent and you use workers or "isolates" which is a lesser thread instead.

But back to Kotlin, CSharp and CPP, these languages are (at least designed to be) more general purposed and they cannot have a default eventloop that's hiding before you call Main function, they don't play that way. There is no single eventloop that's driving everything in the language even if in your system there has to be one. Every single thread drive their own thing. And async/await is just flattened, linearized CPS (Continuation Programming Style), implemented with a re-enterable generator state machine. The semantics of await keyword in those languages are different than those who have eventloop builtin:

C++-ish be like: await means this “task” (not thread) logic stops here (in a way like a generator yield return), the whole following logic is stopped until the following Future is fulfilled, because the result here is needed for the logic to continue. The logic here can be resumed with another function call, in other words, the following logic is a continuation. the calculation or completion of this future may happen in a thread in the thread pool, or this current thread depending on your choice.

JavaScript-ish be like: await means our eventloop put away the current Task (a.k.a Promise or future) into the waiting queue, because it needs certain Futures to be fulfilled, and the loop will continue to process the tasks in the working queue. When the Future is fulfilled, this task will be put back into a pending queue that the eventloop will look at before it starts to process working queue tasks. It's more like a "queue play".

In other words, imagine the joint of running threads doing exactly the same thing as one single eventloop. At least I've dug through the source code of Python's asyncio, the whole point of asyncio is to utilize non-blocking IO like epoll, so that the main thread does not need to wait/block for anything, there is a eventloop and it just polls for IO socket ready state, check if anything is ready every "tick", then continue to execute the working queue tasks, like a pipeline. meanwhile if we use multithreading flavoured languages this polling must be done explicitly by a thread, blocking (then the new thread sleeps) or non-blocking (you have to poll), you can make your own eventloop also, which is literally a must for GUI related frameworks, just like what Android and Unreal Engine and Unity did.

Jasisa

Answer 5

Event loops are great for event driven applicaitons where you literally do nothing until some external event occurs. This was common for early X11 based programs that would display something and then waits for the operator to select or manipulate a widget, such as a slider bar or minimize widget, and the event would be passed to the c event handler. Later we setup up callbacks for events to make the event handler more streamlined. This is very common in IoT programming today.

Async processing is different. Async processing starts a thread to do something and all the process cares about is that it was done and possibly a return value. Basically it prevents the process from being blocked while an action is occurring.

Quinonez