What is the executor pattern in a C++ context?

Question

The author of asio, Christopher Kohlhoff, is working on a library and proposal for executors in C++. His work so far includes this repo and docs. Unfortunately, the rationale portion has yet to be written. So far, the docs give a few examples of what the library does but I don't feel like I'm missing something. Somehow this is more than a family of fancy invoker functions.

Everything I can find on Google is very Java specific and a lot of it is particular to specific frameworks so I'm having trouble figuring out what this "executor pattern" is all about.

What are executors in this context? What do they do? What are the canonical examples of when they would be helpful? What variations exist among executors? What are the alternatives to executors and how do they compare? In particular, there seems to be a lot of overlap with an event loop where the events are initial input events, execution events, and a shutdown event.

When trying to figure out new abstractions I usually find understanding the motivation key. So for executors, what are we trying to abstract and why? What are we trying to make generic? Without executors, what extra work would we have to do?

Etna

Use an overloaded call operator()(... args) operator? – Blinni

@πάνταῥεῖ I don't understand your comment. – Etna

Well, that's the idiomatic way to represent an executor/functor pattern. – Blinni

@πάνταῥεῖ It sounds like you're thinking of the syntax for the function call operator. 'Executor' is an abstraction. I want to understand the general rules and characteristics of objects that can be thought of as executors. – Etna

I think it is something like what this paper says: Parallel Algorithms Need Executors | N4406 – Altorilievo

I don't know about executor pattern but the command pattern is where e.g menu choices in a user interface causes action objects to be sent around, and crucially these support undo functionality. So one can have an undo/redo queue. – Glennaglennie

@Etna Something like a Template Function pattern? Related is Strategy. – Blinni

@Cheersandhth.-Alf I noticed googling "executor pattern" brings up the wiki for command pattern but arg, that wiki doesn't use the word "executor". I think the executor pattern is fed command pattern objects though. So somehow "executor" abstracts away how to execute a command. – Etna

@NickyC Nice link. Once I get a chance to get back to this and read a bit more I'll post an answer. Seems like the rough gist is an abstraction of threadpools. – Etna

Also see the Kohlhoff's n4242 paper – Askwith

Answer 1

The most basic benefit of executors is separating the definition of a program's parallelism from how it's used. Java's executor model exists because, by and large, you don't actually know, when you're first writing code, what parallelism model is best for your scenario. You might have little to gain from parallelism and shouldn't use threads at all, you might do best with a long running dedicated worker thread for each core, or a dynamically scaling pool of threads based on current load that cleans up threads after they've been idle a while to reduce memory usage, context switches, etc., or maybe just launching a thread for every task on demand, exiting when the task is done.

The key here is it's nigh impossible to know which approach is best when you're first writing code. You may know where parallelism might help you, but in traditional threading, you end up intermingling the parallelism "configuration" (when and whether to create threads) with the use of parallelism (determining which functions to call with what arguments). When you do mix the code like this, it's a royal pain to do performance testing of different options, because each and every thread launch is independent, and must be updated separately.

The main benefit of the executor model is that the parallelism configuration is done in one place (where the executor is created), and the users of that executor don't have to know anything about it. They just submit work to the executor, receive a future, and at some later point, retrieve the result (blocking if necessary) from the future. If you want to experiment with other configurations, you change the one line defining the executor and run your code again. Even if you decide you need to use different parallelism models for different sections of your code, refactoring to add a second executor and change some of the users of the first executor to use the second is easy compared to manually rewriting the threading details of every site; as long as the executor's name is (relatively) unique, finding users and changing them to use a different one is pretty easy. Executors both simplify your code (by avoiding intermingling thread creation/management with the tasks the threads do) and simplify performance testing.

As a side-benefit, you also abstract away the complexities of transferring data into and out of a worker thread (the submit method encapsulates the former, the future's result method encapsulates the latter). std::async gets you some of this benefit, but with no real control over the parallelism involved (just a yes/no/maybe choice of whether to force a thread, force deferred execution in the current thread, or let the compiler/library decide, with no fine grained control over whether a thread pool is used, and if so, how it behaves). A true executor framework gives you the control std::async fails to provide, with similar ease of use.

Gorki

This would map to .net's TaskScheduler if I am not mistaken? Or perhaps how Task and TaskScheduler represent the separation of responsibility that Task is a unit of work while scheduler decides the when/how/if of scheduling it to something which can be either a dedicated thread, some thread from thread pool or other context which might be tied to UI frameworks. – Myotonia

@TanveerBadar: Yes, that looks to fill a similar niche. .NET's approach seems to try to encourage use of a TaskFactory to create the tasks piecemeal, vs. Java's approach of accepting a specific interface (Callable<T>) that binds arguments via the constructor and returns results via the interface method, but the idea is roughly the same. C++ might actually be a bit easier to use, thanks to lambdas and variadic templates (no need for TaskFactory or implementing Callable when you can just pass the function reference followed by the arguments directly), but the idea is the same. – Gorki

Thank you for your answer and further explanation. I can safely say I learned something new today. – Myotonia