When to use std::async vs std::threads?

Question

Can anybody give a high level intuition about when to use each of them?

References:

• Is it smart to replace boost::thread and boost::mutex with c++11 equivalents?
• When is it a good idea to use std::promise over the other std::thread mechanisms?

Inquisitorial

Use whichever one you know how to apply to your problem. Some problems map better to one or the other, and this will naturally be reflected in your thinking. – Capitally

But are they equivalent? I do not think so. Are std::async made only to make life easier or to solve a previous C++ problem? – Inquisitorial

Interesting reading (regarding std::async) if using g++: async(f) isn't., and also this – Kella

@JaviV: They solve the same problem (concurrency) at very different levels of abstraction. – Capitally

@JaviV They are mostly unrelated concerns (deferred asyncs won't even execute on a different thread than the one waiting for it). They are sometimes mostly equivalent conceptually (e.g. start a few tasks and immediately get the corresponding futures or join the corresponding threads). Sometimes they are complementary (e.g. creating lots of deferred asyncs where the associated futures are executed on a thread pool). Sometimes only one of the two concepts is relevant to a problem (e.g. threads only as monitoring or GUI threads; deferred asyncs only to implement lazy evaluation) – Bringhurst

Answer 1

It's not really an either-or thing - you can use futures (together with promises) with manually created std::threads. Using std::async is a convenient way to fire off a thread for some asynchronous computation and marshal the result back via a future but std::async is rather limited in the current standard. It will become more useful if the suggested extensions to incorporate some of the ideas from Microsoft's PPL are accepted.

Currently, std::async is probably best suited to handling either very long running computations or long running IO for fairly simple programs. It doesn't guarantee low overhead though (and in fact the way it is specified makes it difficult to implement with a thread pool behind the scenes), so it's not well suited for finer grained workloads. For that you either need to roll your own thread pools using std::thread or use something like Microsoft's PPL or Intel's TBB.

You can also use std::thread for 'traditional' POSIX thread style code written in a more modern and portable way.

Bartosz Milewski discusses some of the limitations of the way std::async is currently specified in his article Async Tasks in C++11: Not Quite There Yet

Wrought

So in case I want to write efficient code (or threads being created and joined frequently) it is better to stick to classical threads? – Inquisitorial

Well threads being created and joined frequently is what you want to avoid for efficient code :) That's the issue with std::async - it is quite likely to be implemented by firing off a new thread rather than with a thread pool. If you want efficient fine grained threaded workloads you generally want something like a thread pool. – Wrought

@mattnewport: Actually, std::async most often does use a threadpool. But it isn't required to. – Capitally

Arguably though Microsoft isn't following the standard by using a threadpool. The article I added a link to at the end of my answer discusses some of the reasons why. Microsoft is pushing extensions to std::future to make it more compatible with a threadpool / task based parallelism approach in a future revision of the standard. – Wrought

@Wrought note the "or" in the parenthesis hehe. I am not experienced with thread pools yet but it would be worthy to explore them as well. Thank you guys. – Inquisitorial

Here is what Bjarne Stroustroup says about async: ...don't even think of using async() to launch tasks that do I/O, manipulate mutexes, or in other ways interact with other tasks. The idea behind async() is the same as the idea behind the range-for statement: Provide a simple way to handle the simplest, rather common, case and leave the more complex examples to the fully general mechanism. – Decolorant

I think Apple's GCD should also be listed as an option, as it is freely available for other platforms. – Lashandralashar

I would be worth reading Item#35 from Effective Moden CPP book. – Dinse

Answer 2

One use-case of using std::future over std::thread is you want to call a function which returns a value. When you want return value of the function, you can call get() method of future.

std::thread doesn't provide a direct way to get the return value of the function.

Nance

Answer 3

One simple reason I've found is the case when you want a way to detect (via polling) whether an asynchronous job is done. With std::thread, you have to manage it yourself. With std::async you can query std::future::valid() (or use std::future::wait_for/wait_until(...)) to know when it is done.

Gruff

Polling is possible in case you use std::thread with std::packaged_task. – Gavrielle

Answer 4

I realise it has been 8 years since this question was posed. The C++ concurrency landscape has shifted quite a bit since then. Recently I too had to wander this landscape, wondering which path to take moving forward. I'd like share some of my thoughts and may be get it validated. I would slightly modify the original question to std::async vs thread pool, instead of just std::thread.

Since 2011 I have been heavily using boost::thread_group and boost::asio::io_service for thread pooling and event looping respectively. My every application starts like this:

int noOfCores = boost::thread::hardware_concurrency();
for (int i = 0; i < noOfCores; i++)
{
    _threadPool.create_thread(boost::bind(&pri_queue::run, &_taskQueue));
}

The task queue _taskQueue is of type pri_queue somewhat similar to this boost example, except my run() function waits on io_service.run_one(). Therefore, I also control the priority in which the tasks are executed, by assigning priority while queuing.

After this, I can throw any function (bound with parameters using boost::bind) at this queue using post() for execution, or schedule it with a delay using boost::asio::deadline_timer::async_wait().

Since everything in my framework is event driven, I am comfortable in dividing any functionality into multiple function objects while awaiting the events like in this boost example of async http client. This model is very time tested, has no thread creation cost since every thread is created upfront.

However, C++ standard has been updated 3 times (14, 17, 20) since I adopted this model across all the products in the company. So you could say I am suffering a bit of FOMO, when I look at all the new changes bandied around. Pardon me, after looking at std::async & coroutines, I don't see how they are helping someone already comfortable using io_service + thread pool model like me. It appears more expensive, and I have no control over priority or thread creation, the implementation differs across compilers.

I see that it is making the functions appear synchronous and structured (all pieces in one place), compared to asynchronous functionality spilt into multiple function objects.

For C++ veterans, I would say thread-pooling is better than std::async or even coroutines. Of course, if the application is not event driven, or if you are new to asynchronous programming, std::async would be easier to deal with.

Obvert

I find this approach very interesting and I'd be interested in a complete example of that event driven framework. Would you mind sharing the code? – Sexuality

An example with priority queue would be too big. That is why I described the process, while pointing to various boost examples. – Obvert

In 2022 I say asio::thread_pool and asio::use_awaitable should be the norm. "For C++ veterans, I would say thread-pooling is better than std::async or even coroutines." - there is a false dichotomy. Asynchrony doesn't require threading at all. (boost.org/doc/libs/1_82_0/doc/html/boost_asio/overview/core/…) – Violence

I get your point, but my applications usually have 100s of threads, in multiple thread groups, each group with different io contexts. I have looked asio::thread_pool, but I had similar question as this fellow: mail-archive.com/[email protected]/msg00211.html and therefore haven't made the switch. – Obvert

Answer 5

I think one huge advantage of std::async/std::future over the general std::thread approach is free exception propagation. std::future::get will throw if your thread function throws. This feature is really convenient. This feature is quite similar to handling the return value (stated by rg665n in a different answer).

If you use std::thread, you need to create a class with members for your return values and possibly occurring exceptions. You need to add mutex/locks for all these members. Usually, your code will be like hundred lines of code longer, if you use std::thread over std::async for simple tasks.

In a condensed way, my answer to your question is: Use std::async/std::future if your child-tasks are not interfering among themselves. Use std::thread if you have to sync sates between your child tasks (because they depend on each other). Use std::thread for observer pattern or threads, that will run the full program lifetime.

Telson

Answer 6

Aside from other great answers, It may be worth reading the Item#35 from Effective Modern CPP book from Scot Meyer in favour of std::async over std::thread.

Quoting the following text from that book.

State-of-the-art thread schedulers employ system-wide thread pools to avoid oversubscription, and they improve load balancing across hardware cores through workstealing algorithms. The C++ Standard does not require the use of thread pools or work-stealing, and, to be honest, there are some technical aspects of the C++11 concurrency specification that make it more difficult to employ them than we’d like. Nevertheless, some vendors take advantage of this technology in their Standard Library implementations, and it’s reasonable to expect that progress will continue in this area. If you take a task-based approach to your concurrent programming, you automatically reap the benefits of such technology as it becomes more widespread. If, on the other hand, you program directly with std::threads, you assume the burden of dealing with thread exhaustion, oversubscription, and load balancing yourself, not to mention how your solutions to these problems mesh with the solutions implemented in programs running in other processes on the same machine

Dinse