c++11 threads vs async

Question

Consider the following two snippets of code where I am trying to launch 10000 threads:

Snippet 1

    std::array<std::future<void>, 10000> furArr_;
    try
    {
        size_t index = 0;
        for (auto & fut : furArr_)
        {
            std::cout << "Created thread # " << index++ << std::endl;
            fut = std::async(std::launch::async, fun);
        }
    }
    catch (std::system_error & ex)
    {
        std::string str = ex.what();
        std::cout << "Caught : " << str.c_str() << std::endl;
    }
    // I will call get afterwards, still 10000 threads should be active by now assuming "fun" is time consuming

Snippet 2

    std::array<std::thread, 10000> threadArr;
    try
    {
        size_t index = 0;
        for (auto & thr : threadArr)
        {
            std::cout << "Created thread # " << index++ << std::endl;
            thr = std::thread(fun);
        }
    }
    catch (std::system_error & ex)
    {
        std::string str = ex.what();
        std::cout << "Caught : " << str.c_str() << std::endl;
    }

The first case always succeeds .i.e. I am able to create 10000 threads and then I have to wait for all of them to finish. In the second case, almost always I end up getting an exception("resource unavailable try again") after creating 1600+ threads.

With a launch policy of std::launch::async, I thought that the two snippets should behave the same way. How different std::async with a launch policy of async is from launching a thread explicitly using std::thread?

I am on Windows 10, VS2015, binary is built in x86 release mode.

Sales

Snippet 1 doesn't create 10000 threads. It creates 10000 units of work, which are queued for execution on the system-maintained pool of a few threads. Have fun print std::this_thread::get_id(), see for yourself. – Overpraise

Wow that is neat. I din't know that!. The thread ids indeed seem to repeat. So potentially using async is like having your own thread pool. Is that analogy correct? – Sales

Well, the standard doesn't specify exactly how std::async is to be implemented; but a typical implementation would in fact employ a thread pool. – Overpraise

Answer 1

Firstly, thanks to Igor Tandetnik for giving me the direction for this answer.

When we use std::async (with async launch policy), we are saying:

“I want to get this work done on a separate thread”.

When we use std::thread we are saying:

“I want to get this work done on a new thread”.

The subtle difference means that async (is usually) implemented using thread pools. Which means if we have invoked a method using async multiple times, often the thread id inside that method will repeat i.e. async allocates multiple jobs to the same set of threads from the pool. Whereas with std::thread, it never will.

This difference means that launching threads explicitly will be potentially more resource intensive (and thus the exception) than using async with async launch policy.

Sales

I think there was some discussion that using a thread-pool to implement std::async was not strictly compliant with the standard: eli.thegreenplace.net/2016/…. I also remember that at least gcc 4.8 does not use a thread-pool, so "usually" may be a bit too optimistic. See e.g. #15666943. – Wherewith

Answer 2

When we use std::async (with async launch policy), we are saying:

“I want to get this work done on a separate thread”.

This is not a given for all implementations.

a typical implementation would in fact employ a thread pool

Igor is wrong in making an assumption about what a typical implementation should be doing.

std::launch::async

is specified such that an implementation could in the most straightforward case create new threads. libstdc++ and libc++ do this. MSVC++ uses a thread pool and that shouldn't imply that it is a typical implementation.

Have a look at these discussions Async ends up creating new threads and this one as well though it may not be directly concerned with your query

Navada