Async future with a callback. C++11

Question

I have a list of futures. The problem is that I have a lot of files and I need to make some long operation after every file will be created. That's why I want to make a callback after each "file save".

E.g.,

  (new thread; saveFile 1.txt -> new thread; do a long operation after the file has been created)
  (new thread; saveFile 2.pdf -> new thread; do a long operation after the file has been created).

I need to do everything in a separate thread. The saving of the file is critical, the second task can't be run before the file will have been created. How can I do it? I have the following code:

 void save_file() {
     // preparing data...
     saving a file
   } 

   std::vector<std::future<void>> saveFileTasks;
   for (int n = 0; n < p.size(); ++n)
   {
      saveFileTasks.push_back(std::async(std::bind(&saveFile, filename)));
   }

   for (auto &e : saveFileTasks) {
      e.get();
   }

How can I make a callback in C++11 with future/promise? I am not allowed to use boost in my project.

I'm really confused, there are so much complicated examples for a very simple task. A lot of examples can't be compilable, e.g., promise.set_wait_callback doesn't exist in C++11 but many functions have been migrated to C++11. I can do it really easy if I use Python or Clojure. How can I do it with C++?

Catanddog

Wouldn't std::future::wait do the job? What's the point of running save_file and long_operation in separate threads if you have to wait until file is saved anyways? – Protuberant

I need some kind of thread pool. I can have lots of tasks. E.g., 1 need to save 10 files. When 5 files are saved, I can do long operations with 5 threads and other 5 threads will save other files. – Catanddog

Answer 1

Unfortunately there is no .then continuation in the current version of std::future - it is proposed alongside similar utilities for a future standard of C++.

If you cannot use boost, you can build your own continuation with function composition:

string save_file(string data)      { /* ... */ return filename; } // step 1
void do_operation(string filename) { /* ... */ }                  // step 2

// ... 

std::vector<std::future<void>> fileTasks;
for(const auto& data : /* ... */)
{
    fileTasks.emplace_back(std::async(std::launch::async, 
        [data]{ do_operation(save_file(data)); });
}

Note that std::async([data]{ do_operation(save_file(data)); } will execute both functions in the same thread. If you want each function to be executed in a separate thread you can call async multiple times:

std::async(std::launch::async, [data]
{
    auto result = save_file(data);
    std::async(std::launch::async, [r = std::move(result)]
    {
        do_operation(std::move(r));
    });
});

---

With either boost::future or a future version of the standard, you could simply say:

std::async(std::launch::async, [data]{ save_file(data); })
    .then([](string filename){ do_operation(filename); );

Ringleader

I have the error: "class std::future<void> has not attribute emplace_back". Could you please explain , what is that method? – Catanddog

@Catanddog fileTasks is a std::vector, not a std::future. You must have mis-typed something. – Chord

From what I understand OP wants to do saving and long operation in separate threads as well. With function composition won't save_file and do_operation be performed in the same thread? – Protuberant

I think it won't, will it? I am thinking about it right now. – Catanddog

@doc: yes, they will. I will improve the answer – Ringleader

Tho' I don't see the point of running these two tasks in separate threads if you have to wait for the results of first one anyways. So your answer makes perfect sense. – Protuberant

@doc: I agree, just adding it for completeness and to avoid unexpected surprises :) – Ringleader

@Yakk: changed & to generally-safer alternatives – Ringleader

Answer 2

In the future, future will have a .then operator that lets you chain tasks.

Lacking it we can write it.

// complete named operator library in about a dozen lines of code:
namespace named_operator {
  template<class D>struct make_operator{ constexpr make_operator() {}; };

  template<class T, class O> struct half_apply { T&& lhs; };

  template<class Lhs, class Op>
  half_apply<Lhs, Op> operator*( Lhs&& lhs, make_operator<Op> ) {
    return {std::forward<Lhs>(lhs)};
  }
  template<class Lhs, class Op, class Rhs>
  decltype(auto) operator*( half_apply<Lhs, Op>&& lhs, Rhs&& rhs )
  {
    return named_invoke( std::forward<Lhs>(lhs.lhs), Op{}, std::forward<Rhs>(rhs) );
  }
}

// create a named operator then:
namespace then_ns {
  static const struct then_t:named_operator::make_operator<then_t> {} then{};

  namespace details {
    template<size_t...Is, class Tup, class F>
    auto invoke_helper( std::index_sequence<Is...>, Tup&& tup, F&& f )
    ->decltype(std::forward<F>(f)( std::get<Is>(std::forward<Tup>(tup))... ))
    {
      return std::forward<F>(f)( std::get<Is>(std::forward<Tup>(tup))... );
    }
  }

  // first overload of A *then* B handles tuple and tuple-like return values:
  template<class Tup, class F>
  auto named_invoke( Tup&& tup, then_t, F&& f )
  -> decltype( details::invoke_helper( std::make_index_sequence< std::tuple_size<std::decay_t<Tup>>{} >{}, std::forward<Tup>(tup), std::forward<F>(f) ) )
  {
    return details::invoke_helper( std::make_index_sequence< std::tuple_size<std::decay_t<Tup>>{} >{}, std::forward<Tup>(tup), std::forward<F>(f) );
  }

  // second overload of A *then* B
  // only applies if above does not:
  template<class T, class F>
  auto named_invoke( T&& t, then_t, F&& f, ... )
  -> std::result_of_t< F(T) >
  {
    return std::forward<F>(f)(std::forward<T>(t));
  }
  // *then* with a future; unpack the future
  // into a call to f within an async:
  template<class X, class F>
  auto named_invoke( std::future<X> x, then_t, F&& f )
  -> std::future< std::decay_t<decltype( std::move(x).get() *then* std::declval<F>() )> >
  {
    return std::async( std::launch::async,
      [x = std::move(x), f = std::forward<F>(f)]() mutable {
        return std::move(x).get() *then* std::move(f);
      }
    );
  }
  // void future, don't try to pass void to f:
  template<class F>
  auto named_invoke( std::future<void> x, then_t, F&& f )
  -> std::future< std::decay_t<decltype( std::declval<F>()() )> >
  {
    return std::async( std::launch::async,
      [x = std::move(x), f = std::forward<F>(f)]() mutable {
        std::move(x).get();
        return std::move(f)();
      }
    );
  }
}
using then_ns::then;

see, that wasn't that hard.

a *then* f, if a is a tuple (or pair or array), will invoke f with the contents of a.

If a isn't tuple-like, or f doesn't accept the contents of a that way, it invokes f with a.

If a is a future, it instead creates a new async future that consumes a.get() using *then*.

Live example.

Suppose you want to increase an atomic int when the file is saved:

std::vector<std::future<void>> saveFileTasks;
for (int n = 0; n < p.size(); ++n)
{
  saveFileTasks.push_back(
    std::async(std::launch::async, [filename]{
      saveFile(filename);
    })
  );
}
std::atomic<int> count;
for (auto &e : saveFileTasks) {
  e = std::move(e) *then* [&count]{
    ++count;
  });
}

Naturally this can all be done without the named operator *then* style syntax, but what is the fun of that?

If the first async returns a tuple, the second one can either take it as a tuple or as unpacked "flat" arguments.

Sawbuck

Not much fun for those, who will have to dig into your code. – Protuberant

@VittorioRomeo it woud be much cleaner to promote built-in types to classes and provide class objects (a bit like Ruby does or a bit like Javascript does). This way you could add methods to class object, which would also allow you to access protected members of the class from the method. – Protuberant

@doc Aka UFCS as Vitt mentioned. ;) (ignoring bit about breaking protection). And if you are asking for the ability to modify classes, go talk to the reflection and reification working group. – Sawbuck

@Yakk AFAIK the whole point of object-oriented programming was to provide clean, well-defined interfaces for the objects. If I get it correctly UFCS makes non-member functions indistingushable from member functions. If so, then it seems to me to be a very bad idea for practical reasons. Normally I look at documentation of a class to see what particular method does and I can see what object is capable of. With UFCS I may need to search for a needle in a haystack. – Protuberant

THIS IS AMAZING then wow I barely could understand your template magic but it definitely worth grasping it ) Thanks so much ) Magic!) – Cythiacyto

@doc I have to agree the code is awful and needlessly complex. This stuff wouldn't pass most code reviews, and I would pity the team that saw this leak into their source tree. – Tabulator

@ram Oh god I'd never propose this for a professional project. Named operators in C++ are a cute toy to play with, but without a standard of practice the clear intention at point of use is overwealmed by the baroque implementation requirements. And without a standard of practice, users would have to understand the implementation, which is a non starter for most C++ programmers. OTOH, it is fun code to write, so when someone asks an impossible question on SO, I sometimes say "look it is possible" with it. – Sawbuck

Answer 3

Even though I'm late to the party I'm going to mention that callbacks can easily be implemented with scope guards.

Long story short:

• Add a scope guard that leverages RAII to call the callback at the end of the scope, which happens after the long-running function exits,

• Create a wrapper function that calls the long-running function that also declares the scope guard with a callback, and

• Create a std::future that launches the wrapped long-running function.

Here's a minimal working example:

#include <iostream>
#include <future>
#include <functional>

class ScopedGuard
{
public:
    ScopedGuard(std::function<void()> callback)
        : m_callback(callback) {}

    ~ScopedGuard() {
        m_callback();
    }
private:
    std::function<void()> m_callback;
};

void my_callback() {
    std::cout << "Calling callback." << std::endl;
}

int my_long_running_task() {
    std::cout << "Calling long running task." << std::endl;
    return 8;
}

int main()
{
    std::cout << "Hello World!" << std::endl;

    std::future<int> f2 = std::async(std::launch::async, []{
        ScopedGuard sg(my_callback);
        return my_long_running_task();
    });
    return 0;
}

With this approach you can also define a specalized ScopeGuard class that wraps both the long-running operation and callback, and pass it as a functor to std::async.

Tabulator