boost::asio::spawn yield as callback

Question

I'm trying to rewrite a project using boost::asio::spawn coroutines. Some parts of the project cannot be changed. For example, the storage protocol library is also written with boost::asio, but without coroutines.

The problem is how to convert yield_context into a normal callback (a boost::function object or a classical functor).

This is what we have in the storage library API:

void async_request_data(uint64_t item_id, boost::function< void(Request_result *) > callback);

As we know from examples, the asio yield context can be used like this:

    my_socket.async_read_some(boost::asio::buffer(data), yield);

In this case a boost::asio::yield_context object serves as a callback for async_read_some. I would like to pass a yield object as the second argument to async_request_data, so i can use it in a synchronous manner.

How can this be done? I think it may be possible via some proxy-object, possibly using an approach based on asio_handler_invoke. But I am having trouble seeing how to do this.

Possess

The accepted answer below no longer works with more recent versions of Boost. But there is an answer here that works: https://mcmap.net/q/904833/-how-can-i-replace-deprecated-handler_type_t-or-boost-asio-handler_type-in-this-example-code-snippet – Hafnium

Answer 1

Looks like the best documentation for this feature can be found in a C++ standard proposal written by the boost asio author:

N4045 – Library Foundations for Asynchronous Operations, Revision 2

See section 9.1 which says:

handler_type_t<CompletionToken, void(error_code, size_t)>   #3
  handler(std::forward<CompletionToken>(token));

3: The completion token is converted into a handler, i.e. a function object to be called when the asynchronous operation completes. The signature specifies the arguments that will be passed to the handler.

I guess in your case the CompletionToken template argument will actually be boost::asio::yield_context and handler_type converts it to a callback object.

---

Here is the code from section 9.1 updated to call your async_request_data function:

template <class CompletionToken>
auto async_foo(uint64_t item_id, CompletionToken&& token)
{
  handler_type_t<CompletionToken, void(Request_result *)>
    handler(std::forward<CompletionToken>(token));

  async_result<decltype(handler)> result(handler);  

  async_request_data(item_id, handler);

  return result.get();  
}

Pyromania

Surely, i need some proxy object to pass it as callback, but its not clear how to write this object's guts. yield_context does not have operator() (othewise it would work "as is" without proxy). It has some guts described here boost.org/doc/libs/1_54_0/doc/html/boost_asio/reference/… , but its not clear how to combine them to do a correct coroutine resume. – Possess

I think handler is the proxy object. Pls see edit. – Pyromania

It worked with some little changes! Thanks for directions! (I posted final code in another answer) – Possess

Thank you @Pyromania and @PSIAlt! This trick will allow my upcoming Asio-based library to provide both handler and coroutine-based APIs without having to implement everything twice! – Hafnium

This answer no longer works with more recent versions of Boost. But there is an answer here that works: https://mcmap.net/q/904833/-how-can-i-replace-deprecated-handler_type_t-or-boost-asio-handler_type-in-this-example-code-snippet – Hafnium

Answer 2

Thanks to @PSIAlt and @free_coffee I know how to use callback functions in stackful coroutine.

Here is a simple example just for asio newbies(like me :D)

https://gist.github.com/chenfengyuan/4d764b0bca82a42c05a9

#include <iostream>
#include <boost/asio.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/asio/spawn.hpp>
#include <memory>

void bar(boost::asio::io_service &io, std::function<void()> cb){
    auto ptr = std::make_shared<boost::asio::deadline_timer>(io, boost::posix_time::seconds(1));
    ptr->async_wait([ptr, cb](const boost::system::error_code&){cb();});
}

template<typename Handler>
void foo(boost::asio::io_service &io, Handler && handler){
    typename boost::asio::handler_type<Handler, void()>::type handler_(std::forward<Handler>(handler));
    boost::asio::async_result<decltype(handler_)> result(handler_);
    bar(io, handler_);
    result.get();
    return;
}

int main()
{
  boost::asio::io_service io;
  boost::asio::spawn(io, [&io](boost::asio::yield_context yield){
      foo(io, yield);
      std::cout << "hello, world!\n";
  });

  io.run();

  return 0;
}

Oster

Nice minimal example! There is a subtle problem. handler_ must invoked through the asio_handler_invoke() hook to properly synchronize with the coroutine. Otherwise, in a multi-threaded environment, a race condition may occur where the coroutine is attempted to be resumed before it has yielded. The asio_handler_invoke hook is overloaded for specific types, so type erasure to std::function<> cannot occur. Here is an updated solution. – Wooster

Answer 3

---

Great thanks to free_coffe i managed this to work. Posting solution for my case, possibly someone need it.

template <class CompletionToken>
RequestResult async_foo(Packet &pkt, CompletionToken&& token) {
   typename boost::asio::handler_type< CompletionToken, void(RequestResult) >::type handler( std::forward<CompletionToken>(token) );
  boost::asio::async_result<decltype(handler)> result(handler);
  storage_api->writePacket(pkt, handler);
  return result.get();
}

Later we can use this proxy:

RequestResult res = async_foo(pkt, std::forward<boost::asio::yield_context>(yield) );

Possess

Great! Doesn't smell quite right using a detail class though. Maybe you can post the errors from when you tried to use handler_type? – Pyromania

@Pyromania error: could not convert ‘result.boost::asio::async_result<Handler>::get<boost::asio::basic_yield_context<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, void (*)(), boost::asio::detail ::is_continuation_if_running> > >()’ from ‘boost::asio::async_result<boost::asio::basic_yield_context<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, void (*)(), boost::asio::detail::is_continuation_if_running> > >: :type {aka void}’ to ‘RequestResult’ – Possess

It looks like the yield_context is somehow being passed as the async_result template parameter, meaning decltype(handler) is not what it should be. Could you post the code? – Pyromania

@Pyromania sure, here is it gist.github.com/PSIAlt/d4c9ccf48b962f797efd – Possess

I think the second template argument to handler_type is supposed to be a function signature, so handler_type<CompletionToken, void(RequestResult)> rather than handler_type<CompletionToken, RequestResult>. – Pyromania

@Pyromania omg it worked! Also works safely with thread pool. Great, thanks! Updated to "correct" version. – Possess

Answer 4

Nowadays, one can treat callback handlers and yield_context (as well as use_awaitable and use_future) uniformly as completions tokens that are accepted via a templatized argument. The token is then transformed into a handler function that is passed to an initiating function via boost::asio::async_initiate:

// This is the API function
template <typename CompletionToken>
auto async_request_db_data(uint64_t item_id, CompletionToken&& token)
{
    // If you can't use C++14 generic lambdas, then use a function
    // object instead with an operator() template member function.
    auto init = [](auto completion_handler, uint64_t item_id)
    {
        init_request_db_data(item_id, std::move(completion_handler));
    }

    // void(database_result) is the signature of the intermediary
    // handler function that is passed to the initiating function.
    return boost::asio::async_initiate<Token, void(database_result)>(
        init, std::forward<CompletionToken>(token));

}

// This is a private function that does the actual work.
template <typename CompletionHandler>
void init_request_db_data(uint64_t item_id, CompletionHandler handler)
{
    // Perform actual work and then post handler when complete,
    // passing the database_result.
    // Note the handler below is moved because some completion handlers
    // are move-only (particularly those originating from use_awaitable).

    auto exec = boost::asio::get_associated_executor(handler);
    boost::asio::post(exec, std::bind(std::move(handler), db_result);
}

int main()
{
    boost::asio::io_context ioctx;

    // API used with stackful coroutine yield_context
    boost::asio::spawn(
        ioctx,
        [](boost::asio_yield_context yield)
        {
            auto result = async_request_db_data(42, yield);
        });

    // API used with traditional callback handler
    async_request_db_data(
        42,
        [](database_result result)
        {
            // Do something with result
        }

    ioctx.run();
}

If you need to type-erase the handler passed to the initiating function (because you want the implementation to be compiled from a cpp file), then check out the experimental any_completion_handler that the author of Boost.Asio has recently written (see related feature request)

Hafnium