Reading JSON from a socket using boost::asio

Question

I am currently trying to transfer some JSON data over the network from a client to a server using the socket API of boost-asio. My client essentially does this:

int from = 1, to = 2;

boost::asio::streambuf buf;
ostream str(&buf);

str << "{"
    << "\"purpose\" : \"request\"" << "," << endl
    << "\"from\" : " << from << "," << endl
    << "\"to\" : " << to << "," << endl
    << "}" << endl;

// Start an asynchronous operation to send the message.
boost::asio::async_write(socket_, buf,
    boost::bind(&client::handle_write, this, _1));

On the server side I have the choice between various boost::asio::async_read* functions. I wanted to use JsonCpp to parse the received data. Studying the JsonCpp API (http://jsoncpp.sourceforge.net/class_json_1_1_reader.html) I found that the Reader operates on top of either a std::string, a char* array or a std::istream which I could operate from the boost::asio::streambuf passed to the functions.

The point is that as far as I know it is not necessarily the case that the entire content is transferred at once, so I would need some kind of confirmation that the buffer contains sufficient data to process the entire document using JsonCpp. How can I assure that the buffer contains enough data?

Slingshot

Answer 1

This is an area for application level protocol

Either

• read until the stream end (the sender disconnects); this doesn't work with connections that are kept alive for more than a single message
• supply a header like Content-Length: 12346\r\n to know in advance how much to read
• supply a delimiter (a bit like mime boundaries, but you could use any sequence that is not allowed/supported as part of the JSON payload) (async_read_until)
• Treat the payload as "binary-style" (BSON e.g.) and supply a (network-order) length field before the text transmission.

The ASIO Http server example contains a pretty nice pattern for parsing HTTP request/headers that you could use. This assumes that your parser can detect completeness and just 'soft-fails' until all information is present.

void connection::handle_read(const boost::system::error_code& e,
    std::size_t bytes_transferred)
{
  if (!e)
  {
    boost::tribool result;
    boost::tie(result, boost::tuples::ignore) = request_parser_.parse(
        request_, buffer_.data(), buffer_.data() + bytes_transferred);

    if (result)
    {
      request_handler_.handle_request(request_, reply_);
      boost::asio::async_write(socket_, reply_.to_buffers(),
          boost::bind(&connection::handle_write, shared_from_this(),
            boost::asio::placeholders::error));
    }
    else if (!result)
    {
      reply_ = reply::stock_reply(reply::bad_request);
      boost::asio::async_write(socket_, reply_.to_buffers(),
          boost::bind(&connection::handle_write, shared_from_this(),
            boost::asio::placeholders::error));
    }
    else
    {
      socket_.async_read_some(boost::asio::buffer(buffer_),
          boost::bind(&connection::handle_read, shared_from_this(),
            boost::asio::placeholders::error,
            boost::asio::placeholders::bytes_transferred));
    }
  }
  else if (e != boost::asio::error::operation_aborted)
  {
    connection_manager_.stop(shared_from_this());
  }
}

I've provided an answer that parses JSON using Boost Spirit earlier Parse a substring as JSON using QJsonDocument; you could use this to detect the end of a proper JSON document (and if it's incomplete, the end will coincide with the start)

Levantine

Well, in any case I would like to write a reply straight away, so closing the connection does not work for me – Slingshot

Is there a generally accepted pattern here? I would provide the server as a service to other developers. I would like to make it as simple for them as I can. – Slingshot

Yes. There are several accepted practices, and I've listed them in the bullets :) Personally, I prefer the content-length header, because it serves as a natural way to detect transport errors/QoI control :) That said, I prefer binary protocols if I have the option – Levantine

@Levantine It seems that the second else would never be fired! – Squinch

@Squinch It is! tribool is not boolean: boost.org/doc/libs/1_76_0/doc/html/tribool/… – Levantine

Answer 2

2 problems here : 1) tell the server how many bytes to read; 2) read the JSON

for 1) you can make your own simple protocol

300#my message here

sends a 300 byte sized message; # is the delimiter between size and message

int write_request(socket_t &socket, const char* buf_json)
{
  std::string buf;
  size_t size_json = strlen(buf_json);
  buf = std::to_string(static_cast<long long unsigned int>(size_json));
  buf += "#";
  buf += std::string(buf_json);
  return (socket.write_all(buf.data(), buf.size()));
}

to read on the server

//parse header, one character at a time and look for for separator #
  //assume size header lenght less than 20 digits
  for (size_t idx = 0; idx < 20; idx++)
  {
    char c;
    if ((recv_size = ::recv(socket.m_sockfd, &c, 1, 0)) == -1)
    {
      std::cout << "recv error: " << strerror(errno) << std::endl;
      return str;
    }
    if (c == '#')
    {
      break;
    }
    else
    {
      str_header += c;
    }
  }

to read JSON, you can use

https://github.com/nlohmann/json

Corunna