Why does Boost property tree write_json save everything as string? Is it possible to change that?

Question

I'm trying to serialize using boost property tree write_json, it saves everything as strings, it's not that data are wrong, but I need to cast them explicitly every time and I want to use them somewhere else. (like in python or other C++ json (non boost) library)

here is some sample code and what I get depending on locale:

boost::property_tree::ptree root, arr, elem1, elem2;
elem1.put<int>("key0", 0);
elem1.put<bool>("key1", true);
elem2.put<float>("key2", 2.2f);
elem2.put<double>("key3", 3.3);
arr.push_back( std::make_pair("", elem1) );
arr.push_back( std::make_pair("", elem2) );
root.put_child("path1.path2", arr);

std::stringstream ss;
write_json(ss, root);
std::string my_string_to_send_somewhare_else = ss.str();

and my_string_to_send_somewhere_else is sth. like this:

{
    "path1" :
    {
       "path2" :
       [
            {
                 "key0" : "0",
                 "key1" : "true"
            },
            {
                 "key2" : "2.2",
                 "key3" : "3.3"
            }
       ]
    }
}

Is there anyway to save them as the values, like: "key1" : true or "key2" : 2.2?

Ferrosilicon

Answer 1

I ended up adding another function to my utils to solve this:

#include <string>
#include <regex>
#include <boost/property_tree/json_parser.hpp>

namespace bpt = boost::property_tree;
typedef bpt::ptree JSON;
namespace boost { namespace property_tree {
    inline void write_jsonEx(const std::string & path, const JSON & ptree)
    {
        std::ostringstream oss;
        bpt::write_json(oss, ptree);
        std::regex reg("\\\"([0-9]+\\.{0,1}[0-9]*)\\\"");
        std::string result = std::regex_replace(oss.str(), reg, "$1");

        std::ofstream file;
        file.open(path);
        file << result;
        file.close();
    }
} }

Hope that helps.

Impertinent

This helped me out with a project of mine. Thank you :) – Patently

for binary value of true and false this patch not working – Arabelle

Answer 2

Ok, I've solved it like this, (of course it won't suite for everybody, as it is a bit of a hack, that need further work).

---

I've wrote my own write_json function (simply copied the files, json_parser.hpp and json_parser_write.hpp to my project) and modified the following lines in json_parser_write.hpp:

1. commented line 37 - escaping the quote '"'
2. changed line 76 - so that it doesn't add quotes anymore: stream << Ch('"') << data << Ch('"'); ==> stream << data;

---

Then values will be saved properly except for strings, so I wrote custom translator for it:

template <typename T>
struct my_id_translator
{
    typedef T internal_type;
    typedef T external_type;

    boost::optional<T> get_value(const T &v) { return  v.substr(1, v.size() - 2) ; }
    boost::optional<T> put_value(const T &v) { return '"' + v +'"'; }
};

and simply saved string using:

elem2.put<std::string>("key2", "asdf", my_id_translator<std::string>());

complete program:

#include <iostream>
#include <string>
#include <sstream>

#include <boost/property_tree/ptree.hpp>

#include "property_tree/json_parser.hpp" // copied the headers

template <typename T>

struct my_id_translator
{
    typedef T internal_type;
    typedef T external_type;

    boost::optional<T> get_value(const T &v) { return  v.substr(1, v.size() - 2) ; }
    boost::optional<T> put_value(const T &v) { return '"' + v +'"'; }
};

int main(int, char *[])
{
    using namespace std;
    using boost::property_tree::ptree;
    using boost::property_tree::basic_ptree;
    try
    {
        ptree root, arr,elem2;
        basic_ptree<std::string, std::string> elem1;
        elem1.put<int>("int", 10 );
        elem1.put<bool>("bool", true);
        elem2.put<double>("double", 2.2);
        elem2.put<std::string>("string", "some string", my_id_translator<std::string>());

        arr.push_back( std::make_pair("", elem1) );
        arr.push_back( std::make_pair("", elem2) );
        root.put_child("path1.path2", arr);

        std::stringstream ss;
        write_json(ss, root);
        std::string my_string_to_send_somewhere_else = ss.str();

        cout << my_string_to_send_somewhere_else << endl;

    }
    catch (std::exception & e)
    {
        cout << e.what();
    }
    return 0;
}

result :)

{
    "path1":
    {
        "path2":
        [
            {
                "int": 10,
                "bool": true
            },
            {
                "double": 2.2,
                "string": "some string"
            }
        ]
    }
}

Ferrosilicon

+1 useful to know there is no clean solution right now, and that a kludge is necessary – Ventriloquist

If the double quote is not escaped then you might run with a problem if this quote happens to be in your string. – Anesthetic

at the end does it work for write and read also. Or is this write-only? – Brutality

@Brutality this is really old. I'm not sure is it still works at all. It was long time ago, but I think it was read-write. – Ferrosilicon

Actually, the code above (with the untouched headers) still works. Maybe it is just a matter of printing the string without the quotes if the results is convertible (lexical_casted) to a numeric type in both direction. This can be done by overloading write/read_json_helper for property_tree<std::string, std::string> instead of modifying the core headers. – Brutality

Answer 3

Boost confirms its implementation has no 100% conformance to JSON standard. Check the following link to see their explanation: Making a ptree variant that preserves JSON types is a future plan, but far off.!

Engross

I think that is a huge understatement. It would be more correct to say that property_tree does not actually support JSON. I unfortunately found this out the hard way. There are more restrictions than the "stringly typing". – Paramnesia

Answer 4

The simplest and cleanest solution that i could come up with was generating the JSON with placeholders and in the end string replacing with the actual value ditching the extra quotes.

static string buildGetOrdersCommand() {
    ptree root;
    ptree element;
    element.put<string>("pendingOnly", ":pendingOnly");
    element.put<string>("someIntValue", ":someIntValue");

    root.put("command", "getOrders");
    root.put_child("arguments", element);

    std::ostringstream buf;
    write_json(buf, root, false);
    buf << std::endl;

    string json = buf.str();
    replace(json, ":pendingOnly", "true");
    replace(json, ":someIntValue", std::to_string(15));

    return json;
}

static void replace(string& json, const string& placeholder, const string& value) {
    boost::replace_all<string>(json, "\"" + placeholder + "\"", value);
}

And the result is

{"command":"getOrders","arguments":{"pendingOnly":true,"someIntValue":15}}

Agreed

Answer 5

As we have typedef basic_ptree<std::string, std::string> ptree; in the boost libraries, boost will always serialize each value as string and parse all values to a string equivalent.

Ballista

Answer 6

From the outputted JSON it is clear that the serializer serializes everything to strings using some sort of .toString() method - that is, its unaware of the type of each member and so encloses everything in " ".

See Creating JSON arrays in Boost using Property Trees for more about this problem .

Purl

Answer 7

All solutions, that require custom translators for strings explicitly, seem to be quite error prone for me since it's likely to forget it sometimes. It would be nice to have some kind of overload way via inheritance for the property tree's put method to handle this implicitly but that's not possible in a robust way since it's a template and you would have to ensure full covariance for all methods of the tree. Also changing boost library stuff as a workaround should be avoided in general if possible.

The most robust way without hacks I found so far is (since C++11):

• use a boost-property tree with <KeyType, std::variant<yourTypes>>
• Provide a translator for your variant (details see link below!) but do not "hackify" the content in terms of JSON-specifics! These are almost totally orthogonal aspects!
• Write an own reader and writer for JSON, should be easily adapted from the Boost ones

Pros:

• no hacks required to affect the property tree in terms of JSON-specific details
• no pollution of the Boost libraries and their namespaces except the specializations of your new own types (the variant translator)
• type safer than the custom string based property tree approach
• should be faster for many runtime scenarios with non-frequent serialization of the tree

Cons:

• requires some efforts for a quite small detail of the behavior
• might be a bit slower in terms of compilation
• might be a bit slower for runtime scenarios with frequent serialization and minor changes of the tree (can be optimized for sure)
• reading the json back into the tree is some kind of philosophic work in doubt to ensure as much symmetry as possible between the used types (for many purposes rather an academic issue)

For more details, for instance see

http://marko-editor.com/articles/property_tree_store_anything/

You can easily adapt this for variant usage.

Spancel