boost serialization 1.5.5 crash when meets Nan and Inf

Asked 24/2, 2015 at 13:8 Answered 28/6, 2017 at 16:52

It seems that boost serialization can't restore the value Nan and inf from text-based archives.

The the program will terminate unless you handle the archive_exception in this case, any solutions to that ?

Emplace answered 24/2, 2015 at 13:8 Comment(1)

Needs a small reproducer. Also, you fail to mention versions, archive types etc. In the valid case, report the bug at the boost trackers – Hoehne 24/2, 2015 at 13:31

The author of the library has this to say:

The simple truth is I never consider this.

When it came up the last time I didn't really think about it very much as I was involved in other things and I hoped intereste[d] parties might come to a consensus without my having to bend my over-stretched brain.

(goes on to discuss workarounds)

This appears to be correct, in my test only binary archives support inf/nan.

Xml and text archives do support the full range of precision, except nan/inf:

Live On Coliru

using BIA = boost::archive::binary_iarchive;
using BOA = boost::archive::binary_oarchive;
using TIA = boost::archive::text_iarchive;
using TOA = boost::archive::text_oarchive;
using XIA = boost::archive::xml_iarchive;
using XOA = boost::archive::xml_oarchive;

int main() {

    // supported:
    assert((perform_test<BIA,  BOA, use_nan, use_inf, use_range>()));
    assert((perform_test<XIA,  XOA, no_nan,  no_inf,  use_range>()));
    assert((perform_test<TIA,  TOA, no_nan,  no_inf,  use_range>()));

    // not supported:
    assert(!(perform_test<XIA, XOA, no_nan,  use_inf>()));
    assert(!(perform_test<TIA, TOA, no_nan,  use_inf>()));

    assert(!(perform_test<XIA, XOA, use_nan, no_inf>()));
    assert(!(perform_test<TIA, TOA, use_nan, no_inf>()));

}

Full Listing

For posterity:

#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/serialization/vector.hpp>
#include <sstream>

using namespace boost::archive;

static bool equal_or_nan(double a, double b) {
    return (std::isnan(a) && std::isnan(b)) || a==b;
}

template <typename IA, typename OA, 
         bool withNan   = true,
         bool withInf   = true,
         bool withRange = true>
bool perform_test() 
{
    std::vector<double> const v {
        withRange? std::numeric_limits<double>::min()       : 0,
        withRange? std::numeric_limits<double>::max()       : 0,
        withRange? std::numeric_limits<double>::epsilon()   : 0,
        withNan?   std::numeric_limits<double>::quiet_NaN() : 0,
        withInf?   std::numeric_limits<double>::infinity()  : 0,
        withInf? - std::numeric_limits<double>::infinity()  : 0,
    };

    std::stringstream ss;
    {
        OA oa(ss);
        oa << boost::serialization::make_nvp("element", v);
    }

    try
    {
        IA ia(ss);
        std::vector<double> w;
        ia >> boost::serialization::make_nvp("element", w);

        return std::equal(v.begin(), v.end(), w.begin(), equal_or_nan);
    } catch(...) {
        return false;
    }
}

static constexpr bool use_inf = true, use_nan = true, use_range = true;
static constexpr bool no_inf  = false, no_nan = false, no_range = false;

using BIA = boost::archive::binary_iarchive;
using BOA = boost::archive::binary_oarchive;
using TIA = boost::archive::text_iarchive;
using TOA = boost::archive::text_oarchive;
using XIA = boost::archive::xml_iarchive;
using XOA = boost::archive::xml_oarchive;

int main() {

    // supported:
    assert((perform_test<BIA,  BOA, use_nan, use_inf, use_range>()));
    assert((perform_test<XIA,  XOA, no_nan,  no_inf,  use_range>()));
    assert((perform_test<TIA,  TOA, no_nan,  no_inf,  use_range>()));

    // not supported:
    assert(!(perform_test<XIA, XOA, no_nan,  use_inf>()));
    assert(!(perform_test<TIA, TOA, no_nan,  use_inf>()));

    assert(!(perform_test<XIA, XOA, use_nan, no_inf>()));
    assert(!(perform_test<TIA, TOA, use_nan, no_inf>()));

}

Hoehne answered 24/2, 2015 at 17:2 Comment(4)

I had a look at these. Would a simple boost::optional< double > suffice? Assuming you don't care to differentiate between NaNs and just want to store all NaNs as null in your archive. – Alleyn 28/6, 2017 at 11:28

@Alleyn Heh. If you don't care for them, don't store them. Optional works, but it doesn't "suffice" for the topic that this Q&A is about. (To avoid head-aches, it "suffices" to stop eating for a week, e.g.) – Hoehne 28/6, 2017 at 11:52

If you happen to have these NaNs in a boost property_tree, you don't happen to control how that writes to XML. Does it work? – Alleyn 28/6, 2017 at 13:46

@Alleyn What? No it doesn't. That's the whole point. But your "solution" appears to simply "don't do that then". Not having NaN does not solve the problem of having NaN. It simply denies the problem. This question is here to ASK whether it can be done, and the answer is no. Removing the problem is a workaround. Many workarounds exist. – Hoehne 29/6, 2017 at 11:18

Firstly, this is not an issue specific to boost archives, it is a feature of iostream i.e. text streaming. You can stream out a NaN value but you cannot read it back. I am not even sure it is "defined" how a NaN will print.

If you are going to "patch" your boost library then the place to do this is in

boost/archive/basic_text_iprimitive.hpp

template< class IStream >
class basic_text_iprimitive

method

 template<class T>
 void load(T & t)

There is an overload for char, signed char and unsigned char but not double.

If you don't like modify the boost header (and I don't like touching those) then take advantage of the fact that they are templated and can be specialized.

Put this patch somewhere in your code in a header and include it before you read the archive.

namespace boost { namespace archive {
template<> template<>
void basic_text_iprimitive< std::istream >::load<double>( double& t )
{
    // fix to handle NaNs
}
} } // close namespaces

From my own test run the templated type is std::istream (or to be more precise std::basic_istream< char, std::char_traits<char> > )

If you find that doesn't work then write similar overloads for other input streams (and of course make them all forward to the one implementation).

What to put in the "fix" section? Well boost does actually create a facet for reading the NaNs, you just have to ensure that you have created the locale for it and imbue it into the archive.

Put the implementation in a C++ file and ensure this locale is created just once:

std::locale infLocale( std::locale(), new boost::math::nonfinite_num_get<char>));

Then you can imbue that into your stream before you read:

is.imbue( infLocale )

(You can also do this imbue at the point you first load and read the XML and if you have only one library for reading XML then do it that way, but if you do this in various places in your code, doing such is not ideal).

Of course, to ensure consistency you will want to use a similar locale for writing NaNs. boost has a nonfinite_num_put facet for that. You can put that in the same locale as nonfinite_num_get and then imbue that into the stream, either when you create the file handle for XML or specialize a template method.

Alleyn answered 28/6, 2017 at 16:52 Comment(2)

I wrote this comment without actually fully testing it and there is a flaw in it. Whilst it hits my overload on some occasions, on others it runs into the boost_serialization library which of course was not built with the "patch". This is when it runs into load_pointer or load_object, essentially the basic_iarchive object (and presumably basic_oarchive too) . Looks like you might have to imbue the locale a bit earlier on. – Alleyn 29/6, 2017 at 14:1

Actually the "flaw" might not exist, it just needs to ensure that all code that streams is built with the patch, so won't work if there is code you cannot rebuild. imbuing the stream at the outset fails. Possibly the library creates temporary streams. – Alleyn 30/6, 2017 at 10:22

Have a look at this topic. There are facets in boost/math/special_functions (see question) and a way to apply them (see answer).

Insurrectionary answered 19/6, 2015 at 13:25 Comment(0)

Helper template functions or overloaded operators (e.g. <<, &) can be used to call stream selector - specialized static template class. Stream selector specializations would then choose correct streaming functions based on the type being serialized:

template <typename T>
boost::archive::xml_iarchive&
stream(boost::archive::xml_iarchive& ia,
       const boost::serialization::nvp<T>& nvp)
{
  return StreamSelector<T>::stream(ia, nvp);
}

template <typename T>
boost::archive::xml_oarchive&
stream(boost::archive::xml_oarchive& oa,
       const boost::serialization::nvp<T>& nvp)
{
  return StreamSelector<T>::stream(oa, nvp);
}

General stream selector static template class can be defined for any type supported by boost::archive::xml_iarchive as the following:

template <typename T>
class StreamSelector {
 public:
  static boost::archive::xml_iarchive&
  stream(boost::archive::xml_iarchive& ia,
         const boost::serialization::nvp<T>& nvp)
  {
    ia.operator >>(nvp);
    return ia;
  }

  static boost::archive::xml_oarchive&
  stream(boost::archive::xml_oarchive& oa,
         const boost::serialization::nvp<T>& nvp)
  {
    oa.operator <<(nvp);
    return oa;
  }
};

Then stream selector can be specialized for double to handle NaN, or to output floating point values in a more human readable format to an archive:

template <>
class StreamSelector<double> {
 public:
  constexpr static double nan = std::numeric_limits<double>::quiet_NaN();
  constexpr static const char* nanCStr = "nan";

  static boost::archive::xml_iarchive&
  stream(boost::archive::xml_iarchive& ia,
         const boost::serialization::nvp<double>& nvp)
  {
    std::string iStr;
    ia >>  boost::serialization::make_nvp(nvp.name(), iStr);
    if(iStr == nanCStr) nvp.value() = nan;
    else nvp.value() = std::stod(iStr);

    return ia;
  }

  static boost::archive::xml_oarchive&
  stream(boost::archive::xml_oarchive& oa,
         const boost::serialization::nvp<double>& nvp)
  {
    if(std::isnan(nvp.value())) {
      std::string nanStr = nanCStr;
      oa << boost::serialization::make_nvp(nvp.name(), nanStr);
    }
    else {
      std::stringstream oStrm;
      oStrm << std::setprecision(std::numeric_limits<double>::digits10 + 1)
            << nvp.value();
      std::string oStr = oStrm.str();
      oa << boost::serialization::make_nvp(nvp.name(), oStr);
    }
    return oa;
  }
};

Other similar stream selector specializations can be added to handle more types (e.g float, long double) and more special cases such as infinities.

Live demo: Open In Coliru

Kanishakanji answered 5/4, 2017 at 16:18 Comment(4)

Does this fix it? This is a patch of course, and really boost should fix it themselves, but it may well work. – Alleyn 28/6, 2017 at 15:45

@Alleyn we use it as a generic solution to specify how specific types should be serialized. – Kanishakanji 29/6, 2017 at 15:41

@Alleyn In double example above, call stream function like stream(ar, boost::serialization::make_nvp("double_var_name", double_var_value)). Also notice, we are using nanCStr = "NaN" string to represent nan numbers. It could be changed to "nan" to make it more C++ stream compliant. – Kanishakanji 29/6, 2017 at 15:53

I actually solved it with my solution, which did work once I had rebuilt ALL the source code that did streaming. The archive library did call back our own objects to find out how to stream them, so as long as those had picked up my patch it worked. – Alleyn 30/6, 2017 at 10:20

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Full Listing

Recommended topics

Hot tags