how to do performance test using the boost library for a custom library

Question

I need to do performance testing of a library written in c++. The library consist of few sets of structures. I have already done the serialization test for these class but not sure how to do perfomance test for these . Below is sample of a struct in library

struct X
{
   public:
      int p;
      double q;

      X();
     ~X();
}

struct Y
{
    float m;
    double n;

     Y();
    ~Y();
}

struct Z
{
public:
   std::map<std::string,boost::shared_ptr<X>> Xtype;
   std::map<std::string,boost::shared_ptr<Y>> Ytype; 

   int i;
   string name; 

   Z();
  ~Z();

}

If any example is provided then it will be really good.

Mortise

Use a profiler. – Uund

@CaptainObvlious - need to use boost only for unit test – Mortise

that's not even a sentence, @user3295725. And why the sudden mention of unit tests? – Sweep

@Sweep -- sorry i mentioned it bymistake.. it should be test only.. – Mortise

Answer 1

Okay, so I added serialization to the types (why did you leave it out?)

struct X
{
    int p;
    double q;
  private: 
    friend boost::serialization::access;
    template <typename Ar>
        void serialize(Ar& ar, unsigned) {
            ar & BOOST_SERIALIZATION_NVP(p);
            ar & BOOST_SERIALIZATION_NVP(q);
        }
};

struct Y
{
    float m;
    double n;
  private: 
    friend boost::serialization::access;
    template <typename Ar>
        void serialize(Ar& ar, unsigned) {
            ar & BOOST_SERIALIZATION_NVP(m);
            ar & BOOST_SERIALIZATION_NVP(n);
        }
};

struct Z
{
   std::map<std::string, boost::shared_ptr<X>> Xtype;
   std::map<std::string, boost::shared_ptr<Y>> Ytype; 

   int i;
   std::string name; 
  private: 
    friend boost::serialization::access;
    template <typename Ar>
        void serialize(Ar& ar, unsigned) {
            ar & BOOST_SERIALIZATION_NVP(i);
            ar & BOOST_SERIALIZATION_NVP(name);
            ar & BOOST_SERIALIZATION_NVP(Xtype);
            ar & BOOST_SERIALIZATION_NVP(Ytype);
        }
};

And now, using the Nonius benchmarking mini-framework, write the following benchmarks:

Z const& fixture(); // forward

#include <nonius/main.h++>
#include <sstream>

NONIUS_BENCHMARK("text archive", [](nonius::chronometer meter) {
    auto const& z = fixture();
    meter.measure([&](int /*i*/) { 
        std::stringstream ss;
        boost::archive::text_oarchive oa(ss);
        oa << z;

        Z clone;
        boost::archive::text_iarchive ia(ss);
        ia >> clone;

        return ss.str().size(); // something observable to thwart the overly smart optimizer
    });
})

NONIUS_BENCHMARK("binary archive", [](nonius::chronometer meter) {
    auto const& z = fixture();
    meter.measure([&](int /*i*/) { 
        std::stringstream ss;
        boost::archive::binary_oarchive oa(ss);
        oa << z;

        Z clone;
        boost::archive::binary_iarchive ia(ss);
        ia >> clone;

        return ss.str().size(); // something observable to thwart the overly smart optimizer
    });
})

NONIUS_BENCHMARK("xml archive", [](nonius::chronometer meter) {
    auto const& z = fixture();
    meter.measure([&](int /*i*/) { 
        std::stringstream ss;
        boost::archive::xml_oarchive oa(ss);
        oa << boost::serialization::make_nvp("root", z);

        Z clone;
        boost::archive::xml_iarchive ia(ss);
        ia >> boost::serialization::make_nvp("root", clone);

        return ss.str().size(); // something observable to thwart the overly smart optimizer
    });
})

The raw output is (for a fixture of 1000 random X and 3000 random Y values):

text archive
mean: 236.069 μs
std dev: 2.54923 μs
variance is unaffected by outliers

binary archive
mean: 92.9736 μs
std dev: 3.35504 μs
variance is moderately inflated by outliers

xml archive
mean: 786.746 μs
std dev: 4.676 μs
variance is unaffected by outliers

Interactive plot: click here

---

The test fixture is actually a lot more work, and is defined as follows:

#include <boost/random.hpp> // for test data
#include <boost/bind.hpp>
#include <boost/make_shared.hpp>
#include <algorithm>

Z const& fixture()
{
    static Z const z = [] {
        Z z;

        boost::random::mt19937 engine;
        auto fgen = boost::bind(boost::random::uniform_real_distribution<float>(), engine);
        auto dgen = boost::bind(boost::random::uniform_real_distribution<double>(), engine);
        auto cgen = boost::bind(boost::random::uniform_int_distribution<char>('a', 'z'), engine);
        auto igen = boost::bind(boost::random::uniform_int_distribution<int>(), engine);

        auto sgen = [&] (int maxlen) { std::string s; std::generate_n(back_inserter(s), igen() % maxlen, cgen); return s; };

        std::generate_n(inserter(z.Ytype, z.Ytype.end()), 1000, [&] { 
                auto py = boost::make_shared<Y>(); 
                py->m = fgen();
                py->n = dgen();
                return std::make_pair(sgen(32), py);
                });
        std::generate_n(inserter(z.Xtype, z.Xtype.end()), 3000, [&] { 
                auto px = boost::make_shared<X>(); 
                px->p = igen();
                px->q = dgen();
                return std::make_pair(sgen(32), px);
                });

        z.i    = igen();
        z.name = sgen(8);

        return z; 
    }();
    return z;
}

Full Code Listing

On Coliru

Sweep