Avoiding attribute copies with karma generators

Question

I'm using karma to generate representations of large structs, but the structs are being copied during generation. I don't think they need to be, so was wondering how to avoid it.

The quick example below prints "Copy!", as the target struct is copied in rule::generate:

namespace karma = spirit::karma;
namespace phoenix = boost::phoenix;

struct foo
{
    foo() { }
    foo( foo const &other ) { std::cout << "Copy!"; }
    int f() const { return 42; }
};

std::string output;
typedef std::back_insert_iterator< std::string > iterator;
karma::rule< iterator, foo() > foo_rule = 
    karma::int_[ karma::_1 = phoenix::bind( &foo::f, karma::_val ) ];
foo my_foo;
iterator it( output );
karma::generate( it, foo_rule, my_foo );

I can stop the copy by declaring foo_rule's attribute by reference:

karma::rule< iterator, foo &() > foo_rule

but that doesn't work with a vector [obviously the foos are therefore copyable, but may be cheap to copy at vector construction, but expensive to copy at generate time :-)]

The example below prints 'Copy!' five times during generation (that is, ignoring copies during vector ctor); 10 times if foo_rule's attribute isn't a reference:

std::vector<foo> my_vec_foo(5);
karma::rule< iterator, std::vector<foo>() > vec_foo_rule = *foo_rule;
karma::generate(it, vec_foo_rule, my_vec_foo);

Having both rules take references doesn't compile with Boost 1.47 on VC 2008. That is, with:

karma::rule< iterator, foo &() > foo_rule /* = ... */;
karma::rule< iterator, std::vector<foo> &() > vec_foo_rule /* = ... */;

I get extract_from_container instantiated with Attribute = std::vector<foo> and Exposed=std::vector<foo> &. On line 131 of extract_from.hpp, it tries to form Exposed const & and the compiler fails when creating refrence-to-reference.

I feel like I'm missing something, so any pointers would be greatly appreciated!

Cantankerous

+1 for excellent minimal problem. Perhaps, you could make it even more copy/pastable (right now people without intimate knowledge of Spirit will not likely get it running). I tested my answer, and the post contains a full minimal sample. Cheers – Kosygin

Answer 1

I'm sure you've tried it, but i'll say it nonetheless. Have you tried as follows:

std::vector<foo> my_vec_foo(5);
karma::rule< iterator, std::vector<foo>&() > vec_foo_rule = *foo_rule;
karma::generate(it, vec_foo_rule, my_vec_foo);

Update I just tested it with the below snippet (g++ 4.6 with Boost 1.47.0). It confirms that the above works. However, there is room for confusion, as the std::vector<foo> my_vec_foo(5) will also show 5 copies being made. See the BIG LETTER warning in the code and the output:

#include <boost/spirit/include/karma.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace karma = boost::spirit::karma;
namespace phoenix = boost::phoenix;

struct foo
{
    foo() { }
    foo( foo const &other ) { std::cerr << "Copy!\n"; }
    int f() const { return 42; }
};

int main()
{
    std::string output;
    typedef std::back_insert_iterator< std::string > iterator;
    iterator it( output );
    karma::rule< iterator, foo&() > foo_rule = 
        karma::int_[ karma::_1 = phoenix::bind( &foo::f, karma::_val ) ];

    foo my_foo;
    karma::generate( it, foo_rule, my_foo );

    std::vector<foo> my_vec_foo(5);

    std::cerr << "\nSTART WATCHING NOW" << std::endl;

    karma::rule< iterator, std::vector<foo>&() > vec_foo_rule = *foo_rule;
    karma::generate(it, vec_foo_rule, my_vec_foo);
}

Output:

Copy!
Copy!
Copy!
Copy!
Copy!

START WATCHING NOW

Kosygin

The five copies should be clear, as that's how resize (and the special ctor) work. They default costruct an element as the second parameter and copy that into all reserved slots. – Erythroblast

@Xeo: do you think I should have explained that? I thought it was obvious :) – Kosygin

Just wanted to point that out to anyone who may be wondering. :) Also, your example misses some includes and it doesn't seem to compile on Ideone. Seems the Boost version they use is too low. – Erythroblast

@Xeo: 'missing includes' seems a bit strongly worded. That is a matter of style: I happen to know that (on linux) all the indirect headers are present and prefer brevity on SO. Note that the OP was missing a lot more (crucial) includes. Those would have been hard to find. Lastly, I never promised it works on ideone :) (I don't know why it matters) – Kosygin

Oh, well, it doesn't really matter, I just wanted to see the exact output of the generator, as I decided to get to know Spirit some more in the near future. :) – Erythroblast

@Xeo: Thanks - I've updated the post to make it a bit clearer that I was ignoring copies during the vector ctor! – Cantankerous

Thanks for the answer - I had tried that without any luck, so it looks like GCC and VC are behaving differently. I've updated the question so it includes the error I get with both rules taking attributes by reference. Apols for omitting the includes; just wanted to keep it short :-) – Cantankerous

@aldous: Seems that VS2008 isn't implementing reference collapsing rules correctly, weird. Try the VS2010 express version (or if you can, get the professional version) and see if your problem persists. – Erythroblast

@Cantankerous If the exact same code fails on MSVC, I'd go and file a bug to the Spirit library (allthough possibly the problem could be in the typetraits implementation) – Kosygin

@Erythroblast and sehe: it turns out that this does compile on VS2010 pro, though the full implementation that I pulled the example from fails in a similar way. I'll spend some time going through the errors and see if I can pin down whether it's a compiler or library bug. This is definitely an answer, though, so thank you! – Cantankerous