Boost Spirit X3 cannot compile repeat directive with variable factor
Asked Answered
A

1

14

I am trying to use the Boost Spirit X3 directive repeat with a repetition factor that is variable. The basic idea is that of a header + payload, where the header specifies the size of the payload. A simple example “3 1 2 3” is interpreted as header = 3, data= {1, 2, 3} (3 integers).

I could only find examples from the spirit qi documentation. It uses boost phoenix reference to wrap the variable factor: http://www.boost.org/doc/libs/1_50_0/libs/spirit/doc/html/spirit/qi/reference/directive/repeat.html

std::string str;
int n;
test_parser_attr("\x0bHello World",
    char_[phx::ref(n) = _1] >> repeat(phx::ref(n))[char_], str);
std::cout << n << ',' << str << std::endl;  // will print "11,Hello World"

I wrote the following simple example for spirit x3 without luck:

#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <string>
#include <iostream>

namespace x3 = boost::spirit::x3;
using x3::uint_;
using x3::int_;
using x3::phrase_parse;
using x3::repeat;
using x3::space;
using std::string;
using std::cout;
using std::endl;

int main( int argc, char **argv )
{
  string data("3 1 2 3");
  string::iterator begin = data.begin();
  string::iterator end = data.end();

  unsigned int n = 0;

  auto f = [&n]( auto &ctx ) { n = x3::_attr(ctx); };
  bool r = phrase_parse( begin, end, uint_[f] >> repeat(boost::phoenix::ref(n))[int_], space );
  if ( r && begin == end  )
    cout << "Parse success!" << endl; 
  else
    cout << "Parse failed, remaining: " << string(begin,end) << endl;

  return 0;
}

Compiling the code above with boost 1.59.0 and clang++ (flags: -std=c++14) gives the following:

boost_1_59_0/boost/spirit/home/x3/directive/repeat.hpp:72:47: error: no matching constructor for

      initialization of 'proto_child0' (aka 'boost::reference_wrapper<unsigned int>')

            typename RepeatCountLimit::type i{};

If I hardcode repeat(3) instead of repeat(boost::phoenix::ref(n)) it works properly, but it is not a possible solution since it should support a variable repetition factor.

Compilation with repeat(n) completes successfully, but it fails parsing with the following output: “Parse failed, remaining: 1 2 3"

Looking at the source code for boost/spirit/home/x3/directive/repeat.hpp:72 it calls the empty constructor for template type RepeatCountLimit::type variable i and then assign during the for loop, iterating over min and max. However since the type is a reference it should be initialized in the constructor, so compilation fails. Looking at the equivalent source code from the previous library version boost/spirit/home/qi/directive/repeat.hpp:162 it is assigned directly:

        typename LoopIter::type i = iter.start();

I am not sure what I am doing wrong here, or if x3 currently does not support variable repetition factors. I would appreciate some help solving this issue. Thank you.

Aho answered 10/11, 2015 at 6:29 Comment(7)
In order to reduce compile times Spirit.X3, amongst other things, isn't based on Boost.Proto anymore. A sad consequence of that choice is that, at the moment, X3 does not know anything about Phoenix(a library heavily based on Proto). Apparently there is work in progress to include a c++14 subset of Phoenix in a future version. In the meantime you could either create a new directive (based on repeat) or do something silly like this.Magnetochemistry
@cv_and_he Timing. From the upvote I wager you, too, have spent some time figuring things out?Procreant
@Procreant Very interesting answer, I had no idea that existed. At the moment the syntax seems a little clumsy but it could be very interesting in the future.Magnetochemistry
@cv_and_he Agreed. I just learned this the other day: livecoding.tv/video/… As you'd expect it is a bit of tinkering with experimental versions of the libraryProcreant
@Procreant BTW I like your suggestion in the mailing list thread I linked a lot (phrase_parse( begin, end, repeat(uint_)[int_], space ); if you don't remember).Magnetochemistry
@cv_and_he I didn't remember ... o.O I actually went and figured out - again - that it doesn't work as of now. Lol. Good call. That's an awesome design right there :)Procreant
Another approach.Magnetochemistry
P
10

From what I gather, reading the source and the mailing list, Phoenix is not integrated into X3 at all: the reason being that c++14 makes most of it obsolete.

I agree that this leaves a few spots where Qi used to have elegant solutions, e.g. eps(DEFERRED_CONDITION), lazy(*RULE_PTR) (the Nabialek trick), and indeed, this case.

Spirit X3 is still in development, so we might see this added¹

For now, Spirit X3 has one generalized facility for stateful context. This essentially replaces locals<>, in some cases inherited arguments, and can be /made to/ validate the number of elements in this particular case as well:

  • x3::with²

Here's how you could use it:

with<_n>(std::ref(n)) 
    [ omit[uint_[number] ] >> 
    *(eps [more] >> int_) >> eps [done] ]

Here, _n is a tag type that identifies the context element for retrieval with get<_n>(cxtx).

Note, currently we have to use a reference-wrapper to an lvalue n because with<_n>(0u) would result in constant element inside the context. I suppose this, too, is a QoI that may be lifted as X# matures

Now, for the semantic actions:

unsigned n;
struct _n{};

auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };

This stores the parsed unsigned number into the context. (In fact, due to the ref(n) binding it's not actually part of the context for now, as mentioned)

auto more   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) >  _val(ctx).size(); };

Here we check that we're actually not "full" - i.e. more integers are allowed

auto done   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };

Here we check that we're "full" - i.e. no more integers are allowed.

Putting it all together:

Live On Coliru

#include <string>
#include <iostream>
#include <iomanip>

#include <boost/spirit/home/x3.hpp>

int main() {
    for (std::string const input : { 
            "3 1 2 3", // correct
            "4 1 2 3", // too few
            "2 1 2 3", // too many
            // 
            "   3 1 2 3   ",
        })
    {
        std::cout << "\nParsing " << std::left << std::setw(20) << ("'" + input + "':");

        std::vector<int> v;

        bool ok;
        {
            using namespace boost::spirit::x3;

            unsigned n;
            struct _n{};

            auto number = [](auto &ctx) { get<_n>(ctx).get() = _attr(ctx); };
            auto more   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) >  _val(ctx).size(); };
            auto done   = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) == _val(ctx).size(); };

            auto r = rule<struct _r, std::vector<int> > {} 
                  %= with<_n>(std::ref(n)) 
                        [ omit[uint_[number] ] >> *(eps [more] >> int_) >> eps [done] ];

            ok = phrase_parse(input.begin(), input.end(), r >> eoi, space, v);
        }

        if (ok) {
            std::copy(v.begin(), v.end(), std::ostream_iterator<int>(std::cout << v.size() << " elements: ", " "));
        } else {
            std::cout << "Parse failed";
        }
    }
}

Which prints:

Parsing '3 1 2 3':          3 elements: 1 2 3 
Parsing '4 1 2 3':          Parse failed
Parsing '2 1 2 3':          Parse failed
Parsing '   3 1 2 3   ':    3 elements: 1 2 3 

¹ lend your support/voice at the [spirit-general] mailing list :)

² can't find a suitable documentation link, but it's used in some of the samples

Procreant answered 10/11, 2015 at 10:34 Comment(5)
I'll think about creating a directive or using some generative trick to make this all more "neatly packaged". I'm afraid the lvalue-requirement for with-context elements currently would imply dynamic allocation... :( I'll keep you postedProcreant
Thanks @cv and @sehe.I tested sehe's solution with gcc 5.2.0 successfully, but with clang (clang-700.1.76) it fails boost/spirit/home/x3/operator/sequence.hpp:61:24: error: no viable overloaded operator[] for type 'const boost::spirit::x3::expect_gen' return left >> expect[right]; repeat.cxx:28:69: note: in instantiation of function template specialization 'boost::spirit::x3::operator><std::__1::reference_wrapper<unsigned int>, unsigned long>' requested here auto more = [](auto &ctx) { _pass(ctx) = get<_n>(ctx) > _val(ctx).size(); };Aho
@GuilhermeSchlinker You can either use static_cast<unsigned>(get<_n>(ctx)) > _val(ctx).size() or get<_n>(ctx).get() > _val(ctx).size().Magnetochemistry
@cv_and_he interestingly, you can use _val(ctx).size() < get<_n>(ctx) :) (Funny that clang and gcc disagree here)Procreant
Much better (paging @GuilhermeSchlinker in case he missed it).Magnetochemistry

© 2022 - 2024 — McMap. All rights reserved.