Boost Spirit lexeme vs no_skip

Question

As like the description on Boost.Spirit, the only difference between lexeme and no_skip is the pre_skip.

But after some test, I'm still confusing about the exactly meaning for pre_skip.

So what kind of condition will make a difference, maybe a example can help me understand it much better.

Thanks!

Liddell

Answer 1

Pre-skip ignores whitespace at the start of the expression.

Contrasting:

Live On Coliru

#include <boost/spirit/include/qi.hpp>

namespace qi = boost::spirit::qi;
static std::string const input = " 42j";

int main() {
    auto run_test = [](auto p) {
        auto f = input.begin(), l = input.end();
        int i;
        return qi::phrase_parse(f, l, p, qi::space, i)
            ? std::to_string(i)
            : "unparsed";
    };

    std::cout << "no_skip: " << run_test(qi::no_skip[ qi::int_ >> 'j' ]) << "\n";
    std::cout << "lexeme: "  << run_test(qi::lexeme[ qi::int_ >> 'j' ]) << "\n";
}

Prints:

no_skip: unparsed
lexeme: 42

As you can see lexeme will silently eat the leading white space. That's the pre-skip.

Rabbin

Excellent explanation! Seems like I test the lexeme and no_skip in a wrong way. Thanks for your explanation! – Liddell

Just realized I could have made the tester more succinct: Live. Doesn't really change anything of course :) – Rabbin

Oh, my standard backgrounder about skippers is here: #17073487 - granted it didn't exactly provide an example like this – Rabbin

I had viewed the question when I searched about this topic, but no succinct example there. Since you provide it, I can realize the difference between these. Actually I had guessed the answer before, but maybe my test code is a little bit complicate so that some another factors affect the test result. I'm going to check out that. Thanks again for your positive response. :) – Liddell

@Rabbin I see that the difference between lexeme and no_skip is in pre_skip (e.g. pre-space is eat or not). They are the same for in_skip (in-space is not eat). And if you want to disable post_skip? I see that in all situations post_skip is performed (eat post-space in input "42j "). See a quick&dirty modification of your example: godbolt.org/z/PfxP7bh15 – Allnight

@ClaudiuCruceanu I don't think I understand what you're trying to say ("They are the same"? "[...] for in_skip" (?) and "in-space is not eat"? If you're saying that to colonize Mars we will need to bring food, then I agree :) Here's some more side-by-side info on skippers in Qi: ichess.org/racer – Rabbin

@Rabbin I only quote from the master :) :"As you can see lexeme will silently eat the leading white space. That's the pre-skip." . And I generalized using expressions like in-skip, post-skip, in-space and post-space. The important issue is that for pre-skip and in-skip we have control but I saw that we cannot disable post-skip. P.S. probably "ichess.org/racer" is not complete/typo. – Allnight

@Rabbin Actualy forget about Marte :)) you can ignore the preamble ;) and pull up a magic :) trick to answer my (more or less ) implicit question: How is possible to disable space skipping after the rule? – Allnight

Oops that's a clipboard failure yes, should have been https://mcmap.net/q/1021010/-boost-spirit-skipper-issues. I still don't really know what in-skip is (other than... just a skipper maybe). Pre/post-skip, yes those make sense. in-space/post-space are very confusing to me. I don't see it in my posts or your code. Did you post a question about disabling post-skipping? – Rabbin

@ClaudiuCruceanu Here's what I think will help you. If you don't want skipping, do not use a skipper: godbolt.org/z/aMzcodofr – Rabbin

Let us continue this discussion in chat. – Allnight

Answer 2

Tecnically speaking skipping happens before calling rules parser. So all directives within a lexeme parsing statement will act as an atom, skipper will be invoked before the begining of the lexeme clause as usual but not between rules within the clause.

The no_skip directive "wont invoke" any skipper at all as stated by the documentation. It will act as lexeme if there is no white space to skip, but that's a subtle difference. It's more appropriate when parsing binary files.

Esperanzaespial