Flex++ Bisonc++ parser

Question

I'm trying to use flex and bison in my project to generate a parser code for a file structure. Main programming language is C++ and project is on an OO design mainly running in parallel.

I heard that flex and bison generated parsers are C codes and they're not reenterant. Googling, I found flex++ and bisonc++. Unfortunately there is no simple tutorial to get started. Most examples are based on bison/flex. Some people somehow integrated bison/flex parsers in their C++ code. They supposed to be "tricky"...

Documentation of flex++ and bisonc++ doesn't help me and. Tutorials and examples, they all get input from stdin and print some messages on stdout.

I need these features in my parser:

1. Parser should be a C++ class, defined in normal manner (a header and a cpp file)
2. Parser receives data from either an std::string or std::stringstream or a null-terminated char*.

I feel so confused. Should I use flex++/bisonc++ or flex/bison? And how to do that, satisfying above conditions?

Marybelle

Answer 1

There are flex/bison, flex++/bison++ and flexc++/bisonc++. I think it's best to pick one of these three pairs, instead of mixing/matching flex++ and bisonc++. Here are the user guides for Flexc++ and Bisonc++.

From the Flexc++ website:

Flexc++, contrary to flex and flex++, generates code that is explicitly intended for use by C++ programs. The well-known flex(1) program generates C source-code and flex++(1) merely offers a C++-like shell around the yylex function generated by flex(1) and hardly supports present-day ideas about C++ software development. Contrary to this, flexc++ creates a C++ class offering a predefined member function lex matching input against regular expressions and possibly executing C++ code once regular expressions were matched. The code generated by flexc++ is pure C++, allowing its users to apply all of the features offered by that language.

From the Bisonc++ website:

Bisonc++ is a general-purpose parser generator that converts a grammar description for an LALR(1) context-free grammar into a C++ class to parse that grammar. Once you are proficient with bisonc++, you may use it to develop a wide range of language parsers, from those used in simple desk calculators to complex programming languages. Bisonc++ is highly comparable to the program bison++, written by Alain Coetmeur: all properly-written bison++ grammars ought to be convertible to bisonc++ grammars after very little or no change. Anyone familiar with bison++ or its precursor, bison, should be able to use bisonc++ with little trouble. You need to be fluent in using the C++ programming in order to use bisonc++ or to understand this manual.

So flexc++/bisonc++ are more than just wrappers around the old flex/bison utilities. They generate complete C++ classes to be used for re-entrant scanning / parsing.

Leacock

I even can't compile flexc++! Just downloaded source code. No makefile, no cmake nothing! – Marybelle

@Marybelle there's a Linux package: packages.debian.org/sid/flexc++ Why would you need a makefile? Just include the headers and build from your own environment. – Leacock

I didn't heard about icmake before. seems to be the build system used to compile flexc++. Thanks – Marybelle

@Marybelle The author Frank Brokken makes a lot of his own utilities. Apart from flexc++, bisonc++, icmake he also did a markup language (yodl) and much more. Oh and he is the author of the C++ Annations document cppannotations.sourceforge.net – Leacock

I'm compiling flexc++. Hope can use it successfully! I really like C++ Annations a lot! It's my main resource for cpp. In fact I always do sudo apt-get install c++-annotations-pdf ;) It has a section for bisonc++/flex (last section in book) Unfortunately it's so complicated for me. – Marybelle

@Marybelle If you mean section 23.9 icce.rug.nl/documents/cplusplus/cplusplus23.html#l576, that's the one. If you follow this line by line, you should get a compilable example by the time you reach 23.9.2. – Leacock

I'll take a look at (4th time!). I think all stuff about lex, flex, yacc, flex, bison, etc, are very complicated. started to study about two weeks ago, since now, no success. yet! – Marybelle

@Marybelle Perseverance will pay off for you, just try to understand one step at the time. I had the good fortune to be a student at the same university where Frank Brokken was a teacher, at the time when the Annotations were being written! That was even before the old C++98 standard was ready, with broken compilers etc. But somehow, I got through that class. Now it's a lot easier, so good luck to you! – Leacock

Answer 2

Flex can generate a reentrant C scanner. See Section 19 Reentrant C scanners in the Flex manual.

Similarly, Bison can generate a reentrant C parser. See Section 3.8.11 A Pure (Reentrant) Parser in the Bison manual for details.

Do you absolutely need to have a C++ parser and std::string/stringstream based parser data?

Have you looked at Boost.Spirit as an alternative?

Crankshaft

No. I can change my code to generate char*s to be parsed. Bu not able to read input from stdin – Marybelle

@sorush-r: You should be able to change the source of your Flex C scanner input by redefining the YY_INPUT() macro. See Section 9 The Generated Scanner in the Flex manual. I'd take a serious look at Boost.Spirit, too. – Crankshaft

Answer 3

The LRSTAR product (LR(k) parser and DFA lexer generator) is C++ based. Runs on Widowns and has six Visual Studio projects. The code also compiles with "gcc" and other compilers. There are classes for lexer and parser, symbol-table, AST. Complete source code is available. It gets good reviews. I should know. I am the author.

Jotham