Bison/Flex declaration mess... how what should I include/declare where?

Question

I'm using Bison and Flex to make a reentrant scanner/parser pair but can't wrap my head around where everything is to be included and declared.

First of all, I'm using reentrant Flex so I need to pass the yyscan_t scanner type first to Bison by declaring %parse-param {yyscan_t scanner} and then make Bison pass this to Flex by declaring %lex-param {yyscan_t scanner}. But yyscan_t is not declared by Bison so I must include the Flex generated scanner header file (which I named scanner.flex.h) in my Bison file. But since my Flex file includes my Bison header, and my Bison header now includes the Flex header, I get circular includes which messes up stuff in unpredictable ways!

And let's say I want to add in location tracking with %locations in my Bison file and %bison-locationsin my Flex file. Now I need to change the declaration of my yyerror and my yylex (it seems I have to define yylex AGAIN even though it's defined in the Flex generated header but I cannot include that, remember?) functions in my Bison file to include the YYLTYPE pointer. But what now? It seems that the default YYLTYPE declaration is placed AFTER the prologue is inserted and thus I cannot use this default YYLTYPE in the declaration of my yyerror and my yylex.

I realize that there are a lot of workarounds to these problems... but how are you supposed to do it properly? It totally escapes me and this just leaves my head a mess...

Wire

Answer 1

In the top of your bison file you need to forward declare these values. You can do this in a common header or in the code file before you include the flex and bison files. yyscan_t is just a typedef of a void * so I declared the parameters as void pointers to get it to work.

CommonHeader.h

int yylex (union YYSTYPE * yyval_param, struct YYLTYPE * yylloc_param, void * yyscanner);
int yyerror(struct YYLTYPE * yylloc_param, void *scanner, const char *s);

Example of yyerror

int yyerror(struct YYLTYPE * yylloc_param, void *scanner, const char *s)
{
    printf("*** Lexical Error %s %d.%d-%d.%d\n", s, 
         yylloc_param->first_line, yylloc_param->first_column, 
         yylloc_param->last_line, yylloc_param->last_column);
}

Ezaria

I know about yyscan_t being void * so I already did that even though it felt a bit like a hack since it's supposed to be opaque. What about YYLTYPE then? It's not declared until after the prologue and forward declaring a whole struct seems a bit much? – Wire

Why does forward declaring a struct seem a bit much. Another option is to add %defines to your bison file. This will generate a header file that you can include. – Ezaria

Total brain fart on my part... I thought defining but said forward declaring. I do generate a header file but that cannot be included in the Bison file as it will lead to infinite include recursion since Bison generated headers have no include guards (but they do have guards against redefining types). I could forward declare YYLTYPE though, something like this?: struct YYLTYPE; typedef struct YYLTYPE YYLTYPE; – Wire

I'm wondering if YYLTYPE is guaranteed to be defined as a struct, though? – Wire

bison does not change very often, I understand your concerns though but you should be safe forward declaring YYLTYPE as a struct. This would only be an issue during a compilation, if anything changes this will happen during compile time and not runtime. Make sure to pass -Wall to gcc/g++ to keep track of warnings. – Ezaria

Answer 2

Came across this while looking for something else.

The answer is that Bison emits the internal declaration for YYLTYPE after the %union spec. Hence, putting function prototypes and other stuff in a prologue section after the %union avoids the problem of declaring YYLTYPE yourself; this is quite OK, Bison allows more than one prologue section:

%{

/* Prologue 1 */

%}

%this

%that

%union {}

%{

/* Prologue 2, YYLTYPE declared */

%}

%%

blah:  talk | blah talk

%%

Imogen