CMake and Flex/Bison

Question

I am converting my build system from configure/make to a cmake system

The system has some autogenerated files, from bison/flex. The original makefile commands are:

bison --defines=tokens.h --output=parser.cpp parser.y
flex --outfile=scanner.cpp scanner.l

I came across this ancient link which seems to explain how to do it, but when i run cmake with the following custom commands, nothing appears to happen (no error messages, no file generation)

FIND_PACKAGE(BISON REQUIRED)
IF(BISON_FOUND)
    ADD_CUSTOM_COMMAND(
      SOURCE ${CMAKE_SOURCE_DIR}/src/rcdgen/parser.y
      COMMAND ${BISON_EXECUTABLE}
      ARGS --defines=${CMAKE_SOURCE_DIR}/src/rcdgen/tokens.h
           -o ${CMAKE_SOURCE_DIR}/src/rcdgen/parser.cpp
           ${CMAKE_SOURCE_DIR}/src/rcdgen/parser.y
      COMMENT "Generating parser.cpp"
      OUTPUT ${CMAKE_SOURCE_DIR}/src/rcdgen/parser.cpp
    )
ENDIF(BISON_FOUND)

FIND_PACKAGE(FLEX REQUIRED)
IF(FLEX_FOUND)
    ADD_CUSTOM_COMMAND(
      SOURCE ${CMAKE_SOURCE_DIR}/src/rcdgen/scanner.l
      COMMAND ${FLEX_EXECUTABLE}
      ARGS -o${CMAKE_SOURCE_DIR}/src/rcdgen/parser.cpp
           ${CMAKE_SOURCE_DIR}/src/rcdgen/scanner.l
      COMMENT "Generating scanner.cpp"
      OUTPUT ${CMAKE_SOURCE_DIR}/src/rcdgen/scanner.cpp
    )
ENDIF(FLEX_FOUND)

I am new to cmake, so it's a bit confusing to me. Does anyone have any idea what a working custom_command would be?

Barca

Can you also post the part of the CMakeList which uses the generated files? – Lossa

Please be aware of tagging. Flex-lexer is for the lexical analyzer; the flex tag is for the Adobe/Apache UI Framework. – Bowel

sorry, 'flex' is what was suggested to me automatically, i didn't think to check the description – Barca

Answer 1

The new hotness for bison usage is actually documented in FindBison So for a simple parser project:

find_package(BISON)
BISON_TARGET(MyParser parser.y ${CMAKE_CURRENT_BINARY_DIR}/parser.cpp
             DEFINES_FILE ${CMAKE_CURRENT_BINARY_DIR}/parser.h)
add_executable(Foo main.cpp ${BISON_MyParser_OUTPUTS})

is what you'd do. Likewise for Flex.

Nucellus

Does not work on older version Flex version should be 2.5.5 or greater for this. Reason are the passed parameter by these functions. – Garber

When using this command, how can the Bison-generated header be used in the code? Neither #include "parser.hpp nor #include "parser.tab.hpp" work for me... – Anabelanabella

@AleksandarStefanović According to the Cmake documentation a header will be made and its filename will be defined by a certain macro. I don't know why it is done that way: Obviously you want to have the name in your code. – Nucellus

Answer 2

The format of your add_custom_commands is not quite right, but they appear to be almost correct. There are two versions of add_custom_command, and the one you want is the one which produces an output file (the parts inside square brackets are optional):

add_custom_command(OUTPUT output1 [output2 ...]
                   COMMAND command1 [ARGS] [args1...]
                   [COMMAND command2 [ARGS] [args2...] ...]
                   [MAIN_DEPENDENCY depend]
                   [DEPENDS [depends...]]
                   [IMPLICIT_DEPENDS <lang1> depend1
                                    [<lang2> depend2] ...]
                   [WORKING_DIRECTORY dir]
                   [COMMENT comment] [VERBATIM] [APPEND])

The idea is that the custom command only executes if the file specified as the OUTPUT of this command is used as an input elsewhere in the same CMakeLists.txt (e.g. in an add_library or add_executable call).

The custom command therefore will only run at build time (i.e. when you run make), not at configure time (when you run CMake), and only if you're building a target which directly or indirectly needs the OUTPUT file.

To fix your commands, I think the following should work (untested):

FIND_PACKAGE(BISON REQUIRED)
SET(BisonOutput ${CMAKE_SOURCE_DIR}/src/rcdgen/parser.cpp)
IF(BISON_FOUND)
    ADD_CUSTOM_COMMAND(
      OUTPUT ${BisonOutput}
      COMMAND ${BISON_EXECUTABLE}
              --defines=${CMAKE_SOURCE_DIR}/src/rcdgen/tokens.h
              --output=${BisonOutput}
              ${CMAKE_SOURCE_DIR}/src/rcdgen/parser.y
      COMMENT "Generating parser.cpp"
    )
ENDIF()

FIND_PACKAGE(FLEX REQUIRED)
SET(FlexOutput ${CMAKE_SOURCE_DIR}/src/rcdgen/scanner.cpp)
IF(FLEX_FOUND)
    ADD_CUSTOM_COMMAND(
      OUTPUT ${FlexOutput}
      COMMAND ${FLEX_EXECUTABLE}
              --outfile=${FlexOutput}
              ${CMAKE_SOURCE_DIR}/src/rcdgen/scanner.l
      COMMENT "Generating scanner.cpp"
    )
ENDIF()

ADD_LIBRARY(MyLib ${BisonOutput} ${FlexOutput})

Frager

This looks good, though i'm not sure what the add_library is for, could you elaborate? – Barca

@Barca - sorry, I could have made that a bit clearer. I was just trying to give an example of actually using the output files as inputs to another command. I don't expect you actually have a library called "MyLib" or that any target uses just these 2 files. I imagine these files will be 2 of many which comprise a lib/exe in your CMakeLists.txt. – Frager

ok, that's what i suspected, and, it seems to work :) Thanks very much – Barca

The new hotness for bison usage is actually documented at cmake.org So bison_target(parser fl.ypp fl.tab.cpp) add_executable(fl ${BISON_parser_OUTPUTS}) is what you'd do. Likewise for Flex. – Nucellus

scanner.l parser.h and parser.tab.h should be added as DEPENDS on the custom command as to regenerate the parser/lexer whenever the .l and .y files are modified. Parser only depends on parser.y but the lexer depends on both scanner.l and whatever header bison generates, usually parser.tab.h – Barrera