How do I handle the global variables and the variable scopes issues?
Inject Global variables into the test via loading the a project's
cache, configure the test CMake file or pushing it via -D command line
option?
Generally speaking all currently existing methods (via cache, via environment variables and via -D command line) are a bad choice in one or another case as involving unpredictable behaviour.
This is a least list of issues i can recall:
- Which one variable could intersect/overlap another one and when?
- Variables load or set can not be applied out of cmake detection stage (for example, in cmake script mode).
- The same unique variable can not hold different values for different OS/compilers/configurations/architectures and so on.
- Variables can not be attached to a package (not scope) term represented by system functions like
Find* or add_subdirectory.
I've used variables inside cmake lists a long time and decided to write my own solution to cut off them all at once out of the cmake lists.
The idea is to write a standalone parser through a cmake script to load variables from a file or set of files and define a set of rules to enable variables set in predefined or strict order with check on collisions and overlapping.
Here a list of several features:
bool A=ON is equal to bool A=TRUE is equal to bool A=1path B="c:\abc" is equal on the Windows to path B="C:\ABC" (explicit path variable instead of a string which is by default)B_ROOT="c:\abc" is equal on the Windows to B_ROOT="C:\ABC" (variable's type detection by the ending of a variable name)LIB1:WIN="c:\lib1" sets only in the Windows, when LIB1:UNIX="/lib/lib1" sets only in the Unix (a variable specialization).LIB1:WIN=c:\lib1, LIB1:WIN:MSVC:RELEASE=$/{LIB1}\msvc_release - variables reusing via expansion and specialization
I can not said everything here, but you can take as an example the tacklelib library (https://github.com/andry81/tacklelib) to research the implementation on your own.
The example of described configuration files is stored here: https://github.com/andry81/tacklelib/tree/HEAD/_config
The implementation:
https://github.com/andry81/tacklelib/tree/HEAD/cmake/tacklelib/SetVarsFromFiles.cmake
As a mandatory the cmake list must be initialized through the configure_environment(...) macro:
https://github.com/andry81/tacklelib/tree/HEAD/CMakeLists.txt
Read the readme file for the details around the tacklelib project:
https://github.com/andry81/tacklelib/tree/HEAD/README_EN.txt
The entire project currently is an experimental.
PS:
Write a parser script on the cmake is a tough task, read at least these issues for the start:
Is there some "official" way of unit-testing your own CMake script code?
Something like a special mode to run CMake in? My goal is "white-box testing" (as much as possible).
I did my own "white-box" or a way of testing my own scripts. I have write a set of modules (which itself dependent to the library) to run test in a separate cmake process:
https://github.com/andry81/tacklelib/tree/HEAD/cmake/tacklelib/testlib
My tests built on it:
https://github.com/andry81/tacklelib/tree/HEAD/cmake_tests
The idea is to put into the tests directory a hierarchy of directories and files with tests and the runner code would just search for the tests in predefined order to execute each test in a separate cmake process:
function(tkl_testlib_enter_dir test_dir)
# Algorithm:
# 1. Iterate not recursively over all `*.include.cmake` files and
# call to `tkl_testlib_include` function on each file, then
# if at least one is iterated then
# exit the algorithm.
# 2. Iterate non recursively over all subdirectories and
# call to the algorithm recursively on each subdirectory, then
# continue.
# 3. Iterate not recursively over all `*.test.cmake` files and
# call to `tkl_testlib_test` function on each file, then
# exit the algorithm.
#
, where set of functions can be used both from a runner cmake script or *.include.cmake file:
Where TestLib.cmake is designed to run cycle over creation external cmake processes with a test module - *.test.cmake and these functions should be called from a runner script or from an include module (groups other include modules - *.include.cmake or test modules - *.test.cmake):
tkl_testlib_enter_dir test_dir
tkl_testlib_include test_dir test_file_name
tkl_testlib_test test_dir test_file_name
Where TestModule.cmake automatically includes in all *.test.cmake modules in which you have to put your testing code.
After that you just use tkl_test_assert_true inside a *.test.cmake module to mark a test as succeeded or failed.
Additionally, you can use filter parameters in the runner scripts in the _scripts subdirectory to filter tests out:
--path_match_filter <[+]regex_include_match_expression> | <-regex_exclude_match_expression>[;...]
--test_case_match_filter <[+]regex_include_match_expression> | <-regex_exclude_match_expression>[;...]
Pros:
- The
TestModule.cmake does traverse the entire directory with tests by predefined rules, you just need to make sure the correct hierarchy and naming to order the testing.
- Usage of a per directory basis separate inclusion file
*.include.cmake to exclusive inclusion or to reorder the tests in the directory and its descedants.
- Existence of a
*.test.cmake file is the only requirement to put the test to running by default. To exclusively include or exclude the test you can start use command line flags --path_match_filter ... and --test_case_match_filter ....
Cons:
- Mostly all test functions has implemented through the
function keyword, which a bit reduces the functionality of several functions. For example, the tkl_test_assert_true can only mark the test is succeeded or failed. To explicitly interrupt the test you have make the branching through the call to tkl_return_if_failed macro.
- All files in the directory with the tests must have the suffix,
.test.cmake - for a test, and .include.cmake - for inclusion commands. All builtin search logic depends on it.
- You have write you own runner to call the script
RunTestLib.cmake. The example of a run all on the unix shell could be found here: https://github.com/andry81/tacklelib/tree/HEAD/cmake_tests/_build/test_all.sh
The entire project currently is an experimental.
