I want to code a state machine which has a reaction list with more than 50 entries.
I found here some python scripts to generate header files for lists with more than 50 entries.
But I cannot manage to generate a single header file. Can someone please explain to me how to use theses scripts? I also didn't find any help in boost documention.
Any help would be appreciated.
Important Update:
A more improved script, which only requires a working python environment is described in this answer to this question.
Original Answer
See my answer to your question on the Boost mailing-list.
The following bash-code will generate the MPL-containers for 51 to 100 entries. If you do not care about the other MPL-container types and just want to generate MPL-lists, adjust the value of CONTAINERS accordingly.
#! /bin/bash
# The name of this script.
SCRIPT_NAME=$(basename $0)
# The usage message for this script.
USAGE="Usage: ${SCRIPT_NAME} <path-to-boost-source> [<new-value-for BOOST_MPL_LIMIT_VECTOR_SIZE>]"
# Script-arguments:
# The (absolute) path to the directory containing the Boost-source.
SOURCE=$(readlink -e "$1")
# The new limit for Boost.MPL's vector.
NEW_BOOST_MPL_LIMIT_VECTOR_SIZE=$2
# Check validity of arguments.
if [[ -z "${SOURCE}" ]] || ! [[ -d "${SOURCE}" ]]; then
echo "Missing path to Boost directory."
echo ${USAGE}
exit
fi
REGEX_INTEGER='^[0-9]+$'
if [[ -n "${NEW_BOOST_MPL_LIMIT_VECTOR_SIZE}" ]] && ! [[ ${NEW_BOOST_MPL_LIMIT_VECTOR_SIZE} =~ ${REGEX_INTEGER} ]]; then
echo "New size for BOOST_MPL_LIMIT_VECTOR_SIZE must be an unsigned integer!"
echo ${USAGE}
exit
fi
# The Boost.MPL-containers for which more preprocessed
# header-files shall be generated.
CONTAINERS="vector list set map"
# The same containers without Boost.MPL-map.
CONTAINERS_WITHOUT_MAP=$(echo ${CONTAINERS} | sed 's/map//g')
# Change into include-directory of Boost.MPL.
pushd ${SOURCE}/boost/mpl >/dev/null 2>&1
# Possibly change the limit for Boost.MPL's vector.
if [[ -n "${NEW_BOOST_MPL_LIMIT_VECTOR_SIZE}" ]]; then
sed -i "s/\(define\s\+BOOST_MPL_LIMIT_VECTOR_SIZE\s\+\)[0-9]\+/\1${NEW_BOOST_MPL_LIMIT_VECTOR_SIZE}/" ./limits/vector.hpp
fi
# Create hpp-files for each MPL-container with 60 to 100 elements
# which will be used during generation.
for container in ${CONTAINERS}; do
for i in {6..10}; do
sed -e "s/50/${i}0/g" \
-e "s/41/$((i-1))1/g" \
-e "s/40/$((i-1))0/g" \
./${container}/${container}50.hpp \
> ./${container}/${container}${i}0.hpp;
done
done
for container in ${CONTAINERS_WITHOUT_MAP}; do
for i in {6..10}; do
sed -e "s/50/${i}0/g" \
-e "s/41/$((i-1))1/g" \
-e "s/40/$((i-1))0/g" \
./${container}/${container}50_c.hpp \
> ./${container}/${container}${i}0_c.hpp;
done
done
# Leave include-directory of Boost.MPL.
popd >/dev/null 2>&1
# Change into source-directory of Boost.MPL.
pushd ${SOURCE}/libs/mpl/preprocessed >/dev/null 2>&1
# Create cpp-files for each MPL-container with 60 to 100 elements
# which will be used during generation.
for container in ${CONTAINERS}; do
for i in {6..10}; do
sed -e "s/50/${i}0/g" \
-e "s/41/$((i-1))1/g" \
-e "s/40/$((i-1))0/g" \
./${container}/${container}50.cpp \
> ./${container}/${container}${i}0.cpp;
done
done
for container in ${CONTAINERS_WITHOUT_MAP}; do
for i in {6..10}; do
sed -e "s/50/${i}0/g" \
-e "s/41/$((i-1))1/g" \
-e "s/40/$((i-1))0/g" \
./${container}/${container}50_c.cpp \
> ./${container}/${container}${i}0_c.cpp;
done
done
# Now generate MPL-preprocessed files using the python-scripts.
python preprocess_vector.py all ${SOURCE}
python preprocess_list.py all ${SOURCE}
python preprocess_set.py all ${SOURCE}
python preprocess_map.py all ${SOURCE}
python preprocess.py all ${SOURCE}
# Leave source-directory of Boost.MPL.
popd >/dev/null 2>&1
For even bigger containers with more than 100 entries, adjust the range in the for-loops.
I hope that helps.
Deniz
Update:
I realized that my original version of the script did only generate the numbered versions of the MPL-container headers. However, it did not update the non-numbered versions.
I fixed it by adding python preprocess.py all ${SOURCE} near the end of the script.
However, this does only generate modified versions of the non-numbered MPL-container headers if during generation the corresponding macro settings (e.g. a higher value for BOOST_MPL_LIMIT_VECTOR_SIZE) are different than the original values.
These modifications can be made in the file user.hpp within each subdirectory of ${SOURCE}/libs/mpl/preprocessed/include.
For example, by adding
#define BOOST_MPL_LIMIT_VECTOR_SIZE 30
to ${SOURCE}/libs/mpl/preprocessed/include/gcc/user.hpp the boost::mpl::vector (included from boost/mpl/vector.hpp) is able to hold 30 elements, now, instead of the default 20.
Note:
However, it might not be a good idea to adjust values like BOOST_MPL_LIMIT_VECTOR_SIZE only for generation and leave its value defined in ${SOURCE}/boost/mpl/limits/vector.hpp untouched.
Possibly, this does not even compile because the underlying boost::mpl::vector21 to boost::mpl::vectorN cannot be found (because not being included automatically).
Best solution is probably to adjust the value in ${SOURCE}/boost/mpl/limits/vector.hpp. The above bash-script can automatically do so, if the second (optional) argument is given.
From Boost 1.59.0 on the below explained scripts are now included in each Boost release.
They are located int <boost-source-dir>/libs/mpl/preprocessed/.
The README.txt in that directory should explain you how to use it. (Or just read on.)
Because I realized that not everyone has a working bash shell at hand (yes, I mean you Windows-guys ;-)) but everyone needs a working python environment, I converted my helper bash-script I mentioned above into python:
boost_mpl_preprocess.py
# Copyright Deniz Bahadir 2015
#
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# http://www.boost.org/LICENSE_1_0.txt)
#
# See http://www.boost.org/libs/mpl for documentation.
# See https://mcmap.net/q/1018298/-boost-mpl-how-to-generate-pre-generated-header-files-for-lists-with-more-than-50-entries for further information.
# See https://mcmap.net/q/1018298/-boost-mpl-how-to-generate-pre-generated-header-files-for-lists-with-more-than-50-entries for further information.
import argparse
import sys
import os
import os.path
import re
import fileinput
import shutil
def create_more_container_files(sourceDir, suffix, maxElements, containers, containers2):
"""Creates additional files for the individual MPL-containers."""
# Create files for each MPL-container with 20 to 'maxElements' elements
# which will be used during generation.
for container in containers:
for i in range(20, maxElements, 10):
# Create copy of "template"-file.
newFile = os.path.join( sourceDir, container, container + str(i+10) + suffix )
shutil.copyfile( os.path.join( sourceDir, container, container + "20" + suffix ), newFile )
# Adjust copy of "template"-file accordingly.
for line in fileinput.input( newFile, inplace=1, mode="rU" ):
line = re.sub(r'20', re.escape(str(i+10)), line.rstrip())
line = re.sub(r'11', re.escape(str(i + 1)), line.rstrip())
line = re.sub(r'10(?![0-9])', re.escape(str(i)), line.rstrip())
print(line)
for container in containers2:
for i in range(20, maxElements, 10):
# Create copy of "template"-file.
newFile = os.path.join( sourceDir, container, container + str(i+10) + "_c" + suffix )
shutil.copyfile( os.path.join( sourceDir, container, container + "20_c" + suffix ), newFile )
# Adjust copy of "template"-file accordingly.
for line in fileinput.input( newFile, inplace=1, mode="rU" ):
line = re.sub(r'20', re.escape(str(i+10)), line.rstrip())
line = re.sub(r'11', re.escape(str(i + 1)), line.rstrip())
line = re.sub(r'10(?![0-9])', re.escape(str(i)), line.rstrip())
print(line)
def create_input_for_numbered_sequences(headerDir, sourceDir, containers, maxElements):
"""Creates additional source- and header-files for the numbered sequence MPL-containers."""
# Create additional container-list without "map".
containersWithoutMap = containers[:]
try:
containersWithoutMap.remove('map')
except ValueError:
# We can safely ignore if "map" is not contained in 'containers'!
pass
# Create header/source-files.
create_more_container_files(headerDir, ".hpp", maxElements, containers, containersWithoutMap)
create_more_container_files(sourceDir, ".cpp", maxElements, containers, containersWithoutMap)
def adjust_container_limits_for_variadic_sequences(headerDir, containers, maxElements):
"""Adjusts the limits of variadic sequence MPL-containers."""
for container in containers:
headerFile = os.path.join( headerDir, "limits", container + ".hpp" )
regexMatch = r'(define\s+BOOST_MPL_LIMIT_' + container.upper() + r'_SIZE\s+)[0-9]+'
regexReplace = r'\g<1>' + re.escape( str(maxElements) )
for line in fileinput.input( headerFile, inplace=1, mode="rU" ):
line = re.sub(regexMatch, regexReplace, line.rstrip())
print(line)
def to_positive_multiple_of_10(string):
"""Converts a string into its encoded positive integer (greater zero) or throws an exception."""
try:
value = int(string)
except ValueError:
msg = '"%r" is not a positive multiple of 10 (greater zero).' % string
raise argparse.ArgumentTypeError(msg)
if value <= 0 or value % 10 != 0:
msg = '"%r" is not a positive multiple of 10 (greater zero).' % string
raise argparse.ArgumentTypeError(msg)
return value
def to_existing_absolute_path(string):
"""Converts a path into its absolute path and verifies that it exists or throws an exception."""
value = os.path.abspath(string)
if not os.path.exists( value ) or not os.path.isdir( value ):
msg = '"%r" is not a valid path to a directory.' % string
raise argparse.ArgumentTypeError(msg)
return value
def main():
"""The main function."""
# Prepare and run cmdline-parser.
cmdlineParser = argparse.ArgumentParser(description="A generator-script for pre-processed Boost.MPL headers.")
cmdlineParser.add_argument("-v", "--verbose", dest='verbose', action='store_true',
help="Be a little bit more verbose.")
cmdlineParser.add_argument("-s", "--sequence-type", dest='seqType', choices=['variadic', 'numbered', 'both'],
default='both',
help="Only update pre-processed headers for the selected sequence types, "
"either 'numbered' sequences, 'variadic' sequences or 'both' sequence "
"types. (Default=both)")
cmdlineParser.add_argument("--no-vector", dest='want_vector', action='store_false',
help="Do not update pre-processed headers for Boost.MPL Vector.")
cmdlineParser.add_argument("--no-list", dest='want_list', action='store_false',
help="Do not update pre-processed headers for Boost.MPL List.")
cmdlineParser.add_argument("--no-set", dest='want_set', action='store_false',
help="Do not update pre-processed headers for Boost.MPL Set.")
cmdlineParser.add_argument("--no-map", dest='want_map', action='store_false',
help="Do not update pre-processed headers for Boost.MPL Map.")
cmdlineParser.add_argument("--num-elements", dest='numElements', metavar="<num-elements>",
type=to_positive_multiple_of_10, default=100,
help="The maximal number of elements per container sequence. (Default=100)")
cmdlineParser.add_argument(dest='sourceDir', metavar="<source-dir>",
type=to_existing_absolute_path,
help="The source-directory of Boost.")
args = cmdlineParser.parse_args()
# Some verbose debug output.
if args.verbose:
print "Arguments extracted from command-line:"
print " verbose = ", args.verbose
print " source directory = ", args.sourceDir
print " num elements = ", args.numElements
print " sequence type = ", args.seqType
print " want: vector = ", args.want_vector
print " want: list = ", args.want_list
print " want: set = ", args.want_set
print " want: map = ", args.want_map
# The directories for header- and source files of Boost.MPL.
# NOTE: Assuming 'args.sourceDir' is the source-directory of the entire boost project.
headerDir = os.path.join( args.sourceDir, "boost", "mpl" )
sourceDir = os.path.join( args.sourceDir, "libs", "mpl", "preprocessed" )
# Check that the header/source-directories exist.
if not os.path.exists( headerDir ) or not os.path.exists( sourceDir ):
# Maybe 'args.sourceDir' is not the source-directory of the entire boost project
# but instead of the Boost.MPL git-directory, only?
headerDir = os.path.join( args.sourceDir, "include", "boost", "mpl" )
sourceDir = os.path.join( args.sourceDir, "preprocessed" )
if not os.path.exists( headerDir ) or not os.path.exists( sourceDir ):
cmdlineParser.print_usage()
print "error: Cannot find Boost.MPL header/source files in given Boost source-directory!"
sys.exit(0)
# Some verbose debug output.
if args.verbose:
print "Chosen header-directory: ", headerDir
print "Chosen source-directory: ", sourceDir
# Create list of containers for which files shall be pre-processed.
containers = []
if args.want_vector:
containers.append('vector')
if args.want_list:
containers.append('list')
if args.want_set:
containers.append('set')
if args.want_map:
containers.append('map')
if containers == []:
print "Nothing to do. (Why did you prevent generating pre-processed headers for all Boost.MPL container types?"
sys.exit(0)
# Some verbose debug output.
if args.verbose:
print "Containers for which to pre-process headers: ", containers
# Create (additional) input files for generating pre-processed headers of numbered sequence MPL containers.
if args.seqType == "both" or args.seqType == "numbered":
create_input_for_numbered_sequences(headerDir, sourceDir, containers, args.numElements)
# Modify settings for generating pre-processed headers of variadic sequence MPL containers.
if args.seqType == "both" or args.seqType == "variadic":
adjust_container_limits_for_variadic_sequences(headerDir, containers, args.numElements)
# Generate MPL-preprocessed files.
os.chdir( sourceDir )
if args.seqType == "both" or args.seqType == "numbered":
if args.want_vector:
if args.verbose:
print "Pre-process headers for Boost.MPL numbered vectors."
os.system( "python " + os.path.join( sourceDir, "preprocess_vector.py" ) + " all " + args.sourceDir )
if args.want_list:
if args.verbose:
print "Pre-process headers for Boost.MPL numbered lists."
os.system( "python " + os.path.join( sourceDir, "preprocess_list.py" ) + " all " + args.sourceDir )
if args.want_set:
if args.verbose:
print "Pre-process headers for Boost.MPL numbered sets."
os.system( "python " + os.path.join( sourceDir, "preprocess_set.py" ) + " all " + args.sourceDir )
if args.want_map:
if args.verbose:
print "Pre-process headers for Boost.MPL numbered maps."
os.system( "python " + os.path.join( sourceDir, "preprocess_map.py" ) + " all " + args.sourceDir )
if args.seqType == "both" or args.seqType == "variadic":
if args.verbose:
print "Pre-process headers for Boost.MPL variadic containers."
os.system( "python " + os.path.join( sourceDir, "preprocess.py" ) + " all " + args.sourceDir )
if __name__ == '__main__':
main()
Running it with
python boost_mpl_preprocess.py --help
gives some usage information.
But you can also just run it with a single command-line parameter which determines the path to the Boost source-directory (e.g. python boost_mpl_preprocess.py ./boost-source) and use the default behavior.
This results in creating preprocessed headers for all four Boost.MPL container types (vector, list, set and map) with their numbered and variadic form.
Note:
As a side effect when preprocessing Boost.MPL containers in their variadic form, the value of their associated limit-macros BOOST_MPL_LIMIT_container_SIZE will be modified, too. It gets the number of elements determined when calling boost_mpl_preprocess.py.
For example, the following call preprocesses only the variadic form of Boost.MPL vector and also sets the macro BOOST_MPL_LIMIT_VECTOR_SIZE in ./boost-source/boost/mpl/limits/vector.hpp to 70.
$ python boost_mpl_preprocess.py --no-list --no-set --no-map \
--sequence-type=variadic --num-elements=70 ./boost-source
Since Boost 1.56 pre-processing Boost.MPL containers even with the above script did no longer work, because of changes in the header-comments, the original python-scripts relied on.
The following script can be used to fix this problem. It just adds the missing information into the header-comments.
fix_boost_mpl_preprocess.py
# Copyright Deniz Bahadir 2015
#
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# http://www.boost.org/LICENSE_1_0.txt)
#
# See http://www.boost.org/libs/mpl for documentation.
# See https://mcmap.net/q/1018298/-boost-mpl-how-to-generate-pre-generated-header-files-for-lists-with-more-than-50-entries for further information.
import argparse
import sys
import os.path
import re
import fileinput
import datetime
import glob
def fix_header_comment(filename, timestamp):
"""Fixes the header-comment of the given file."""
# Fix input file.
name = os.path.basename( filename )
for line in fileinput.input( filename, inplace=1, mode="rU" ):
# If header-comment already contains anything for '$Id$', remove it.
line = re.sub(r'\$Id:[^$]+\$', r'$Id$', line.rstrip())
# Replace '$Id$' by a string containing the file's name (and a timestamp)!
line = re.sub(re.escape(r'$Id$'), r'$Id: ' + name + r' ' + timestamp.isoformat() + r' $', line.rstrip())
print(line)
def fix_input_files_for_variadic_seq(sourceDir, timestamp):
"""Fixes files used as input when pre-processing MPL-containers in their variadic form."""
files = glob.glob( os.path.join( sourceDir, "src", "*" ) )
for currentFile in sorted( files ):
fix_header_comment( currentFile, timestamp )
def fix_input_files_for_numbered_seq(sourceDir, suffix, timestamp, containers):
"""Fixes files used as input when pre-processing MPL-containers in their numbered form."""
# Fix input files for each MPL-container type.
for container in containers:
files = glob.glob( os.path.join( sourceDir, container, container + '*' + suffix ) )
for currentFile in sorted( files ):
fix_header_comment( currentFile, timestamp )
def fix_input_files(headerDir, sourceDir, containers=['vector', 'list', 'set', 'map'], verbose='false'):
"""Fixes source- and header-files used as input when pre-processing MPL-containers."""
# The new modification time.
timestamp = datetime.datetime.now();
# Fix the input files.
if verbose:
print "Fix input files for pre-processing Boost.MPL variadic containers."
fix_input_files_for_variadic_seq(sourceDir, timestamp)
if verbose:
print "Fix input files for pre-processing Boost.MPL numbered containers."
fix_input_files_for_numbered_seq(headerDir, ".hpp", timestamp, containers)
fix_input_files_for_numbered_seq(sourceDir, ".cpp", timestamp, containers)
def to_existing_absolute_path(string):
"""Converts a path into its absolute path and verifies that it exists or throws an exception."""
value = os.path.abspath(string)
if not os.path.exists( value ) or not os.path.isdir( value ):
msg = '"%r" is not a valid path to a directory.' % string
raise argparse.ArgumentTypeError(msg)
return value
def main():
"""The main function."""
# Prepare and run cmdline-parser.
cmdlineParser = argparse.ArgumentParser(
description="Fixes the input files used for pre-processing of Boost.MPL headers.")
cmdlineParser.add_argument("-v", "--verbose", dest='verbose', action='store_true',
help="Be a little bit more verbose.")
cmdlineParser.add_argument(dest='sourceDir', metavar="<source-dir>",
type=to_existing_absolute_path,
help="The source-directory of Boost.")
args = cmdlineParser.parse_args()
# Some verbose debug output.
if args.verbose:
print "Arguments extracted from command-line:"
print " verbose = ", args.verbose
print " source directory = ", args.sourceDir
# The directories for header- and source files of Boost.MPL.
# NOTE: Assuming 'args.sourceDir' is the source-directory of the entire boost project.
headerDir = os.path.join( args.sourceDir, "boost", "mpl" )
sourceDir = os.path.join( args.sourceDir, "libs", "mpl", "preprocessed" )
# Check that the header/source-directories exist.
if not os.path.exists( headerDir ) or not os.path.exists( sourceDir ):
# Maybe 'args.sourceDir' is not the source-directory of the entire boost project
# but instead of the Boost.MPL git-directory, only?
headerDir = os.path.join( args.sourceDir, "include", "boost", "mpl" )
sourceDir = os.path.join( args.sourceDir, "preprocessed" )
if not os.path.exists( headerDir ) or not os.path.exists( sourceDir ):
cmdlineParser.print_usage()
print "error: Cannot find Boost.MPL header/source files in given Boost source-directory!"
sys.exit(0)
# Some verbose debug output.
if args.verbose:
print "Chosen header-directory: ", headerDir
print "Chosen source-directory: ", sourceDir
# Fix input file for generating pre-processed headers.
fix_input_files(headerDir, sourceDir, verbose = args.verbose)
if __name__ == '__main__':
main()
Run it like so:
$ python fix-boost_mpl_preprocess.py ./boost-source
© 2022 - 2024 — McMap. All rights reserved.