Boost parse_config_file, empty key value

Question

I am using Boost program_options to parse a config file in the standard way as
shown in the multiple_sources.cpp example file of program_options.

    ifstream ini_file("config.ini");  
    po::store(po::parse_config_file(ini_file, desc, true), vm);  
    po::notify(vm);

The config.ini file however has empty key=value pairs such as:

key1=value1  
key2=value2  
key3=  
key4=  
key5=value5  

Trying to read this file results in a Boost error:

boost::program_options::invalid_option_value

Is there any mechanism in boost::program_options to read such config files with empty entires?

Any help would be much appreciated.

---

I am editing this Question since the the problem has not yet been solved. I will just cite the example from Boost (1.53).

This is the full multiple_sources.cpp file:

#include <boost/program_options.hpp>
namespace po = boost::program_options;


#include <iostream>
#include <fstream>
#include <iterator>
using namespace std;

// A helper function to simplify the main part.
template<class T>
ostream& operator<<(ostream& os, const vector<T>& v)
{
    copy(v.begin(), v.end(), ostream_iterator<T>(os, " ")); 
    return os;
}


int main(int ac, char* av[])
{
    try {
        int opt;
        string config_file;

        // Declare a group of options that will be 
        // allowed only on command line
        po::options_description generic("Generic options");
        generic.add_options()
            ("version,v", "print version string")
            ("help", "produce help message")
            //("optimization","optimization level")      
            ("config,c", po::value<string>(&config_file)->default_value("multiple_sources.cfg"),
                  "name of a file of a configuration.")
            ;

        // Declare a group of options that will be 
        // allowed both on command line and in
        // config file
        po::options_description config("Configuration");
        config.add_options()
            ("optimization", po::value<int>(&opt)->default_value(10), 
                  "optimization level")
            ("include-path,I", po::value< vector<string> >()->composing(), 
                 "include path")
            ;

        // Hidden options, will be allowed both on command line and
        // in config file, but will not be shown to the user.
        po::options_description hidden("Hidden options");
        hidden.add_options()
            ("input-file", po::value< vector<string> >(), "input file")
            ;

        po::options_description cmdline_options;
        cmdline_options.add(generic).add(config).add(hidden);

        po::options_description config_file_options;
        config_file_options.add(config).add(hidden);

        po::options_description visible("Allowed options");
        visible.add(generic).add(config);

        po::positional_options_description p;
        p.add("input-file", -1);

        po::variables_map vm;
        store(po::command_line_parser(ac, av).
              options(cmdline_options).positional(p).run(), vm);
        notify(vm);

        ifstream ifs(config_file.c_str());
        if (!ifs)
        {
            cout << "can not open config file: " << config_file << "\n";
            return 0;
        }
        else
        {
            store(parse_config_file(ifs, config_file_options), vm);
            notify(vm);
        }

        if (vm.count("help")) {
            cout << visible << "\n";
            return 0;
        }

        if (vm.count("version")) {
            cout << "Multiple sources example, version 1.0\n";
            return 0;
        }

        if (vm.count("include-path"))
        {
            cout << "Include paths are: " 
                 << vm["include-path"].as< vector<string> >() << "\n";
        }

        if (vm.count("input-file"))
        {
            cout << "Input files are: " 
                 << vm["input-file"].as< vector<string> >() << "\n";
        }

        cout << "Optimization level is " << opt << "\n";                
    }
    catch(exception& e)
    {
        cout << e.what() << "\n";
        return 1;
    }    
    return 0;
}

And the corresponding configuration file (multiple_sources.cfg) is:

#
# Comment out this line to use hard-coded default value of 10
# 
optimization = 1
include-path = /opt

If this file is now modified to:

#
# Comment out this line to use hard-coded default value of 10
# 
optimization = 
include-path = /opt

The following error message is thrown:

the argument for option 'optimization' is invalid

The proposed solutions with validation overloading etc. seem unnecessarily complicated, especially since one might have to write a validation function for each data type, incorporating the possibility of recognizing '\n' other white-space.

Any suggestions? Thank you everyone for taking the time.

---

Following Aditya's suggestion I have replaced the following line :

            ("optimization", po::value<int>(&opt)->default_value(10), 
                  "optimization level")

with the following :

            ("optimization", po::value<int>(&opt)->zero_tokens(), 
                  "optimization level")

and :

            ("optimization", po::value<int>(&opt)->implicit_value(10), 
                  "optimization level")

and neither of them read blank options. Boost example program "parser_test.cpp" bypasses the use of zero_tokens(), or implicit_value() by reading the key-value pairs into a vector as follows:

void test_config_file(const char* config_file)
{
    options_description desc;
    desc.add_options()
        ("gv1", new untyped_value)
        ("gv2", new untyped_value)
        ("empty_value", new untyped_value)
        ("plug*", new untyped_value)
        ("m1.v1", new untyped_value)
        ("m1.v2", new untyped_value)
        ("b", bool_switch())
    ;

    const char content1[] =
    " gv1 = 0#asd\n"
    "empty_value = \n"
    "plug3 = 7\n"
    "b = true\n"
    "[m1]\n"
    "v1 = 1\n"
    "\n"
    "v2 = 2\n"    
    ;

    vector<option> a2 = parse_config_file<char>(config_file, desc).options;
    BOOST_REQUIRE(a2.size() == 6);
    check_value(a2[0], "gv1", "0");
    check_value(a2[1], "empty_value", "");
    check_value(a2[2], "plug3", "7");
    check_value(a2[3], "b", "true");
    check_value(a2[4], "m1.v1", "1");
    check_value(a2[5], "m1.v2", "2");
}

Erma

Have a look at: #4460150 – Willywillynilly

Thanks for the link ypnos, however I already had a look at that solution and I added "true" as the 3rd parameter in the parse_config_file arguments. This still does not resolve the issue. Did you encounter this problem and did the proposed solution fix it? – Erma

Sorry, then I am of no help for you. :/ Would it work if you had key3=1, key4=1 in the config file? – Willywillynilly

Thanks for replying ypnos. If I have key3=1, key4=1, then there are no issues. The problem is in the case of blank entries – Erma

I thought you were talking about redundant keys that you do not want to be processed anyways. If you have a definition for your keys in your options description, you can add a custom validator that ignores empty entries and instead returns a default value. See #8820609 – Willywillynilly

@EssGee This question is being flagged as duplicate of the one ypnos mentioned above. I understand it isn't actually a duplicate? Could you perhaps add a link to the question to your own question and explain what is different (perhaps similar to what you did in the comment above)? – Pyelitis

jogojapan, I did't quite follow what you want me to do. Should I post the explanation as a new question? As you correctly understood, I don't believe the question is a duplicate. – Erma

did you look at zero_tokens()? – Filagree

Thanks for the tip Aditya. I tried zero_tokens() as well as implicit_value() and still no success. Were you able to run it? – Erma

What are you trying to accomplish with key= that wouldn't be possible by just omitting that key in the config file? – Gaitan

@MarkB But can you ensure that a customer is always putting key=value pairs into the config file of your software? – Deepfreeze

@MarkB in my case ini-file is automatically generated during build process from the template file with lines of the form nameN={valueN}. It would be very convenient and natural to have default empty strings, if some replacement value valueN is not defined. – Touraine

Answer 1

I'd recommend you to surround with try/catch this exception and throw errors only for those fields who are really necessary for your program and can't be empty, otherwise the field will be ignored.

Carreno

Answer 2

No, there is currently no way to handle this within boost::program_options. Having an empty key:value pair in your INI file is the same as specifying the option on the command line without providing the argument. The approache of writing custom validators suggested by ypnos may work but seems impractical as you would need to apply it to every option you expect may be left blank. You'll have to write your own implementation of po::parse_config_file that would ignore lines without a value (unless the corresponding option is marked as zero_token) to get the result you are looking for.

Requisition

Actually, program_options allow to specify an option on the command line without an argument, using the implicit_value() helper. But this does not work for INI files, unfortunately. – Touraine

Answer 3

I stumbled on this problem several times and finally solved it by just removing invalid lines from the initial INI-file before passing it to the program_options::parse_config_file. It can be done by creating a temporary modified INI-file on filesystem or reading the whole original INI-file to memory and modifying it there, and then create istringstream to pass to parse_config_file.

I, personally, created a boost::iostreams filtering stream class for this purpose to filter ifstream on the fly. This does not create any new dependency, since I already use Boost, and boost::iostreams can be used header-only in my case.

This is how it is used:

#include <fstream>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/program_options.hpp>
#include "inifilefilter.h" // IniFileInputFilter
...
std::ifstream file("config.ini");
if (file)
{
   // filtering out empty values, i.e. name=
   boost::iostreams::filtering_istream filteredIniFile; 
   filteredIniFile.push(IniFileInputFilter());
   filteredIniFile.push(file);
   po::parsed_options parsedFileOpts =
         po::parse_config_file(filteredIniFile, configFileOptions, true);
   ...
}

I won't show the code from the inifilefilter.h here, because it somewhat lengthy with all the checks and corner cases, but at the same time it's pretty simple in it's idea. The IniFileInputFilter's read() method buffers each input line, and then at the end-of-line passes the line further only if it's valid. And 'valid' means that it is either a [section] line, or have some non-space characters between the first '=' and the end-of-line.

Touraine