Using a file to store optparse arguments

Question

I've been using optparse for a while now, and would like to add the ability to load the arguments from a config file.

So far the best I can think of is a wrapper batch script with the arguments hardcoded... seems clunky.

What is the most elegant way to do this?

Cushiony

Answer 1

I agree with S.Lott's idea of using a config file, but I'd recommend using the built-in ConfigParser (configparser in 3.0) module to parse it, rather than a home-brewed solution.

Here's a brief script that illustrates ConfigParser and optparse in action.

import ConfigParser
from optparse import OptionParser

CONFIG_FILENAME = 'defaults.cfg'

def main():
    config = ConfigParser.ConfigParser()
    config.read(CONFIG_FILENAME)

    parser = OptionParser()
    parser.add_option("-l",
                      "--language",
                      dest="language",
                      help="The UI language",
                      default=config.get("Localization", "language"))
    parser.add_option("-f",
                      "--flag",
                      dest="flag",
                      help="The country flag",
                      default=config.get("Localization", "flag"))

    print parser.parse_args()

if __name__ == "__main__":
    main()

Output:

(<Values at 0x2182c88: {'flag': 'japan.png', 'language': 'Japanese'}>, [])

Run with "parser.py --language=French":

(<Values at 0x2215c60: {'flag': 'japan.png', 'language': 'French'}>, [])

Help is built in. Run with "parser.py --help":

Usage: parser.py [options]

Options:
  -h, --help            show this help message and exit
  -l LANGUAGE, --language=LANGUAGE
                        The UI language
  -f FLAG, --flag=FLAG  The country flag

The config file:

[Localization]
language=Japanese
flag=japan.png

Younglove

Answer 2

You can use argparse module for that:

>>> open('args.txt', 'w').write('-f\nbar')
>>> parser = argparse.ArgumentParser(fromfile_prefix_chars='@')
>>> parser.add_argument('-f')
>>> parser.parse_args(['-f', 'foo', '@args.txt'])
Namespace(f='bar')

It might be included in stdlib, see pep 389.

Frontal

PEP 389 was approved and argparse is already part of Python 2.7 – Prelate

Answer 3

I had a similar problem, but also wanted to specific the config file as an argument. Inspired by S. Lott's answer, I came up with the following code.

Example terminal session:

$ python defaultconf.py # use hard-coded defaults
False
$ python defaultconf.py --verbose # verbose on command line
True
$ python defaultconf.py --loadconfig blah # load config with 'verbose':True
True
$ python defaultconf.py --loadconfig blah --quiet # Override configured value
False

Code:

#!/usr/bin/env python2.6
import optparse

def getParser(defaults):
    """Create and return an OptionParser instance, with supplied defaults
    """
    o = optparse.OptionParser()
    o.set_defaults(**defaults)
    o.add_option("--verbose", dest = "verbose", action="store_true")
    o.add_option("--quiet", dest = "verbose", action="store_false")

    o.add_option("--loadconfig", dest = "loadconfig")

    return o


def main():
    # Hard coded defaults (including non-command-line-argument options)
    my_defaults = {'verbose': False, 'config_only_variable': 42}

    # Initially parse arguments
    opts, args = getParser(my_defaults).parse_args()

    if opts.loadconfig is not None:
        # Load config from disk, update the defaults dictionary, and reparse
        # Could use ConfigParser, simplejson, yaml etc.

        config_file_values = {'verbose': True} # the dict loaded from disk

        my_defaults.update(config_file_values)
        opts, args = getParser(my_defaults).parse_args()

    print opts.verbose

if __name__ == '__main__':
    main()

A practical implementation can be found on Github: The defaults dictionary, the argument parser and the main function

Chum

That's great. I've extended it to also show the default values from the config file in --help: – Stacte

Answer 4

That's what the set_defaults function is for. http://docs.python.org/library/optparse.html#optparse.OptionParser.set_defaults

Create a file that's the dictionary of default values.

{ 'arg1': 'this',
'arg2': 'that'
}

Then read this file, eval it to convert the text to a dictionary, and provide this dictionary as the arguments to set_defaults.

If you're really worried about eval, then use JSON (or YAML) notation for this file. Or you could even make an .INI file out of it and use configparser to get your defaults.

Or you can use a simple list of assignment statements and exec.

Config File.

arg1 = 'this'
arg2 = 'that'

Reading the config file.

defaults= {}
with open('defaults.py','r') as config
    exec config in {}, defaults

Tennilletennis

There's no reason to use eval here - you can just use marshal. – Reconstitute

@Nick Bastin: Eval is not Evil, only the sociopathic end-users who are always trying to hack the application by exploiting the use of eval. – Tennilletennis

Wagging your finger at an attacker taking advantage of a chink in your armor neither eliminates the chink or stops the attack... :) – Bawl

@Kevin Litle: Who's this mysterious "attacker"? A co-worker? The person who purchased the application? Who specifically? Please provide the name of this mysterious "attacker" has. – Tennilletennis

@S. Lot ;) : I don't know who this mysterious attacker is -- that's part of the problem! Neither do I know where this tiny, "safe" Python applet, into which I'm going stick a naked "eval()" of an externally accessible file, is going to end up in, say, three years. Maybe it will become part of a nifty, public service I never even envisioned, artfully mashed together by trusting developers in the company/community I left a year after I wrote it. But, we digress; sorry! – Bawl

@Kevin Little: Your threat scenarios -- "maybe it will become part of..." is silly. This is Python. The mysterious sociopath has access to the source already. – Tennilletennis

Answer 5

Read the arguments in the same commandline format from a file e.g. @commands, then use your original parser to parse them.

options, args = parser.parse_args()

if args[0][0] == '@': # script.py @optfile 
    with open(args[0][1:]) as f:
        fa = [l.strip() for l in f]
    fa = fa + args[1:] # put back any other positional arguments
    # Use your original parser to parse the new options
    options, args = parser.parse_args(args=fa, values=options)

Arhna

Answer 6

I've built a lot of scripts with flags and options lately, and I've come up with the solution described here.

Basically I instance an optionparser with a special flag that tells to try and load options from a file, so you can use normally your script specifying options from command line or provide them (or a set of them) from a file.

Update: i have shared code on GitHub

Cymatium