Parsing empty options in Python

Question

I have an application that allows you to send event data to a custom script. You simply lay out the command line arguments and assign what event data goes with what argument. The problem is that there is no real flexibility here. Every option you map out is going to be used, but not every option will necessarily have data. So when the application builds the string to send to the script, some of the arguments are blank and python's OptionParser errors out with "error: --someargument option requires an argument"

Being that there are over 200 points of data, it's not like I can write separate scripts to handle each combination of possible arguments (it would take 2^200 scripts). Is there a way to handle empty arguments in python's optionparser?

Panelboard

Answer 1

Sorry, misunderstood the question with my first answer. You can accomplish the ability to have optional arguments to command line flags use the callback action type when you define an option. Use the following function as a call back (you will likely wish to tailor to your needs) and configure it for each of the flags that can optionally receive an argument:

import optparse

def optional_arg(arg_default):
    def func(option,opt_str,value,parser):
        if parser.rargs and not parser.rargs[0].startswith('-'):
            val=parser.rargs[0]
            parser.rargs.pop(0)
        else:
            val=arg_default
        setattr(parser.values,option.dest,val)
    return func

def main(args):
    parser=optparse.OptionParser()
    parser.add_option('--foo',action='callback',callback=optional_arg('empty'),dest='foo')
    parser.add_option('--file',action='store_true',default=False)
    return parser.parse_args(args)

if __name__=='__main__':
    import sys
    print main(sys.argv)

Running from the command line you'll see this:

# python parser.py
(<Values at 0x8e42d8: {'foo': None, 'file': False}>, [])

# python parser.py --foo
(<Values at 0x8e42d8: {'foo': 'empty', 'file': False}>, [])

# python parser.py --foo bar
(<Values at 0x8e42d8: {'foo': 'bar', 'file': False}>, [])

Whichsoever

The problem with this variant is that --for=bar will not work: parser.py: error: --foo option does not take a value – Prescind

Answer 2

Yes, there is an argument to do so when you add the option:

from optparse import OptionParser
parser = OptionParser()
parser.add_option("--SomeData",action="store", dest="TheData", default='')

Give the default argument the value you want the option to have it is to be specified but optionally have an argument.

Whichsoever

I am using the default option, but that only is used when --SomeData isn't specified in the commandline. If you do: --SomeData --SomeMoreData blah, then --SomeData errors because it wasn't passed a value. – Panelboard

@Panelboard sorry, I misunderstood your question. A post another answer with a solution that should work for you. – Whichsoever

Answer 3

I don't think optparse can do this. argparse is a different (non-standard) module that can handle situations like this where the options have optional values.

With optparse you have to either have to specify the option including it's value or leave out both.

Davilman

Answer 4

Optparse already allows you to pass the empty string as an option argument. So if possible, treat the empty string as "no value". For long options, any of the following work:

my_script --opt=   --anotheroption
my_script --opt='' --anotheroption
my_script --opt="" --anotheroption
my_script --opt '' --anotheroption
my_script --opt "" --anotheroption

For short-style options, you can use either of:

my_script -o '' --anotheroption
my_script -o "" --anotheroption

Caveat: this has been tested under Linux and should work the same under other Unixlike systems; Windows handles command line quoting differently and might not accept all of the variants listed above.

Houseboy

Of course this works, but it is not the same as simply using my_script --opt. – Prescind

Answer 5

Mark Roddy's solution would work, but it requires attribute modification of a parser object during runtime, and has no support for alternative option formattings other than - or --. A slightly less involved solution is to modify the sys.argv array before running optparse and insert an empty string ("") after a switch which doesn't need to have arguments. The only constraint of this method is that you have your options default to a predictable value other than the one you are inserting into sys.argv (I chose None for the example below, but it really doesn't matter).

The following code creates an example parser and set of options, extracts an array of allowed switches from the parser (using a little bit of instance variable magic), and then iterates through sys.argv, and every time it finds an allowed switch, it checks to see if it was given without any arguments following it . If there is no argument after a switch, the empty string will be inserted on the command line. After altering sys.argv, the parser is invoked, and you can check for options whose values are "", and act accordingly.

#Instantiate the parser, and add some options; set the options' default values to None, or something predictable that
#can be checked later.
PARSER_DEFAULTVAL = None
parser = OptionParser(usage="%prog -[MODE] INPUT [options]")
#This method doesn't work if interspersed switches and arguments are allowed.
parser.allow_interspersed_args = False
parser.add_option("-d", "--delete", action="store", type="string", dest="to_delete", default=PARSER_DEFAULTVAL)
parser.add_option("-a", "--add", action="store", type="string", dest="to_add", default=PARSER_DEFAULTVAL)

#Build a list of allowed switches, in this case ['-d', '--delete', '-a', '--add'] so that you can check if something
#found on sys.argv is indeed a valid switch. This is trivial to make by hand in a short example, but if a program has
#a lot of options, or if you want an idiot-proof way of getting all added options without modifying a list yourself,
#this way is durable. If you are using OptionGroups, simply run the loop below with each group's option_list field.
allowed_switches = []
for opt in parser.option_list:
    #Add the short (-a) and long (--add) form of each switch to the list.
    allowed_switches.extend(opt._short_opts + opt._long_opts)

#Insert empty-string values into sys.argv whenever a switch without arguments is found.
for a in range(len(sys.argv)):
    arg = sys.argv[a]
    #Check if the sys.argv value is a switch
    if arg in allowed_switches:
        #Check if it doesn't have an accompanying argument (i.e. if it is followed by another switch, or if it is last
        #on the command line)
        if a == len(sys.argv) - 1 or argv[a + 1] in allowed_switches:
            sys.argv.insert(a + 1, "")

options, args = parser.parse_args()

#If the option is present (i.e. wasn't set to the default value)
if not (options.to_delete == PARSER_DEFAULTVAL):
    if options.droptables_ids_csv == "":
        #The switch was not used with any arguments.
        ...
    else:
        #The switch had arguments.
        ...

Bellaude

Answer 6

After checking that the cp command understands e.g. --backup=simple but not --backup simple, I answered the problem like this:

import sys
from optparse import OptionParser

def add_optval_option(pog, *args, **kwargs):
    if 'empty' in kwargs:
        empty_val = kwargs.pop('empty')
        for i in range(1, len(sys.argv)):
            a = sys.argv[i]
            if a in args:
                sys.argv.insert(i+1, empty_val)
                break
    pog.add_option(*args, **kwargs)

def main(args):
    parser = OptionParser()
    add_optval_option(parser,
                      '--foo', '-f',
                      default='MISSING',
                      empty='EMPTY',
                      help='"EMPTY" if given without a value. Note: '
                      '--foo=VALUE will work; --foo VALUE will *not*!')
    o, a = parser.parse_args(args)
    print 'Options:'
    print '  --foo/-f:', o.foo
    if a[1:]:
        print 'Positional arguments:'
        for arg in a[1:]:
            print ' ', arg
    else:
        print 'No positional arguments'

if __name__=='__main__':
    import sys
    main(sys.argv)

Self-advertisement: This is part of the opo module of my thebops package ... ;-)

Prescind