How can I get optparse's OptionParser to ignore invalid options?

Question

In python's OptionParser, how can I instruct it to ignore undefined options supplied to method parse_args?

e.g.
I've only defined option --foo for my OptionParser instance, but I call parse_args with list: [ '--foo', '--bar' ]

I don't care if it filters them out of the original list. I just want undefined options ignored.

The reason I'm doing this is because I'm using SCons' AddOption interface to add custom build options. However, some of those options guide the declaration of the targets. Thus I need to parse them out of sys.argv at different points in the script without having access to all the options. In the end, the top level Scons OptionParser will catch all the undefined options in the command line.

Secretory

Ummm... Extra arguments are an error, by definition. What are you trying to do? – Constant

see edits to original post above. – Secretory

It would be nice to be able to use the optparse module to parse an arbitrary list of arguments. I'm wanting to do this to write a command line frontend to a REST service without having to specify all the arguments in the service's API. – Vinni

@RossRogers consider marking my answer as accepted here if it satisfies your requirements. – Gangling

Answer 1

Per synack's request in a different answer's comments, I'm posting my hack of a solution which sanitizes the inputs before passing them to the parent OptionParser:

import optparse
import re
import copy
import SCons

class NoErrOptionParser(optparse.OptionParser):
    def __init__(self,*args,**kwargs):
        self.valid_args_cre_list = []
        optparse.OptionParser.__init__(self, *args, **kwargs)

    def error(self,msg):
        pass

    def add_option(self,*args,**kwargs):
        self.valid_args_cre_list.append(re.compile('^'+args[0]+'='))
        optparse.OptionParser.add_option(self, *args, **kwargs)

    def parse_args(self,*args,**kwargs):
        # filter out invalid options
        args_to_parse = args[0]
        new_args_to_parse = []
        for a in args_to_parse:
            for cre in self.valid_args_cre_list:
                if cre.match(a):
                    new_args_to_parse.append(a)


        # nuke old values and insert the new
        while len(args_to_parse) > 0:
            args_to_parse.pop()
        for a in new_args_to_parse:
            args_to_parse.append(a)

        return optparse.OptionParser.parse_args(self,*args,**kwargs)


def AddOption_and_get_NoErrOptionParser( *args, **kwargs):
    apply( SCons.Script.AddOption, args, kwargs)
    no_err_optparser = NoErrOptionParser(optparse.SUPPRESS_USAGE)
    apply(no_err_optparser.add_option, args, kwargs)

    return no_err_optpars

Secretory

Interesting solution. Thanks for taking the time to post it back. – Christabella

Answer 2

Here's one way to have unknown arguments added to the result args of OptionParser.parse_args, with a simple subclass.

from optparse import (OptionParser,BadOptionError,AmbiguousOptionError)

class PassThroughOptionParser(OptionParser):
    """
    An unknown option pass-through implementation of OptionParser.

    When unknown arguments are encountered, bundle with largs and try again,
    until rargs is depleted.  

    sys.exit(status) will still be called if a known argument is passed
    incorrectly (e.g. missing arguments or bad argument types, etc.)        
    """
    def _process_args(self, largs, rargs, values):
        while rargs:
            try:
                OptionParser._process_args(self,largs,rargs,values)
            except (BadOptionError,AmbiguousOptionError), e:
                largs.append(e.opt_str)

And here's a snippet to show that it works:

# Show that the pass-through option parser works.
if __name__ == "__main__": #pragma: no cover
    parser = PassThroughOptionParser()
    parser.add_option('-k', '--known-arg',dest='known_arg',nargs=1, type='int')
    (options,args) = parser.parse_args(['--shazbot','--known-arg=1'])    
    assert args[0] == '--shazbot'
    assert options.known_arg == 1

    (options,args) = parser.parse_args(['--k','4','--batman-and-robin'])
    assert args[0] == '--batman-and-robin'
    assert options.known_arg == 4

Gangling

Pretty nice solution! I was trying to create a custom script that disguises as another complex script and I only need to support a small subset of its options, so this solution seems like it would work very well. – Kirt

Answer 3

By default there is no way to modify the behavior of the call to error() that is raised when an undefined option is passed. From the documentation at the bottom of the section on how optparse handles errors:

If optparse‘s default error-handling behaviour does not suit your needs, you’ll need to subclass OptionParser and override its exit() and/or error() methods.

The simplest example of this would be:

class MyOptionParser(OptionParser):
    def error(self, msg):
        pass

This would simply make all calls to error() do nothing. Of course this isn't ideal, but I believe that this illustrates what you'd need to do. Keep in mind the docstring from error() and you should be good to go as you proceed:

Print a usage message incorporating 'msg' to stderr and exit. If you override this in a subclass, it should not return -- it should either exit or raise an exception.

Christabella

After further testing, this isn't working. The error is masked, but the parser stops parsing the arguments. This only works if the invalid flags are at the end of the command line. – Secretory

I ended up sanitizing the inputs before passing them to OptionParser. At least it is done in one place. – Secretory

Interesting find. Would you mind posting your solution, given it doesn't contain sensitive data? I am curious how you solved this. – Christabella

The solution isn't 100% bullet proof and it doesn't allow flags with args to have a space between them. Only a =. I've posted the hack below as an "answer" :-P – Secretory

This probably shouldn't be the top answer. justind's answer below is more useful, i.e. not throwing errors on unknown options, but still checking the known options. – Foreclose

Answer 4

Python 2.7 (which didn't exist when this question was asked) now provides the argparse module. You may be able to use ArgumentParser.parse_known_args() to accomplish the goal of this question.

Vaporific

Answer 5

This is pass_through.py example from Optik distribution.

#!/usr/bin/env python

# "Pass-through" option parsing -- an OptionParser that ignores
# unknown options and lets them pile up in the leftover argument
# list.  Useful for programs that pass unknown options through
# to a sub-program.

from optparse import OptionParser, BadOptionError

class PassThroughOptionParser(OptionParser):

    def _process_long_opt(self, rargs, values):
        try:
            OptionParser._process_long_opt(self, rargs, values)
        except BadOptionError, err:
            self.largs.append(err.opt_str)

    def _process_short_opts(self, rargs, values):
        try:
            OptionParser._process_short_opts(self, rargs, values)
        except BadOptionError, err:
            self.largs.append(err.opt_str)


def main():
    parser = PassThroughOptionParser()
    parser.add_option("-a", help="some option")
    parser.add_option("-b", help="some other option")
    parser.add_option("--other", action='store_true',
                      help="long option that takes no arg")
    parser.add_option("--value",
                      help="long option that takes an arg")
    (options, args) = parser.parse_args()
    print "options:", options
    print "args:", args

main()

Dorsiventral

This parser collapses an unrecognised "-abc" to "-a" – Vinni

@Nick: long options are specified as "--abc" in optparse syntax. I am afraid there is nothing that can be done about this even with PassThroughOptionParser subclass. "-abc" in the example above is valid -a option with bc argument (which it requires, BTW). – Dorsiventral

In general optparse treats "-abc" the same as "-a -b -c". In this case though, "-abc" is treated the same as "-a" (when "-a", "-b", and "-c" are unrecognised). – Vinni