How do you convert command line args in python to a dictionary?

Question

I'm writing an application that takes arbitrary command line arguments, and then passes them onto a python function:

$ myscript.py --arg1=1 --arg2=foobar --arg1=4

and then inside myscript.py:

import sys
argsdict = some_function(sys.argv)

where argsdict looks like this:

{'arg1': ['1', '4'], 'arg2': 'foobar'}

I'm sure there is a library somewhere that does this, but I can't find anything.

EDIT: argparse/getopt/optparse is not what I'm looking for. These libraries are for defining an interface that is the same for each invocation. I need to be able to handle arbitrary arguments.

Unless, argparse/optparse/getopt has functionality that does this...

Brandonbrandt

docs.python.org/dev/library/argparse.html – Mediant

argparse is completely different, its for defining a commandline interface. I'm trying to parse an arbitrary commandline interface. Each invocation of this script will have different arguments each time. – Brandonbrandt

I think it's still possible to do that with argparse. If you don't want to use it, then AFAIK you have no option but to write a parser for the arguments yourself. – Intrusion

Yes I also can't figure out how to parse arbitrary arguments with the standard libraries... – Regeneration

Is this still unsolved 5 years later? Do I need to write this thing?? – Lustre

Answer 1

You can use something like this:

myscript.py

import sys
from collections import defaultdict

d=defaultdict(list)
for k, v in ((k.lstrip('-'), v) for k,v in (a.split('=') for a in sys.argv[1:])):
    d[k].append(v)

print dict(d)

Result:

C:\>python myscript.py  --arg1=1 --arg2=foobar --arg1=4
{'arg1': ['1', '4'], 'arg2': ['foobar']}

Note: the value will always be a list, but I think this is more consistent. If you really want the final dictionary to be

{'arg1': ['1', '4'], 'arg2': 'foobar'}

then you could just run

for k in (k for k in d if len(d[k])==1):
    d[k] = d[k][0]

afterwards.

Resuscitator

Answer 2

Here's an example using argparse, although it's a stretch. I wouldn't call this complete solution, but rather a good start.

class StoreInDict(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        d = getattr(namespace, self.dest)
        for opt in values:
            k,v = opt.split("=", 1)
            k = k.lstrip("-")
            if k in d:
                d[k].append(v)
            else:
                d[k] = [v]
        setattr(namespace, self.dest, d)

# Prevent argparse from trying to distinguish between positional arguments
# and optional arguments. Yes, it's a hack.
p = argparse.ArgumentParser( prefix_chars=' ' )

# Put all arguments in a single list, and process them with the custom action above,
# which convertes each "--key=value" argument to a "(key,value)" tuple and then
# merges it into the given dictionary.
p.add_argument("options", nargs="*", action=StoreInDict, default=dict())

args = p.parse_args("--arg1=1 --arg2=foo --arg1=4".split())
print args.options

Mcdade

Answer 3

Something like this?

import sys

argsdict = {}

for farg in sys.argv:
    if farg.startswith('--'):
        (arg,val) = farg.split("=")
        arg = arg[2:]

        if arg in argsdict:
            argsdict[arg].append(val)
        else:
            argsdict[arg] = [val]     

Slightly different from specified, the value is always a list.

Jessalin

I recommend using a defaultdict for your variable argsdict. This way you get rid of the in test and can append without worrying. It makes your code more concise and expressive and therefore pythonic. – Bayberry

Answer 4

This is what I used today, it accounts for:

--key=val, --key, -key, -key val

def clean_arguments(args):
    ret_args = defaultdict(list)

    for index, k in enumerate(args):
        if index < len(args) - 1:
            a, b = k, args[index+1]
        else:
            a, b = k, None

        new_key = None

        # double hyphen, equals
        if a.startswith('--') and '=' in a:
            new_key, val = a.split('=')

        # double hyphen, no equals
        # single hyphen, no arg
        elif (a.startswith('--') and '=' not in a) or \
                (a.startswith('-') and (not b or b.startswith('-'))):
            val = True

        # single hypen, arg
        elif a.startswith('-') and b and not b.startswith('-'):
            val = b

        else:
            if (b is None) or (a == val):
                continue

            else:
                raise ValueError('Unexpected argument pair: %s, %s' % (a, b))

        # santize the key
        key = (new_key or a).strip(' -')
        ret_args[key].append(val)

    return ret_args

Diplomate

Answer 5

..may I ask why are you trying to rewrite (a bunch of) wheels, when you have:

• http://docs.python.org/library/getopt.html
• http://docs.python.org/library/optparse.html
• http://docs.python.org/library/argparse.html
• ...etc, etc, etc...

?

EDIT:

In reply to your edit, optparse/argparse (the later one only available in >=2.7) are flexible enough to extend to suit your needs, while maintaining a consistent interface (eg. a user expects to be able to use both --arg=value and --arg value, -a value and -avalue, etc.. using a pre-existent library, you don't have to worry about supporting all those syntaxes, etc.).

Heeling

how would you extend argparse, for instance, to parse arbitrary arguments? can't figure that out. – Regeneration

Answer 6

Or something like this) Sorry, if this is stupid, I am a newbie:)

$ python3 Test.py a 1 b 2 c 3

import sys

def somefunc():
    keys = []
    values = []
    input_d = sys.argv[1:]

    for i in range(0, len(input_d)-1, 2):
        keys.append(input_d[i])
        values.append(input_d[i+1])

    d_sys = dict(zip(keys, values))

somefunc()

Clarey

Answer 7

If you really want to write something of your own instead of a proper command-line parsing library, for your input this should work:

dict(map(lambda x: x.lstrip('-').split('='),sys.argv[1:]))

You'll want to add something to catch arguments without an '=' in them.

Roadway

This will not give the desired results. Using the OPs example, arg1 would not map to 1 and 4, but only to 4. – Resuscitator