Proper way to use **kwargs in Python

Question

What is the proper way to use **kwargs in Python when it comes to default values?

kwargs returns a dictionary, but what is the best way to set default values, or is there one? Should I just access it as a dictionary? Use get function?

class ExampleClass:
    def __init__(self, **kwargs):
        self.val = kwargs['val']
        self.val2 = kwargs.get('val2')

People do it different ways in code that I've seen and it's hard to know what to use.

Amongst

Answer 1

You can pass a default value to get() for keys that are not in the dictionary:

self.val2 = kwargs.get('val2',"default value")

However, if you plan on using a particular argument with a particular default value, why not use named arguments in the first place?

def __init__(self, val2="default value", **kwargs):

Dewayne

I like to use positional arguments only for required arguments, and kwargs for arguments that may or may not be specified, but it is helpful to have a default value. kwargs is nice because you can submit your args in any order you choose. Positional arguments don't give you that freedom. – Amongst

You can pass named arguments in any order you like. You only need to adher to the positions if you don't use the names -- which in the case of kwargs, you have to. Rather, using named arguments as opposed to kwargs gives you the additional freedom of not using the names -- then, however, you have to keep the order. – Dewayne

@Kekoa: You can always submit named arguments in any order you choose. You don't have to use **kwargs to get this flexibility. – Monochromatism

pylint flags it as bad form to use kwargs in __init__(). Can someone explain why this is a lint-worthy transgression? – Sibell

Does a default value for a named argument in the __init__() method not override the same attribute set in the class's namespace (defined before any of the methods)? It doesn't seem to for me. – Undertaker

@Sibell Probably because a simple self.__dict__update(**kwargs) can redefine methods and cause other bugs – Apulia

@Sibell I'd suggest a read of PEP20 - Zen of Python. Barring that, perhaps some articles on OOP? Python's dynamic nature shouldn't be used as a crutch for bad OO design. The relevant line, btw, from PEP20: "Explicit is better than implicit." – Finzer

the self dict update could be wrapped in something that checks to see if it is redefining methods – Corporator

@Dewayne ideally i would like to use named args with default values, because they can be passed in any order, but I don't want to give my caller the freedom to not name them (i.e. to pass them positionally) for readability reasons. I wish python had a way to use named args and force the caller to use the names. Without such a feature in python, I am stuck using kwargs in order to force my callers to use the names :( Thoughts? – Gonidium

Answer 2

While most answers are saying that, e.g.,

def f(**kwargs):
    foo = kwargs.pop('foo')
    bar = kwargs.pop('bar')
    ...etc...

is "the same as"

def f(foo=None, bar=None, **kwargs):
    ...etc...

this is not true. In the latter case, f can be called as f(23, 42), while the former case accepts named arguments only -- no positional calls. Often you want to allow the caller maximum flexibility and therefore the second form, as most answers assert, is preferable: but that is not always the case. When you accept many optional parameters of which typically only a few are passed, it may be an excellent idea (avoiding accidents and unreadable code at your call sites!) to force the use of named arguments -- threading.Thread is an example. The first form is how you implement that in Python 2.

The idiom is so important that in Python 3 it now has special supporting syntax: every argument after a single * in the def signature is keyword-only, that is, cannot be passed as a positional argument, but only as a named one. So in Python 3 you could code the above as:

def f(*, foo=None, bar=None, **kwargs):
    ...etc...

Indeed, in Python 3 you can even have keyword-only arguments that aren't optional (ones without a default value).

However, Python 2 still has long years of productive life ahead, so it's better to not forget the techniques and idioms that let you implement in Python 2 important design ideas that are directly supported in the language in Python 3!

Clytemnestra

@Alex Martelli: I haven't found a single answer that claims kwargs to be identical to named arguments, let alone superior. But good discourse to the Py3k changes, so +1 – Dewayne

@Alex Martelli : thanks a lot for your answer, it learned me that python 3 allows mandatory keyword-arguments, which the lack of was often resulting in insatisfying architecture in my code and functions. – Meara

Answer 3

I suggest something like this

def testFunc( **kwargs ):
    options = {
            'option1' : 'default_value1',
            'option2' : 'default_value2',
            'option3' : 'default_value3', }

    options.update(kwargs)
    print options

testFunc( option1='new_value1', option3='new_value3' )
# {'option2': 'default_value2', 'option3': 'new_value3', 'option1': 'new_value1'}

testFunc( option2='new_value2' )
# {'option1': 'default_value1', 'option3': 'default_value3', 'option2': 'new_value2'}

And then use the values any way you want

dictionaryA.update(dictionaryB) adds the contents of dictionaryB to dictionaryA overwriting any duplicate keys.

Haukom

Thanks @AbhinavGupta! Exactly what I was looking for. Just added for key in options: self.__setattr__(key, options[key]) and I'm good to go. : ) – Aver

Answer 4

You'd do

self.attribute = kwargs.pop('name', default_value)

or

self.attribute = kwargs.get('name', default_value)

If you use pop, then you can check if there are any spurious values sent, and take the appropriate action (if any).

Spiritless

Can you clarify what you mean by suggesting .pop would help you “check if there are any spurious values sent”? – Dispassion

@Alan H.: if there's anything left over in kwargs after all the popping is done, then you've got spurious values. – Spiritless

@VinaySajip: Ok, that is a great point on .pop "vs" .get, but I still don't see why pop is preferable over named arguments, besides forcing the caller not to use positional parameters. – Catechu

@MestreLion: It depends on how many keyword arguments your API allows for. I don't claim that my suggestion is better than named arguments, but Python allows you to capture unnamed arguments in kwargs for a reason. – Spiritless

So, just checking. Does pop return a dictionary value if the key exists and if not it returns the default_value passed? And removes that key afterwards? – Nevlin

Answer 5

Using **kwargs and default values is easy. Sometimes, however, you shouldn't be using **kwargs in the first place.

In this case, we're not really making best use of **kwargs.

class ExampleClass( object ):
    def __init__(self, **kwargs):
        self.val = kwargs.get('val',"default1")
        self.val2 = kwargs.get('val2',"default2")

The above is a "why bother?" declaration. It is the same as

class ExampleClass( object ):
    def __init__(self, val="default1", val2="default2"):
        self.val = val
        self.val2 = val2

When you're using **kwargs, you mean that a keyword is not just optional, but conditional. There are more complex rules than simple default values.

When you're using **kwargs, you usually mean something more like the following, where simple defaults don't apply.

class ExampleClass( object ):
    def __init__(self, **kwargs):
        self.val = "default1"
        self.val2 = "default2"
        if "val" in kwargs:
            self.val = kwargs["val"]
            self.val2 = 2*self.val
        elif "val2" in kwargs:
            self.val2 = kwargs["val2"]
            self.val = self.val2 / 2
        else:
            raise TypeError( "must provide val= or val2= parameter values" )

Monochromatism

I like that little brainteaser! I kept thinking, "But you could just use get or pop with--oh, they're co-dependent..." – Receptor

Answer 6

Since **kwargs is used when the number of arguments is unknown, why not doing this?

class Exampleclass(object):
  def __init__(self, **kwargs):
    for k in kwargs.keys():
       if k in [acceptable_keys_list]:
          self.__setattr__(k, kwargs[k])

Aviculture

yes, this is elegant and powerful... not too sure about the square brackets around acceptable_keys_list though: I'd make this a tuple or a list and then drop those brackets in the "if" statement – Soonsooner

I slightly modified this for cases when all keys are expected: #1099049 – Paradox

Answer 7

Here's another approach:

def my_func(arg1, arg2, arg3):
    ... so something ...

kwargs = {'arg1': 'Value One', 'arg2': 'Value Two', 'arg3': 'Value Three'}
# Now you can call the function with kwargs like this:

my_func(**kwargs)

Interlope

Used a lot in Django CBVs (e.g. get_form_kwargs()). ccbv.co.uk/projects/Django/1.5/django.views.generic.edit/… – Receptor

The get_form() method shows how to obtain keyword arguments extensively by deferring to another method (get_form_kwargs as mentioned above). It instantiates the form as follows: form_class(**self.get_form_kwargs()). – Receptor

It's then easy to override get_form_kwargs() in a subclass view and add/remove kwargs based on specific logic. But that's for a Django tutorial. – Receptor

Answer 8

I think the proper way to use **kwargs in Python when it comes to default values is to use the dictionary method setdefault, as given below:

class ExampleClass:
    def __init__(self, **kwargs):
        kwargs.setdefault('val', value1)
        kwargs.setdefault('val2', value2)

In this way, if a user passes 'val' or 'val2' in the keyword args, they will be used; otherwise, the default values that have been set will be used.

Callimachus

Answer 9

Following up on @srhegde suggestion of using setattr:

class ExampleClass(object):
    __acceptable_keys_list = ['foo', 'bar']

    def __init__(self, **kwargs):
        [self.__setattr__(key, kwargs.get(key)) for key in self.__acceptable_keys_list]

This variant is useful when the class is expected to have all of the items in our acceptable list.

Paradox

That's not a use case for a list comprehension, you should use a for loop in your init method. – Hungarian

Answer 10

You could do something like this

class ExampleClass:
    def __init__(self, **kwargs):
        arguments = {'val':1, 'val2':2}
        arguments.update(kwargs)
        self.val = arguments['val']
        self.val2 = arguments['val2']

Gurevich

Answer 11

If you want to combine this with *args you have to keep *args and **kwargs at the end of the definition.

So:

def method(foo, bar=None, *args, **kwargs):
    do_something_with(foo, bar)
    some_other_function(*args, **kwargs)

Davin

Answer 12

Another simple solution for processing unknown or multiple arguments can be:

class ExampleClass(object):

    def __init__(self, x, y, **kwargs):
      self.x = x
      self.y = y
      self.attributes = kwargs

    def SomeFunction(self):
      if 'something' in self.attributes:
        dosomething()

Humblebee

Answer 13

**kwargs gives the freedom to add any number of keyword arguments. One may have a list of keys for which he can set default values. But setting default values for an indefinite number of keys seems unnecessary. Finally, it may be important to have the keys as instance attributes. So, I would do this as follows:

class Person(object):
listed_keys = ['name', 'age']

def __init__(self, **kwargs):
    _dict = {}
    # Set default values for listed keys
    for item in self.listed_keys: 
        _dict[item] = 'default'
    # Update the dictionary with all kwargs
    _dict.update(kwargs)

    # Have the keys of kwargs as instance attributes
    self.__dict__.update(_dict)

Celom

Answer 14

@AbhinavGupta and @Steef suggested using update(), which I found very helpful for processing large argument lists:

args.update(kwargs)

What if we want to check that the user hasn't passed any spurious/unsupported arguments? @VinaySajip pointed out that pop() can be used to iteratively process the list of arguments. Then, any leftover arguments are spurious. Nice.

Here's another possible way to do this, which keeps the simple syntax of using update():

# kwargs = dictionary of user-supplied arguments
# args = dictionary containing default arguments

# Check that user hasn't given spurious arguments
unknown_args = user_args.keys() - default_args.keys()
if unknown_args:
    raise TypeError('Unknown arguments: {}'.format(unknown_args))

# Update args to contain user-supplied arguments
args.update(kwargs)

unknown_args is a set containing the names of arguments that don't occur in the defaults.

Astred