Use of *args and **kwargs [duplicate]

Question

So I have difficulty with the concept of *args and **kwargs.

So far I have learned that:

• *args = list of arguments - as positional arguments
• **kwargs = dictionary - whose keys become separate keyword arguments and the values become values of these arguments.

I don't understand what programming task this would be helpful for.

Maybe:

I think to enter lists and dictionaries as arguments of a function AND at the same time as a wildcard, so I can pass ANY argument?

Is there a simple example to explain how *args and **kwargs are used?

Also the tutorial I found used just the "*" and a variable name.

Are *args and **kwargs just placeholders or do you use exactly *args and **kwargs in the code?

Kerouac

Once you get a grasp on these, you'll never want to miss them (especially, if you ever had to deal with PHP's func_*_args()). – Doughnut

The docs are at docs.python.org/faq/programming.html#id13, btw. – Woolpack

Actually, these 2 argument formats can be added to any function declaration as long as they are the last 2. Note the order: explicit args, then *args, then **kwargs. e.g. def foo (arg1, arg2, *args, **kwargs): ... – Oust

(If you are in Javascript there's arguments array that's similar) – Lecythus

In some other languages the *args form would be known as variadic arguments. – Fray

Answer 1

The syntax is the * and **. The names *args and **kwargs are only by convention but there's no hard requirement to use them.

You would use *args when you're not sure how many arguments might be passed to your function, i.e. it allows you pass an arbitrary number of arguments to your function. For example:

>>> def print_everything(*args):
        for count, thing in enumerate(args):
...         print( '{0}. {1}'.format(count, thing))
...
>>> print_everything('apple', 'banana', 'cabbage')
0. apple
1. banana
2. cabbage

Similarly, **kwargs allows you to handle named arguments that you have not defined in advance:

>>> def table_things(**kwargs):
...     for name, value in kwargs.items():
...         print( '{0} = {1}'.format(name, value))
...
>>> table_things(apple = 'fruit', cabbage = 'vegetable')
cabbage = vegetable
apple = fruit

You can use these along with named arguments too. The explicit arguments get values first and then everything else is passed to *args and **kwargs. The named arguments come first in the list. For example:

def table_things(titlestring, **kwargs)

You can also use both in the same function definition but *args must occur before **kwargs.

You can also use the * and ** syntax when calling a function. For example:

>>> def print_three_things(a, b, c):
...     print( 'a = {0}, b = {1}, c = {2}'.format(a,b,c))
...
>>> mylist = ['aardvark', 'baboon', 'cat']
>>> print_three_things(*mylist)
a = aardvark, b = baboon, c = cat

As you can see in this case it takes the list (or tuple) of items and unpacks it. By this it matches them to the arguments in the function. Of course, you could have a * both in the function definition and in the function call.

Youthen

how would you look this up in the python help/documentation? – Petrous

@Alex: Here – Dyedinthewool

It appears impossible to expand a list passed as a positional argument in the middle of a function call as in function_call(arg1,arg2,*expanded_list_args,arg4,arg5). The expanded list may only be followed by keyword arguments I believe. Is there a way to get around that? – Poaceous

@Poaceous yes, and this works with python3, only. But what is really a bit weird: this kinda works on assignments: a, b, *c, d, e = 1, 2, 3, 4, 5, 6 assigns c to [3, 4]. A bit confusing – Mihe

In the example above, mylist is like an array in JS. Our normal method of print_three_things takes 3 args. Passing it to print_three_things(*mylist), the * annotation is more or less like the spreading operator in ES6. Let me know if my consideration is okay or wrong? Thanks – Manchukuo

Thanks for pointing out you can send tuples/lists in *args.. super helpful – Pascal

You can also use both in the same function definition but *args must occur before **kwargs. Why? – Propagandism

Is there any performance difference b/w both of them? – Lothair

There is no example of using two asterisks when calling a function. It would look similar to whats there at the bottom but instead of mylist it would take mydict with names for each entry and use the doubled asterisks when calling the method. – Elmerelmina

Answer 2

One place where the use of *args and **kwargs is quite useful is for subclassing.

class Foo(object):
    def __init__(self, value1, value2):
        # do something with the values
        print value1, value2

class MyFoo(Foo):
    def __init__(self, *args, **kwargs):
        # do something else, don't care about the args
        print 'myfoo'
        super(MyFoo, self).__init__(*args, **kwargs)

This way you can extend the behaviour of the Foo class, without having to know too much about Foo. This can be quite convenient if you are programming to an API which might change. MyFoo just passes all arguments to the Foo class.

Morrie

This answer really only makes sense if you already understand * and **. – Commorant

I don't get it. If your API changes, you still have to change all the places you instantiate the child classes. And if you're changing all those places, then you might as well have the signature of the child class be fixed, instead of hacked together with args and kwargs for no reason. Your reasoning about this not requiring knowledge of Foo is meaningless, because as soon as the set signature of the Foo constructor changes, all your MyFoo instantiation calls will have to change as well. This requires knowledge of Foo and the parameters its constructor requires. – Characharabanc

@ZoranPavlovic An example where this could be used is in a situation where you are providing an add-on for an existing product and want to override/extend some functionality. By using *args and **kwargs this add-on will keep functioning if the underlying product changes. However, instantiating MyFoo happens outside the add-on. Does that make (more) sense? (Having said this: a decorator is a better example of when you can use *args and **kwargs.) – Morrie

how do you extend this for specific arguments of the subclass? – Tereasaterebene

@HananShteingart For example like this: def __init__(self, foo, *args, **kwargs) – Morrie

For this particular example, there's no guarantee that value1 and value2 will get passed. Also, if more than 2 arguments are passed, an exception is raised. To conclude, this is not a good example of argument forwarding. – Vardhamana

... and then one can create objects like f = MyFoo((1,2,3),{'third':4,'fifth':5}) . Here, first tuple (1,2,3) will go as argument to 'args' and dictionary later will go to 'kwargs'. It is important to note that a call like f = MyFoo(1,2,3,4,5) becomes wrong as Python interpreter cannt figure out which argument should be part of tuple and which should belong to kwargs (dictionary). So when using *args and **kwargs together you pretty much have to first form the arguments to be passed. If there was no **kwargs there , second call would have been perfectly fine. – Gainsborough

This is a very limited use case and most people will not use this pattern in the scenario you've described. They'll use it as a hack - I've seen it and it can cause a lot of grief. You are essentially even bypassing the limited checks that the Python interpreter provides. – Whisenant

@kavinyao: you clearly do not know Python and do not tested what you wrote. value2 goes as first element in args, so no exception. – Nikolia

@VIFI: the same to you... The dict will go as the second element of args. To have them as kwargs, you have to write MyFoo(1, 2, 3, third=4, fifth=5). And MyFoo(1,2,3,4,5) will create an args=(1,2,3,4,5) Please, you both, read the docs and test before saying completely wrong affirmations. – Nikolia

@ZoranPavlovic: see this example: https://mcmap.net/q/21791/-can-i-have-a-dictionary-with-same-name-keys As you can see, I don't need to know the firm of dict constructor to create my own map. If the dict API changes, my MultiDict will continue to work. Of course you have to change your code, but also who used dict have to change it. But the code of MultiDict will need no change. – Nikolia

How would you tackle this in C++? – Avatar

Answer 3

Here's an example that uses 3 different types of parameters.

def func(required_arg, *args, **kwargs):
    # required_arg is a positional-only parameter.
    print required_arg

    # args is a tuple of positional arguments,
    # because the parameter name has * prepended.
    if args: # If args is not empty.
        print args

    # kwargs is a dictionary of keyword arguments,
    # because the parameter name has ** prepended.
    if kwargs: # If kwargs is not empty.
        print kwargs

>>> func()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: func() takes at least 1 argument (0 given)

>>> func("required argument")
required argument

>>> func("required argument", 1, 2, '3')
required argument
(1, 2, '3')

>>> func("required argument", 1, 2, '3', keyword1=4, keyword2="foo")
required argument
(1, 2, '3')
{'keyword2': 'foo', 'keyword1': 4}

Hoyle

Answer 4

Here's one of my favorite places to use the ** syntax as in Dave Webb's final example:

mynum = 1000
mystr = 'Hello World!'
print("{mystr} New-style formatting is {mynum}x more fun!".format(**locals()))

I'm not sure if it's terribly fast when compared to just using the names themselves, but it's a lot easier to type!

Merissameristem

hey, this guy invented Python 3.6 f-strings before it was cool – Kimikokimitri

@Kimikokimitri please explain? what are you referring to exactly? didn't the format method already exist even in Python 2? – Trough

@Trough f-strings, not format strings (e.g. f'{mystr} New-style formatting is {mynum}x more fun!') – Merissameristem

@Trough congrats on being one of todays lucky 10k – Merissameristem

@Trough see python.org/dev/peps/pep-0498 – Kimikokimitri

Please make sure you really don't use any string from user, otherwise this will surely create a big security hole. – Tout

@Tout in what contexts? If you're using string formatting for SQL queries then you're either stupid or you haven't learned about parameterized queries yet. – Merissameristem

@WayneWerner nobody was talking about SQL queries (though I understand they play a big role of example in this field). I think I thought about allowing the user to even partially set the string which will be formatted. Very simplified case: string = '{locale_username}: ' + input(); string.format(**locals()). Also, programmers should only .format() the string only once. – Tout

Answer 5

One case where *args and **kwargs are useful is when writing wrapper functions (such as decorators) that need to be able accept arbitrary arguments to pass through to the function being wrapped. For example, a simple decorator that prints the arguments and return value of the function being wrapped:

def mydecorator( f ):
   @functools.wraps( f )
   def wrapper( *args, **kwargs ):
      print "Calling f", args, kwargs
      v = f( *args, **kwargs )
      print "f returned", v
      return v
   return wrapper

Tadich

Answer 6

*args and **kwargs are special-magic features of Python. Think of a function that could have an unknown number of arguments. For example, for whatever reasons, you want to have function that sums an unknown number of numbers (and you don't want to use the built-in sum function). So you write this function:

def sumFunction(*args):
  result = 0
  for x in args:
    result += x
  return result

and use it like: sumFunction(3,4,6,3,6,8,9).

**kwargs has a diffrent function. With **kwargs you can give arbitrary keyword arguments to a function and you can access them as a dictonary.

def someFunction(**kwargs):
  if 'text' in kwargs:
    print kwargs['text']

Calling someFunction(text="foo") will print foo.

Misgive

Finally, a simple-enough use case that can help me understand *args and **kwargs XD. Thank you sir. – Isaiah

Answer 7

Just imagine you have a function but you don't want to restrict the number of parameter it takes. Example:

>>> import operator
>>> def multiply(*args):
...  return reduce(operator.mul, args)

Then you use this function like:

>>> multiply(1,2,3)
6

or

>>> numbers = [1,2,3]
>>> multiply(*numbers)
6

Preposition

Answer 8

The names *args and **kwargs or **kw are purely by convention. It makes it easier for us to read each other's code

One place it is handy is when using the struct module

struct.unpack() returns a tuple whereas struct.pack() uses a variable number of arguments. When manipulating data it is convenient to be able to pass a tuple to struck.pack() eg.

tuple_of_data = struct.unpack(format_str, data)
# ... manipulate the data
new_data = struct.pack(format_str, *tuple_of_data)

without this ability you would be forced to write

new_data = struct.pack(format_str, tuple_of_data[0], tuple_of_data[1], tuple_of_data[2],...)

which also means the if the format_str changes and the size of the tuple changes, I'll have to go back and edit that really long line

Hautegaronne

Answer 9

Note that *args/**kwargs is part of function-calling syntax, and not really an operator. This has a particular side effect that I ran into, which is that you can't use *args expansion with the print statement, since print is not a function.

This seems reasonable:

def myprint(*args):
    print *args

Unfortunately it doesn't compile (syntax error).

This compiles:

def myprint(*args):
    print args

But prints the arguments as a tuple, which isn't what we want.

This is the solution I settled on:

def myprint(*args):
    for arg in args:
        print arg,
    print

Horton

Of course there's always from __future__ import print_function :) – Guillaume

print is a function in Python3:) docs.python.org/3.0/whatsnew/3.0.html – Underhanded

Answer 10

These parameters are typically used for proxy functions, so the proxy can pass any input parameter to the target function.

def foo(bar=2, baz=5):
    print bar, baz

def proxy(x, *args, **kwargs): # reqire parameter x and accept any number of additional arguments
    print x
    foo(*args, **kwargs) # applies the "non-x" parameter to foo

proxy(23, 5, baz='foo') # calls foo with bar=5 and baz=foo
proxy(6)# calls foo with its default arguments
proxy(7, bar='asdas') # calls foo with bar='asdas' and leave baz default argument

But since these parameters hide the actual parameter names, it is better to avoid them.

Gerfalcon

Answer 11

You can have a look at python docs (docs.python.org in the FAQ), but more specifically for a good explanation the mysterious miss args and mister kwargs (courtesy of archive.org) (the original, dead link is here).

In a nutshell, both are used when optional parameters to a function or method are used. As Dave says, *args is used when you don't know how many arguments may be passed, and **kwargs when you want to handle parameters specified by name and value as in:

myfunction(myarg=1)

Epistemology

Another nice tutorial: freepythontips.wordpress.com/2013/08/04/… – Underhanded

'Here is a link, and also what that other person said' is not a useful answer. About the only thing this answer actually says on its own is to imply that **kwargs is required to use named arguments, which is false. It's only needed to handle arbitrary named arguments. – Chavez