Execute function on all possible combinations of parameters

Question

I have sets of values that I want to apply as parameters to a function:

params = {
    'a': [1, 2, 3],
    'b': [5, 6, 7],
    'x': [None, 'eleven', 'f'],
    # et cetera
}

I want to run myfunc() with all possible combinations, so myfunc(a=1, b=5, x=None ...), myfunc(a=2, b=5, x=None ...) ... myfunc(a=3, b=7, x='f' ...). Is there something (for example in itertools) that can help? I thought about using itertools.product() but that doesn't keep the names of the parameters and just gives me tuples of the combinations.

Fingerprint

Answer 1

You can use itertools.product to get all combinations of arguments:

>>> import itertools
>>> for xs in itertools.product([1,2], [5,6], ['eleven', 'f']):
...     print(xs)
... 
(1, 5, 'eleven')
(1, 5, 'f')
(1, 6, 'eleven')
(1, 6, 'f')
(2, 5, 'eleven')
(2, 5, 'f')
(2, 6, 'eleven')
(2, 6, 'f')

With Argument list unpacking, you can call myfunc with all combinations of keyword arguments:

params = {
    'a': [1, 2, 3],
    'b': [5, 6, 7],
    'x': [None, 'eleven', 'f'],
}

def myfunc(**args):
    print(args)

import itertools
keys = list(params)
for values in itertools.product(*map(params.get, keys)):
    myfunc(**dict(zip(keys, values)))

output:

{'a': 1, 'x': None, 'b': 5}
{'a': 1, 'x': None, 'b': 6}
{'a': 1, 'x': None, 'b': 7}
{'a': 1, 'x': 'eleven', 'b': 5}
{'a': 1, 'x': 'eleven', 'b': 6}
{'a': 1, 'x': 'eleven', 'b': 7}
{'a': 1, 'x': 'f', 'b': 5}
...

Mvd

That's very neat! I'm concerned about the order being off though, since dicts are unordered (right?). That could mean that itertools.product() returns the parameters in a different order than you have the keys, which would lead to a mismatch. – Fingerprint

@Bluefire, myfunc(a=1, b=2, c=3) , myfunc(b=2, c=3, a=1) both okay. – Mvd

map(params.get, keys) is a needlessly verbose way of writing params.values() here; it's guaranteed that .keys() and .values() will align unless the dictionary is modified. – Sybarite

@Mvd I understand that, my concern is a mismatch where you pass the value of a into b and vice versa. – Fingerprint

@DSM, I couldn't be sure. Also I wanted the code works both for Python 2 and Python 3. – Mvd

@Bluefire, That's why I separated keys = list(params) and get values by using map(params.get, keys) (to match keys orders with values orders) – Mvd

@falsetru: nothing you're doing behaves differently between 2 and 3. – Sybarite

@Bluefire, I could go agressive (?) like other answer or DSM's comment. But I'm not sure whether it's guaranteed or just CPython implementation details. – Mvd

@DSM, My fault. I thought dict.keys() in Python 3.x returns a kind of iterator which is consumed once it's being iterated. – Mvd

Answer 2

Ordering of .keys and .values are guaranteed across all Python versions (unless dict is altered which does not happen here), so this might be a bit trivial:

from itertools import product

for vals in product(*params.values()):
    myfunc(**dict(zip(params, vals)))

You can find the gurantee in the docs:

If keys, values and items views are iterated over with no intervening modifications to the dictionary, the order of items will directly correspond.

---

Demo:

for vals in product(*params.values()):
    print(dict(zip(params, vals)))

---

{'a': 1, 'x': None, 'b': 5}
{'a': 1, 'x': None, 'b': 6}
{'a': 1, 'x': None, 'b': 7}
{'a': 1, 'x': 'eleven', 'b': 5}
{'a': 1, 'x': 'eleven', 'b': 6}
{'a': 1, 'x': 'eleven', 'b': 7}
{'a': 1, 'x': 'f', 'b': 5}
{'a': 1, 'x': 'f', 'b': 6}
{'a': 1, 'x': 'f', 'b': 7}
...

Hartford

Maybe add a link to the guarantee? – Sybarite

@Sybarite Yes sir, included. Thanks. – Hartford

Answer 3

I developed combu which is that solution.

1. Install combu

pip install combu

2. Use combu

# Case 1: Directly call
import combu

for res, param in combu.execute(myfunc, params):
    print(res, params)

# Case 2: Use class
from combu import Combu

comb = Combu(myfunc)

for res, param in comb.execute(params):
    print(res, params)

Instrumentalism