How to parametrize a Pytest fixture?

Question

Consider the following Pytest:

import pytest

class TimeLine(object):
    instances = [0, 1, 2]

@pytest.fixture
def timeline():
    return TimeLine()

def test_timeline(timeline):
    for instance in timeline.instances:
        assert instance % 2 == 0

if __name__ == "__main__":
    pytest.main([__file__])

The test test_timeline uses a Pytest fixture, timeline, which itself has the attribute instances. This attribute is iterated over in the test, so that the test only passes if the assertion holds for every instance in timeline.instances.

What I actually would like to do, however, is to generate 3 tests, 2 of which should pass and 1 of which would fail. I've tried

@pytest.mark.parametrize("instance", timeline.instances)
def test_timeline(timeline):
    assert instance % 2 == 0

but this leads to

AttributeError: 'function' object has no attribute 'instances'

As I understand it, in Pytest fixtures the function 'becomes' its return value, but this seems to not have happened yet at the time the test is parametrized. How can I set up the test in the desired fashion?

Abomasum

Does this answer your question? Pass a parameter to a fixture function – Hickok

Answer 1

From Using fixtures in pytest.mark.parametrize pytest issue, it would appear that it is currently not possible to use fixtures in pytest.mark.parametrize.

Abomasum

this is incorrect, the parametrize mark accepts indirect=True or indirect=['specific', 'named'] to say which arguments will have respective fixtures invoked. – Abelmosk

The proposal referenced in your post has been moved to GitHub at the fillowing URL github.com/pytest-dev/pytest/issues/349. – Strongwilled

As commented by @msudder, look up indirect parameterization – Tumular

Answer 2

This is actually possible via indirect parametrization.

This example does what you want with pytest 3.1.2:

import pytest

class TimeLine:
    def __init__(self, instances):
        self.instances = instances

@pytest.fixture
def timeline(request):
    return TimeLine(request.param)

@pytest.mark.parametrize(
    'timeline',
    ([1, 2, 3], [2, 4, 6], [6, 8, 10]),
    indirect=True
)
def test_timeline(timeline):
    for instance in timeline.instances:
        assert instance % 2 == 0

if __name__ == "__main__":
    pytest.main([__file__])

Also see this similar question.

Flake

Your test fails with FAILED indir-param.py::test_timeline[timeline0] - assert (1 % 2) == 0 :-) (I know it doesn't matter for the answer). – Butcherbird

Yes, it should fail. 1 divided by 2 does not have a remainder (modulus) of 0. – Affusion

Answer 3

instead of indirect parametrization, or my hacky solution below involving inheritance, you can also just use the params argument to @pytest.fixture() -- i think this is the simplest solution maybe?

import pytest

class TimeLine:
    def __init__(self, instances=[0, 0, 0]):
        self.instances = instances


@pytest.fixture(params=[
    [1, 2, 3], [2, 4, 5], [6, 8, 10]
])
def timeline(request):
    return TimeLine(request.param)


def test_timeline(timeline):
    for instance in timeline.instances:
        assert instance % 2 == 0

https://docs.pytest.org/en/latest/how-to/fixtures.html#parametrizing-fixtures

Prankster

Answer 4

This is a confusing topic, because people tend to think that fixtures and parameters are the same thing. They are not and are not even collected in the same phase of the pytest run. By design, parameters are meant to be collected when tests are collected (the list of test nodes is under construction), while fixtures are meant to be executed when test nodes are run (the list of test nodes is fixed and cannot be modified). So fixture contents can not be used to change the number of test nodes - this is the role of parameters. See also my detailed answer here.

This is the reason why your question has no solution "as is": you should put the Timeline instances in a parameter, not in a fixture. Like this:

import pytest

class TimeLine(object):
    instances = [0, 1, 2]

@pytest.fixture(params=TimeLine().instances)
def timeline(request):
    return request.param

def test_timeline(timeline):
    assert timeline % 2 == 0

Kurt Peek's answer mentions another topic that IMHO adds confusion here because it is not directly related to your question. Since it is mentioned, and even accepted as the solution, let me elaborate: indeed in pytest, you can not use a fixture in a @pytest.mark.parametrize. But even if you could that would not change anything.

Since this feature is now available in pytest-cases (I'm the author), you can try yourself. The only way to make your example work even with that feature, is still the same: to remove the list from the fixture. So you end up with:

import pytest
from pytest_cases import parametrize

class TimeLine(object):
    instances = [0, 1, 2]

@pytest.fixture(params=TimeLine().instances)
def timeline(request):
    return request.param

@parametrize("t", [timeline])
def test_timeline(t):
    assert t % 2 == 0

Which is the same than previous example with an extra, probably useless, layer. Note: see also this discussion.

Aleciaaleck

Important aspect here is that it the parameter should be called request in the test method signature. Mentioning that because I thought this name was arbitrary. – Sverige

Answer 5

The use of indirect parametrization works, but I find the need to use request.param as a magic, unnamed variable a little awkard.

Here's a pattern I've used. It's awkward in a different way, arguably, but perhaps you'll prefer it too!

import pytest

class TimeLine:
    def __init__(self, instances):
        self.instances = instances


@pytest.fixture
def instances():
    return [0, 0, 0]


@pytest.fixture
def timeline(instances):
    return TimeLine(instances)


@pytest.mark.parametrize('instances', [
    [1, 2, 3], [2, 4, 5], [6, 8, 10]
])
def test_timeline(timeline):
    for instance in timeline.instances:
        assert instance % 2 == 0

the timeline fixture depends on another fixture called instances, which has a default vale of [0,0,0], but in the actual test, we use parametrize to inject a series of different values for instances.

the advantage as I see it is that everything has a good name, plus you don't need to pass that indirect=True flag.

Prankster

Answer 6

It is possible with vanilla pytest

test_numbers.py

import pytest


@pytest.fixture(params=[1, 2, 3, 4, 5])
def number(request):
    yield request.param


def test_number_is_integer(number):
    assert isinstance(number, int)

Output

test_numbers.py::test_number_is_integer[1] PASSED
test_numbers.py::test_number_is_integer[2] PASSED
test_numbers.py::test_number_is_integer[3] PASSED
test_numbers.py::test_number_is_integer[4] PASSED
test_numbers.py::test_number_is_integer[5] PASSED

Socinian

Works well but in more complex cases (for instance when the fixture alters its request.param), how to retrieve the current parameter value from within the test function? – Benignant

@scūriolus not sure what you mean.. number is the current parameter value. But in anycase you can just yield whatever you need for your case e.g. yield request, request.param – Socinian

Indeed, thanks! (although I prefer to avoid returning/yielding multiple values, implicit tuples, when it's possible) – Benignant

Answer 7

Your parametrize paramset is referenceing the function. Perhaps you mean to reference Timeline.instances, not the fixture function /timeline/:

@pytest.mark.parametrize("instance", Timeline.instances)
def test_func(instance):
    pass

I think you are looking to add a mark xfail to one set of parameters, perhaps with a reason?

def make_my_tests():
    return [0, 2, mark.xfail(1, reason='not even')]

@mark.parametrize('timeline', paramset=make_my_tests(), indirect=True)
def test_func(timeline):
    assert timeline % 2 == 0

Abelmosk

Answer 8

For example, setting the fixture name fruits and indirect=True to @pytest.mark.parametrize() can pass Apple, Orange and Banana to test() indirectly through fruits() fixture as shown below. *My answer explains the difference between indirect=True and indirect=False in @pytest.mark.parametrize():

import pytest

@pytest.fixture
def fruits(request):
    return f'fruits {request.param}'

@pytest.mark.parametrize(
    "fruits", 
    ("Apple", "Orange", "Banana"), 
    indirect=True # Here
)
def test(fruits):
    print(fruits)
    assert True

Output:

$ pytest -q -rP
...                              [100%]
=============== PASSES ================ 
_____________ test[Apple] _____________ 
-------- Captured stdout call --------- 
fruits Apple
____________ test[Orange] _____________ 
-------- Captured stdout call --------- 
fruits Orange
____________ test[Banana] _____________ 
-------- Captured stdout call --------- 
fruits Banana
3 passed in 0.10s

Salpa