Why does mock patch only work when running specific test and not whole test suite?

Question

I'm using Django and Pytest specifically to run the test suite and am trying to test that a specific form shows up with expected data when a user hits the site (integration test).

This particular view uses a stored procedure, which I am mocking since the test would never have access to that.

My test code looks like this:

#test_integrations.py

from my_app.tests.data_setup import setup_data, setup_sb7_data
from unittest.mock import patch

...

# Setup to use a non-headless browser so we can see whats happening for debugging
@pytest.mark.usefixtures("standard_browser")
class SeniorPageTestCase(StaticLiveServerTestCase):
    """
    These tests surround the senior form
    """

    @classmethod
    def setUpClass(cls):
        cls.host = socket.gethostbyname(socket.gethostname())
        super(SeniorPageTestCase, cls).setUpClass()

    def setUp(self):
        # setup the dummy data - this works fine
        basic_setup(self)
        # setup the 'results'
        self.sb7_mock_data = setup_sb7_data(self)

    @patch("my_app.utils.get_employee_sb7_data")
    def test_senior_form_displays(self, mock_sb7_get):
        # login the dummy user we created
        login_user(self, "futureuser")
        # setup the results
        mock_sb7_get.return_value = self.sb7_mock_data
        # hit the page for the form
        self.browser.get(self.live_server_url + "/my_app/senior")
        form_id = "SeniorForm"
        # assert that the form displays on the page
        self.assertTrue(self.browser.find_element_by_id(form_id))

# utils.py

from django.conf import settings
from django.db import connections


def get_employee_sb7_data(db_name, user_number, window):
    """
    Executes the stored procedure for getting employee data

    Args:
        user_number: Takes the user_number
        db (db connection): Takes a string of the DB to connect to

    Returns:

    """
    cursor = connections[db_name].cursor()
    cursor.execute(
        'exec sp_sb7 %s, "%s"' % (user_number, window.senior_close)
    )
    columns = [col[0] for col in cursor.description]
    results = [dict(zip(columns, row)) for row in cursor.fetchall()]
    return results

# views.py

from myapp.utils import (
    get_employee_sb7_data,
)

...

###### Senior ######
@login_required
@group_required("user_senior")
def senior(request):

    # Additional Logic / Getting Other Models here

    # Execute stored procedure to get data for user
    user_number = request.user.user_no
    results = get_employee_sb7_data("production_db", user_number, window)
    if not results:
        return render(request, "users/senior_not_required.html")

    # Additional view stuff

    return render(
        request,
        "users/senior.html",
        {
            "data": data,
            "form": form,
            "results": results,
        },
    )

If I run this test itself with:

pytest my_app/tests/test_integrations.py::SeniorPageTestCase

The tests pass without issue. The browser shows up - the form shows up with the dummy data as we would expect and it all works.

However, if I run:

pytest my_app

All other tests run and pass - but all the tests in this class fail because it's not patching the function.

It tries to call the actual stored procedure (which fails because it's not on the production server yet) and it fails.

Why would it patch correctly when I call that TestCase specifically - but not patch correctly when I just run pytest on the app or project level?

I'm at a loss and not sure how to debug this very well. Any help is appreciated

Salesgirl

Are you sure that's the cause? @pytest.mark.usefixtures("standard_browser") is a class attribute. So it gets reused by all tests. Could that cause the same symptoms? The same applies to setUp() without a tearDown() that does any necessary cleanup. – Tempo

The tests work when running individually - just not when running at the app or project level. That fixture just opens a browser and does nothing else - closing when the test finishes - so I don't believe that has any impact. It works when tested by itself. Additionally, the setUp() sets up the data - which is automatically truncated between tests by Django - so there is nothing left to be cleaned up. The error in the logs specifically shows it not running that fixture and instead trying to run the stored procedure - so the patch isn't being applied when the whole suite is ran. – Salesgirl

Is this method imported from a different place when you run the full suite? Reason – Tempo

Not that I'm aware of unless pytest does something different when running the full suite vs. just particular class of tests (which I don't believe it does); I've added the utility function and the view which calls that function to the question for easier understanding of what is happening. – Salesgirl

Well, what it does show me is that there 2 different symbols: get_employee_sb7_data and my_app.utils.get_employee_sb7_data. They're from the same place, but different names. So the difference would be that views.py gets imported before test_integrations.py and is somehow still in scope, but when run separately, it first imports the test case (which it does). The way to test the theory is to add views.py as import in the test case and then both should exhibit the same (not working) behaviour. – Tempo

You're correct, if I add import views.py to my test_integrations.py file - the same error occurs when I run the that test class specifically (as well as when I run the whole suite). Am I mocking/patching it completely wrong in this case then? – Salesgirl

Answer 1

So what's happening is that your views are imported before you're patching.

Let's first see the working case:

1. pytest imports the test_integrations file
2. the test is executed and patch decorator's inner function is run
3. there is no import of the utils yet and so patch imports and replaces the function
4. test body is executed, which passes a url to the test client
5. the test client imports the resolver and in turn it imports the views, which imports the utils.
6. Since the utils are already patched, everything works fine

If another test case runs first, that also imports the same views, then that import wins and patch cannot replace the import.

Your solution is to reference the same symbol. So in test_integrations.py:

@patch("myapp.views.get_employee_sb7_data")

Tempo

I added the import at the top of the file (test_integrations.py) as you suggested, then changed the decorator on the functions of that test class to be @patch("get_employee_sb7_data") but recieved the error: TypeError: Need a valid target to patch. You supplied: 'get_employee_sb7_data' – Salesgirl

My bad. Updated. The symbol exists as absolute import, but as the "name", not the target. – Tempo

Thank you for the great explanation as well as correction as to what needs to happen. I now have a greater understanding of what is going on (although Mocks are still confusing a little to me - the documentation seems as clear as mud! ha!). I appreciate it! – Salesgirl