How does @mock.patch know which parameter to use for each mock object?

Question

Looking at this webpage: http://www.toptal.com/python/an-introduction-to-mocking-in-python -- The author talks about Mocking and Patching in Python and gives a pretty solid "real-world" example. The part that is tripping me up is understanding how the unit testing frame work knows which mock object gets passed to which patch.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import os.path

def rm(filename):
    if os.path.isfile(filename):
        os.remove(filename)

Code sample is pretty easy to understand. Hard-coded dependency on the OS library/module. First checks if the file exists using the os.path.isfile() method and if so, removes it using os.remove()

Test/Mock code is as follows:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from mymodule import rm

import mock
import unittest

class RmTestCase(unittest.TestCase):

    @mock.patch('mymodule.os.path')
    @mock.patch('mymodule.os')
    def test_rm(self, mock_os, mock_path):
        # set up the mock
        mock_path.isfile.return_value = False

        rm("any path")

        # test that the remove call was NOT called.
        self.assertFalse(mock_os.remove.called, "Failed to not remove the file if not present.")

        # make the file 'exist'
        mock_path.isfile.return_value = True

        rm("any path")

        mock_os.remove.assert_called_with("any path")

I guess what's confusing me is that there are 2 @Patch calls and 2 parameters passed in the test. How does the unit testing framework know that mymodule.os.path is patching os.path and that it is mapped to mock_path? And where is mymodule.os.path defined?

(There appears to be a lot of "magic" going on and I'm not following it.)

Gaselier

Answer 1

it goes by the order of the execution of the decorators and that is also the order of the parameters passed on to your test method...

order of decorators execution is shown here: https://thadeusb.com/weblog/2010/08/23/python_multiple_decorators/

When you use patch the way you wrote it, a Mock instance it is automatically created for you and passed as a parameter to your test method. there is another version of it:

@mock.patch("subprocess.check_output", mock.MagicMock(return_value='True'))
def test_mockCheckOutput(self):
    self.assertTrue(subprocess.check_output(args=[])=='True')

in this case you pass your own Mock object and in this example, when you call subprocess.check_output(), it will return 'True'

you could however do:

def test_mockCheckOutput(self):
    m = mock.MagicMock(return_value='True')
    with mock.patch("subprocess.check_output", m):
        self.assertTrue(subprocess.check_output(args=[])=='True')

and in this case you can pass any mock item you want because it will be evaluated during runtime... :)

Albuminate

Ok, that answers half my question. (Actually, I'd say that it goes in the backwards order, but I see what you mean in the page you linked to.) Next: Where is the Mock object created? I see that I can access it within the test (because it's passed in my the decorator.) But what if I wanted to create it outside this method explicitly, say in a Setup() method? How would I do this? – Gaselier

When you use patch the way you wrote it, it is automatically created for you. there is another version of it: @mock.patch("subprocess.check_output", mock.MagicMock(return_value='True')) def test_mockCheckOutput(self): in this case you pass your own Mock object and in this example, when you call subprocess.check_output(), it will return 'True' – Albuminate

Feel free to edit your answer so your code shows up properly formatted (to help anyone else who reads this later.) :) – Gaselier

Ahh, so I could create a m = mock.MagicMock() and then modify it to my heart's content and the insert it with the @mock.patch("mymodule.os.path", m) line. Yes? – Gaselier

you'll have to finish modifying m to your heart's content in the global namespace before it is assigned to the patch and you can't modify it once it was assigned... – Albuminate

Interesting. Ok, thanks for the help. You get the prize for the best answer. :) – Gaselier

@Gaselier I hope my last edit gave you what you want. – Albuminate

Link got broken – Concentre

For the benefit of others who find the link about decorators broken, execution order is bottom up i.e. inside out. In the original question that would be mymodule.os first and then mymodule.os.path second. Decorators are just syntax sugar for functions that wrap (nest your function). In the case of mock, it calls your function with an extra positional arg, which you have to add to your function signature. It's positional,so the name is not important, but the order is, and that ordering comes from the inside-out order of the decorators, hence why mock_os comes before mock_path. – Refit

fixed the broken link – Albuminate

Answer 2

When applying a decorator, it is good to look at it like this

<wrapper1>
    <wrapper2>
        <wrapper3>
           **Your Function**
        </wrapper3>
    </wrapper2>
</wrapper1>

Basically your function is required to interact with the wrappers in this order:

wrapper3-> wrapper2->wrapper1

@wrapper1
@wrapper2
@wrapper3
def your_func(wrapper1.input, wrapper2.input, wrapper3.input):

NOTE wrapper1.input isn't how you would actually reference its input

To answer the second part of your question, how mymodule.os knows to refer to os. When Patching you are effectively intercepting calls to that specific name. When you call os in mymodule you are effectively calling mymodule.os. When patching you must refer to the class that is being mocked by the way it is being called in the actual code, not from the test modules perspective

Operative

Answer 3

The order of your patches should be reversed, as they are applied bottom up. See this comment in the python docs on nested mock arguments:

Note When you nest patch decorators the mocks are passed in to the decorated function in the same order they applied (the normal python order that decorators are applied). This means from the bottom up, so in the example above the mock for module.ClassName1 is passed in first.

Terraqueous