I have a function with a decorator that I'm trying test with the help of the Python Mock library. I'd like to use mock.patch to replace the real decorator with a mock 'bypass' decorator which just calls the function.
What I can't figure out is how to apply the patch before the real decorator wraps the function. I've tried a few different variations on the patch target and reordering the patch and import statements, but without success. Any ideas?
Decorators are applied at function definition time. For most functions, this is when the module is loaded. (Functions that are defined in other functions have the decorator applied each time the enclosing function is called.)
So if you want to monkey-patch a decorator, what you need to do is:
module.decorator = mymockdecoratorIf the module that contains the decorator also contains functions that use it, those are already decorated by the time you can see them, and you're probably S.O.L.
Edit to reflect changes to Python since I originally wrote this: If the decorator uses functools.wraps() and the version of Python is new enough, you may be able to dig out the original function using the __wrapped__ attribute and re-decorate it, but this is by no means guaranteed, and the decorator you want to replace also may not be the only decorator applied.
reload function in order to regenerate the python binary code docs.python.org/2/library/functions.html#reload and monkeypatch your decorator –
Aerodonetics __init__. That ensured that the patch was loaded before any test file. We have an isolated tests folder so the strategy works for us, but this may not work for every folder layout. –
Exempt while hasattr(target_func,'__wrapped__'): target_func=target_func.__wrapped__ to dig out the core function and test it. –
Photosensitive __wrapped__ , i think it is the most elegant way to do this and also the most 'local' way, meaning you just choose not to apply it a the function level –
Wensleydale importlib.reload (of the module which uses the decorator, after having injected a mock decorator). But don't you then need to have a teardown function which restores the status quo ante? Aren't you otherwise stuck with a mock decorator for all eternity (or at least until the present test suite finishes)? A true mock/patch would end when the test ends, wouldn't it? –
Understood inspect.unwrap() recursively looks for __wrapped__ values and returns the unwrapped function. –
Didst It should be noted that several of the answers here will patch the decorator for the entire test session rather than a single test instance; which may be undesirable. Here's how to patch a decorator that only persists through a single test.
Our unit to be tested with the undesired decorator:
# app/uut.py
from app.decorators import func_decor
@func_decor
def unit_to_be_tested():
# Do stuff
pass
From decorators module:
# app/decorators.py
def func_decor(func):
def inner(*args, **kwargs):
print "Do stuff we don't want in our test"
return func(*args, **kwargs)
return inner
By the time our test gets collected during a test run, the undesired decorator has already been applied to our unit under test (because that happens at import time). In order to get rid of that, we'll need to manually replace the decorator in the decorator's module and then re-import the module containing our UUT.
Our test module:
# test_uut.py
from unittest import TestCase
from app import uut # Module with our thing to test
from app import decorators # Module with the decorator we need to replace
from importlib import reload # Library to help us reload our UUT module
from mock import patch
class TestUUT(TestCase):
def setUp(self):
# Do cleanup first so it is ready if an exception is raised
def kill_patches(): # Create a cleanup callback that undoes our patches
patch.stopall() # Stops all patches started with start()
reload(uut) # Reload our UUT module which restores the original decorator
self.addCleanup(kill_patches) # We want to make sure this is run so we do this in addCleanup instead of tearDown
# Now patch the decorator where the decorator is being imported from
patch('app.decorators.func_decor', lambda x: x).start() # The lambda makes our decorator into a pass-thru. Also, don't forget to call start()
# HINT: if you're patching a decor with params use something like:
# lambda *x, **y: lambda f: f
reload(uut) # Reloads the uut.py module which applies our patched decorator
The cleanup callback, kill_patches, restores the original decorator and re-applies it to the unit we were testing. This way, our patch only persists through a single test rather than the entire session -- which is exactly how any other patch should behave. Also, since the clean up calls patch.stopall(), we can start any other patches in the setUp() we need and they will get cleaned up all in one place.
The important thing to understand about this method is how the reloading will affect things. If a module takes too long or has logic that runs on import, you may just need to shrug and test the decorator as part of the unit. :( Hopefully your code is better written than that. Right?
If one doesn't care if the patch is applied to the whole test session, the easiest way to do that is right at the top of the test file:
# test_uut.py
from mock import patch
patch('app.decorators.func_decor', lambda x: x).start() # MUST BE BEFORE THE UUT GETS IMPORTED ANYWHERE!
from app import uut
Make sure to patch the file with the decorator rather than the local scope of the UUT and to start the patch before importing the unit with the decorator.
Interestingly, even if the patch is stopped, all the files that already imported will still have the patch applied to the decorator, which is the reverse of the situation we started with. Be aware that this method will patch any other files in the test run that are imported afterwards -- even if they don't declare a patch themselves.
lambda *x, **y: lambda f: f –
Perinephrium Decorators are applied at function definition time. For most functions, this is when the module is loaded. (Functions that are defined in other functions have the decorator applied each time the enclosing function is called.)
So if you want to monkey-patch a decorator, what you need to do is:
module.decorator = mymockdecoratorIf the module that contains the decorator also contains functions that use it, those are already decorated by the time you can see them, and you're probably S.O.L.
Edit to reflect changes to Python since I originally wrote this: If the decorator uses functools.wraps() and the version of Python is new enough, you may be able to dig out the original function using the __wrapped__ attribute and re-decorate it, but this is by no means guaranteed, and the decorator you want to replace also may not be the only decorator applied.
reload function in order to regenerate the python binary code docs.python.org/2/library/functions.html#reload and monkeypatch your decorator –
Aerodonetics __init__. That ensured that the patch was loaded before any test file. We have an isolated tests folder so the strategy works for us, but this may not work for every folder layout. –
Exempt while hasattr(target_func,'__wrapped__'): target_func=target_func.__wrapped__ to dig out the core function and test it. –
Photosensitive __wrapped__ , i think it is the most elegant way to do this and also the most 'local' way, meaning you just choose not to apply it a the function level –
Wensleydale importlib.reload (of the module which uses the decorator, after having injected a mock decorator). But don't you then need to have a teardown function which restores the status quo ante? Aren't you otherwise stuck with a mock decorator for all eternity (or at least until the present test suite finishes)? A true mock/patch would end when the test ends, wouldn't it? –
Understood inspect.unwrap() recursively looks for __wrapped__ values and returns the unwrapped function. –
Didst When I first ran across this problem, I use to rack my brain for hours. I found a much easier way to handle this.
This will fully bypass the decorator, like the target wasn't even decorated in the first place.
This is broken down into two parts. I suggest reading the following article.
http://alexmarandon.com/articles/python_mock_gotchas/
Two Gotchas that I kept running into:
1.) Mock the Decorator before the import of your function/module.
The decorators and functions are defined at the time the module is loaded. If you do not mock before import, it will disregard the mock. After load, you have to do a weird mock.patch.object, which gets even more frustrating.
2.) Make sure you are mocking the correct path to the decorator.
Remember that the patch of the decorator you are mocking is based on how your module loads the decorator, not how your test loads the decorator. This is why I suggest always using full paths for imports. This makes things a lot easier for testing.
Steps:
1.) The Mock function:
from functools import wraps
def mock_decorator(*args, **kwargs):
def decorator(f):
@wraps(f)
def decorated_function(*args, **kwargs):
return f(*args, **kwargs)
return decorated_function
return decorator
2.) Mocking the decorator:
2a.) Path inside with.
with mock.patch('path.to.my.decorator', mock_decorator):
from mymodule import myfunction
2b.) Patch at top of file, or in TestCase.setUp
mock.patch('path.to.my.decorator', mock_decorator).start()
Either of these ways will allow you to import your function at anytime within the TestCase or its method/test cases.
from mymodule import myfunction
2.) Use a separate function as a side effect of the mock.patch.
Now you can use mock_decorator for each decorator you want to mock. You will have to mock each decorator separately, so watch out for the ones you miss.
We tried to mock a decorator that sometimes gets another parameter like a string, and some times not, eg.:
@myDecorator('my-str')
def function()
OR
@myDecorator
def function()
Thanks to one of the answers above, we wrote a mock function and patch the decorator with this mock function:
from mock import patch
def mock_decorator(f):
def decorated_function(g):
return g
if callable(f): # if no other parameter, just return the decorated function
return decorated_function(f)
return decorated_function # if there is a parametr (eg. string), ignore it and return the decorated function
patch('path.to.myDecorator', mock_decorator).start()
from mymodule import myfunction
Note that this example is good for a decorator that doesn't run the decorated function, only do some stuff before the actual run. In case the decorator also runs the decorated function, and hence it needs to transfer the function's parameters, the mock_decorator function has to be a bit different.
Hope this will help others...
The following worked for me:
It worked like a charm.
To patch a decorator, you need to either import or reload the module which uses that decorator after patching it OR redefine the module's reference to that decorator altogether.
Decorators are applied at the time that a module is imported. This is why if you imported a module which uses a decorator you want to patch at the top of your file and attempt it to patch it later on without reloading it, the patch would have no effect.
Here is an example of the first way mentioned of doing this - reloading a module after patching a decorator it uses:
import moduleA
...
# 1. patch the decorator
@patch('decoratorWhichIsUsedInModuleA', examplePatchValue)
def setUp(self)
# 2. reload the module which uses the decorator
reload(moduleA)
def testFunctionA(self):
# 3. tests...
assert(moduleA.functionA()...
Helpful References:
This may sound a bit odd but one can patch sys.path, with a copy of itself, and perform an import within the scope of the test function. The following code shows the concept.
from unittest.mock import patch
import sys
@patch('sys.modules', sys.modules.copy())
def testImport():
oldkeys = set(sys.modules.keys())
import MODULE
newkeys = set(sys.modules.keys())
print((newkeys)-(oldkeys))
oldkeys = set(sys.modules.keys())
testImport() -> ("MODULE") # Set contains MODULE
newkeys = set(sys.modules.keys())
print((newkeys)-(oldkeys)) -> set() # An empty set
MODULE may then be substituted with the module you are testing. (This works in Python 3.6 with MODULE substituted with xml for example)
For your case, let's say the decorator function resides in the module pretty and the decorated function resides in present, then you would patch pretty.decorator using the mock machinery and substitute MODULE with present. Something like the following should work (Untested).
class TestDecorator(unittest.TestCase) : ...
@patch(`pretty.decorator`, decorator)
@patch(`sys.path`, sys.path.copy())
def testFunction(self, decorator) :
import present
...
This works by providing a "clean" sys.path for each test function, using a copy of the current sys.path of the test module. This copy is made when the module is first parsed ensuring a consistent sys.path for all the tests.
There are a few implications, however. If the testing framework runs multiple test modules under the same python session any test module that imports MODULE globally breaks any test module that imports it locally. This forces one to perform the import locally everywhere. If the framework runs each test module under a separate python session then this should work. Similarly you may not import MODULE globally within a test module where you're importing MODULE locally.
The local imports must be done for each test function within a subclass of unittest.TestCase. It is perhaps possible to apply this to the unittest.TestCase subclass directly making a particular import of the module available for all of the test functions within the class.
Those messing with builtin imports will find replacing MODULE with sys, os etc. will fail, since these are alread on sys.path when you try to copy it. The trick here is to invoke Python with the builtin imports disabled, I think python -X test.py will do it but I forget the appropriate flag (See python --help). These may subsequently be imported locally using import builtins, IIRC.
I seem to have got somewhere with this. It's important that a test should always leave things as it found them... but only one of the other answers here seems to address that point: if you are substituting a mock or fake for a real decorator, you have to restore that real decorator after the test.
In module thread_check.py I have a decorator called thread_check which (this is a PyQt5 context) checks to see that a function or method is called in the "right thread" (i.e. Gui or non-Gui). It looks like this:
def thread_check(gui_thread: bool):
def pseudo_decorator(func):
if not callable(func):
raise Exception(f'func is type {type(func)}')
def inner_function(*args, **kwargs):
if QtWidgets.QApplication.instance() != None:
app_thread = QtWidgets.QApplication.instance().thread()
curr_thread = QtCore.QThread.currentThread()
if gui_thread != None:
if (curr_thread == app_thread) != gui_thread:
raise Exception(f'method {func.__qualname__} should have been called in {"GUI thread" if gui_thread else "non-GUI thread"}')
return func(*args, **kwargs)
return inner_function
return pseudo_decorator
In practice, in my case here, it makes more sense in most cases to patch out this decorator completely, for all tests, with a "do-nothing decorator" at the start of each run. But to illustrate how it can be done on a per-test basis, see below.
The problem posed is that a method such as is_thread_interrupt_req of class AbstractLongRunningTask (in fact it's not abstract: you can instantiate it) must be run in a non-Gui thread. So the method looks like this:
@thread_check(False) # i.e. non-Gui thread
def is_thread_interrupt_req(self):
return self.thread.isInterruptionRequested()
This is how I solved the question of patching the thread_check decorator, in a way which cleans up the "module space" to restore the real decorator for the next test:
@pytest.fixture
def restore_tm_classes():
yield
importlib.reload(task_manager_classes)
@pytest.mark.parametrize('is_ir_result', [True, False]) # return value from QThread.isInterruptionRequested()
@mock.patch('PyQt5.QtCore.QThread.isInterruptionRequested')
def test_ALRT_is_thread_interrupt_req_returns_val_of_thread_isInterruptionRequested(mock_is_ir, request, qtbot, is_ir_result, restore_tm_classes):
print(f'\n>>>>>> test: {request.node.nodeid}')
print(f'thread_check.thread_check {thread_check.thread_check}')
def do_nothing_decorator(gui_thread):
def pseudo_decorator(func):
return func
return pseudo_decorator
with mock.patch('thread_check.thread_check', side_effect=do_nothing_decorator):
importlib.reload(task_manager_classes)
with mock.patch('PyQt5.QtCore.QThread.start'): # NB the constructor calls QThread.start(): must be mocked!
tm = task_manager_classes.TaskManager(task_manager_classes.AbstractLongRunningTask)
mock_is_ir.return_value = is_ir_result
assert tm.task.is_thread_interrupt_req() == is_ir_result
def test_another(request):
print(f'\n>>>>>> test: {request.node.nodeid}')
print(f'thread_check.thread_check {thread_check.thread_check}')
... in test_another we get the following printed out:
thread_check.thread_check <function thread_check at 0x000002234BEABE50>
... which is the same object as was printed out at the start of the test_ALRT... test.
The key here is to use side_effect in your patch in combination with importlib.reload to reload your module which is itself going to use the decorator.
Note the context manager indenting here: the patch on thread_check.thread_check only needs to apply to the reload... by the time the actual method (is_thread_interrupt_req) is called, the fake decorator is in place.
There is something quite strange going on here if you don't use this teardown fixture restore_tm_classes: in fact in the next test method, it then appears (from my experiments) that the decorator will neither be the real one nor the do_nothing_decorator, as I ascertained by putting in print statements in both. So if you don't restore by reloading the tweaked module it appears that the app code in the task_manager_classes module is then left, for the duration of the test suite, with a "zombie decorator" (which appears to do nothing).
Caveat
There are big potential problems when you use importlib.reload in the middle of a test run.
In particular it can then turn out that the app code is using class X with a certain id value (i.e. id(MyClass)) but the test code (in this and subsequently run modules) is using supposedly the same class X but having another id value! Sometimes this may not matter, other times it can lead to some rather baffling failed tests, which can probably be solved, but may require you to
prefer to avoid mock.patching objects which have not been created actually inside the test: when for example a class itself (I'm not thinking here of an object of a class, but the class as variable itself) is imported or created outside any tests and thus is created in the test collection phase: in this case the class object will not be the same as the one after the reload.
even to use importlib.reload(...) inside some fixtures in various modules which had previously worked without this!
Always use pytest-random-order (with multiple runs) to reveal the full extent of such (and other) problems.
As I said, the decorator could simply be patched out at the start of the run. Whether it's therefore worth doing this is another matter. I have in fact implemented the reverse situation: where the thread_check decorator is patched out at the start of the run, but then patched back in, using the above importlib techniques, for one or two tests which need the decorator to be operative.
Maybe you can apply another decorator onto the definitions of all your decorators that basically checks some configuration variable to see if testing mode is meant to be used.
If yes, it replaces the decorator it is decorating with a dummy decorator that does nothing.
Otherwise, it lets this decorator through.
Those messing with builtin imports will find replacing MODULE with sys, os etc. will fail, since these are alread on sys.path when you try to copy it. The trick here is to invoke Python with the builtin imports disabled, I think python -X test.py will do it but I forget the appropriate flag (See python --help). These may subsequently be imported locally using import builtins, IIRC.
In short:
def remove_decorators(obj):
for attr_name in dir(obj):
# attr_name.startswith('func_name_pattern') if you only need a specific function
if callable(getattr(obj, attr_name)):
while hasattr(getattr(obj, attr_name), '__wrapped__'):
setattr(obj, attr_name, getattr(obj, attr_name).__wrapped__)
# this should be called before testing
remove_decorators(class)
Note that it might remove all decorators so feel free to check for its name :)
I like to make a trick simpler and easier to understand. Take advantage of the decorator's functionality and create a bypass.
The mock function:
from functools import wraps
def the_call(*args, **kwargs):
def decorator(function):
@wraps(function)
def wrapper(*args, **kwargs):
if kwargs.pop("bypass", None) is True:
return function(*args, **kwargs)
# You will probably do something that will change the response or the arguments here below
args = ("bar")
kwargs = {"stuff": "bar"}
return function(*args, **kwargs)
return wrapper
return decorator
Your function with the decorator:
@the_call()
def my_simple_function(stuff: str):
return stuff
print(my_simple_function(stuff="Hello World"))
Will return:
"bar"
So in your tests, simply pass the parameter bypass = True
print(my_simple_function(stuff="Hello World", bypass=True))
Will return:
"Hello World"
for @lru_cache(max_size=1000)
class MockedLruCache(object):def __init__(self, maxsize=0, timeout=0): pass def __call__(self, func): return funccache.LruCache = MockedLruCache
if use decorator which haven't params, you should:
def MockAuthenticated(func):
return func
from tornado import web
web.authenticated = MockAuthenticated
© 2022 - 2025 — McMap. All rights reserved.