Skipping all unit tests but one in Python by using decorators and metaclasses

Question

I am writing unit tests for an MCU that communicates commands through the USB port and checks their response. If one unit test fails it makes sense for me to do some debugging in the MCU. Therefore I would like to disable all unittests except for the one that I would like to debug on the MCU side because if I set a breakpoint somewhere it might get triggered by another unittest with different commands.

I went to the python docs and found this code which is a decorator that will skip all unittests that don't have an attribute.

def skipUnlessHasattr(obj, attr):
    if hasattr(obj, attr):
        return lambda func: func
return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))

In order to make it more simple I deleted the attr argument and statically changed it to 'StepDebug' which is an attribute that I want to set in only one unittest in order to debug it.

So the next step is for me to apply this for all my class methods automatically. After reading around on the webs I found the following code that uses a metaclass in order to decorate all methods with the above decorator. https://mcmap.net/q/210897/-how-to-decorate-all-functions-of-a-class-without-typing-it-over-and-over-for-each-method-duplicate

def decorating_meta(decorator):
class DecoratingMetaclass(type):
    def __new__(self, class_name, bases, namespace):
        for key, value in list(namespace.items()):
            if callable(value):
                namespace[key] = decorator(value)
        return type.__new__(self, class_name, bases, namespace)

return DecoratingMetaclass

So my minimum working example is

import unittest

def decorating_meta(decorator):
    class DecoratingMetaclass(type):
        def __new__(self, class_name, bases, namespace):
            for key, value in list(namespace.items()):
                if callable(value):
                    namespace[key] = decorator(value)
            return type.__new__(self, class_name, bases, namespace)

    return DecoratingMetaclass

def skipUnlessHasattr(obj):
    if hasattr(obj, 'StepDebug'):
        return lambda func : func
    return unittest.skip("{!r} doesn't have {!r}".format(obj, 'StepDebug'))

class Foo(unittest.TestCase):
    __metaclass__ = decorating_meta(skipUnlessHasattr)
    def test_Sth(self):
        self.assertTrue(False)

if __name__ == '__main__':
    unittest.main()

and the error that I get is:

AttributeError: 'Foo' object has no attribute '__name__'

From what I read this is something that happens when instead of a class you have an instance but I don't quite understand how I can use this information to solve my problem.

Can someone please help?

Catalano

Sorry I cannot understan why you cannot use something like @unittest.skipUnlessHasattr("StepDebug") before your class test declaration do decoarate all method. Moreover you can define you own decorator as def myskip(): return unittest.skipUnlessHasattr("StepDebug"). – Choli

If I understand your question correctly I don't want to just add a decorator before each test_* method because I will have many unittests and I don't want to add it manually everytime if there is an option to do it automatically. – Catalano

I'm not sure but maybe @unittest.skipUnlessHasattr() work as class decorator too. I'm sure that you can apply @skip docorator to a class but it is not clear if it is valid for skipUnlessHasattr() too. – Choli

I'm probably misunderstanding something but why don't you just run only the test that you want to debug rather than the whole suite? – Briton

I didn't do it that way because I thought that then I would have to comment/uncomment a lot of code. My idea was that whenever I want to debug something just flip a switch on the test that failed. – Catalano

Answer 1

Ok I found a way to have the functionality that I want. I changed the decorator to:

def skipUnlessHasattr(obj):
    if hasattr(obj, 'StepDebug'):
        def decorated(*a, **kw):
            return obj(*a, **kw)
        return decorated
    else:
        def decorated(*a, **kw):
            return unittest.skip("{!r} doesn't have {!r}".format(obj, 'StepDebug'))
        return decorated

Now I get all tests skipped except the ones that I have added an attribute StepDebug.

The only minor thing with this is that the output doesn't report all the other tests as skipped but rather as a success.

..F..
======================================================================
FAIL: test_ddd (__main__.Foo)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./Documents/programs/Python/mwe.py", line 23, in     decorated
    return obj(*a, **kw)
  File "./Documents/programs/Python/mwe.py", line 39, in     test_ddd
    self.assertTrue(False)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 5 tests in 0.014s

FAILED (failures=1)

P.S. Why when copying the output after indenting 4 spaces it doesn't go into a code block? I tried 8 spaces as well and it wouldn't work. Eventually I added 4 spaces to each line. Is there a smarter way?

Catalano

Answer 2

https://mcmap.net/q/235171/-python-unittest-how-to-run-only-part-of-a-test-file uses decorators, initializing via environment parameters

MCU = os.getenv('MCU', False)

and decorating test classes and/or methods to be excluded via f.ex.

@unittest.skipIf(MCU)

this can be called as

MCU=1 python # test file etc

The only minor thing with this is that the output doesn't report all the other tests as skipped but rather as a success.

This marks the skipped tests as s.

Selda

This is what I do as well. In Bash, many other shells, remember to export the environment variable so that Python can see it. – Gypsum

Answer 3

When I need to debug only one test and I don't want to deal with command line parameters or modification of unittest.main() I do the following.

I add a file focused_unittests.py:

import unittest
import ast
import inspect

# src: https://stackoverflow.com/a/31197273
def get_decorators(cls):
    target = cls
    decorators = {}

    def visit_FunctionDef(node):
        decorators[node.name] = []
        for n in node.decorator_list:
            name = ''
            if isinstance(n, ast.Call):
                name = n.func.attr if isinstance(n.func, ast.Attribute) else n.func.id
            else:
                name = n.attr if isinstance(n, ast.Attribute) else n.id

            decorators[node.name].append(name)

    node_iter = ast.NodeVisitor()
    node_iter.visit_FunctionDef = visit_FunctionDef
    node_iter.visit(ast.parse(inspect.getsource(target)))
    return decorators

# The rest of the code was created by ChatGPT.
def focus(test_func):
    def wrapper(*args, **kwargs):
        # Set a flag indicating the focused test
        test_func.__unittest_focus__ = True
        return test_func(*args, **kwargs)
    return wrapper

# Extend the unittest.TestCase to skip non-focused tests
class FocusedTestCase(unittest.TestCase):
    def setUp(self):
        # Get decorators for the current class
        decorators = get_decorators(type(self))
        
        # Check if there are focused tests
        focused_tests = [name for name, decs in decorators.items() if 'focus' in decs]
        if focused_tests and self._testMethodName not in focused_tests:
            self.skipTest('Skipped due to focus')

and I use it like this:

# test_example.py
import unittest
from focused_unittests import FocusedTestCase, focus

class TestExample(FocusedTestCase):

    @focus
    def test_add(self):
        self.assertEqual(2 + 3, 5)

    @focus
    def test_subtract(self):
        self.assertEqual(3 - 2, 1)

    def test_multiply(self):
        self.assertEqual(3 * 2, 6)

    def test_divide(self):
        self.assertEqual(6 / 2, 3)

if __name__ == '__main__':
    unittest.main()

So I change unittest.TestCase to FocusedTestCase and I add @focus decorator to the methods I need to debug. Now I can debug with just F5 in vscode.

Amendatory