Python unit test with base and sub class

Question

I currently have a few unit tests which share a common set of tests. Here's an example:

import unittest

class BaseTest(unittest.TestCase):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

The output of the above is:

Calling BaseTest:testCommon
.Calling BaseTest:testCommon
.Calling SubTest1:testSub1
.Calling BaseTest:testCommon
.Calling SubTest2:testSub2
.
----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK

Is there a way to rewrite the above so that the very first testCommon is not called?

EDIT: Instead of running 5 tests above, I want it to run only 4 tests, 2 from the SubTest1 and another 2 from SubTest2. It seems that Python unittest is running the original BaseTest on its own and I need a mechanism to prevent that from happening.

Priedieu

I see noone has mentioned it but do you have the option to change main part and run a test suite that has all subclasses of BaseTest? – Jequirity

Is there still no great solution for this in 2022? Multiple inheritance is awkward and leads to linting issues. setUpClass with raising SkipTest is pretty good but the test runner shows skipped tests. Other frameworks solve these kinds of issues by adding an __abstract__ = True. Is there no clean way to do this still? – Dar

Answer 1

Use multiple inheritance, so your class with common tests doesn't itself inherit from TestCase.

import unittest

class CommonTests(object):
    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(unittest.TestCase, CommonTests):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(unittest.TestCase, CommonTests):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

Nairobi

This method only works for setUp and tearDown methods if you reverse the order of the base classes. Because the methods are defined in unittest.TestCase, and they don't call super(), then any setUp and tearDown methods in CommonTests need to be first in the MRO, or they won't be called at all. – Ruffin

Just to clarify Ian Clelland's remark so that it will be clearer for people like me: if you add setUp and tearDown methods to CommonTests class, and you want them to be called for each test in derived classes, you have to reverse the order of the base classes, so that it will be: class SubTest1(CommonTests, unittest.TestCase). – Achromatous

I'm not really a fan of this approach. This establishes a contract in the code that classes must inherit from both unittest.TestCase and CommonTests. I think the setUpClass method below is the best and is less prone to human error. Either that or wrapping the BaseTest class in a container class which is a bit more hacky but avoids the skip message in the test run printout. – Poling

The problem with this one is pylint has a fit because CommonTests is invoking methods which don't exist in that class. – Corie

I like this one the best. I argue CommonTests should not inherit from TestCase because it is not really a test case (i.e. you don't want it to run with your other tests). If you really don't like the "contract" that requires inheritance from both, consider a MixIn: `class CommonMixIn( CommonTests, unittest.TestCase): – Lefthander

i had to switch the order of the multiple inheritance to get my CommonTests.tearDown() to run. not sure why. – Nihility

Don't do this, use the 'blank class wrap' trick in https://mcmap.net/q/127998/-python-unit-test-with-base-and-sub-class – Reminiscent

Answer 2

Do not use multiple inheritance, it will bite you later.

Instead you can just move your base class into the separate module or wrap it with the blank class:

class BaseTestCases:

    class BaseTest(unittest.TestCase):

        def testCommon(self):
            print('Calling BaseTest:testCommon')
            value = 5
            self.assertEqual(value, 5)


class SubTest1(BaseTestCases.BaseTest):

    def testSub1(self):
        print('Calling SubTest1:testSub1')
        sub = 3
        self.assertEqual(sub, 3)


class SubTest2(BaseTestCases.BaseTest):

    def testSub2(self):
        print('Calling SubTest2:testSub2')
        sub = 4
        self.assertEqual(sub, 4)

if __name__ == '__main__':
    unittest.main()

The output:

Calling BaseTest:testCommon
.Calling SubTest1:testSub1
.Calling BaseTest:testCommon
.Calling SubTest2:testSub2
.
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

Sexology

This is my favorite. It is the least hacky means and doesn't interfere with overriding methods, doesn't alter the MRO and allows me to define setUp, setUpClass etc. in the base class. – Thaxton

I seriously don't get it (where does the magic come from ?), but it's far the best solution according to me :) Coming from Java, I hate Multiple Inheritance... – Doe

@Edouardb unittest only runs module-level classes that inherit from TestCase. But BaseTest is not module-level. – Aristaeus

As a very similar alternative, you could define the ABC inside a no-args function that returns the ABC when called – Haustorium

This is a nice solution as long as unittest behaves like this (only running module-level classes). A more future-proof solution is to override the run(self, result=None) method in BaseTest: if self.__class__ is BaseTest: return result; else: return super().run(result). – Broaddus

Answer 3

You can solve this problem with a single command:

del(BaseTest)

So the code would look like this:

import unittest

class BaseTest(unittest.TestCase):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

del(BaseTest)

if __name__ == '__main__':
    unittest.main()

Hammerhead

BaseTest is a member of the module while it is being defined, so it is available for use as the base class of the SubTests. Just before the definition is complete, del() removes it as a member, so the unittest framework won't find it when it searches for TestCase subclasses in the module. – Ozonize

this is an awesome answer! I like it more than @MatthewMarshall 's because in his solution, you'll get syntax errors from pylint, because the self.assert* methods do not exist in a standard object. – Insomnia

Doesn't work if BaseTest is referenced anywhere else in the base class or its subclasses, e.g. when calling super() in method overrides: super( BaseTest, cls ).setUpClass( ) – Thaxton

@Thaxton At least in python 3, BaseTest can be referenced through super(self.__class__, self) or just super() in the subclasses, although apparently not if you were to inherit constructors. Maybe there is also such an "anonymous" alternative when the base class needs to reference itself (not that I have any idea when a class needs to reference itself). – Marchese

Answer 4

Matthew Marshall's answer is great, but it requires that you inherit from two classes in each of your test cases, which is error-prone. Instead, I use this (python>=2.7):

class BaseTest(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        if cls is BaseTest:
            raise unittest.SkipTest("Skip BaseTest tests, it's a base class")
        super(BaseTest, cls).setUpClass()

Achromatous

That's neat. Is there a way to get around having to use a skip? To me, skips are undesireable and are used to indicate a problem in the current test plan (either with the code or the test)? – Journal

@ZacharyYoung I don't know, maybe other answers can help. – Achromatous

@ZacharyYoung I've tried to fix this problem, see my answer. – Epispastic

it's not immediately clear what is inherently error-prone about inheriting from two classes – Formwork

@Formwork see comments to the accepted answer :) You need to inherit each of your test classes from the two base classes; you need to preserve the correct order of them; should you like to add another base test class, you'd need to inherit from it too. There is nothing wrong with mixins, but in this case they can be replaced with a simple skip. – Achromatous

I like the approach of checking whether you're running in the BaseTest class. You can use some other documented TestCase features to avoid the skip result. – Stour

Answer 5

You can add __test__ = False in BaseTest class, but if you add it, be aware that you must add __test__ = True in derived classes to be able to run tests.

import unittest

class BaseTest(unittest.TestCase):
    __test__ = False

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):
    __test__ = True

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):
    __test__ = True

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

Simplex

This solution does not work with unittest's own test discovery/test runner. (I believe it requires using an alternate test runner, like nose.) – Stour

Answer 6

What are you trying to achieve? If you have common test code (assertions, template tests, etc), then place them in methods which aren't prefixed with test so unittest won't load them.

import unittest

class CommonTests(unittest.TestCase):
      def common_assertion(self, foo, bar, baz):
          # whatever common code
          self.assertEqual(foo(bar), baz)

class BaseTest(CommonTests):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(CommonTests):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)

class SubTest2(CommonTests):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

Tetrabasic

Under your suggestion, would common_assertion() still be run automatically when testing the subclasses? – Maurene

@Maurene No it would not. The default setting is to only run methods starting with "test". – Hanako

Answer 7

Matthew's answer is the one I needed to use since I'm on 2.5 still. But as of 2.7 you can use the @unittest.skip() decorator on any test methods you want to skip.

http://docs.python.org/library/unittest.html#skipping-tests-and-expected-failures

You'll need to implement your own skipping decorator to check for the base type. Haven't used this feature before, but off the top of my head you could use BaseTest as a marker type to condition the skip:

def skipBaseTest(obj):
    if type(obj) is BaseTest:
        return unittest.skip("BaseTest tests skipped")
    return lambda func: func

Chart

Answer 8

A way I've thought of solving this is by hiding the test methods if the base class is used. This way the tests aren't skipped, so the test results can be green instead of yellow in many test reporting tools.

Compared to the mixin method, ide's like PyCharm won't complain that unit test methods are missing from the base class.

If a base class inherits from this class, it will need to override the setUpClass and tearDownClass methods.

class BaseTest(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        cls._test_methods = []
        if cls is BaseTest:
            for name in dir(cls):
                if name.startswith('test') and callable(getattr(cls, name)):
                    cls._test_methods.append((name, getattr(cls, name)))
                    setattr(cls, name, lambda self: None)

    @classmethod
    def tearDownClass(cls):
        if cls is BaseTest:
            for name, method in cls._test_methods:
                setattr(cls, name, method)
            cls._test_methods = []

Epispastic

Answer 9

Another option is not to execute

unittest.main()

Instead of that you can use

suite = unittest.TestLoader().loadTestsFromTestCase(TestClass)
unittest.TextTestRunner(verbosity=2).run(suite)

So you only execute the tests in the class TestClass

Humiliate

This is the least hacky solution. Instead of modifying what unittest.main() collects into default suite you form explicit suite and run its tests. – Wakeup

Answer 10

Here is a solution that uses only documented unittest features, and that avoids having a "skip" status in your test results:

class BaseTest(unittest.TestCase):

    def __init__(self, methodName='runTest'):
        if self.__class__ is BaseTest:
            # don't run these tests in the abstract base implementation
            methodName = 'runNoTestsInBaseClass'
        super().__init__(methodName)

    def runNoTestsInBaseClass(self):
        pass

    def testCommon(self):
        # everything else as in the original question

How it works: per the unittest.TestCase documentation, "Each instance of TestCase will run a single base method: the method named methodName." The default "runTests" runs all the test* methods on the class—that's how TestCase instances normally work. But when running in the abstract base class itself, you can simply override that behavior with a method that does nothing.

A side effect is your test count will increase by one: the runNoTestsInBaseClass "test" gets counted as a successful test when it's run on BaseClass.

(This also works in Python 2.7, if you're still on that. Just change super() to super(BaseTest, self).)

Stour

A better way is to override the run(result=None) method and just return the result in case of BaseTest. Then also the correct number of tests is reported. – Broaddus

Answer 11

Just rename the testCommon method to something else. Unittest (usually) skips anything that doesn't have 'test' in it.

Quick and simple

  import unittest

  class BaseTest(unittest.TestCase):

   def methodCommon(self):
       print 'Calling BaseTest:testCommon'
       value = 5
       self.assertEquals(value, 5)

  class SubTest1(BaseTest):

      def testSub1(self):
          print 'Calling SubTest1:testSub1'
          sub = 3
          self.assertEquals(sub, 3)


  class SubTest2(BaseTest):

      def testSub2(self):
          print 'Calling SubTest2:testSub2'
          sub = 4
          self.assertEquals(sub, 4)

  if __name__ == '__main__':
      unittest.main()`

Thiazole

This would have the result of not running the methodCommon test in either of the SubTests. – Long

Answer 12

I made about the same than @Vladim P. (https://mcmap.net/q/127998/-python-unit-test-with-base-and-sub-class) but slightly modified:

import unittest2


from some_module import func1, func2


def make_base_class(func):

    class Base(unittest2.TestCase):

        def test_common1(self):
            print("in test_common1")
            self.assertTrue(func())

        def test_common2(self):
            print("in test_common1")
            self.assertFalse(func(42))

    return Base



class A(make_base_class(func1)):
    pass


class B(make_base_class(func2)):

    def test_func2_with_no_arg_return_bar(self):
        self.assertEqual("bar", func2())

and there we go.

Echevarria

Answer 13

As of Python 3.2, you can add a test_loader function to a module to control which tests (if any) are found by the test discovery mechanism.

For example, the following will only load the original poster's SubTest1 and SubTest2 Test Cases, ignoring Base:

def load_tests(loader, standard_tests, pattern):
    suite = TestSuite()
    suite.addTests([SubTest1, SubTest2])
    return suite

It ought to be possible to iterate over standard_tests (a TestSuite containing the tests the default loader found) and copy all but Base to suite instead, but the nested nature of TestSuite.__iter__ makes that a lot more complicated.

Schweinfurt

Answer 14

So this is kind of an old thread but I came across this problem today and thought of my own hack for it. It uses a decorator that makes the values of the functions None when acessed through the base class. Don't need to worry about setup and setupclass because if the baseclass has no tests they won't run.

import types
import unittest


class FunctionValueOverride(object):
    def __init__(self, cls, default, override=None):
        self.cls = cls
        self.default = default
        self.override = override

    def __get__(self, obj, klass):
        if klass == self.cls:
            return self.override
        else:
            if obj:
                return types.MethodType(self.default, obj)
            else:
                return self.default


def fixture(cls):
    for t in vars(cls):
        if not callable(getattr(cls, t)) or t[:4] != "test":
            continue
        setattr(cls, t, FunctionValueOverride(cls, getattr(cls, t)))
    return cls


@fixture
class BaseTest(unittest.TestCase):
    def testCommon(self):
        print('Calling BaseTest:testCommon')
        value = 5
        self.assertEqual(value, 5)


class SubTest1(BaseTest):
    def testSub1(self):
        print('Calling SubTest1:testSub1')
        sub = 3
        self.assertEqual(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print('Calling SubTest2:testSub2')
        sub = 4
        self.assertEqual(sub, 4)

if __name__ == '__main__':
    unittest.main()

Pulsatile

Answer 15

Change the BaseTest method name to setUp:

class BaseTest(unittest.TestCase):
    def setUp(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)


class SubTest1(BaseTest):
    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):
    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

Output:

Ran 2 tests in 0.000s

Calling BaseTest:testCommon Calling
SubTest1:testSub1 Calling
BaseTest:testCommon Calling
SubTest2:testSub2

From the documentation:

TestCase.setUp()
Method called to prepare the test fixture. This is called immediately before calling the test method; any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing.

Gratifying

That would work, what if I have n testCommon, should I place them all under setUp? – Priedieu

Yes you should put all code that is not an actual test case under setUp. – Gratifying

But if a subclass has more than one test... method, setUp gets executed over and over and over again, once per such method; so it's NOT a good idea to put tests there! – Castano

Not really sure what OP wanted in terms of when executed in a more complex scenario. – Gratifying