Removing the work from __init__ to aid unit testing

Question

The key to this question is aiding unit-testing. If I have a busy __init__ (i.e. __init__ that does complex initialization), I cannot simply instantiate an object of a class, but I need to mock/stub out all methods invoked on dependencies within the __init__.

To illustrate this problem, here is example:

class SomeClass(object):
    def __init__(self, dep1, dep2, some_string):
        self._dep1 = dep1
        self._dep2 = dep2
        self._some_string = some_string

        # I would need to mock everything here (imagine some even more
        # complicated example)
        for dep2element in self._dep2:
            dep2element.set_dep(dep1)
        self._dep1.set_some_string(some_string)

    def fun1(self):
        ...
    def fun2(self):
        ...
    def fun3(self):
        ...

To test the fun* functions, every test must perform the complex construction.

class TestSomeClass(TestCase):
    def create_SomeClass(self, some_string):
        dep1 = Mock()
        # mock everything required by SomeClass' constructor

        dep2 = Mock()
        # mock everything required by SomeClass' constructor

        return SomeClass(dep1, dep2, some_string)

    def test_fun1(self):
        sc = self.create_SomeClass('some string')
        ...

    def test_fun2(self):
        sc = self.create_SomeClass('some other string')
        ...

    def test_fun3(self):
        sc = self.create_SomeClass('yet another string')
        ...

I find this redundant and would like to know how this problems can be elegantly handled in python, if not by moving the work from the constructor.

SOLUTION:

As @ecatmur suggested, to test some specific function, this code should do the trick:

def test_some_method():
    mobject = Mock(SomeClass)
    SomeClass.fun1(mobject)

With this approach all the methods will be mocked out. If fun1 calls some other method you want executed (e.g. fun2) you can do it like this:

def test_some_method():
    mobject = Mock(SomeClass)
    mobject.fun2 = SomeClass.fun2.__get__(mobject)
    SomeClass.fun1(mobject)

SomeClass.fun2.__get__(mobject) will produce instancemethod which will provide the correct binding.

¡Viva el Python!

ORIGINAL QUESTION:

Original question was centered around moving the work done in __init__ to the separate init method and different problems revolving that approach. My usual approach is to make this

class SomeClass(object):
    def __init__(self, dep1, dep2, some_string)
        self._dep1 = dep1
        self._dep2 = dep2

        # lots of mumbo-jumbo here...

become this

class SomeClass(object):
    def __init__(self, dep1, dep2)
        self._dep1 = dep1
        self._dep2 = dep2

    def initiate(self, some-string)
        # lots of mumto-jumbo here...

General sentiment was that moving work from __init__ is not a common practice and would be meaningless to seasoned python developers.

Larena

For starters, returning anything from init except None will raise an exception. But it's ok- you can still do your one line initialization. Init doesn't return anything, but the constructor still does. – Interdictory

A minor semantic point - in Python, the __init__() method isn't quite a "constructor". __new__() and __init__() share the function of a usual constructor. Whenever you do a = MyClass(), MyClass.__new__() is called, which creates and returns the object instance, on which __init__() is called, before returning the instance to be assigned to a (in this case). – Puffer

I guess the exact wording would be 'constructor expression', but I already fixed that in my question... – Larena

This question made me understand why def method(self): is good in python ! ¡Viva el Python! :) – Phonic

Answer 1

If you write initialization functions separate from __init__ then experienced developers will certainly see your code as a kid's playground.

If you're concerned about being able to create objects that look like instances of your class without running the __init__ method, then use Mock:

def test_some_method():
    mock_object = Mock(MyClass)
    MyClass.some_method(mock_object)

Herring

The problem is the other way around: when I need to mock dependencies that SomeClass depends on. I expanded my post. Please, check it out. Thnx – Larena

@Vedran I don't understand; a half-constructed object isn't any use, so why would having one help with testing? You'd better post an example of an actual test you want to run on your class. – Herring

I rephrased my question entirely and hope the question is more to the point now. – Larena

@Vedran your question still doesn't make any sense; if your tests require a fully constructed SomeClass object then you need to run the constructor in full anyway; if they don't then a Mock object will do. – Herring

This is about testing SomeClass itself, and not about using SomeClass object to test something else. The point here is to restrict testing to only single unit of code (like functions fun1, fun2 and fun3) in isolation from the other functions in a class or beyond. This comes down to removing all unnecessary code out of the requirements of a single unit test. In my example, this would mean that fun1, fun2, or fun3 don't require full initialization, thus making the initialization code redundant from the perspective of a single unit test. – Larena

@Vedran so pass Mock(SomeClass) to fun1 etc. – Herring

If I'm testing a function fun1 which calls function fun2, both of which don't require the complex initialization, can I somehow install the fun2 on the mock? Something like this: mock.fun2 = MyClass.fun2? – Larena

@Vedran almost, you have to write mock.fun2 = MyClass.fun2.__get__(mock), to get the binding working right. – Herring

This will actually produce an instancemethod object? Kewl... Thnx for your help. – Larena

Answer 2

__init__ can't actually return anything; if you think about how it is used, this should be obvious:

class Example(object):
    def __init__(self, x):
        self.x = x
        return ANYTHING

e = Example(1)   # how are you going to get ANYTHING back?

Using an initialize() method, separate from __init__, seems kind of silly - the whole point is that your initializer should automatically run when the object is created, so your example initialization should look like

scobj = SomeClass(dep1, dep2, 'abcdef')
# scobj.initialize('abcdef')     # <= separate call not needed!

Edit:

If you really need to factor the code in __init__, I suggest putting it in private methods and calling those from __init__, like so:

class Example2(object):
    def __init__(self, a, b, c):
        self._init_connection(a)
        self._init_display(b)
        self.c = c

    def _init_connection(self, a):
        self.conn = make_connection(a)

    def _init_display(self, b):
        self.disp = make_display(b)

... this is good because

• all initialization is done on creating the object (you don't need to remember to make follow-up initialization calls)
• you don't have to keep a flag to remember whether your object has been initialized or not - if it exists, it has been initialized.

Gonfalon

I'll expand my intro to the question. – Larena

Answer 3

class Klass:
    def __init__(self):
        initialize instance here, no return

__init__ will automatically run at object creation, so you don't need to check if it run; it was run if you have an instance!

About returning self from init(): I wouldn't, as it is an in-place operation. Read the first answer here, as it explains this pretty well.

Slantwise

I already know about the init, so I'll just expand my examples. Thnx for the answer, – Larena

Then use it? what's the problem about __init__? – Slantwise

The problem with constructor is that i don't want to have too much work in it. The key is here is to aid unit testing, and I don't want to have to mock everything that might be needed in the busy constructor. – Larena

You need to put it somewhere, what's the problem with putting it in __init__? – Slantwise

Answer 4

There's nothing wrong with splitting __init__() out into separate methods. If you can do it in a way which makes those methods appropriately named and reusable, then all the better - e.g.:

class SomeClass(object):
    def __init__(self, dep1, dep2, some_string):
        self.set_dependencies(dep1, dep2, some_string)

    def set_dependencies(self, dep1, dep2, some_string):
        self._dep1 = dep1
        self._dep2 = dep2
        self._some_string = some_string

        for dep2element in self._dep2:
            dep2element.set_dep(dep1)
        self._dep1.set_some_string(some_string)

Here, a remove_dependencies() method could be added in the future to revert the instance to a neutral state, before setting new dependencies. The functionality is decoupled from the initialisation process.

Note I've called this from __init__(), even though we don't want this to run in testing. If we don't call this from __init__(), someone using this class would have to call set_dependencies() after instantiating - which is an unnecessary complication to your class's API.

What we can do is stub the method out for testing, e.g.:

class TestSomeClass(TestCase):
    def __init__(self):
        SomeClass._old_set_dependencies = SomeClass.set_dependencies
        SomeClass.set_dependencies = lambda *args: None

    ...

This is a pretty brash way of stubbing out a method, just to prove a point really - there are some good libraries out there for doing it more diligently - see this discussion, but I'd also recommend Mockstar which is an extension to Mock designed to simplify the semantics.

Puffer

This is great. :) Can you name some of those libraries? I guess that mock.patch would work, but I guess there are other solutions too. – Larena

I've added some relevant links to my answer. – Puffer

Answer 5

What about

class SomeClass(object):
    def __init__(self, dep1, dep2, st=None):
        self._dep1 = dep1
        self._dep2 = dep2
        if st is None:
            self._initialized = False
        else:
            self.initialize(st)

    def initialize(self, st):
        ...
        self._initialized = True
        ...

?

Skvorak

Why not putting self._initialized away? – Slantwise