What is a Pythonic way for Dependency Injection?

Question

Introduction

For Java, Dependency Injection works as pure OOP, i.e. you provide an interface to be implemented and in your framework code accept an instance of a class that implements the defined interface.

Now for Python, you are able to do the same way, but I think that method was too much overhead right in case of Python. So then how would you implement it in the Pythonic way?

Use Case

Say this is the framework code:

class FrameworkClass():
    def __init__(self, ...):
        ...

    def do_the_job(self, ...):
        # some stuff
        # depending on some external function

The Basic Approach

The most naive (and maybe the best?) way is to require the external function to be supplied into the FrameworkClass constructor, and then be invoked from the do_the_job method.

Framework Code:

class FrameworkClass():
    def __init__(self, func):
        self.func = func

    def do_the_job(self, ...):
        # some stuff
        self.func(...)

Client Code:

def my_func():
    # my implementation

framework_instance = FrameworkClass(my_func)
framework_instance.do_the_job(...)

Question

The question is short. Is there any better commonly used Pythonic way to do this? Or maybe any libraries supporting such functionality?

UPDATE: Concrete Situation

Imagine I develop a micro web framework, which handles authentication using tokens. This framework needs a function to supply some ID obtained from the token and get the user corresponding to that ID.

Obviously, the framework does not know anything about users or any other application specific logic, so the client code must inject the user getter functionality into the framework to make the authentication work.

Superlative

Why don't you "provide an interface to be implemented and in your framework code accept an instance of a class that implements the defined interface"? In Python you would do this in an EAFP style (i.e. assume it meets that interface and an AttributeError or TypeError gets raised otherwise), but otherwise it's the same. – Vaporize

It is easy to do that using abs's ABCMeta metaclass with @abstractmethod decorator, and no manual validation. Just want to get couple of options and suggestions. The one you quoted is the most clean one, but I think with more overhead. – Superlative

Then I don't know what question you're trying to ask. – Vaporize

Ok, I will try in other words. The problem is clear. The question is how to do that in Pythonic way. Option 1: The way you quoted, Option 2: The basic approach I described in the question. So the question is, any other Pythonic ways to do that? – Superlative

"There should be one-- and preferably only one --obvious way to do it." – Vaporize

Thanks, I know that :) but here, there are obviously more than one ways. – Superlative

This is obviously highly application-specific. Is there a particular reason why, in your application, myfunc isn't wrapped up in mysubclass? (Then you don't need to assign it in __init__) – Arawak

Thanks for the comment. I have updated the question to answer your question. – Superlative

Here you can find a good sample: code.activestate.com/recipes/413268 – Dilorenzo

Answer 1

See Raymond Hettinger - Super considered super! - PyCon 2015 for an argument about how to use super and multiple inheritance instead of DI. If you don't have time to watch the whole video, jump to minute 15 (but I'd recommend watching all of it).

Here is an example of how to apply what's described in this video to your example:

Framework Code:

class TokenInterface():
    def getUserFromToken(self, token):
        raise NotImplementedError

class FrameworkClass(TokenInterface):
    def do_the_job(self, ...):
        # some stuff
        self.user = super().getUserFromToken(...)

Client Code:

class SQLUserFromToken(TokenInterface):
    def getUserFromToken(self, token):      
        # load the user from the database
        return user

class ClientFrameworkClass(FrameworkClass, SQLUserFromToken):
    pass

framework_instance = ClientFrameworkClass()
framework_instance.do_the_job(...)

This will work because the Python MRO will guarantee that the getUserFromToken client method is called (if super() is used). The code will have to change if you're on Python 2.x.

One added benefit here is that this will raise an exception if the client does not provide a implementation.

Of course, this is not really dependency injection, it's multiple inheritance and mixins, but it is a Pythonic way to solve your problem.

Trefler

This answer considered super() :) – Superlative

Raymond called it CI while, I thought, that it is a pure mixin. But could it be that in Python mixin and CI are virtually the same? The only difference is the level of indection. Mixin injects dependency into a class level while CI injects dependency into an instance. – Denverdeny

I think constructor level injection is pretty easy to do in Python anyway, like how OP described it. this pythonic way looks to be very interesting though. it just requires a bit more wiring than simple constructor injection IMO. – Lourdeslourie

While I find it very elegant, I have two problems with this approach: 1. What happens when you need serveral items being injected into your class? 2. Inheritance is most often used in a "is a"/specialization-sense. Using it for DI defies that idea (for example if I want to inject a Service into a Presenter). – Nutt

One of the advantages of DI is that I can inject other dependencies in test scenarios. How does this approach deal with that use case? – Cornwall

Why not write: def do_the_job(tokenf = getUserFromToken, **kwargs): user = tokenf(**kwargs); ... – Miun

With this approach, how can one write a unit test for FrameworkClass providing a mock for TokenInterface? – Preferential

As @Nutt says, this method gets confusing when you need to have two separate DI furthermore using inheritance rather than method signature is limiting. It's a fancier and limited version of the constructor-based DI as seen in @Bryan Roach's answer – Brietta

Answer 2

The way we do dependency injection in our project is by using the inject lib. Check out the documentation. I highly recommend using it for DI. It kinda makes no sense with just one function but starts making lots of sense when you have to manage multiple data sources etc, etc.

Following your example it could be something similar to:

# framework.py
class FrameworkClass():
    def __init__(self, func):
        self.func = func

    def do_the_job(self):
        # some stuff
        self.func()

Your custom function:

# my_stuff.py
def my_func():
    print('aww yiss')

Somewhere in the application you want to create a bootstrap file that keeps track of all the defined dependencies:

# bootstrap.py
import inject
from .my_stuff import my_func

def configure_injection(binder):
    binder.bind(FrameworkClass, FrameworkClass(my_func))

inject.configure(configure_injection)

And then you could consume the code this way:

# some_module.py (has to be loaded with bootstrap.py already loaded somewhere in your app)
import inject
from .framework import FrameworkClass

framework_instance = inject.instance(FrameworkClass)
framework_instance.do_the_job()

I'm afraid this is as pythonic as it can get (the module has some python sweetness like decorators to inject by parameter etc - check the docs), as python does not have fancy stuff like interfaces or type hinting.

So to answer your question directly would be very hard. I think the true question is: does python have some native support for DI? And the answer is, sadly: no.

Stone

Thanks for you answer, seems pretty much interesting. I will checkout the decorators part. In the meanwhile, let's wait for more answers. – Superlative

Thanks for the link to the 'inject' library. This is the closest I've found so far to filling the gaps that I wanted filled by DI -- and bonus, it's actually being maintained! – Cp

Answer 3

Some time ago I wrote dependency injection microframework with a ambition to make it Pythonic - Dependency Injector. That's how your code can look like in case of its usage:

"""Example of dependency injection in Python."""

import logging
import sqlite3

import boto.s3.connection

import example.main
import example.services

import dependency_injector.containers as containers
import dependency_injector.providers as providers


class Platform(containers.DeclarativeContainer):
    """IoC container of platform service providers."""

    logger = providers.Singleton(logging.Logger, name='example')

    database = providers.Singleton(sqlite3.connect, ':memory:')

    s3 = providers.Singleton(boto.s3.connection.S3Connection,
                             aws_access_key_id='KEY',
                             aws_secret_access_key='SECRET')


class Services(containers.DeclarativeContainer):
    """IoC container of business service providers."""

    users = providers.Factory(example.services.UsersService,
                              logger=Platform.logger,
                              db=Platform.database)

    auth = providers.Factory(example.services.AuthService,
                             logger=Platform.logger,
                             db=Platform.database,
                             token_ttl=3600)

    photos = providers.Factory(example.services.PhotosService,
                               logger=Platform.logger,
                               db=Platform.database,
                               s3=Platform.s3)


class Application(containers.DeclarativeContainer):
    """IoC container of application component providers."""

    main = providers.Callable(example.main.main,
                              users_service=Services.users,
                              auth_service=Services.auth,
                              photos_service=Services.photos)

Here is a link to more extensive description of this example - http://python-dependency-injector.ets-labs.org/examples/services_miniapp.html

Hope it can help a bit. For more information please visit:

• GitHub https://github.com/ets-labs/python-dependency-injector
• Docs http://python-dependency-injector.ets-labs.org/

Dorelle

Thank you @Roman Mogylatov. I'm curious to know how you configure/adapt these containers at runtime, say from a configuration file. It seems like these dependencies are hardcoded into the given container (Platform and Services). Is the solution to create a new container for every combination of injectable library classes? – Intellectuality

Hi @BillDeRose. While my answer was considered as too long for being an SO comment, I've created an github issue and post my answer there - github.com/ets-labs/python-dependency-injector/issues/197 :) Hope it helps, Thanks, Roman – Dorelle

Answer 4

Dependency injection is a simple technique that Python supports directly. No additional libraries are required. Using type hints can improve clarity and readability.

Framework Code:

class UserStore():
    """
    The base class for accessing a user's information.
    The client must extend this class and implement its methods.
    """
    def get_name(self, token):
        raise NotImplementedError

class WebFramework():
    def __init__(self, user_store: UserStore):
        self.user_store = user_store

    def greet_user(self, token):
        user_name = self.user_store.get_name(token)
        print(f'Good day to you, {user_name}!')

Client Code:

class AlwaysMaryUser(UserStore):
    def get_name(self, token):      
        return 'Mary'

class SQLUserStore(UserStore):
    def __init__(self, db_params):
        self.db_params = db_params

    def get_name(self, token):
        # TODO: Implement the database lookup
        raise NotImplementedError

client = WebFramework(AlwaysMaryUser())
client.greet_user('user_token')

The UserStore class and type hinting are not required for implementing dependency injection. Their primary purpose is to provide guidance to the client developer. If you remove the UserStore class and all references to it, the code still works.

Adjacent

Answer 5

After playing around with some of the DI frameworks in python, I've found they have felt a bit clunky to use when comparing how simple it is in other realms such as with .NET Core. This is mostly due to the joining via things like decorators that clutter the code and make it hard to simply add it into or remove it from a project, or joining based on variable names.

I've recently been working on a dependency injection framework that instead uses typing annotations to do the injection called Simple-Injection. Below is a simple example

from simple_injection import ServiceCollection


class Dependency:
    def hello(self):
        print("Hello from Dependency!")

class Service:
    def __init__(self, dependency: Dependency):
        self._dependency = dependency

    def hello(self):
        self._dependency.hello()

collection = ServiceCollection()
collection.add_transient(Dependency)
collection.add_transient(Service)

collection.resolve(Service).hello()
# Outputs: Hello from Dependency!

This library supports service lifetimes and binding services to implementations.

One of the goals of this library is that it is also easy to add it to an existing application and see how you like it before committing to it as all it requires is your application to have appropriate typings, and then you build the dependency graph at the entry point and run it.

Hope this helps. For more information, please see

• github: https://github.com/BradLewis/simple-injection
• docs: https://simple-injection.readthedocs.io/en/latest/
• pypi: https://pypi.org/project/simple-injection/

Digestive

Answer 6

A very easy and Pythonic way to do dependency injection is importlib.

You could define a small utility function

def inject_method_from_module(modulename, methodname):
    """
    injects dynamically a method in a module
    """
    mod = importlib.import_module(modulename)
    return getattr(mod, methodname, None)

And then you can use it:

myfunction = inject_method_from_module("mypackage.mymodule", "myfunction")
myfunction("a")

In mypackage/mymodule.py you define myfunction

def myfunction(s):
    print("myfunction in mypackage.mymodule called with parameter:", s)

You could of course also use a class MyClass iso. the function myfunction. If you define the values of methodname in a settings.py file you can load different versions of the methodname depending on the value of the settings file. Django is using such a scheme to define its database connection.

Appointed

Answer 7

I think that DI and possibly AOP are not generally considered Pythonic because of typical Python developers preferences, rather that language features.

As a matter of fact you can implement a basic DI framework in <100 lines, using metaclasses and class decorators.

For a less invasive solution, these constructs can be used to plug-in custom implementations into a generic framework.

Raila

Answer 8

There is also Pinject, an open source python dependency injector by Google.

Here is an example

>>> class OuterClass(object):
...     def __init__(self, inner_class):
...         self.inner_class = inner_class
...
>>> class InnerClass(object):
...     def __init__(self):
...         self.forty_two = 42
...
>>> obj_graph = pinject.new_object_graph()
>>> outer_class = obj_graph.provide(OuterClass)
>>> print outer_class.inner_class.forty_two
42

And here is the source code

Conroy

Answer 9

Due to Python OOP implementation, IoC and dependency injection are not standard practices in the Python world. But the approach seems promising even for Python.

• To use dependencies as arguments is a non-pythonic approach. Python is an OOP language with beautiful and elegant OOP model, that provides more straightforward ways to maintain dependencies.
• To define classes full of abstract methods just to imitate interface type is weird too.
• Huge wrapper-on-wrapper workarounds create code overhead.
• I also don't like to use libraries when all I need is a small pattern.

So my solution is:

# Framework internal
def MetaIoC(name, bases, namespace):
    cls = type("IoC{}".format(name), tuple(), namespace)
    return type(name, bases + (cls,), {})


# Entities level                                        
class Entity:
    def _lower_level_meth(self):
        raise NotImplementedError

    @property
    def entity_prop(self):
        return super(Entity, self)._lower_level_meth()


# Adapters level
class ImplementedEntity(Entity, metaclass=MetaIoC):          
    __private = 'private attribute value'                    

    def __init__(self, pub_attr):                            
        self.pub_attr = pub_attr                             

    def _lower_level_meth(self):                             
        print('{}\n{}'.format(self.pub_attr, self.__private))


# Infrastructure level                                       
if __name__ == '__main__':                                   
    ENTITY = ImplementedEntity('public attribute value')     
    ENTITY.entity_prop         

EDIT:

Be careful with the pattern. I used it in a real project and it showed itself a not that good way. My post on Medium about my experience with the pattern.

Daemon

Of course IOC and DI are commonly used, what isn't commonly used are DI frameworks, for better or for worse. – Fabria

Answer 10

I found it easy to make Dependency Injection in python application without any additional libraries. However the code needs some tedious maintenance efforts and after a while the fun is gone. I tried few thing and found Injector library doing its job very well. I wrote the article about my experience and it contains the link to example applications source code on GitHUB Dependency Injection in Python with Injector. Here the example of code which can help to run application in few modes (development, production, test etc). The proper dependency implementation initiated and placed in application context and gets injected.

class ApplicationContext:

    def __init__(self, profile):
        # Injector instance initiated
        self.__injector = Injector()
        # Profile value can be bound to Injector and later injected while creating managed classes instances.
        self.__injector.binder.bind(Profiles, profile)

        # The HTTPService implementation defined by profile. This done with binding the HTTPService implementation.
        if Profiles.DEFAULT == profile:
            self.__injector.binder.bind(HTTPService, HTTPDefaultService)
        if Profiles.DEVELOPMENT == profile:
            self.__injector.binder.bind(HTTPService, HTTPDevelopmentService)

        # Injector does not need explicit binding here because we have just one ProcessorService class
        self.__processor_service = self.__injector.get(ProcessorService)

    def run(self):
        self.__processor_service.run()

class HTTPService:

    def get_fundamentals(self, ticker):
        pass


class HTTPDefaultService(HTTPService):

    def __init__(self):
        pass

    def get_fundamentals(self, ticker):
        url = f'https://eodhd.com/api/fundamentals/{ticker}?api_token=demo'
        fundamentals_data = requests.get(url).json()
        return fundamentals_data

class ProcessorService:

    @inject
    def __init__(self, http_service: HTTPService):
        self.__http_service = http_service

    def run(self):
        # AAPL ticker represents Apple company
        fundamentals_json = self.__http_service.get_fundamentals('AAPL')

Wirra