Django: Signal/Method called after "AppConfig.ready()"

Question

I have an AppConfig.ready() implementation which depends on the readiness of an other application.

Is there a signal or method (which I could implement) which gets called after all application ready() methods have been called?

I know that django processes the signals in the order of INSTALLED_APPS.

But I don't want to enforce a particular ordering of INSTALLED_APPS.

Example:

INSTALLED_APPS=[
   'app_a',
   'app_b',
   ...
]

How can "app_a" receive a signal (or method call) after "app_b" processed AppConfig.ready()?

(reordering INSTALLED_APPS is not a solution)

Cowes

Django processes apps in the order of INSTALLED_APPS, so you should be able to put your code in the ready() method as long as you order your apps appropriately. – Hospitalization

@Hospitalization thank you for your hint. I updated the question. – Cowes

Answer 1

I'm afraid the answer is No. Populating the application registry happens in django.setup(). If you look at the source code, you will see that neither apps.registry.Apps.populate() nor django.setup() dispatch any signals upon completion.

Here are some ideas:

• You could dispatch a custom signal yourself, but that would require that you do that in all entry points of your Django project, e.g. manage.py, wsgi.py and any scripts that use django.setup().

• You could connect to request_started and disconnect when your handler is called.

• If you are initializing some kind of property, you could defer that initialization until the first access.

If any of these approaches work for you obviously depends on what exactly you are trying to achieve.

Napery

Answer 2

So there is a VERY hackish way to accomplish what you might want...

Inside the django.apps.registry is the singleton apps which is used by Django to populate the applications. See setup in django.__init__.py.

The way that apps.populate works is it uses a non-reentrant (thread-based) locking mechanism to only allow apps.populate to happen in an idempotent, thread-safe manner.

The stripped down source for the Apps class which is what the singleton apps is instantiated from:

class Apps(object):

    def __init__(self, installed_apps=()):
        # Lock for thread-safe population.
        self._lock = threading.Lock()

    def populate(self, installed_apps=None):
        if self.ready:
            return

        with self._lock:
            if self.ready:
                return

            for app_config in self.get_app_configs():
                app_config.ready()

            self.ready = True

With this knowledge, you could create some threading.Thread's that await on some condition. These consumer threads will utilize threading.Condition to send cross-thread signals (which will enforce your ordering problem). Here is a mocked out example of how that would work:

import threading

from django.apps import apps, AppConfig

# here we are using the "apps._lock" to synchronize our threads, which
# is the dirty little trick that makes this work
foo_ready = threading.Condition(apps._lock)

class FooAppConfig(AppConfig):
    name = "foo"

    def ready(self):
        t = threading.Thread(name='Foo.ready', target=self._ready_foo, args=(foo_ready,))
        t.daemon = True
        t.start()

    def _ready_foo(self, foo_ready):
        with foo_ready:
            # setup foo
            foo_ready.notifyAll() # let everyone else waiting continue

class BarAppConfig(AppConfig):
    name = "bar"

    def ready(self):
        t = threading.Thread(name='Bar.ready', target=self._ready_bar, args=(foo_ready,))
        t.daemon = True
        t.start()

    def _ready_bar(self, foo_ready):
        with foo_ready:
            foo_ready.wait() # wait until foo is ready
            # setup bar

Again, this ONLY allows you to control the flow of the ready calls from the individual AppConfig's. This doesn't control the order models get loaded, etc.

But if your first assertion was true, you have an app.ready implementation that depends on another app being ready first, this should do the trick.

Reasoning:

Why Conditions? The reason this uses threading.Condition over threading.Event is two-fold. Firstly, conditions are wrapped in a locking layer. This means that you will continue to operate under controlled circumstances if the need arises (accessing shared resources, etc). Secondly, because of this tight level of control, staying inside the threading.Condition's context will allow you to chain the configurations in some desirable ordering. You can see how that might be done with the following snippet:

lock = threading.Lock()
foo_ready = threading.Condition(lock)
bar_ready = threading.Condition(lock)
baz_ready = threading.Condition(lock)

Why Deamonic Threads? The reason for this is, if your Django application were to die sometime between acquiring and releasing the lock in apps.populate, the background threads would continue to spin waiting for the lock to release. Setting them to daemon-mode will allow the process to exit cleanly without needing to .join those threads.

Trainload

(Ab)using the lock is quite clever! I wonder if it wouldn't be enough to spawn a thread in Bar that waits for app._lock to be released. Also, my understanding of the problem was that the app being waited for cannot be modified. If the apps can work together there is probably a simpler solution. Maybe @Cowes can clarify. – Napery

@DanielHepper hahah "abuse" is definitely the right word! If what you said is true, the app being "waited on" is not able to be modified, this would certainly not be the solution directly. I think, as you said, just waiting for the lock might be sufficient for that. – Trainload

Yes, you are right. This is very hackish. Nevertheless, thank you for your answer. – Cowes

They don't have to be modified. Third-party AppConfigs can be overridden, though it may mean some duplicate code (or some hacky method copying: can you set a super call as the target of a thread?). – Conchoid

Answer 3

You can add a dummy app which only purpose is to fire a custom all_apps_are_ready signal (or method call on AppConfig).

Put this app at the end of INSTALLED_APPS.

If this app receives the AppConfig.ready() method call, you know that all other apps are ready.

Cowes

Answer 4

An alternative solution:

Subclass AppConfig and send a signal at the end of ready. Use this subclass in all your apps. If you have a dependency on one being loaded, hook up to that signal/sender pair.

If you need more details, don't hesitate!

There are some subtleties to this method:

1) Where to put the signal definition (I suspect in manage.py would work, or you could even monkey-patch django.setup to ensure it gets called everywhere that it is). You could put in a core app that is always the first one in installed_apps or somewhere where django will always load it before any AppConfigs are loaded.

2) Where to register the signal receiver (you should be able to do this in AppConfig.__init__ or possibly just globally in that file).

See https://docs.djangoproject.com/en/dev/ref/applications/#how-applications-are-loaded

Therefore, the setup is as follow:

• When django first starts up, register the signal.
• at the end of every app_config.ready send the signal (with the AppConfig instance as the sender)
• in AppConfigs that need to respond to the signal, register a receiver in __init__ with the appropriate sender.

Let me know how it goes!

If you need it to work for third-party apps, keep in mind that you can override the AppConfigs for these apps (convention is to place these in a directory called apps). Alternatively, you could monkey patch AppConfig

Conchoid

I updated the question and added an example. Do you think you question is still valid? – Cowes

@Cowes I've added some more details. It's a bit more complicated then I initially thought, but I suspect it should work based on the Django docs. The trick is getting everything in the right places ;) – Conchoid