Adding Celery settings - what does it do?

Question

We upgrade Celery and started to get a warning:

CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine whether broker connection retries are made during startup in Celery 6.0 and above. If you wish to retain the existing behavior for retrying connections on startup, you should set broker_connection_retry_on_startup to True.

We are using redis as broker. I try to understand what does means: "broker connection retries are made during startup" - Do we need this? Do we dont need this? What is this? Why we should use it?

Skycap

Answer 1

This is a deprecation warning for a change that will come up with Celery 6. Apparently they decided to introduce a new, boolean configuration parameter broker_connection_retry_on_startup that can be used to tell Celery whether it should retry to establish connection during the startup. As you know, when you start Celery worker, it will almost immediately try to communicate with the broker. If the broker is not running or there are issues with network, worker will keep retrying to establish communication with the broker (in your case Redis).

Digitigrade

what are the effects of not turning this on? If the network is down at the moment of creating the worker, what will happen? will it shut down? – Earsplitting

Answer 2

Besides the excellent response from @DejanLekic, you can utilize the following settings to silence the warning by putting the below code into celery.py.

celery_app.conf.broker_connection_retry_on_startup = True

Or using django settings file

CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True

Duty