Running Django-Celery in Production

Question

I've built a Django web application and some Django-Piston services. Using a web interface a user submits some data which is POSTed to a web service and that web service in turn uses Django-celery to start a background task.

Everything works fine in the development environment using manage.py. Now I'm trying to move this to production on a proper apache server. The web application and web services work fine in production but I'm having serious issues starting celeryd as a daemon. Based on these instructions: http://docs.celeryproject.org/en/latest/tutorials/daemonizing.html#running-the-worker-as-a-daemon I've created a celeryconfig.py file and stuck it in the /usr/bin directory (this is where celeryd is location on my arch linux server).

CELERYD_CHDIR="/srv/http/ControllerFramework/"
DJANGO_SETTINGS_MODULE="settings"
CELERYD="/srv/http/ControllerFramework/manage.py celeryd"

However when I try to start celeryd from the command line I get the following error:

 "Missing connection string! Do you have "
celery.exceptions.ImproperlyConfigured: Missing connection string! Do you have    CELERY_RESULT_DBURI set to a real value?

Not sure where to go from here. Below is my settings.py section as it pertains to this problem:

BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "controllerFramework"
BROKER_PASSWORD = "******"
BROKER_VHOST = "localhost"

Lackaday

Is CELERY_RESULT_DBURI actually set somewhere? – Combative

I tried setting that value to the following: CELERY_RESULT_BACKEND = 'amqp' But I get an error when I try to run celeryd: [2010-07-23 12:46:13,138: WARNING/MainProcess] raise KeyError("Missing hostname for AMQP connection.") [2010-07-23 12:46:13,138: WARNING/MainProcess] KeyError [2010-07-23 12:46:13,138: WARNING/MainProcess] : [2010-07-23 12:46:13,139: WARNING/MainProcess] 'Missing hostname for AMQP connection.' Since I got this error and I couldn't find anywhere in the documentation for django-celery that this value needed to be set I assumed that I was doing something wrong, again. – Lackaday

I'm gonna take a guess that maybe when running in production the settings are supposed to be in celeryconfig.py? and not in settings.py. In that case, were is the celeryconfig.py file supposed to be added? – Lackaday

So I've added the line for CELERY_RESULT_DBURI, even though none of the documentation I've seen asks me to. Now when I run celeryd I get this error: 'Missing hostname for AMQP connection.' I've tried tinkering with the settings file but no matter what I do I get that error. – Lackaday

Answer 1

So I ended up having a chat with the project lead on django-celery. Couple of things. First off celery must be run using 'manage.py celeryd'. Secondly, in the settings.py file you have to 'import djcelery' This import issue may be fixed in the next version but for now you have to do this.

Lackaday