How to write an Ubuntu Upstart job for Celery (django-celery) in a virtualenv

Question

I really enjoy using upstart. I currently have upstart jobs to run different gunicorn instances in a number of virtualenvs. However, the 2-3 examples I found for Celery upstart scripts on the interwebs don't work for me.

So, with the following variables, how would I write an Upstart job to run django-celery in a virtualenv.

Path to Django Project:

/srv/projects/django_project

Path to this project's virtualenv:

/srv/environments/django_project

Path to celery settings is the Django project settings file (django-celery):

/srv/projects/django_project/settings.py

Path to the log file for this Celery instance:

/srv/logs/celery.log

For this virtual env, the user:

iamtheuser

and the group:

www-data

I want to run the Celery Daemon with celerybeat, so, the command I want to pass to the django-admin.py (or manage.py) is:

python manage.py celeryd -B

It'll be even better if the script starts after the gunicorn job starts, and stops when the gunicorn job stops. Let's say the file for that is:

/etc/init/gunicorn.conf

Jacobian

Answer 1

You may need to add some more configuration, but this is an upstart script I wrote for starting django-celery as a particular user in a virtualenv:

start on started mysql
stop on stopping mysql

exec su -s /bin/sh -c 'exec "$0" "$@"' user -- /home/user/project/venv/bin/python /home/user/project/django_project/manage.py celeryd

respawn

It works great for me.

I know that it looks ugly, but it appears to be the current 'proper' technique for running upstart jobs as unprivileged users, based on this superuser answer.

I thought that I would have had to do more to get it to work inside of the virtualenv, but calling the python binary inside the virtualenv is all it takes.

Salzhauer

might be nice to add your tweak... I posted mine here: #14276321 – Sinus

Answer 2

Here is my working config using the newer systemd running on Ubuntu 16.04 LTS. Celery is in a virtualenv. App is a Python/Flask.

Systemd file: /etc/systemd/system/celery.service

You'll want to change the user and paths.

[Unit]
Description=Celery Service
After=network.target

[Service]
Type=forking
User=nick
Group=nick
EnvironmentFile=-/home/nick/myapp/server_configs/celery_env.conf
WorkingDirectory=/home/nick/myapp
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
  --pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \
  -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
  --logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'

[Install]
WantedBy=multi-user.target 

Environment file (referenced above):/home/nick/myapp/server_configs/celery_env.conf

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/home/nick/myapp/venv/bin/celery"

# App instance to use
CELERY_APP="myapp.tasks"

# How to call manage.py
CELERYD_MULTI="multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
#   and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"

To automatically create the log and run folder with the correct permissions for your user, create a file in /usr/lib/tmpfiles.d. I was having trouble with the /var/run/celery folder being deleted on rebooting and then celery could not start properly.

My /usr/lib/tmpfiles.d/celery.conf file:

d /var/log/celery 2775 nick nick -
d /var/run/celery 2775 nick nick -

To enable: sudo systemctl enable celery.service

Now you'll need to reboot your system for the /var/log/celery and /var/run/celery folders to be created. You can check to see if celery started after rebooting by checking the logs in /var/log/celery.

To restart celery: sudo systemctl restart celery.service Debugging: tail -f /var/log/syslog and try restarting celery to see what the error is. It could be related to the backend or other things.

Hope this helps!

Baerman

Bro take a beer! You just saved my as** – Carnify