Celery worker and beat load in one command

Question

Is there a way to start the celery worker and beat in one command? I would like to add celery to my automated deployment procedure with Fabric.

I am currently running:

celery -A prj worker -B

followed by

celery -A prj beat -l info -S django

However, the first command starts the worker and does not allow the next command (starting the beat) to be run because the worker start-up messages appear.

Is there a way to avoid the start-up messages appearing? Or do both of these actions in one command? Perhaps there is even a way to start these from my Django config?

Thanks!

Clarhe

Answer 1

Celery allows for you to run the worker and beat in the same process (mostly used for development purposes). Check out the help for celery worker:

> celery worker -h

...

Embedded Beat Options:
  -B, --beat            Also run the celery beat periodic task scheduler. Please note that there must only be
                        one instance of this service. .. note:: -B is meant to be used for development
                        purposes. For production environment, you need to start celery beat separately.
  -s SCHEDULE_FILENAME, --schedule-filename SCHEDULE_FILENAME, --schedule SCHEDULE_FILENAME
                        Path to the schedule database if running with the -B option. Defaults to celerybeat-
                        schedule. The extension ".db" may be appended to the filename. Apply optimization
                        profile. Supported: default, fair
  --scheduler SCHEDULER
                        Scheduler class to use. Default is celery.beat.PersistentScheduler

So the combined command, including your use of the django scheduler, would be the following:

celery -A prj worker --beat --scheduler django --loglevel=info

Glanti

Important to notice: -B is meant to be used for development purposes. For production environment, you need to start celery beat separately. (even if I can't explain why) – Overskirt

@DavidDahan that's probably to avoid accidental launching of several beat instances when scaling workers – Kayo

Answer 2

celery -A proj worker -l info -B

Kaela

Please put your answer always in context instead of just pasting code. See here for more details. – Cutup