Celery beat queue includes obsolete tasks

Question

I'm using periodic celery tasks with Django. I used to have the following task in my app/tasks.py file:

@periodic_task(run_every=timedelta(minutes=2))
def stuff():
  ...

But now this task has been removed from my app/tasks.py file. However, I keep seeing call to this task in my celery logs:

[2013-05-21 07:08:37,963: ERROR/MainProcess] Received unregistered task of type u'app.tasks.stuff'.

It seems that the celery beat scheduler that I use does not update its queue. This is how the scheduler is defined in my project/settings.py file:

CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"

Restarting the celery worker does not help. FYI, I use a Redis broker.

How can I either clear or update the celery beat queue so that older tasks are not sent to my celery worker?

Expository

I think you have to remove the database entry for the schedule as well. You can do so from the django admin. – Casas

Hi Regis, did you manage to resolve this problem. I have the same issue. @Casas - your solution is – Dysphagia

The problem could be in the redis server. Since the celery tasks are a pickled object that were pushed into the redis server, the redis keep holding the obsolete tasks. Restart the redis server, or call to the purge function of the task. – Typhogenic

Answer 1

Install django-celery.

As cited, this project is not needed to use celery but yet you need this to enable the admin interface at /admin/djcelery/ for managing periodic tasks. Initially there won't be no registered or periodic tasks.

Restart the beat and check the table Periodic tasks again. Beat would have added the existing scheduled tasks into that table with the interval or crontab defined in the settings or the decorators. There you can delete the unwanted tasks.

UPDATE: From celery4, it's recommended to use this package. https://github.com/celery/django-celery-beat

Program

Answer 2

Delete the .pyc file for where the task was originally written. Or, just delete all .pyc files in your projects directory.

This command should work:

find . -name "*.pyc" -exec rm -rf {} \;

How do I remove all .pyc files from a project?

Stereoscope

This doesn't work! I also tried removing celerybeat-schedule too. – Program