trouble in setting celery tasks backend in Python

Question

I followed all the steps given in [ http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html ] This is the code:

from __future__ import absolute_import
from celery import Celery

#app = Celery('tasks', broker='pyamqp://guest@localhost//')
app = Celery('tasks', backend='redis://localhost', broker='pyamqp://guest@localhost//')
@app.task
def add(x, y):
   return x + y

When I run celery worker using the following command

celery -A tasks worker --loglevel=info

I get a syntax error for setting the backend. This is the error message:

[2018-07-10 16:37:21,970: CRITICAL/MainProcess] Unrecoverable error: SyntaxError('invalid syntax', ('c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\celery\backends\redis.py', 22, 19, 'from . import async, base\n'))Traceback (most recent call last): File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\kombu\utils\objects.py", line 42, in get return obj.dict[self.name] KeyError: 'backend' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\celery\worker\worker.py", line 205, in start self.blueprint.start(self) File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\celery\bootsteps.py", line 115, in start self.on_start() File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\celery\apps\worker.py", line 139, in on_start self.emit_banner() File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\celery\apps\worker.py", line 154, in emit_banner ' \n', self.startup_info(artlines=not use_image))), File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\celery\apps\worker.py", line 217, in startup_info results=self.app.backend.as_uri(), File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\kombu\utils\objects.py", line 44, in get value = obj.dict[self.name] = self.get(obj) File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\celery\app\base.py", line 1196, in backend return self.get_backend() File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\celery\app\base.py", line 914, in get_backend self.loader) File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\celery\app\backends.py", line 70, in by_url return by_name(backend, loader), url File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\celery\app\backends.py", line 50, in by_name cls = symbol_by_name(backend, aliases) File "c:\users\user_\appdata\local\programs\python\python37-32\lib\site-packages\kombu\utils\imports.py", line 56, in symbol_by_name module = imp(module_name, package=package, **kwargs) File "c:\users\user_\appdata\local\programs\python\python37-32\lib\importlib_init_.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1006, in _gcd_import File "", line 983, in _find_and_load File "", line 967, in _find_and_load_unlocked File "", line 677, in _load_unlocked File "", line 724, in exec_module File "", line 860, in get_code File "", line 791, in source_to_code File "", line 219, in call_with_frames_removed File "c:\users\user\appdata\local\programs\python\python37-32\lib\site-packages\celery\backends\redis.py", line 22 from . import async, base ^ SyntaxError: invalid syntax

However, when I use the commented line instead I have no issues just that the results backend is disabled and I need to set the results backend to redis-server

Fajardo

Answer 1

I solved the problem. The main cause of the problem was that I was using Python 3.7. But, to my knowledge, Celery currently works with Python 3.6 and lower. I made the following changes to the Celery code:

1. Renamed "C:\Users\myusername\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\celery\backends\async.py" to "C:\Users\myusername\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\celery\backends\asynchronous.py"

2. Opened redis.py and changed every line that had the keyword "async" to "asynchronous".

Apparently,

async

is now a keyword in Python 3.

You can also read this link: https://github.com/celery/celery/issues/4500

Hopefully, this answer will help all those who have the same problem till a newer version of Celery is released.

---

UPDATE: This is the issue of Python 3.7. You could use Python 3.6 instead without such an issue. But, if you'd like to conitnue using Python 3.7 and celery[redis] you can use the above solution to resolve the issue.

Fajardo

"Celery currently works with Python 2 only". Incorrect. Issue is with Python 3.7, not Python 3 in general. – Nicolis

@C S Correct! if you use Python 3.6 you shouldn't have such an issue. But, Python 3.7 has the issue. – Fajardo

Besides your solution, some people may prefer pip install --upgrade https://github.com/celery/celery/tarball/master . Anyway, thanks for your question and answer – Nicolis

for rabbitmq you need to make modification in rpc.py accordingly. – Elizabetelizabeth

Also you may have to edit async.asynchronousBackendMixin in redis.py to asynchronous.AsyncBackendMixin. I had to for celery 4.2.2 – Bead

Answer 2

support @Ai Da,and you should maintain AsyncBackendMixin in redis.py.

Trickery

Answer 3

Did you try adding the redis port and db in the backend parameter? Or In the Celery config file adding the paramaters CELERY_REDIS_HOST CELERY_REDIS_PORT CELERY_REDIS_DB CELERY_RESULT_BACKEND CELERY_RESULT_PASSWORD

Kelley

Where can I find the celery config file? I am using Windows. – Fajardo

You have to make your own file for the app's configurations and call them in your main celery file. – Kelley

Alright. I added a celeryconfig.py and imported it in my main file. the config file content is BROKER_URL = 'pyamqp://guest@localhost//' CELERY_RESULT_BACKEND = 'redis://localhost' then I set app = Celery('tasks') but the backend is disabled – Fajardo

Use Something like: CELERY_REDIS_HOST = 'localhost' CELERY_REDIS_PORT = 6379 # Whatever the port redis is running on CELERY_REDIS_DB = 0 # usually defaults to 0 CELERY_RESULT_BACKEND = 'redis' CELERY_RESULT_PASSWORD = "your_password" – Kelley

Thanks but I still get "results: disabled://" – Fajardo

Answer 4

look like install with

pip install git+https://github.com/vBlackOut/django-celery.git --upgrade

work for me

Mesnalty