Django Celery results set task id to something human readable?

Question

After many days I have a working celery and celery beat task list, and the results are stored using django_celery_results. However when I look at the table record, it hasn't got any useful information in there.

is it possible to set the task id to something human readable?

one example would be using the demo task, it returns the no, but an unreadable task id

tasks.py

@app.task
def test(a,b):
    return a + b

scheduler in app.settings

CELERYBEAT_SCHEDULE = {
    'test_task': {
        'task': 'home.tasks.test',
        'schedule': crontab(minute='*/1'),
    },

Cycloplegia

Answer 1

The easy answer is No. The task_id attribute is generated automatically. If you follow the code backwards the core function that generates ID's is in kombu.utils.uuid.uuid(..), which interestingly is just a thin wrapper around the builtin uuid.uuid4(..).

However, if you look at the function signature:

def uuid(_uuid=uuid4):
    """Generate unique id in UUID4 format.
    See Also:
        For now this is provided by :func:`uuid.uuid4`.
    """
    return str(_uuid())

..it appears like it would be possible to supply your own function that generates the ID, and as long as they're unique they should work. Fundamentally you'd have to patch: celery.__init__.uuid, celery.utils.__init__.uuid, and celery.utils.__init__.gen_unique_id.

I don't think that you'd be able to apply worthwhile names since this function is called with no parameters to simply return something unique.

BUT

If you look at Task.apply and Task.apply_async there's an undocumented parameter task_id!

This would allow you to manually specify the task_id before the Task is called/created, and as long as they're globally unique you'll still get all of the reporting and metrics. Unfortunately I don't see any easy way to change the task_id from a Result, to make it more useful after the fact...kinda making your UI's textbox a little silly.

---

If your project is set up for it, you can use Task Inheritance and modify task behavior that way. In theory you should be able to overwrite apply and apply_async to inject a task's ID.

import time

from celery import Task

 class NamedTask(Task):
    id_gen = lambda: int(time.time() * 1000)


    def _gen_task_id(self):
        return {
            'task_id': '%s-%s' % (
                self.name,
                self.id_gen())}

    def apply(self, *args, **kwargs):
        kwargs.update(self._gen_task_id())
        return Task.apply(self, *args, **kwargs)

    def apply_async(self, *args, **kwargs):
        kwargs.update(self._gen_task_id())
        return Task.apply_async(self, *args, **kwargs)



@task(base=NamedTask)
def add(x, y):
    return x + y

Should make your Task ID look like: project.tasks.add-15073315189870

Salubrious

Hi I dont want to alter the task ID, that is fine, I was wondering if I can add another field to the model where I can give it a description and see it in the django_celery_resutls DB view? – Cycloplegia

@AlexW, your question should have been clear on that. Because now the meaning of the question itself changes. And your title clearly stated set task id to something human readable?. – Bunn

apologies for the confusion, re reading this I can see how the question needs improvment – Cycloplegia

Answer 2

Implement a custom result backend that overwrites _store_result to decide what is saved as result to the database.

Depending on which backend you're using find the related class in celery.backends.

This example extends the result for an amqp backend.

class UsefulInfoBackend(AMQPBackend):
    def store_result(self, task_id, result, state,
                     traceback=None, request=None, **kwargs):
        result = super(UsefulInfoBackend, self).store_result(task_id, result, state,
                     traceback=None, request=None, **kwargs)
        result['useful_info'] = 'Very Useful! :)' # determine the rules for extraneous information here contains. 
        return result

When initializing Celery pass your result backend class.

celery.Celery(backend=UsefulInfoBackend)

Sinapism

Answer 3

In my case , I tried the next line of code and it worked like charm

>>> my_celery_task.apply_async((2, 2), task_id='human-readable')
>>> result = AsyncResult(my_celery_task.task_id)
>>> result
human-readable

Orifice

Answer 4

Try:

@app.task(name="periodic_test")
def test(a, b):
    return a+b

This may help to find the task status in the UI, but Celery always needs a unique id for each task execution.

Mithras