What is the different between the get logger functions from celery.utils.log and logging?

Question

I have been trying to figure out the difference between the python logger and the celery logger, specifically the difference between the commands below, but cannot find a good answer.

I am using celery v3, with django 1.10.

from celery.utils.log import get_task_logger
logger = get_logger(__name__)
...
from celery.utils.log import get_task_logger
logger = get_task_logger(__name__)
...
import logging
logger = logging.get_logger(__name__)

The celery documentation (latest, v3.1) is very lacking on this topic. I have looked at similar questions such as this one, but it is still it unclear which to use, why to use that one, and specifically what the differences are. I am looking for a clear, concise answer.

I am also using sentry in my production environment. How does this choice affect your sentry logs? i.e. these common settings

Makhachkala

Answer 1

From experience using get_task_logger seems to get you a few things of importance, especially with Sentry.

• Auto prepending task names to your log output
• The ability to set log handling rules at a higher level than just module (I believe it's actually setting the logger name to celery.task)
• Probably, most importantly for Sentry setup, is it hooks the logging into their log handlers which Sentry makes use of.

Important: There is a bit of extra config that needs to go into Celery registration for Sentry:

https://docs.sentry.io/clients/python/integrations/celery/

You may be able to get errors to flow into Sentry without some of this setup, but I think this will give you the best traces and details + ensure that things like expected exceptions declared via throws are properly ignored.

Britneybritni

This is helpful. I am going to test a bit before I accept. Do you know if the logger in functions (i.e. util functions, manager functions etc) called in tasks will also have this property? My impression is no unless they are also called with get_task_logger. – Makhachkala

Just to clear I need to put something like 'celery.task': { 'level': 'INFO', 'handlers': ['console', 'sentry'], 'propagate': False, }, in my config when I use get_task_logger? – Makhachkala

@Makhachkala "Do you know if the logger in functions (i.e. util functions, manager functions etc) called in tasks will also have this property?" Good question. Based on what i'm seeing in Sentry, it looks like functions from further down the stack report to the logger they are using and although they do make it to Sentry are somewhat lacking in context versus if the same entry was captured in normal execution. I can also say, we do not have any special handling for celery.task in our loggers. This setup got things flowing: docs.sentry.io/clients/python/integrations/celery – Britneybritni

@Makhachkala In combo with the mentioned celery config, adding sentry as a root handler seemed to be sufficient: gist.github.com/kcolton/a4a7ef2c2a77fd0872efe0323d190779 – Britneybritni