Python/Django: log to console under runserver, log to file under Apache

Question

How can I send trace messages to the console (like print) when I'm running my Django app under manage.py runserver, but have those messages sent to a log file when I'm running the app under Apache?

I reviewed Django logging and although I was impressed with its flexibility and configurability for advanced uses, I'm still stumped with how to handle my simple use-case.

Blairblaire

Simplest solution is to have different settings.py files for main server and development environment, see deploydjango.com/django_project_structure – Xenocrates

Answer 1

Text printed to stderr will show up in httpd's error log when running under mod_wsgi. You can either use print directly, or use logging instead.

python 3:

print("Goodbye cruel world!", file=sys.stderr)

python 2:

print >>sys.stderr, 'Goodbye, cruel world!'

Howlond

It's technically not valid WSGI though, and will trigger errors in more strict environments. – Xerophthalmia

There is nothing whatsoever wrong with using 'print' with 'sys.stderr' as far as WSGI goes and it should not trigger errors. – Hairsplitter

I imported sys but this doesn't seem to work for me. – Drury

This does not work in Python 3 have a look here. You need print("Goodbye cruel world!", file=sys.stderr) – Fadil

@Fadil did the trick ty. Crazy that even printing a line can be controversial. – Photoelectrotype

Answer 2

Here's a Django logging-based solution. It uses the DEBUG setting rather than actually checking whether or not you're running the development server, but if you find a better way to check for that it should be easy to adapt.

LOGGING = {
    'version': 1,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        'file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/path/to/your/file.log',
            'formatter': 'simple'
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    }
}

if DEBUG:
    # make all loggers use the console.
    for logger in LOGGING['loggers']:
        LOGGING['loggers'][logger]['handlers'] = ['console']

see https://docs.djangoproject.com/en/dev/topics/logging/ for details.

Mightily

also try LOGGING['loggers'][logger]['handlers'] += ['console'] – Selfgovernment

@m01: After configuring this into settings.py, how to use this for printing purpose? Thanks – Delict

I put the code from my answer into my settings.py towards the bottom, and set DEBUG = True (look for that setting near the top in the same file). Then, I run python manage.py runserver from a terminal (see django docs for details), and the log messages will appear in the terminal window. In production, I'd use a different settings.py, where DEBUG = False - the log messages go to /path/to/your/file.log. – Mightily

Your indentation gave me a headache. Thanks for the info though, it works! – Adamski

Thanks! I've made some changes to the indentation, I hope it's better now – Mightily

it doens't work for me can not print in console anything – Universality

if I want to add request's query_param into the log,how can I implement?? – Green

Please read documentation: docs.djangoproject.com/en/dev/topics/logging/… "When DEBUG is True: The django logger sends messages in the django hierarchy (except django.server) at the INFO level or higher to the console." – Ales

Answer 3

You can configure logging in your settings.py file.

One example:

if DEBUG:
    # will output to your console
    logging.basicConfig(
        level = logging.DEBUG,
        format = '%(asctime)s %(levelname)s %(message)s',
    )
else:
    # will output to logging file
    logging.basicConfig(
        level = logging.DEBUG,
        format = '%(asctime)s %(levelname)s %(message)s',
        filename = '/my_log_file.log',
        filemode = 'a'
    )

However that's dependent upon setting DEBUG, and maybe you don't want to have to worry about how it's set up. See this answer on How can I tell whether my Django application is running on development server or not? for a better way of writing that conditional. Edit: the example above is from a Django 1.1 project, logging configuration in Django has changed somewhat since that version.

Dantzler

I don't want to rely on DEBUG; I'd rather depend on the dev-server detection mechanism linked in that other post. But the other post's detection mechanism relies on having access to a request instance. How can I get a request instance in settings.py? – Blairblaire

Answer 4

I use this:

logging.conf:

[loggers]
keys=root,applog
[handlers]
keys=rotateFileHandler,rotateConsoleHandler

[formatters]
keys=applog_format,console_format

[formatter_applog_format]
format=%(asctime)s-[%(levelname)-8s]:%(message)s

[formatter_console_format]
format=%(asctime)s-%(filename)s%(lineno)d[%(levelname)s]:%(message)s

[logger_root]
level=DEBUG
handlers=rotateFileHandler,rotateConsoleHandler

[logger_applog]
level=DEBUG
handlers=rotateFileHandler
qualname=simple_example

[handler_rotateFileHandler]
class=handlers.RotatingFileHandler
level=DEBUG
formatter=applog_format
args=('applog.log', 'a', 10000, 9)

[handler_rotateConsoleHandler]
class=StreamHandler
level=DEBUG
formatter=console_format
args=(sys.stdout,)

testapp.py:

import logging
import logging.config

def main():
    logging.config.fileConfig('logging.conf')
    logger = logging.getLogger('applog')

    logger.debug('debug message')
    logger.info('info message')
    logger.warn('warn message')
    logger.error('error message')
    logger.critical('critical message')
    #logging.shutdown()

if __name__ == '__main__':
    main()

Gelsemium

Answer 5

You can do this pretty easily with tagalog (https://github.com/dorkitude/tagalog)

For instance, while the standard python module writes to a file object opened in append mode, the App Engine module (https://github.com/dorkitude/tagalog/blob/master/tagalog_appengine.py) overrides this behavior and instead uses logging.INFO.

To get this behavior in an App Engine project, one could simply do:

import tagalog.tagalog_appengine as tagalog
tagalog.log('whatever message', ['whatever','tags'])

You could extend the module yourself and overwrite the log function without much difficulty.

Magnetohydrodynamics

Answer 6

This works quite well in my local.py, saves me messing up the regular logging:

from .settings import *

LOGGING['handlers']['console'] = {
    'level': 'DEBUG',
    'class': 'logging.StreamHandler',
    'formatter': 'verbose'
}
LOGGING['loggers']['foo.bar'] = {
    'handlers': ['console'],
    'propagate': False,
    'level': 'DEBUG',
}

Graph