Google App Engine/Python - Change logging formatting

Question

How can one change the formatting of output from the logging module in Google App Engine?

I've tried, e.g.:

  log_format = "* %(asctime)s %(levelname)-8s %(message)s"
  date_format = "%a, %d %b %Y %H:%M:%S"

  console = logging.StreamHandler()
  fr = logging.Formatter(log_format)
  console.setFormatter(fr)

  logger = logging.getLogger()
  logger.addFilter(SuperfluousFilter())
  logger.addHandler(console)

  logger.setLevel(logging.DEBUG)
  console.setLevel(logging.DEBUG)

  logging.error("Reconfiguring logging")

However this results in duplicate logging output: One with the logging handler from google/appengine/tools/dev_appserver.py (or somewhere in the Google code), and one from my new StreamHandler above. The above code outputs:

ERROR    2010-06-23 20:46:18,871 initialize.py:38] Reconfiguring logging
2010-06-23 20:46:18,871 ERROR    Reconfiguring logging

Where the top line is clearly from dev_appserver.py, the bottom line from my code.

So I guess the corollary question is: How can change the formatting of Google App Engine, yet avoid the duplicate output?

Madge

Answer 1

Here is one way you can change the logging format without duplicating output:

# directly access the default handler and set its format directly
logging.getLogger().handlers[0].setFormatter(fr)

This is a bit of a hack because you have to directly access the handlers list stored in the root logger. The problem is GAE automatically uses logging before your code is ever run - this creates a default handler. Unfortunately, I don't see how you can get a reference to this handler without directly accessing the handlers list as above.

Borszcz

That's very helpful. Another option I've considered running setLevel(logging.CRITICAL) on the root logger, and creating a new logger to stderr. – Madge