How can I change the logging format in Google AppEngine

Question

AppEngine uses JUL for its logging and I have configured the logging.properties file and reference to that in appengine-web.xml

The problem is that the format that AppEngine presents the data in the console log gets truncated after about 180 chars on each log line. And since a large part of that is taken up with the method and class name (including package) and date there isn't much of the actual log message that comes out.

I have tried to configure my own Formatter, both programatically and via logging.properties without luck.

I realise that I could push all my log through slf4j, logback or log4j but I believe that doing so causes all such log to appear as stdout in the AppEngine log console which has its own style of verbosity.

Is there a way to define a particular format for the AppEngine log and if so how? t would be enough if each log line wasn't getting truncated to 180 chars.

Affenpinscher

I'm having this same problem. I've created a custom Formatter, and specified it in my logging.properties, but it seems to be completely ignored by appengine. I'm fairly sure my logging.properties file is being picked up because changing the log level does work. – Pulley

Good question, but I think the summary is misleading: the problem is that the log lines displayed in app engine console are truncated. – Music

Answer 1

Have you tried using the log download feature: It does not truncate the logs, you get to see everything, and can specify the number of days and severity of which log entries to download.

C:\eclipse\plugins\com.google.appengine.eclipse.sdkbundle_1.7.2.1\appengine-java-sdk-1.7.2.1\bin>appcfg.cmd --severity=DEBUG --num_days=2 request_logs C:\workspace\my_app\war C:\workspace\MY_LOGS.TXT

<path to SDK>/appcfg.cmd --severity=DEBUG --num_days=2 request_logs <path to war> <path to log>

That is the only way I found to be able to see my log details. The default log level is INFO, so you really have to use the severity and num_days arguments for it to be useful.

Additionally, if you want to download the logs for a backend, then you need to specify the backend version using the following option on the command line. Where worker is the name of the backend.

 --version=worker

Inessive

Here is the link to the documentation for the log download feature. developers.google.com/appengine/docs/java/tools/… – Inessive

Thanks Greg, downloading the log gives me the ability to read long log lines. But it's still a pain in the ass to read and I can only access them from a machine that also has an app-engine-sdk on it (which is only about 30% of the time). I'm really looking for a way to configure the log format so that I have ready to read logs from the browser. – Affenpinscher

Actually, if you look inside that script file I mentioned you will see that it is really only starting the java application, and that you only need to carry with you that single JAR file. – Inessive

java -cp "%~dp0\..\lib\appengine-tools-api.jar" com.google.appengine.tools.admin.AppCfg %* This jar can easily be "installed" on any machine you wish. – Inessive

Answer 2

After digging into this a bit further, I couldn't get a custom JUL Handler or Formatter to get picked up by the appengine runtime. So instead, I created a ServletContextListener to initialize the root logger with my own custom formatter, as follows:

<listener>
   <listener-class>MyLoggingListener</listener-class>
</listener>

Here's a sample ServletContextListener:

public class MyLoggingListener implements ServletContextListener
{
    private static Logger LOG;

    @Override
    public void contextInitialized(ServletContextEvent sce)
    {
        LOG = Logger.getLogger("");
        Handler[] handlers = LOG.getHandlers();
        for (Handler handler : handlers)
        {
            handler.setFormatter(new MyLogFormatter());
        }
    }

    @Override
    public void contextDestroyed(ServletContextEvent sce)
    {}
}

Update: The above code works in the Dev Environment, but seems to be ignored on Appengine. Back to the drawing board on this one.

Pulley

Answer 3

Looking through the source code for the App engine configuration tool itself, I have seen that all it is doing it calling the same LogService which you would have access to yourself through the LogService API.

Therefore, the answer to your question really is as simple as dropping the code from the following link into your code base, securing it to only the Admin user and problem solved.

https://developers.google.com/appengine/docs/java/logservice/overview

Inessive

Well it seems like mass overkill to hardcode a page to retrieve the logs in a better format because AppEngine doesn't seem capable (or no-one knows how) to configure a Formatter for the logger that works. But hell I guess that answers my questions thanks. – Affenpinscher

Final note: I realised that clicking on the plus icon expands the log lines to show the full log record. So I do have access to my full record, it's just in a somewhat clunky format. – Affenpinscher