Request Logs for All Versions of Google App Engine Application

Question

Does anyone know how to request all logs of a Google App Engine application? By all logs, I mean all app versions logs.

We increment the version of our app with every deployment. I have a script that pulls the log files every 30 minutes for some offline analytics.

This is kinda what my script is calling right now. appcfg.py --append --num_days=0 --include_all request_logs /opt/gae/appname/ /var/log/gae/appname.log

This does not get all app versions logs files, but only the version that's specified in the /opt/gae/appname/app.yaml file. The problem is that whatever is in that app.yaml file might not be live, but rather still in development.

It would be nice if we had --version=0 flag that pulled the logs for all the versions, like it does for --num_days=0.

Nymphalid

Why not just keep the same major version ID, if you need this functionality? – Metsky

We use versioning so we can test different versions in production and roll back and forward between them if we need to. – Nymphalid

Answer 1

You can fetch the logs using the LogService Api, see parameter version_ids. I'm sure you could modify appcfg.py, to download all version logs.

You could add an issue asking for this feature at http://code.google.com/p/googleappengine/issues/list.

Thermos

Thanks for your post, but that's not any different than passing "--version" thru appcfg.py request_logs. I will probably add it as an issue. Thanks again. – Nymphalid

So even better (somehow I missed that option), you could easily create a script that calls appcfg for each version you have ever uploaded. Bu t in practice you only want to call request_logs for the active versions. – Thermos

What we ended up querying a web page on which we display the current version. So, we fetch that via CURL and then we use the --version flag to pass that info to the script that pulls the logs. Kind of a hack, but works pretty well. We will write up a blog post with the details and share it here soon. – Nymphalid

The original problem still exists. Is there an issue for this now? – Jeffryjeffy