Tapestry Web App on Tomcat occasionally spits out garbage

Question

We have a Tapestry-Spring-Hibernate webapp running on Tomcat 6, handing about a few 1000 requests a second. Randomly, for no apparent reason, a page just displays a bunch of random characters on the browser. However, when the page is refreshed, it displays fine. Here is a screen-shot of the source of the garbled page on Chrome:

Here is what I have found so far:

1. It doesn't seem to be browser specific. I have witnessed this on Chrome and Firefox, but users have also reported this on IE 7 and up.
2. Load on the server seems to have no correlation to when this happens.
3. Refreshing the page displays the page normally, as if nothing ever happened.
4. I don't see anything relevant in the server or the application logs
5. The content-type tag for the page is <meta content="text/html; charset=utf-8" http-equiv="content-type"/>
6. There are a couple other webapps deployed on the same container, one being Alfresco, but they don't seem to exhibit this at all.

My question is, has someone run into this before, and if so, can they point me to where I should start looking? Is this a problem with the page having something like the incorrect content-type or the server not being able to handle it for some reason? Or could this be a framework bug in Tapestry or in the application itself? Any pointers are welcome. At this point, I am not sure where the problem is, so I wasn't sure if this goes on ServerFault or stays here.

Parlous

Answer 1

This seems to be related to gzip compression issues in the Tapestry framework (as suggested by @barnyr) and is possibly a regression bug in Tapestry 5.3. To quote Howard from a mailing list thread:

I believe this was a bug where under certain circumstances, a corrupt GZIP stream of page content would be streamed to the client; this is fixed in 5.2.6 for sure, but I thought it was fixed in 5.2.5 as well.

The quick fix is to add the following configuration symbol in the contributeApplicationDefaults method of the app's module class:

configuration.add(SymbolConstants.GZIP_COMPRESSION_ENABLED, "false");

This of course disables gzip compression, but might a trade-off worth making.

Possibly related issues:

• GZip compression should be disabled if the request is over http 1.0
• Is GZIP compression compatible with XmlHttpRequest?

Parlous

You can just as easily enable gzip compression at container level, e.g. viralpatel.net/blogs/enable-gzip-compression-in-tomcat or google for more info. – Lionize

Answer 2

It is most likely a bug in the application. (Most bugs are ... despite the natural tendency of programmers to blame something else.)

However, could be a bit tricky to track down this problem. I suggest that you start with the standard things:

• Look at the server error logs to see if anything strange shows up at the time when one of these "events" occur.
• Look at the server access logs to see if you can identify the request that produced the garbage data.
• Enable your browser's debugger and see if you can track down the bad request that way.

If you can figure out what the request that produced the bad response was, you'll have more traction in finding the cause.

---

FWIW - that doesn't look like the result of a character encoding problem. That looks more like binary or compressed data.

Photogene

Does your app compress the response? It looks like zipped data without the headers to tell the browser that it's compressed. Is there a web server in front of your app? – Noctambulous

Yes, the app gzips the response. And no, we don't have a web server fronting it. We used to, but that was a whole different set of problems. I think you might be on to something there with the gzip headers. I'll start looking there thanks! – Parlous

Answer 3

Here's one situation that has led me to see a garbled page. On the error page Tapestry sets a response header called X-Tapestry-ErrorMessage. Evidently newlines aren't allowed in headers (at least on some browsers) so if that header has a newline in it, you get the gibberish. One error message we were setting happened to have a trailing new line. I changed it to remove any newlines before setting that header and then the error page showed correctly.

Haggar