Disadvantages to rack-cache vs. Varnish in Heroku cedar stack?

Question

The previous 2 Heroku application stacks came with a Varnish layer which automatically reverse-proxy-cached content based on http headers.

The new Heroku cedar stack doesn't have this Varnish layer. Heroku suggests using rack-cache and memcache instead.

Does this have disadvantages compared to the previous stacks with the varnish layer? With rack-cache, aren't there fewer servers serving the caching layer, and in a less optimized way?

• http://devcenter.heroku.com/articles/http-routing
• http://devcenter.heroku.com/articles/http-caching
• http://devcenter.heroku.com/articles/cedar

Greenebaum

If you're using Rails > 3.1, you can throw your static assets up on a bamboo stack app so you get the benefits of Varnish for that, and leave the rest as a Cedar app. Or you can put them on a CDN described here: devcenter.heroku.com/articles/cdn-asset-host-rails31 – Bullfighter

Answer 1

There is absolutely no question that removing the varnish layer is a huge downgrade in cache performance -- both latency and throughput -- from Heroku's bamboo to cedar stack. For one, your application requests are competing with, and may be queued behind, cache hits for dyno time.

The disadvantages, to name a few, are: interpreted ruby (vs. compiled C) application level (vs. proxy level) memcached based (vs. in process memory based) blocking I/O based (vs. nonblocking I/O based). Any suggestion that the two caching strategies could compete at scale is ridiculous. You won't see much difference on a small scale. But if you have hundreds or even thousands of cache hits per second, varnish will not substantially degrade, while the cedar stack would require many dynos just to serve static assets in a performant fashion. (I'm seeing ~ 5-10ms dyno processing time for cache hits on cedar).

Cedar was built the way it was built not to improve performance, but to introduce new levels of flexibility over your application. While it allows you to do non blocking I/O operations like long polling, and fine grained control over static asset caching parameters, it's clear that heroku would like you to bring your own caching/bandwidth if your goal is internet scale.

Avesta

Answer 2

I don't know how the rack-cache performance compares to Varnish in terms of raw requests. The best bet would be to create a simple app benchmark it and switch stacks.

It's worth bearing in mind that because the heroku.com stack was Nginx->Varnish->App as long as you've set the correct HTTP headers you're App layer will be doing much less work. As most of the delivery will be handled by Varnish, as well as Varnish being pretty fast, this frees up your Dyno for actual app-handling-requests.

With the herokuapp.com stack hitting your app earlier, it's down to you and your app to handling caching efficiently, this might mean you choose to use rack-cache to cache full page output or you might choose to use memcached to deal with database requests or both.

In the end it depends on what your app is doing, if it's serving the same static content to a lot of users then you'd benefit from Varnish, but if you've got an application where users login and interact with content then you won't see might benefit from Varnish so caching partial content or raw database queries might be more efficient. If you install the New Relic addon you'll be able to take a peek under the hood and see what's slowing your app down.

Entasis

Okay, you've confirmed what I suspected, that the app dyno is doing the work of serving the cache instead of an independent layer (which is sort of obvious but never at all mentioned in the heroku docs so I wanted to confirm). – Greenebaum

@Entasis - "you choose to use rack-cache to cache full page output or you might choose to use memcached to deal with database requests or both." What commands would be used to specify a page shoudl be served via rack-cache vs. memcached? If I do a "caches_action :show" will that go into rack-cache or memcached? Similarly, if I do a Rails.cache.fetch {...} will that use Memcached for both the meta & entity information? Is there any way to use Rack-cache to store the results of rails.cache.fetch so that I don't fill up my memcache with entity information (i.e. meta only)? – Earwitness

@Earwitness might be better to throw that into a new question, think rack-cache and rails.cache.fetch behave differently and I am certainly no rails expert :) – Entasis

@Entasis thanks for the reply! I have tried in a separate question with not much luck so far, so trying to fish around :) If you do know anyone who might be able to answer that one I'd appreciate it if you could pass me along. – Earwitness

Answer 3

There are also 3rd party options for hosted Varnish. I wrote a quick post about setting it up with Fastly/Varnish.

Fastly is a hosted Varnish solution that will sit in front of your Heroku app and serve cached responses.

Update link: https://medium.com/@harlow/scale-rails-with-varnish-http-caching-layer-6c963ad144c6

I've seen really great response times. If you can get a good cache hit-rate with Varnish you should be able to throttle back a good percentage of your dyno's.

Spelldown

Should link to medium.com/@harlow/… I think ? – Torpid

Answer 4

A more modern answer, with 20/20 hindsight:

To get the caching performance approaching that of bamboo on cedar-14, this is the usual pattern:

1. Configure Rails to generate appropriate caching headers (i.e., Etags, Cache-Control, Last-Modified, etc.)
2. Stick fastly (varnish as a service) or cloudflare in front of your app. If app headers were set correctly, profile-like pages that need to be fresh won't be cached, as opposed to static assets.

I would recommend redis-rails as a rack cache backend, if you're doing caching at multiple layers (FE (CF/FY), page, action, fragment, etc.).

Vigorous