Month: May 2012

I’ve had a bit of trouble understanding how to get Heroku to cache pages from Swing Out London, so here’s my explanation – hopefully it’ll help someone.

The TL:DR

Varnish Caching on Heroku IS as simple as the docs state but because Heroku has multiple Varnish servers, any given request is unlikely to return a cached copy unless your site has a lot of traffic, or you set max-age to a large number.

The Basics

Some of the Heroku stacks (currently Aspen and Bamboo) support basic caching using Varnish – a service which sits in front of your application server and serves up cached content. It essentially works in the same way as your browser cache, except that when the varnish cache serves up your content, it

Telling Varnish to cache your content is the same as telling a user’s browser to cache content for a limited time:

class WebsiteController < ApplicationController

  def index
    response.headers['Cache-Control'] = 'public, max-age=300'
    ...
  end

In this case it says “this page won’t change in the next 5 minutes (300 seconds), so you don’t need to request it again from the App”.

For further info, check the Heroku docs.

The Problem

When I first tried this, it didn’t seem to be working at all – the response header was being set on the page, and Varnish was being used, but the request was reaching the application. Here are some example http headers:

HTTP/1.1 200 OK
Server: nginx
Date: Mon, 14 May 2012 14:59:14 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Cache-Control: public, max-age=1200
Etag: "913fba41febb1c5111ac375f09e88f46"
X-Ua-Compatible: IE=Edge,chrome=1
X-Runtime: 7.055496
Content-Length: 45804
Accept-Ranges: bytes
X-Varnish: 878887772
Age: 0
Via: 1.1 varnish

Note that Age is 0, indicating that this is a fresh page retrieved from the application rather than the cache. Also worth noting is that the X-Varnish header is the ID that Varnish assigns to the request.

After making about 10 requests I found one which was returned from the cache:

HTTP/1.1 200 OK
Server: nginx
Date: Mon, 14 May 2012 14:59:17 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Cache-Control: public, max-age=1200
Etag: "913fba41febb1c5111ac375f09e88f46"
X-Ua-Compatible: IE=Edge,chrome=1
X-Runtime: 1.824720
Content-Length: 45804
Accept-Ranges: bytes
X-Varnish: 793821349 793811181
Age: 40
Via: 1.1 varnish

This says “This page is 40 seconds old, and was originally cached as part of request 793811181”.

So it seemed like Varnish was caching, but only intermittently. A number of other people seemed to have the same problem.

The explanation

It looks like Heroku has multiple Varnish instances running independently, so when you request a page you could be hitting any of those instances. Here’s a likely scenario for a site with low traffic:

Let’s assume there are 3 varnish instances: A, B and C (Heroku maybe has 10) and represent the age of the cached pages on those instances as [a, b, c]. Now if the max-age is set to 25 seconds, and I make a number of repeated requests, I might see the following:

1. [-, -, -] On the first request I get sent to A. A doesn’t have a cached copy so requests the page from the server and caches it:
2. [10, -, -] Ten seconds later I make another request and get sent to B. Ditto.
3. [20, 10 , -] Ten seconds later I make another request and I get sent to C. Ditto.
4. [30, 20 , 30] Ten seconds later I make another request and I get sent to A again, it has a cached copy, but it’s older than the max-age so I discard it and request the page again from the server.

If on the other hand those requests were made every 5 seconds (i.e. if the site had higher traffic) then on the fourth request the cached copy would only be 15 seconds old so would be returned.

The solution

Increasing the max-age makes a big difference – I’ve set mine to 20 minutes for the time being. In slow periods not many pages will be served from the cache, but when traffic increases, the number of requests to the server will remain relatively similar.

The thing to understand is this: if the number of varnish instances is N and you’ve set the max age to A, then of the requests made in any period of A seconds, on average N of them will hit your server.

Many thanks to Garry Shulter for helping me understand.

Caveat: my understanding may not be entirely correct – if I’ve made any mistakes, please let me know in the comments!