One second of micro-caching absorbs a traffic spike

The claim

Caching a dynamic page for one second sounds pointless. It is the single highest-leverage configuration change available to a small site, because it converts an unbounded number of origin requests per second into exactly one. A homepage that collapses at 90 concurrent users will serve 5,000 with the same hardware and a five-line Nginx block.

The arithmetic

Suppose your homepage takes 180 ms of application time and you have 8 worker processes. Your theoretical ceiling is roughly:

8 workers / 0.180 s = 44 requests per second

A local news mention or a mailing list send can produce 400 requests per second for ten minutes. At 44 per second the queue grows without bound and everyone gets a timeout, including the customers who were already mid-checkout.

With a one-second cache, the origin serves at most one request per second per cached URL, regardless of arrival rate. The other 399 are served from memory in under a millisecond. Your application's load during the spike is lower than on a quiet Tuesday.

The configuration

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=micro:10m
                 max_size=1g inactive=10m use_temp_path=off;

server {
    location / {
        proxy_pass http://app;
        proxy_cache micro;
        proxy_cache_valid 200 301 1s;
        proxy_cache_lock on;
        proxy_cache_use_stale updating error timeout http_500 http_502 http_503;
        proxy_cache_background_update on;
        add_header X-Cache $upstream_cache_status;
    }
}

Three of those lines do the important work.

proxy_cache_lock on means that when an entry expires, one request goes to the origin and the rest wait for it. Without this, all 400 requests arrive at your application the instant the entry expires — a thundering herd once per second, which is worse than no cache at all.

proxy_cache_use_stale with the error conditions listed turns your cache into a safety net. If the application starts returning 502s, visitors keep receiving the last good copy instead of an error page. This has covered more deploy mistakes than any amount of testing.

proxy_cache_background_update refreshes the stale entry without making anyone wait for it. Combined with the previous directive, no visitor ever pays the full render cost.

Not caching logged-in users

The obvious risk is serving one customer's session to another. Bypass on the session cookie:

proxy_cache_bypass $cookie_session_id;
proxy_no_cache     $cookie_session_id;

Both directives are needed and they do different things: bypass skips reading from the cache, no_cache skips writing to it. Setting only the first means an authenticated response gets stored and later handed to an anonymous visitor, which is the failure that ends up in a privacy report.

Also ensure your application sets Cache-Control: private on any authenticated response, so that a misconfiguration upstream cannot cause the same problem.

Vary, and the query string

Two other settings decide whether your hit rate is 98% or 3%. The first is the cache key. By default Nginx keys on the full URI including the query string, which means a campaign link carrying tracking parameters produces a distinct cache entry for every visitor. Strip the parameters you do not use before they reach the cache key, or normalise the key explicitly.

The second is the Vary header. A response that varies on User-Agent is effectively uncacheable, since there are thousands of distinct values in the wild. Varying on Accept-Encoding is fine and expected. Check what your framework emits before assuming.

Verify it is actually working

curl -sI https://example.ca/ | grep -i x-cache
for i in $(seq 1 5); do curl -sI https://example.ca/ | grep -i x-cache; sleep 0.3; done

You should see MISS followed by HIT, HIT, HIT. If every request is a MISS, something in the response is preventing caching — usually a Set-Cookie header on every request, which Nginx respects by refusing to store the response. Many frameworks issue a session cookie to anonymous visitors by default; turning that off for anonymous traffic is often the only application change required.

Choosing the duration

One second is the right default for a homepage or a busy listing page, because it is short enough that nobody perceives staleness and long enough to flatten any realistic spike. Ten seconds is reasonable for a catalogue. Sixty seconds is reasonable for an article. The only pages that should never be cached this way are cart, checkout, and account.

The configuration takes fifteen minutes. It is the cheapest capacity increase available to you, and unlike buying a larger instance, it costs nothing per month.