Put Varnish Cache in Front of Django

Problem statement

A content-heavy Django site, such as a news site, a CMS, or a catalog, serves the same anonymous pages thousands of times, and each render costs database queries and template time. A shared HTTP cache in front of Gunicorn can serve those pages in microseconds. Varnish is built for that, with a programmable configuration language (VCL). The hard parts are not caching personalized pages, handling cookies Django and analytics set, and invalidating pages when content changes.

1. Put Varnish between Nginx (TLS) and Gunicorn: Nginx → Varnish → Gunicorn.
2. Send Cache-Control: public, max-age=... from views that are safe to share; Django's cache_control and cache_page decorators help.
3. In VCL, pass requests with session or CSRF cookies, non-GET methods, and the admin; strip analytics cookies.
4. Invalidate with PURGE or BAN requests from Django signals when content changes.
5. Use Vary carefully; vary on Accept-Language only if you need it, never on Cookie for cached pages.

Step-by-step solution

1. Arrange the processes

client → Nginx :443 (TLS) → Varnish :6081 → Gunicorn :8000

Varnish does not terminate TLS itself (Varnish Enterprise and Hitch can), so keep Nginx or another proxy in front. Nginx passes X-Forwarded-Proto through Varnish to Django.

2. Send cacheable headers from Django

from django.views.decorators.cache import cache_control
from django.views.decorators.vary import vary_on_headers

@cache_control(public=True, max_age=60, s_maxage=600)
def article_detail(request, slug):
    ...

s-maxage applies to shared caches like Varnish, while max-age applies to browsers. Make sure views that depend on the user (request.user, messages, CSRF tokens in forms) are not marked public. Django's UpdateCacheMiddleware adds Vary: Cookie when the session is accessed, which correctly prevents caching shared copies of personalized pages.

3. Write the VCL

vcl 4.1;

backend django { .host = "127.0.0.1"; .port = "8000"; }

acl purgers { "127.0.0.1"; "10.10.0.0"/24; }

sub vcl_recv {
    if (req.method == "PURGE" || req.method == "BAN") {
        if (client.ip !~ purgers) { return (synth(405)); }
        if (req.method == "BAN") { ban("req.url ~ " + req.url); return (synth(200, "Banned")); }
        return (purge);
    }
    if (req.method != "GET" && req.method != "HEAD") { return (pass); }
    if (req.url ~ "^/(admin|accounts)/") { return (pass); }
    # drop analytics cookies, keep Django's
    set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_ga|_gid|_fbp)[^;]*", "");
    if (req.http.Cookie ~ "(sessionid|csrftoken)=") { return (pass); }
    unset req.http.Cookie;
    return (hash);
}

sub vcl_backend_response {
    if (beresp.http.Set-Cookie) { set beresp.uncacheable = true; return (deliver); }
    set beresp.grace = 1h;
}

grace lets Varnish serve a slightly stale page while it fetches a fresh one, which smooths traffic spikes. Never cache responses that set cookies.

4. Invalidate on content changes

import requests
from django.db.models.signals import post_save
from django.dispatch import receiver

@receiver(post_save, sender=Article)
def purge_article(sender, instance, **kwargs):
    for url in (instance.get_absolute_url(), "/", "/articles/"):
        requests.request("PURGE", f"http://127.0.0.1:6081{url}", headers={"Host": "news.example.com"}, timeout=2)

Run purges from a Celery task after commit in larger setups. BAN with a regex invalidates groups of URLs, such as every page of a category.

5. Monitor hit rate

varnishstat -1 | grep -E "cache_hit|cache_miss" shows hits and misses; varnishlog -g request -q "ReqURL ~ '^/articles'" shows why a request was passed. A low hit rate usually means cookies are reaching vcl_recv or views do not send public cache headers.

Explanation

Varnish stores responses in memory keyed by URL and host, and answers repeat requests without contacting the backend. By default it refuses to cache requests with cookies or responses that set them, which in a Django site means almost nothing gets cached, because analytics and CSRF cookies are everywhere. VCL lets you decide precisely which cookies matter. Django supplies the other half: correct Cache-Control and Vary headers that describe what may be shared.

Edge cases / notes

• CSRF tokens in cached pages. Forms on cached pages embed a token from whoever triggered the cache fill. Load forms or tokens via JavaScript, or exclude those pages.
• Memory sizing. Set -s malloc,1G to fit your hot pages; Varnish evicts least recently used objects when full.
• Alternatives. Nginx proxy_cache or a CDN often covers the same need with less configuration.

Next steps

• Set headers correctly with Django Cache-Control Headers for CDNs.
• A simpler option: Nginx Proxy Cache for Django.
• Plan layers with Django Caching Strategy for Production.

FAQ

Why is Varnish not caching my Django pages?

Usually cookies on the request or Set-Cookie on the response. Strip analytics cookies and pass only requests with Django session cookies.

Can Varnish cache pages for logged-in users?

Not shared copies. Pass them to Django, or use edge-side includes for small personalized fragments.

How do I clear a page after editing it?

Send a PURGE request for its URL, or a BAN for a URL pattern, from a Django signal or task.

Does Varnish handle HTTPS?

Not in the open-source version. Terminate TLS in Nginx or another proxy in front.