Liveness and Readiness Probes for Django on Kubernetes

Problem statement

Probes decide when Kubernetes sends traffic to a Django pod and when it kills and restarts it. Configured carelessly, they cause outages instead of preventing them: a liveness probe that checks the database restarts every pod when PostgreSQL has a hiccup, a probe rejected by ALLOWED_HOSTS crash-loops a healthy app, and a slow startup gets killed before Gunicorn finishes loading. Each probe type answers a different question and needs a different endpoint and thresholds.

1. startupProbe: gives slow boots time (for example 30 × 5 s) before other probes start.
2. readinessProbe: checks the app can serve, optionally including the database; failing removes the pod from Service endpoints, without restarting it.
3. livenessProbe: checks only that the process responds; failing restarts the container. Never include the database or external services.
4. Expose two endpoints: /healthz/live (no dependencies) and /healthz/ready (database and cache).
5. Make probes pass Django's host and HTTPS checks: send a Host header and exempt the paths from SECURE_SSL_REDIRECT.
6. Set timeouts above the endpoint's worst case and failure thresholds that tolerate brief slowness.

Step-by-step solution

1. Two endpoints

# core/health.py
from django.db import connection
from django.http import JsonResponse
from django.core.cache import cache

def live(request):
    return JsonResponse({"status": "alive"})

def ready(request):
    try:
        with connection.cursor() as c:
            c.execute("SELECT 1")
        cache.get("healthcheck")
    except Exception as exc:
        return JsonResponse({"status": "unready", "error": exc.__class__.__name__}, status=503)
    return JsonResponse({"status": "ready"})

urlpatterns += [path("healthz/live", live), path("healthz/ready", ready)]
SECURE_REDIRECT_EXEMPT = [r"^healthz/"]

Keep both fast and free of authentication. Do not log every probe request at info level.

2. Probe configuration

containers:
  - name: web
    ports: [{ containerPort: 8000 }]
    startupProbe:
      httpGet: { path: /healthz/live, port: 8000, httpHeaders: [{ name: Host, value: example.com }] }
      periodSeconds: 5
      failureThreshold: 30          # up to 150 s to boot
    readinessProbe:
      httpGet: { path: /healthz/ready, port: 8000, httpHeaders: [{ name: Host, value: example.com }] }
      periodSeconds: 5
      timeoutSeconds: 3
      failureThreshold: 3
    livenessProbe:
      httpGet: { path: /healthz/live, port: 8000, httpHeaders: [{ name: Host, value: example.com }] }
      periodSeconds: 10
      timeoutSeconds: 5
      failureThreshold: 6           # about a minute of unresponsiveness before restart

The Host header makes the request pass ALLOWED_HOSTS; without it Kubernetes sends the pod IP and Django returns 400.

3. Why liveness must not check dependencies

If the database is unreachable for 30 seconds and liveness includes it, Kubernetes restarts every Django pod. They come back cold, all reconnect at once, and the outage gets longer. Readiness handles dependency failures correctly: pods stop receiving traffic until the database returns, then resume without restarts.

4. Tune for Gunicorn behaviour

Probes are requests like any other: if all Gunicorn workers are busy with slow requests, probes queue and time out. A liveness failure then kills a pod that was merely busy. Mitigations: run enough workers or threads, keep liveness thresholds generous, and consider a lightweight check that bypasses the worker pool (for example Gunicorn's own status via a sidecar) only if needed.

5. Graceful shutdown

When a pod terminates, it is removed from endpoints while Gunicorn may still be finishing requests. Add a short preStop sleep so the load balancer stops sending new traffic before SIGTERM; see Graceful Shutdown for Django and Celery.

Verification check: kubectl describe pod shows probes passing; stopping the database in staging marks pods 0/1 Ready without restarts and they recover automatically; a deliberately hung process (for example a debug endpoint that sleeps forever on all workers) is restarted by liveness.

Explanation

Kubernetes uses readiness to manage Service endpoints, liveness to restart unhealthy containers, and startup probes to delay the other two while the app boots. These have very different consequences, so they must ask different questions. Readiness can reflect dependencies because removing a pod from rotation is harmless and reversible. Liveness should detect only states that a restart fixes, such as a deadlocked process; dependency failures are not among them. Django-specific details (host validation, HTTPS redirects, worker saturation) determine whether the probes see the app the way users do.

Edge cases / notes

• Readiness during deploys gates rolling updates: new pods only receive traffic once ready.
• Database checks in readiness should be fast (SELECT 1) with short timeouts, or they themselves cause flapping.
• Celery workers have no HTTP server; use celery inspect ping in an exec probe sparingly, or a file-touch heartbeat.
• gRPC and TCP probes exist but HTTP probes test Django's request path best.

Next steps

• Build endpoints with Django Health Check Endpoint Guide.
• Diagnose restarts with Fix a Django Pod in CrashLoopBackOff.

FAQ

What is the difference between liveness and readiness probes?

Readiness controls whether a pod receives traffic; liveness restarts the container when it fails.

Should the Django liveness probe check the database?

No. A database outage would restart every pod. Check the database only in readiness.

Why do my probes return 400?

The probe sends the pod IP as Host, which ALLOWED_HOSTS rejects. Add a Host header to the probe.

When do I need a startupProbe?

When the app takes longer to boot than liveness tolerates, for example with heavy imports or warm-up.