Django Kubernetes Production Checklist

Problem statement

Getting Django to run on Kubernetes takes an afternoon. Running it reliably takes many details that manifests copied from tutorials usually lack: probes that fail pods for the wrong reasons, no resource requests, migrations racing in init containers, Celery Beat running twice after a rollout, uploads written to pod filesystems, and pods killed mid-request during node maintenance.

This checklist reviews a Django deployment on Kubernetes area by area. It assumes a managed cluster (EKS, GKE, AKS, or similar), a managed PostgreSQL database, and Redis, which is the most common setup for small and medium teams.

1. Build immutable images tagged by commit, running as non-root, with static files collected at build time.
2. Configure readiness, liveness, and startup probes that match Django's behaviour, with ALLOWED_HOSTS-compatible Host headers.
3. Set CPU and memory requests (and memory limits) from measurements; size Gunicorn workers to the pod's CPU.
4. Run migrations as a Job before each rollout; roll out with maxUnavailable: 0.
5. Keep secrets in Secrets or an external secret manager; keep media in object storage.
6. Run Celery workers as their own Deployment, Beat as a single-replica Deployment with Recreate strategy.
7. Add PodDisruptionBudgets, graceful shutdown settings, autoscaling, and centralised logs and metrics.

Checklist

1. Images

• Images are built in CI and tagged with the commit SHA or version; manifests never reference latest.
• The container runs as a non-root user; the root filesystem can be read-only (with emptyDir for /tmp if needed).
• collectstatic runs during the image build; static files are served by WhiteNoise or a CDN.
• One image serves web, workers, Beat, and migrations, with different commands.

2. Probes

• Readiness probe hits a lightweight endpoint (for example /healthz/) and removes the pod from the Service when it fails.
• Liveness probe checks only that the process responds; it does not check the database, or a database outage would restart every pod.
• Startup probe (or generous initialDelaySeconds) covers slow application start so liveness does not kill booting pods.
• Probes send a Host header that is in ALLOWED_HOSTS, and the probe path is exempt from SECURE_SSL_REDIRECT.

3. Resources and workers

• Every container has CPU and memory requests based on observed usage; memory limits prevent one pod from exhausting a node.
• Gunicorn worker count matches the CPU request (for example 2 to 3 workers per CPU), not the node's total CPUs, which os.cpu_count() reports inside containers.
• --max-requests with jitter contains memory growth; memory limits are above peak usage to avoid OOM kills.

4. Rollouts and migrations

• Deployment strategy uses maxUnavailable: 0 and maxSurge: 1 (or more) so capacity never drops during rollouts.
• Migrations run in a Job (or Helm/Argo hook) once per release, before the Deployment is updated, and a failed Job stops the release.
• Migrations are backward compatible, so old pods keep working against the new schema during rollouts and rollbacks.
• Rollback (kubectl rollout undo, helm rollback, or a Git revert) has been tested.

5. Configuration and secrets

• Non-secret configuration lives in ConfigMaps; secrets in Secrets or an external secret manager (External Secrets Operator, CSI driver), not in images or manifests in Git.
• Changes to ConfigMaps or Secrets trigger a rollout (checksum annotations or a controller), because pods read environment variables only at start.
• SECURE_PROXY_SSL_HEADER, ALLOWED_HOSTS, and CSRF_TRUSTED_ORIGINS match the ingress setup.

6. Storage

• Media files go to object storage; pods do not write user data to local or emptyDir volumes.
• The database is outside the cluster (managed service) or on a properly operated stateful setup with backups and failover.
• Database connections are bounded: replicas × workers × connections per worker stays below the database limit, or a pooler is used.

7. Celery

• Workers run in their own Deployment(s), per queue where useful, with resource requests and limits.
• terminationGracePeriodSeconds exceeds typical task duration so warm shutdown can finish running tasks.
• Beat runs as exactly one replica with strategy: Recreate, so a rollout never runs two schedulers at once.
• Worker liveness uses a meaningful check (for example celery inspect ping against the local node, with a generous timeout), or relies on process exit rather than a probe that flaps.

8. Availability

• At least two web replicas, spread across nodes and zones (topologySpreadConstraints or anti-affinity).
• PodDisruptionBudgets keep a minimum number of web pods available during node drains and upgrades.
• Graceful shutdown: a short preStop sleep lets the ingress stop routing to a terminating pod before Gunicorn receives SIGTERM; Gunicorn's graceful timeout fits within the termination grace period.
• HorizontalPodAutoscaler scales web (and optionally workers) on CPU or custom metrics, with sensible minimum and maximum replicas.

9. Networking and ingress

• TLS terminates at the ingress or load balancer with automated certificates (cert-manager or the provider).
• Request body size and timeouts at the ingress match the application (upload limits, long requests).
• NetworkPolicies restrict which pods can reach the database and Redis, if your cluster enforces them.

10. Observability

• Logs go to stdout in structured form and are collected centrally.
• Metrics cover request rate, errors, latency, pod restarts, OOM kills, and queue lengths.
• Alerts exist for crash loops, failing rollouts, pending pods, and migration Job failures.
• Errors are sent to an error tracker tagged with the release.

Explanation

Kubernetes runs your processes, restarts them, and routes traffic based on what you declare: probes decide when pods receive traffic or get restarted, requests decide scheduling, and rollout settings decide how capacity changes during releases. Django adds its own requirements: schema changes that must happen once and before new code, a scheduler that must never run twice, and settings that depend on the proxy in front. Most production incidents on Kubernetes come from the gap between those two sets of rules, which is what this checklist covers.

Edge cases / notes

ASGI and WebSockets

Channels or long-lived connections need longer ingress timeouts, sticky sessions only if your design needs them, and graceful handling of connection drops during rollouts.

Cost

Requests determine how many nodes you pay for. Right-size requests from metrics after launch; over-requesting is a common source of wasted spend.

When to automate this

Encode these settings in a Helm chart or Kustomize base shared across services, validate manifests in CI (policy tools can enforce requests, probes, and non-root users), and run rollouts through GitOps so every change is reviewed.

Next steps

• Build the base setup with Deploy Django on Kubernetes.
• Run schema changes safely with Run Django Migrations as a Kubernetes Job.
• Package everything with Deploy Django to Kubernetes with a Helm Chart.
• If probes fail, see Fix Load Balancer Health Checks Failing for Django.

FAQ

Should the Django liveness probe check the database?

No. If the database is down, restarting pods does not help and can make recovery harder. Check dependencies in readiness or monitoring instead.

How many Gunicorn workers should a Django pod run?

Base it on the pod's CPU request, not the node's CPU count. Two to three workers per requested CPU is a common starting point.

How do I stop Celery Beat running twice on Kubernetes?

Run it as a single-replica Deployment with the Recreate strategy, so the old pod stops before the new one starts.

Where should Django migrations run on Kubernetes?

In a Job (or a Helm or Argo CD hook) once per release, before the Deployment rolls out.