Django Monitoring and Alerting Checklist

Problem statement

Monitoring gaps are only visible during incidents. A team discovers that Celery has not processed tasks for two days, that error emails went to an unused mailbox, that logs rotated away before anyone could read them, or that the alert for high error rates never fired because it was based on a metric nobody was collecting.

Good monitoring for a Django deployment covers five questions:

1. Is the site up and working for users?
2. When something breaks, do we find out quickly and know where to look?
3. Is the system trending toward a problem (memory, disk, connections, queues)?
4. Did the last deploy make things better or worse?
5. When we are paged, can we diagnose without guesswork?

This checklist reviews each layer, from logs to alert routing, with the specific signals that matter for Django applications.

Quick answer

1. Logs: Django, Gunicorn, Celery, Nginx, and the database log to a place with retention and search.
2. Errors: an error tracker captures unhandled exceptions with request context and release tags.
3. Metrics: request rate, error rate, latency percentiles, plus host, database, cache, and queue metrics.
4. Uptime: external checks for key URLs, certificate expiry, and heartbeats for scheduled jobs.
5. Background work: queue depth, task failures, and scheduler liveness.
6. Deploys: releases annotated on dashboards and tagged in the error tracker.
7. Alerts: few, actionable, routed to a person, with runbook links and an escalation path.

Checklist

1. Logging

What to check:

• Django logs to stdout or a managed location with a LOGGING configuration; the django.request logger records unhandled exceptions.
• Gunicorn access and error logs go to the same destination (--access-logfile -, --error-logfile -).
• Celery worker and Beat logs are captured.
• Nginx access logs include useful fields: status, $request_time, $upstream_response_time, and the real client IP.
• PostgreSQL logs slow queries and lock waits.
• Retention is long enough to investigate reports that arrive days later; journal and file logs have size limits so they cannot fill the disk.
• Logs are searchable: at minimum journalctl and grep, ideally a central system.
• Personal data and secrets are not logged; django.security warnings are visible.

A request ID in every log line makes multi-service investigations much easier:

import logging
import uuid

logger = logging.getLogger("app.request")


class RequestIDMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.id = request.headers.get("X-Request-ID") or uuid.uuid4().hex
        response = self.get_response(request)
        response["X-Request-ID"] = request.id
        return response

2. Error tracking

What to check:

• An error tracker (Sentry or similar) receives unhandled exceptions from web and worker processes.
• Events include the release version and environment, so regressions map to deploys.
• Sensitive data scrubbing is configured; request bodies and headers with credentials are not sent.
• Alert rules exist for new issues and for spikes in existing ones, not for every occurrence.
• Someone triages the issue list regularly; unresolved noise is fixed or muted deliberately.

3. Application metrics

Collect and dashboard at least:

Metrics can come from an APM product, Prometheus with django-prometheus, or derived from Nginx access logs when a full metrics stack is not justified.

4. Infrastructure metrics

• CPU, memory, disk usage and growth, disk I/O, and network per server.
• Disk usage alerts with enough warning (for example at 80 percent) to act before a full disk breaks the database.
• Container or pod restarts and OOM kills.
• Certificate expiry (also covered by external monitoring).

5. Database monitoring

• Connections used versus max_connections.
• Slow queries (pg_stat_statements top statements by total time).
• Lock waits and long-running transactions.
• Replication lag if replicas exist.
• Table and index growth, dead tuples, and autovacuum activity.
• Backup success and age of the most recent backup.

6. Cache and broker

• Redis memory used versus maxmemory, evictions, and connected clients.
• Cache hit ratio trend (a sudden drop often follows a bad deploy).
• Broker queue lengths per queue and the age of the oldest queued task.

7. Background jobs

• Celery workers reporting as alive (for example inspect ping from a monitoring job, or worker heartbeat metrics).
• Task failure rate and retry counts per task.
• Celery Beat liveness through a heartbeat task, since a stopped scheduler produces no errors.
• Scheduled management commands (systemd timers, cron) report success to a heartbeat monitor.
• Dead letter handling: repeatedly failing tasks are visible rather than silently retried forever.

8. External checks

• Uptime checks from outside your infrastructure for the homepage, a health endpoint, and a critical flow.
• Keyword checks so a 200 response containing an error message counts as down.
• Certificate and domain expiry monitoring.
• Checks from more than one region if your users are distributed.

9. Deploy visibility

• Release version exposed by the application (for example a header or health endpoint field) and sent to the error tracker.
• Deploy events annotated on dashboards so a latency change can be matched to a release.
• Post-deploy smoke tests run automatically and fail the deploy if they do not pass.

10. Alerting practices

• Alerts correspond to user-visible problems or imminent ones: site down, high error rate, latency far above normal, queue backlog growing, disk nearly full, backup missing, certificate expiring.
• Each alert links to a runbook with first steps.
• Alerts go to a channel someone watches, with paging for severity that justifies waking someone.
• Escalation exists if the first responder does not acknowledge.
• Thresholds are based on observed baselines, with a duration (for 10m) to avoid flapping.
• Alert volume is reviewed periodically; alerts that never require action are removed or downgraded.

11. Security signals

• Failed login spikes and admin login attempts.
• django.security warnings (DisallowedHost, SuspiciousOperation) at a level you can review.
• Rate limit rejections (429 counts) per endpoint.
• Unexpected changes in traffic patterns or user agents.

Verify

Test the monitoring itself, not just the application:

# Error tracking: raise a test exception in staging
python manage.py shell -c "raise RuntimeError('monitoring test')"

# Uptime alerting: stop the app in staging and confirm an alert arrives
sudo systemctl stop myapp

# Heartbeat: skip a scheduled job run and confirm the missing-ping alert

Verification check:

• a deliberate staging outage produces an alert within the expected time
• an unhandled exception appears in the error tracker with release and request context
• dashboards show traffic, errors, latency, queues, and database health on one screen
• every alert has an owner and a runbook link
• logs from web, worker, proxy, and database can be searched for a single request ID

Explanation

Monitoring layers answer different questions: external checks tell you whether users can reach the site, metrics show trends and saturation, error tracking explains failures with code-level context, logs provide detail for a specific request, and traces connect the pieces across services. Alerting turns a subset of those signals into action. The most common gaps are the quiet parts of a system, especially background jobs and backups, which fail without producing any error a user or a web dashboard would show.

Edge cases / notes

Small projects

A solo developer can cover most of this with an error tracker, an uptime service with heartbeat monitors, and journal logs with retention. Add metrics when traffic grows enough that trends matter.

Cost control

Log and trace volume drives cost. Sample traces, keep high-volume debug logs short-lived, and retain aggregated metrics longer than raw data.

Privacy

Logs and error events can capture personal data. Configure scrubbing, and set retention in line with your privacy commitments.

Common mistakes

• Monitoring the web tier but not Celery or scheduled jobs.
• Alerting on every exception instead of on rates and user impact.
• Logs with no retention limits, filling the disk they were meant to help protect.
• Never testing that alerts actually reach someone.

Next steps

• Build metrics dashboards with Monitor Django with Prometheus and Grafana.
• Add external checks with Set Up Uptime Monitoring and Alerts for Django.
• Capture exceptions with Set Up Sentry Error Monitoring for Django.
• Get logging right with Django Logging Setup for Production.

FAQ

What should I monitor in a Django application?

User-facing availability and errors, request latency, background job health, database and cache saturation, host resources, and backup success, with alerts on the signals that indicate user impact.

How many alerts should a Django project have?

Few enough that each one is acted upon. Start with site down, elevated error rate, queue backlog, disk space, and backup failure, then add alerts as you learn which failures recur.

Do I need Prometheus and Grafana for a small Django site?

Not necessarily. An error tracker plus external uptime and heartbeat monitoring covers the most common failures. Add metrics when you need trends and capacity planning.

How do I monitor Celery in production?

Track queue lengths and oldest task age, task failure rates, worker liveness, and a Beat heartbeat task, and alert when queues grow or the heartbeat stops.