Find and Fix a Django Memory Leak in Production

Problem statement

Gunicorn workers start at 120 MB and drift upwards for days. Monitoring shows a sawtooth that resets only on deploys, or a steady climb until the OOM killer takes workers out. Restarting the service always helps, which is why the problem keeps coming back.

There are two different problems that look alike. Real leaks are objects kept alive forever: a module-level dictionary that grows per request, signal receivers registered repeatedly, or DEBUG = True storing every SQL query. Normal Python growth is a worker that once handled a huge request, grew to hold it, and keeps the memory even after freeing the objects. Both are contained by recycling workers; only leaks need code changes.

1. Graph per-worker RSS over at least a day; a plateau is a high-water mark, a constant climb is a leak.
2. Add --max-requests 1000 --max-requests-jitter 100 to Gunicorn to recycle workers.
3. Confirm DEBUG = False; with debug on, every query is kept in memory.
4. Profile with tracemalloc snapshots on a staging copy to find what grows.
5. Fix the usual causes: unbounded module-level caches, querysets loaded whole, and per-request registration of signals or handlers.

Step-by-step solution

1. Measure per worker

watch -n 60 'ps -o pid,rss,etime,cmd -C gunicorn --sort=-rss | head'

Better, export process memory to your monitoring (node exporter's process metrics, Datadog, or a simple cron that logs ps output) and graph it per PID. Note when the jumps happen and correlate them with request logs: a jump right after a large export points to a high-water mark from that view.

2. Recycle workers

This is the immediate mitigation and belongs in every production config:

# gunicorn.conf.py
max_requests = 1000
max_requests_jitter = 100

Gunicorn replaces a worker after it has served 1,000 (±100) requests, finishing the current request first. Jitter prevents all workers restarting at the same moment. Pick a value that recycles each worker a few times per day at your traffic level.

3. Rule out DEBUG

python manage.py shell -c "from django.conf import settings; print(settings.DEBUG)"

With DEBUG = True, django.db.connection.queries records every query. In normal requests it is reset at the start of each request, but in long-running code paths such as management commands, background threads, or streaming responses it grows without bound. Production must run with DEBUG = False, and long loops should call django.db.reset_queries() if debug is ever enabled.

4. Profile with tracemalloc

Reproduce on staging with production-like data, and take snapshots before and after a batch of requests:

# a temporary debug view, protected and removed afterwards
import tracemalloc
from django.http import HttpResponse

tracemalloc.start(25)
_baseline = tracemalloc.take_snapshot()

def memory_diff(request):
    snap = tracemalloc.take_snapshot()
    top = snap.compare_to(_baseline, "traceback")[:10]
    return HttpResponse("\n\n".join(str(s) + "\n" + "\n".join(s.traceback.format()) for s in top),
                        content_type="text/plain")

Hit the suspicious endpoints a few thousand times with a load tool, then request the diff. Allocations that keep growing between calls, with a traceback into your code, are the leak. Tools such as memray give the same information with less setup and flame graphs.

5. Fix the common causes

# Leak: module-level cache that is never pruned
_RATES = {}
def get_rate(currency, day):
    key = (currency, day)
    if key not in _RATES:
        _RATES[key] = fetch_rate(currency, day)
    return _RATES[key]

Use functools.lru_cache(maxsize=1024) or Django's cache with a timeout instead. Other frequent culprits:

• list(Model.objects.all()) or iterating big querysets without .iterator().
• Signal receivers or logging handlers connected inside a view or function called per request (dispatch_uid prevents duplicates).
• Third-party clients created per request that start background threads.
• Building whole files in memory instead of using StreamingHttpResponse or FileResponse.

Verification check: after the fix, graph worker RSS for several days with max_requests temporarily raised. Memory should level off at a stable size instead of climbing, and the kernel log should show no OOM kills.

Explanation

CPython frees an object as soon as nothing references it, so a leak in Python code means something still holds a reference: a global, a class attribute, a closure, or a C extension. Separately, the memory allocator keeps freed memory in arenas for reuse rather than returning it to the operating system, so a process's RSS reflects its peak, not its current needs. Worker recycling handles both by starting fresh processes regularly, which is why it is standard practice for long-running Python web servers.

Edge cases / notes

• Preloading. With preload_app = True, workers share memory pages copy-on-write; RSS overstates unique memory. Use PSS (smem) for an accurate per-worker figure.
• Threads and async. Threaded or async workers hold many requests at once, so peak memory is higher. Size max_requests accordingly.
• Native libraries. Image, PDF, and XML libraries can leak outside Python's view; tracemalloc will not see it, but RSS will. Isolate that work in Celery with aggressive recycling.
• Containers. Memory limits apply to the whole container; set them with headroom above workers times peak RSS.

Next steps

• If workers are already being killed, start with Fix Gunicorn Workers Killed by the OOM Killer.
• Choose worker counts that fit memory with How to Optimize Gunicorn Workers for Django.

FAQ

Is it normal for Gunicorn workers to grow over time?

Some growth to a plateau is normal. Continuous growth that never levels off indicates a leak.

What max_requests value should I use?

Typically 500 to 5,000, with jitter around 10 percent. Lower values recycle more often at a small CPU cost.

Does gc.collect() fix memory leaks?

No. It frees unreachable cycles, but a leak is memory that is still reachable.

Can I profile memory in production?

Briefly and carefully with sampling tools, but reproducing on staging is safer and easier.