Continuous Profiling for Django With Pyroscope

Problem statement

CPU usage on your Django servers crept up 30 percent over a few releases, or one endpoint got slower without any query changes. Traces tell you which request was slow, but not which Python function burned the CPU. Attaching a profiler to production by hand is awkward and only captures a moment. Continuous profiling samples stack traces from every process all the time with low overhead, so you can see flame graphs for any time window, per view or task, and compare before and after a deploy.

1. Run Grafana Pyroscope (self-hosted or Grafana Cloud) as the profile store.
2. Add the pyroscope-io Python SDK to Django and Celery, configured with an application name and server address.
3. Tag profiles with the view name, Celery task, and release version.
4. Read flame graphs in Grafana: wide bars are where time goes; compare a slow period with a normal one.
5. Keep overhead low with the default sample rate, and initialize after Gunicorn forks.

Step-by-step solution

1. Run Pyroscope

services:
  pyroscope:
    image: grafana/pyroscope:latest
    ports: ["127.0.0.1:4040:4040"]
    volumes: [pyroscope-data:/data]
volumes: { pyroscope-data: {} }

Add it as a data source in Grafana, or use Grafana Cloud Profiles. Pin a version for production use.

2. Instrument Django

# gunicorn.conf.py
def post_fork(server, worker):
    import pyroscope
    pyroscope.configure(
        application_name="shop.web",
        server_address=os.environ["PYROSCOPE_URL"],
        sample_rate=100,
        tags={"env": "production", "version": os.environ.get("GIT_SHA", "dev")},
        oncpu=True,
        gil_only=True,
    )

Initializing in post_fork gives each worker its own sampler. The SDK samples stacks in a native thread with low overhead (typically a few percent at 100 Hz).

3. Tag by view and task

Tags let you filter flame graphs to one endpoint:

import pyroscope

class ProfilingTagMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        with pyroscope.tag_wrapper({"view": request.path_info.split("/")[1] or "root"}):
            return self.get_response(request)

Use low-cardinality tags: route names or the first path segment, never full URLs with IDs. For Celery, wrap task execution with the task name using the task_prerun and task_postrun signals or a base task class.

4. Read flame graphs

In Grafana's profiles view, each bar is a function; width is the share of samples where it was on the stack. Look for:

• Unexpectedly wide bars in serializers, template rendering, or JSON encoding.
• Regex compilation, copy.deepcopy, or logging formatting in hot paths.
• Time under database driver calls (psycopg) when queries are slow, which points back to SQL.

Use the diff view to compare a time range before and after a release; functions that grew stand out.

5. Keep it safe

• Default sample rates keep overhead low; avoid raising them in production.
• Profiles contain function names and file paths, not request data, but treat them as internal.
• Retention settings in Pyroscope control storage use; a few days to a few weeks is typical.

Explanation

A sampling profiler periodically records the call stack of running threads. Aggregating many samples shows where a program spends its time, with overhead proportional to the sample rate rather than to the amount of code executed. Continuous profiling does this all the time and ships aggregated profiles to a store, where they can be queried by time range and tags. For Django, that means CPU hotspots per view, per task, and per release, available after the fact when a regression is noticed.

Edge cases / notes

• Wall time vs CPU. CPU profiles show computation; waiting on I/O or locks shows up only in wall-clock profiles. Use traces for I/O waits.
• Async views. Profiles of async code show event loop frames; tags help separate requests.
• Containers. eBPF profilers need kernel support and privileges on the node; SDKs work anywhere.

Next steps

• Combine with one-off profiling in Profile Slow Django Endpoints in Production.
• Diagnose CPU spikes with Fix Django High CPU Usage.
• Add traces with Django OpenTelemetry Tracing.

FAQ

What is continuous profiling?

Always-on, low-overhead sampling of stack traces from production processes, stored so you can inspect any time window.

How much overhead does Pyroscope add?

At the default sample rate, typically a small single-digit percentage of CPU.

Why initialize it in post_fork?

Each Gunicorn worker needs its own sampler; starting it in the master before forking does not carry over reliably.

Can I profile Celery tasks?

Yes. Configure the SDK in workers and tag samples with the task name.