Fix Django StreamingHttpResponse and SSE Buffered by Nginx

Problem statement

A Django view returns a StreamingHttpResponse: progress updates for an import, server-sent events (SSE) for notifications, or tokens from an LLM API as they are generated. With runserver the output trickles in as expected. In production, the browser shows nothing for a long time and then everything at once, or the SSE connection never delivers events until it closes.

Something between Django and the browser is collecting the response before forwarding it. Nginx buffers upstream responses by default, GZipMiddleware or Nginx gzip compress in blocks, and CDNs may buffer too. Each layer must be told to pass chunks through as they arrive.

1. Add the header X-Accel-Buffering: no to the streaming response so Nginx passes it through unbuffered.
2. Or set proxy_buffering off; in an Nginx location dedicated to streaming endpoints.
3. Exclude streaming responses from compression (GZipMiddleware, Nginx gzip), or flush-friendly compress them.
4. For SSE, send Content-Type: text/event-stream and Cache-Control: no-cache.
5. Serve long-lived streams from an ASGI server with async generators, so each stream does not occupy a sync worker.

Step-by-step solution

1. Confirm where buffering happens

Test each layer with curl -N (no buffering on the curl side):

curl -N http://127.0.0.1:8000/events/          # straight to Gunicorn or Uvicorn
curl -N https://example.com/events/            # through Nginx and any CDN

If the first streams and the second does not, the proxy layer buffers. If neither streams, the problem is in Django or middleware.

2. Disable Nginx buffering for the response

The most targeted fix is a response header that Nginx understands:

from django.http import StreamingHttpResponse

def import_progress(request, job_id):
    def stream():
        for pct in run_import(job_id):
            yield f"{pct}%\n"
    resp = StreamingHttpResponse(stream(), content_type="text/plain")
    resp["X-Accel-Buffering"] = "no"
    resp["Cache-Control"] = "no-cache"
    return resp

Nginx removes the header before sending it to the client. Alternatively, configure a location:

location /events/ {
    proxy_pass http://unix:/run/gunicorn.sock;
    proxy_buffering off;
    proxy_cache off;
    proxy_read_timeout 1h;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
}

Keep buffering on for normal views: it frees Gunicorn workers quickly while Nginx feeds slow clients.

3. Keep compression away from streams

GZipMiddleware compresses streaming responses, and compressors emit output in blocks, so small events wait until enough data accumulates. The simplest fix is not to enable GZipMiddleware at all and let Nginx compress normal responses, with compression switched off where streams are served:

location /events/ {
    gzip off;
    proxy_buffering off;
    proxy_pass http://unix:/run/gunicorn.sock;
}

If you keep GZipMiddleware, know that it has no per-view opt-out: it compresses every response unless one already carries a Content-Encoding header. Moving compression to Nginx, where it can be switched off per location, is the cleaner design.

4. Format server-sent events correctly

import json, time

def notifications(request):
    def events():
        yield "retry: 5000\n\n"
        while True:
            for note in pending_notifications(request.user):
                yield f"id: {note.id}\nevent: note\ndata: {json.dumps(note.as_dict())}\n\n"
            yield ": keepalive\n\n"
            time.sleep(15)
    resp = StreamingHttpResponse(events(), content_type="text/event-stream")
    resp["Cache-Control"] = "no-cache"
    resp["X-Accel-Buffering"] = "no"
    return resp

Each event ends with a blank line. The comment line (: keepalive) keeps proxies from closing idle connections.

5. Do not tie up sync workers

Under Gunicorn's sync workers, each open stream holds a whole worker. Ten SSE clients and three workers means the site stops responding. Serve streams from an ASGI server with an async generator:

import asyncio

async def notifications(request):
    async def events():
        while True:
            yield ": keepalive\n\n"
            await asyncio.sleep(15)
    resp = StreamingHttpResponse(events(), content_type="text/event-stream")
    resp["X-Accel-Buffering"] = "no"
    return resp

Django 4.2+ supports async iterators in StreamingHttpResponse under ASGI.

Verification check: curl -N https://example.com/events/ should print each event as it is sent, and the browser's EventSource should fire message events immediately. Load a normal page while several streams are open to confirm workers are not exhausted.

Explanation

Nginx's proxy_buffering reads the upstream response into memory and temporary files so the backend is released as fast as possible, then sends it to the client at the client's pace. That is ideal for normal pages and fatal for streams, because nothing is forwarded until a buffer fills or the response ends. X-Accel-Buffering: no lets the application opt individual responses out. Compression has a similar effect at a smaller scale, and a sync WSGI worker can only hold one response at a time, which turns every stream into a capacity problem.

Edge cases / notes

• Cloudflare streams responses for most content types, but buffers some; test through it and consider a separate hostname with caching bypassed.
• HTTP/2 between client and Nginx does not affect buffering; the upstream side is HTTP/1.1.
• Timeouts. Long streams need proxy_read_timeout above the gap between chunks and Gunicorn --timeout above the stream duration for sync workers.
• WebSockets are an alternative for two-way traffic; SSE is simpler for server-to-client updates.

Next steps

• Review proxy defaults in Configure Nginx for Django in Production.
• Decide whether to move to ASGI with Django WSGI vs ASGI.

FAQ

Why does streaming work with runserver but not in production?

runserver has no proxy in front. In production, Nginx buffers the response by default.

What does X-Accel-Buffering: no do?

It tells Nginx not to buffer that specific response, so chunks go to the client as Django yields them.

Should I turn off proxy_buffering for the whole site?

No. Buffering protects workers from slow clients. Disable it only for streaming endpoints.

Can Gunicorn serve server-sent events?

It can, but each open stream occupies a sync worker. Use an ASGI server or async workers for many clients.