Fix "Canceling Statement Due to Conflict With Recovery" on Django Replicas

Problem statement

You added a read replica and routed reports or API reads to it with a database router. Long queries now fail at random:

django.db.utils.OperationalError: canceling statement due to conflict with recovery
DETAIL:  User query might have needed to see row versions that must be removed.

The replica is a hot standby that continuously replays changes from the primary. Sometimes replaying a change would remove rows your query is still reading, usually because vacuum on the primary cleaned them up. The standby has to choose between pausing replay, which makes it fall behind, and cancelling your query. After max_standby_streaming_delay (30 seconds by default) it cancels the query.

1. Identify the conflict type with SELECT * FROM pg_stat_database_conflicts; on the replica.
2. For snapshot conflicts, enable hot_standby_feedback = on on the replica so the primary keeps rows the replica still needs.
3. Or raise max_standby_streaming_delay on a replica dedicated to reports, accepting more lag.
4. Keep short API reads on a low-lag replica, and send long reports to a separate one.
5. Retry read-only queries that fail with this error; they are safe to repeat.

Step-by-step solution

1. See which conflict you have

On the replica:

SELECT datname, confl_snapshot, confl_lock, confl_bufferpin, confl_deadlock
FROM pg_stat_database_conflicts
WHERE datname = current_database();

confl_snapshot counts cancellations because of removed row versions, the common case with vacuum. confl_lock points to ACCESS EXCLUSIVE locks replayed from the primary, for example from migrations, TRUNCATE, or VACUUM truncating a table's end.

2. Enable hot_standby_feedback

With feedback on, the replica tells the primary the oldest transaction it still needs, and vacuum on the primary does not remove those rows:

# postgresql.conf on the replica
hot_standby_feedback = on

Reload the replica. On managed services, set the parameter in the replica's parameter group (RDS), database flags (Cloud SQL), or server parameters (Azure). The cost is that long replica queries can hold back vacuum on the primary, which causes table bloat if reports run for hours.

3. Give reports their own replica

Mixing short API reads and hour-long exports on one standby forces a bad compromise. Use two aliases in Django settings:

DATABASES = {
    "default": {...},                       # primary
    "replica": {...},                       # low lag, hot_standby_feedback on
    "reporting": {...},                     # max_standby_streaming_delay = -1
}

On the reporting replica, max_standby_streaming_delay = -1 lets queries run as long as they need; replay waits. Monitor its lag and never send user-facing reads there. In code, choose explicitly: Order.objects.using("reporting").filter(...).

4. Retry reads that are cancelled

Read-only queries can be repeated safely. Wrap report jobs in a small retry:

from django.db import OperationalError
import time

def run_report(qs, attempts=3):
    for attempt in range(attempts):
        try:
            return list(qs)
        except OperationalError as exc:
            if "conflict with recovery" not in str(exc) or attempt == attempts - 1:
                raise
            time.sleep(2 ** attempt)

Do not retry inside a transaction that already did other work; start fresh.

5. Avoid lock conflicts from migrations

confl_lock cancellations line up with deploys. Operations such as adding a column with a volatile default or VACUUM truncating empty pages take ACCESS EXCLUSIVE locks that replay on the standby. Run large migrations when report jobs are idle, and consider vacuum_truncate = off on very large tables.

Explanation

PostgreSQL's MVCC keeps old row versions until no transaction can see them, then vacuum removes them. On the primary, vacuum knows about every running transaction. A standby's queries are invisible to the primary unless feedback is enabled, so vacuum may remove rows a replica query still reads. When the replica replays that cleanup, it must either wait or cancel. max_standby_streaming_delay sets how long it waits. Neither choice is wrong; they trade replica freshness against query completion.

Edge cases / notes

• Aurora PostgreSQL. Aurora replicas share storage and behave differently but still cancel long queries in similar situations. Check the Aurora-specific parameters.
• Read-your-writes. A replica with a large delay can serve data minutes old. Keep requests that just wrote data on the primary.
• Statement timeouts. Set a statement_timeout on the replica connection for user-facing reads so slow queries fail fast instead of waiting for a conflict.

Next steps

• Route reads correctly with Use Read Replicas With Django.
• Keep vacuum healthy with Tune PostgreSQL Autovacuum for Django.
• Limit query duration with Set PostgreSQL statement_timeout for Django.

FAQ

Is my replica broken?

No. Cancelling queries is normal standby behavior when replay would remove rows they need.

Should I turn on hot_standby_feedback everywhere?

It is a good default for replicas serving short reads. For replicas that run very long queries, it can bloat the primary; use a delayed reporting replica instead.

Can Django retry the query automatically?

Not by default. Add retries in report jobs or API views that read from replicas.

Why do cancellations increase during deploys?

Migrations and maintenance take exclusive locks that replay on the standby, cancelling queries on those tables.