Fix Django Migrations Hanging on a Database Lock

Problem statement

The deploy runs python manage.py migrate and it never finishes. There is no error and no progress. Worse, the site starts timing out at the same moment: requests that touch the table being migrated pile up until Gunicorn workers are exhausted and users see 502 or 504 errors.

This is PostgreSQL lock queueing. Many schema changes (ALTER TABLE ... ADD COLUMN ... DEFAULT on older versions, adding constraints, renaming, changing types) need an ACCESS EXCLUSIVE lock on the table. If any other session holds even a weak lock on it, typically a long-running query or a transaction left open, the migration waits. And while it waits in the queue, every new query on that table waits behind it. This guide gets you out of the situation and prevents it.

1. In psql, find the migration's session and what blocks it with pg_blocking_pids().
2. If the site is suffering, cancel the migration first (pg_cancel_backend) to release the queue, or terminate the blocker if it is safe to do so.
3. Identify why the blocker exists: an idle-in-transaction connection, a long report query, a Celery task, a manual psql session, or an admin action.
4. Rerun the migration with a lock_timeout so it fails fast instead of queueing indefinitely, and retry at a quieter moment.
5. Prevent recurrence with idle_in_transaction_session_timeout, statement_timeout for web traffic, and a lock timeout for migrations.

Step-by-step solution

1. See what the migration is waiting for

SELECT pid, state, wait_event_type, wait_event,
       now() - query_start AS running_for,
       left(query, 80) AS query
FROM pg_stat_activity
WHERE datname = current_database() AND wait_event_type = 'Lock';

Then find who blocks it:

SELECT a.pid AS blocked_pid,
       left(a.query, 60) AS blocked_query,
       b.pid AS blocking_pid,
       b.state AS blocking_state,
       now() - b.xact_start AS blocking_xact_age,
       left(b.query, 60) AS blocking_query,
       b.application_name, b.client_addr
FROM pg_stat_activity a
JOIN LATERAL unnest(pg_blocking_pids(a.pid)) AS bp(pid) ON true
JOIN pg_stat_activity b ON b.pid = bp.pid
WHERE a.wait_event_type = 'Lock';

Typical results:

• blocking_state = 'idle in transaction' with an old xact_start: a connection opened a transaction and never committed. Its query column shows the last statement it ran, not what it is doing now.
• blocking_state = 'active' with a long-running SELECT: a report, export, or analytics query.
• application_name = 'psql': someone's interactive session with an open BEGIN.

2. Stop the damage

If users are affected, the fastest relief is to cancel the waiting migration. The queue behind it drains immediately:

SELECT pg_cancel_backend(<migration_pid>);

The migration fails, and because PostgreSQL migrations run in a transaction by default, nothing is half-applied. Your deploy script should then stop without switching to the new code.

Alternatively, if the blocker is clearly safe to kill (an abandoned psql session, a stuck report), terminate it and let the migration proceed:

SELECT pg_terminate_backend(<blocking_pid>);

Terminating an application connection mid-transaction rolls that transaction back; make sure it is not doing important work.

3. Rerun with a lock timeout

Make the migration give up quickly if it cannot get its lock, instead of stalling traffic:

PGOPTIONS="-c lock_timeout=5s" python manage.py migrate --noinput

libpq, and therefore psycopg, reads PGOPTIONS and applies the settings to the session. If the lock is not acquired within five seconds, the migration fails with canceling statement due to lock timeout. Retry it a few times with a pause:

for attempt in 1 2 3 4 5; do
  PGOPTIONS="-c lock_timeout=5s" python manage.py migrate --noinput && break
  echo "migration blocked, retrying in 15s (attempt $attempt)"
  sleep 15
done

Each failed attempt causes at most five seconds of queueing for other queries.

You can also set it per migration with a leading RunSQL:

from django.db import migrations


class Migration(migrations.Migration):
    dependencies = [("orders", "0041_previous")]
    operations = [
        migrations.RunSQL("SET LOCAL lock_timeout = '5s';", reverse_sql=migrations.RunSQL.noop),
        migrations.AddField(...),
    ]

SET LOCAL lasts until the end of the migration's transaction.

4. Remove the source of long transactions

• Idle in transaction usually comes from code that opens a transaction and then waits: a management command in transaction.atomic() calling an external API, a Celery task holding a transaction across network calls, or a psql session someone left open. Fix the code, and set a server-side limit:

ALTER DATABASE myapp SET idle_in_transaction_session_timeout = '60s';

• Long queries from web traffic should be capped with statement_timeout for the application role. Reports and exports that legitimately run long belong on a replica or in a background task with its own role and limits.
• Autovacuum normally yields to a waiting lock request, except for anti-wraparound vacuums, which do not; check pg_stat_activity for autovacuum: entries on the table.

5. Schedule risky migrations

Migrations that need an exclusive lock on a busy table should run at a low-traffic time, with a lock timeout and retries, and ideally be restructured to need only brief locks (for example adding a nullable column, then backfilling in batches, then adding a constraint as NOT VALID and validating separately).

Verification check:

SELECT count(*) FROM pg_stat_activity WHERE wait_event_type = 'Lock';
SELECT pid, now() - xact_start AS age, state FROM pg_stat_activity
WHERE xact_start IS NOT NULL ORDER BY age DESC LIMIT 5;

After the fix, no sessions should be waiting on locks, and no transaction should be minutes old. python manage.py showmigrations should show the migration applied.

Explanation

PostgreSQL grants locks in queue order. A request for an ACCESS EXCLUSIVE lock conflicts with every other lock mode, so it waits for all current holders to finish, and any later request that conflicts with it (which is every normal read and write) waits behind it. The migration itself may need only milliseconds once it gets the lock; the outage comes from the wait. lock_timeout bounds that wait, and removing long-lived transactions removes the reason to wait at all.

Edge cases / notes

Non-atomic migrations

Migrations with atomic = False (for example those using AddIndexConcurrently) are not wrapped in a transaction. If one is cancelled partway, some operations may have been applied; check the table's state before rerunning. A failed concurrent index build leaves an INVALID index that must be dropped first.

MySQL

MySQL has similar metadata lock queueing. The session variable lock_wait_timeout limits the wait, and SHOW PROCESSLIST shows sessions Waiting for table metadata lock.

When to automate this

Put the lock_timeout and retry loop in the deploy script so every migration run is protected, and add a pre-migration check that aborts if any transaction older than a few minutes exists. Alert on idle in transaction sessions so their source is fixed before the next deploy.

Next steps

• Restructure heavy changes with Run Schema Changes on Large Tables in Django Safely.
• If a migration failed partway, follow Django Migration Failed During Deploy: Recovery Playbook.
• Find the code leaving transactions open with Fix Idle in Transaction Connections from Django.
• Cap long queries with Set PostgreSQL Statement Timeouts for Django.

FAQ

Why does Django migrate hang without an error?

The migration is waiting for a table lock held by another session. PostgreSQL waits indefinitely unless lock_timeout is set.

Why did the whole site slow down while the migration was waiting?

Queries that need the table queue behind the migration's pending exclusive lock request. Cancel the migration or remove the blocker to release them.

Is it safe to cancel a running Django migration?

On PostgreSQL, normal migrations run in a transaction and roll back completely when cancelled. Non-atomic migrations can be partially applied.

How do I stop migrations from blocking production traffic?

Run them with lock_timeout and retries, eliminate long-running and idle transactions, and restructure changes that need long exclusive locks.