Fix Django Migration "Column Contains Null Values"

Problem statement

A deploy runs migrate and stops:

django.db.utils.IntegrityError: column "country" of relation "accounts_profile"
contains null values

The migration either adds a field without null=True and without a default, or alters an existing field from null=True to null=False. PostgreSQL tries to add a NOT NULL constraint, finds existing rows where the column is NULL, and refuses. Locally the table was empty or every row had a value, so the migration passed.

1. Find the rows: SELECT count(*) FROM accounts_profile WHERE country IS NULL;
2. Split the change: first migration adds or keeps the field as null=True.
3. Second migration backfills values with RunPython or an UPDATE in batches.
4. Third migration, ideally in a later deploy, changes the field to null=False.
5. For new fields with a constant default, add default= (and db_default= on Django 5.0+) so existing rows get a value.

Step-by-step solution

1. Recover the failed deploy

On PostgreSQL, migrations run in a transaction, so the failed one was rolled back. Check the state:

python manage.py showmigrations accounts

The failing migration is unchecked. Do not fake it. Leave the previous release running, or roll back the code if the new code expects the column to be non-null.

2. Split the migration

Edit the model back to null=True, generate migrations, and add a data migration between them:

# accounts/migrations/0042_backfill_country.py
from django.db import migrations

def backfill(apps, schema_editor):
    Profile = apps.get_model("accounts", "Profile")
    Profile.objects.filter(country__isnull=True).update(country="US")

class Migration(migrations.Migration):
    dependencies = [("accounts", "0041_profile_country")]
    operations = [migrations.RunPython(backfill, migrations.RunPython.noop)]

Then, in a later release, set null=False and generate the final migration.

3. Backfill large tables in batches

A single UPDATE on millions of rows holds row locks for a long time and generates a lot of WAL. Loop over primary key ranges:

def backfill(apps, schema_editor):
    Profile = apps.get_model("accounts", "Profile")
    batch = 5000
    last_id = 0
    while True:
        ids = list(
            Profile.objects.filter(id__gt=last_id, country__isnull=True)
            .order_by("id").values_list("id", flat=True)[:batch]
        )
        if not ids:
            break
        Profile.objects.filter(id__in=ids).update(country="US")
        last_id = ids[-1]

class Migration(migrations.Migration):
    atomic = False
    dependencies = [("accounts", "0041_profile_country")]
    operations = [migrations.RunPython(backfill, migrations.RunPython.noop)]

atomic = False lets each batch commit on its own. For very large tables, run the backfill as a management command outside the deploy and keep only the constraint change in migrations; see Run a Django Data Migration on a Large Table.

4. Use defaults for new fields

If every existing row should get the same value, a default avoids the backfill. On Django 5.0+, db_default sets it in the database:

country = models.CharField(max_length=2, db_default="US")

On PostgreSQL 11 and later, adding a column with a constant default does not rewrite the table. A plain default= without db_default makes Django add the column with the default, fill existing rows, and then drop the database default.

5. Add NOT NULL without a long lock

ALTER COLUMN ... SET NOT NULL scans the whole table while holding an ACCESS EXCLUSIVE lock. On large tables, add a CHECK (country IS NOT NULL) NOT VALID constraint, run VALIDATE CONSTRAINT (which takes a weaker lock), and PostgreSQL 12+ can then set NOT NULL without scanning again. Run Django Schema Changes on Large Tables shows the migration code.

Explanation

NOT NULL is a constraint on the whole table, so PostgreSQL must prove that no current row violates it. If any row has NULL, the constraint cannot be added, and the migration fails. Django's makemigrations asks for a one-off default when you add a non-nullable field, but if you choose "add the default in models.py later" or change an existing nullable field, nothing fills the old rows.

Edge cases / notes

• makemigrations prompt. When Django asks for a one-off default, the value is used only to fill existing rows during that migration. It is a quick fix for small tables and constant values.
• MySQL. MySQL in strict mode fails similarly; in non-strict mode it silently converts NULLs to empty values, which hides bad data.
• Foreign keys. Making a ForeignKey non-nullable requires a valid related row for every existing record; create placeholder records consciously or keep it nullable.

Next steps

• Recover a broken release with Fix Django Migration Failed During Deploy.
• Review the next migration with the Django Migration Review Checklist.
• Plan large backfills with Run a Django Data Migration on a Large Table.

FAQ

Why did the migration pass on my machine?

Your local table had no NULLs, usually because it was empty or created after the field existed.

Can I just add a default to fix it?

For a new field with a constant value, yes. For existing nullable fields, you still need to update the NULL rows before adding the constraint.

Is it safe to run the backfill inside the migration?

For small tables, yes. For large tables, run it in batches with atomic = False or as a separate command so deploys stay fast.

What does the one-off default in makemigrations do?

It fills existing rows during that migration only. The model itself stays without a default.