Run Django's Built-In Tasks Framework in Production

Problem statement

Django 6.0 added a built-in tasks framework, django.tasks: a standard way to define background tasks with @task and enqueue them with .enqueue(). Many small projects that only need to send emails or generate thumbnails in the background no longer need to adopt Celery and a broker on day one.

The catch is that Django defines the API, not the worker. The built-in backends run tasks immediately in the request (the default) or discard them (for tests). Production needs a backend that stores tasks and a worker process that executes them, and that part has to be deployed, supervised, and monitored like any other service. This guide covers that production side.

1. Define tasks with from django.tasks import task and @task on module-level functions with JSON-serialisable arguments.
2. Do not use the default ImmediateBackend in production if the goal is to get work out of the request.
3. Install a backend that persists tasks and provides a worker; the django-tasks reference package includes a database backend and a db_worker command.
4. Enqueue inside transaction.on_commit() when the task depends on data written in the same request.
5. Run the worker under systemd (or as a separate container) with restarts and graceful shutdown.
6. Monitor queue depth and failed tasks, prune old results, and design tasks to be safe to run again.

Step-by-step solution

1. Define and enqueue a task

# notifications/tasks.py
from django.core.mail import send_mail
from django.tasks import task


@task
def send_welcome_email(user_id):
    from accounts.models import User

    user = User.objects.get(pk=user_id)
    send_mail("Welcome", "Thanks for signing up.", None, [user.email])
    return user.email

# accounts/views.py
from functools import partial

from django.db import transaction

from notifications.tasks import send_welcome_email


def signup(request):
    ...
    user = form.save()
    transaction.on_commit(partial(send_welcome_email.enqueue, user.pk))
    ...

Pass IDs, not model instances: arguments and return values must be JSON-serialisable, and the worker should read current data from the database. Enqueueing after commit prevents a worker from picking up the task before the user row exists.

Tasks can be customised per call with .using(), for example send_welcome_email.using(queue_name="email", priority=10).enqueue(user.pk); which options take effect depends on the backend.

2. Configure a production backend

The setting is TASKS:

# settings/base.py
TASKS = {
    "default": {
        "BACKEND": "django.tasks.backends.immediate.ImmediateBackend",
    }
}

ImmediateBackend runs the task synchronously inside enqueue(). It is useful in development and for proving the API works, but it keeps the work in the request. DummyBackend stores nothing and runs nothing, which suits tests that assert a task was enqueued.

For production, install a backend that persists tasks. The django-tasks package, which the built-in framework grew out of, provides a database backend that stores tasks in your existing database, so no broker is needed:

pip install django-tasks

# settings/production.py
INSTALLED_APPS += ["django_tasks", "django_tasks.backends.database"]

TASKS = {
    "default": {
        "BACKEND": "django_tasks.backends.database.DatabaseBackend",
        "QUEUES": ["default", "email"],
    }
}

Run migrate to create its tables. Package layout and backend paths can change between releases, so follow the README of the version you pin. Other community backends target Redis or existing queue systems; the task code stays the same whichever backend you choose.

3. Run the worker under systemd

# /etc/systemd/system/myapp-tasks.service
[Unit]
Description=myapp background task worker
After=network.target postgresql.service

[Service]
User=myapp
Group=myapp
WorkingDirectory=/srv/myapp/current
EnvironmentFile=/etc/myapp.env
ExecStart=/srv/myapp/current/.venv/bin/python manage.py db_worker --queue-name default --queue-name email
Restart=always
RestartSec=5
KillSignal=SIGTERM
TimeoutStopSec=60

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapp-tasks
journalctl -u myapp-tasks -f

• Restart=always brings the worker back after crashes or deploy restarts.
• TimeoutStopSec gives a running task time to finish on SIGTERM before systemd kills it. Set it longer than your longest typical task.
• Check python manage.py db_worker --help for the options your installed version supports.

In containers, run the worker as a separate service using the same image with a different command.

4. Restart workers on every deploy

Workers import your code at start-up. After deploying new code, restart them, or they keep running the old task implementations:

sudo systemctl restart myapp-tasks

Make task signatures backward compatible across one release: a task enqueued by the old code may be executed by the new worker, and the other way round during a rolling deploy.

5. Scale workers

For more throughput, run several worker processes. With systemd, use a template unit:

sudo cp /etc/systemd/system/myapp-tasks.service /etc/systemd/system/myapp-tasks@.service
sudo systemctl enable --now myapp-tasks@1 myapp-tasks@2

A database-backed queue adds load to your primary database: every worker polls and every task writes status updates. That is fine for moderate volumes. For thousands of tasks per minute, a broker-based backend or Celery is a better fit.

6. Handle failures deliberately

A failed task is recorded with a failed status and exception details; the framework itself does not retry automatically. Design tasks so a re-run is safe (idempotent), and decide explicitly where retries happen:

@task
def sync_invoice(invoice_id):
    invoice = Invoice.objects.get(pk=invoice_id)
    if invoice.synced_at:
        return "already synced"
    external_api.push(invoice)          # may raise
    invoice.synced_at = timezone.now()
    invoice.save(update_fields=["synced_at"])

For transient errors, catch them and enqueue a new attempt with a delay (.using(run_after=...)) up to a maximum, or rely on retry features of your chosen backend if it provides them.

7. Monitor and prune

• Queue depth: count tasks waiting to run; alert if it keeps growing.
• Oldest waiting task: alert when it exceeds your latency expectations (for example 5 minutes for emails).
• Failures: send task exceptions to your error tracker. Sentry's Django integration captures exceptions raised in the worker process if the SDK is initialised there too.
• Retention: stored results accumulate. Prune completed results older than a few days with the backend's cleanup command or a scheduled task.

Verification check:

# python manage.py shell
from notifications.tasks import send_welcome_email
result = send_welcome_email.enqueue(1)
result.id, result.status

Enqueue a task, confirm the worker log shows it running, then refresh the result (result.refresh()) and check that its status is successful.

Explanation

django.tasks separates three concerns: the task definition (plain functions with a decorator), the backend (where tasks are stored and how they are dispatched), and the worker (the process that executes them). Django ships the first and simple backends; production backends and workers come from packages. Because application code depends only on the Django API, you can start with a database backend and switch to a broker-backed one later without rewriting tasks.

Edge cases / notes

Periodic tasks

The framework does not schedule recurring tasks. Use systemd timers or cron to run management commands (or to enqueue tasks) on a schedule.

Celery or django.tasks?

Celery offers mature retries, routing, rate limits, periodic tasks with Beat, and monitoring tools. django.tasks with a database backend is simpler to operate for low-to-moderate volumes. Migrating from Celery is optional; existing Celery deployments keep working.

When to automate this

Add the worker restart to your deploy script, a health check that alerts when no task has completed recently, and a scheduled cleanup of old results. Those three steps prevent the most common silent failures of background work.

Next steps

• Compare with a broker-based setup in Configure Redis and Celery for Django Production.
• Borrow the supervision patterns from Run Celery Workers with systemd for Django.
• Design safe retries with Make Celery Tasks Idempotent and Safe to Retry.
• For tasks that never start, the diagnosis steps in Fix Celery Tasks Stuck in PENDING apply to any queue.

FAQ

Does Django 6 include a background worker?

No. Django provides the tasks API and simple backends. A production backend and worker come from third-party packages such as django-tasks.

Why do my Django tasks run inside the request?

The default ImmediateBackend runs tasks synchronously. Configure a backend that stores tasks and run a worker.

Can I pass a model instance to a Django task?

No. Arguments must be JSON-serialisable. Pass the primary key and load the object inside the task.

Should I replace Celery with django.tasks?

Only if its features cover your needs. It is a good fit for simple background work; Celery remains stronger for complex routing, retries, and high throughput.