Run Dramatiq Workers for Django in Production

Problem statement

You chose Dramatiq over Celery for its smaller API and sensible defaults: automatic retries with backoff, time limits, and acknowledgement after processing. Running it in production still needs decisions about the broker, worker processes and threads, database connections in long-lived workers, periodic jobs (Dramatiq has no built-in scheduler), and graceful shutdown during deploys.

1. Install django-dramatiq and dramatiq[redis] (or dramatiq[rabbitmq]), and add django_dramatiq to INSTALLED_APPS.
2. Configure DRAMATIQ_BROKER with the broker URL and middleware, including django_dramatiq.middleware.DbConnectionsMiddleware.
3. Run workers with python manage.py rundramatiq --processes 2 --threads 4 under systemd or as a container.
4. Use APScheduler, periodiq, or system cron to enqueue periodic jobs; run the scheduler once.
5. Give workers time to finish on SIGTERM, and make actors idempotent because retries are on by default.

Step-by-step solution

1. Configure the broker and middleware

INSTALLED_APPS += ["django_dramatiq"]

DRAMATIQ_BROKER = {
    "BROKER": "dramatiq.brokers.redis.RedisBroker",
    "OPTIONS": {"url": os.environ["REDIS_URL"]},
    "MIDDLEWARE": [
        "dramatiq.middleware.AgeLimit",
        "dramatiq.middleware.TimeLimit",
        "dramatiq.middleware.Callbacks",
        "dramatiq.middleware.Retries",
        "django_dramatiq.middleware.DbConnectionsMiddleware",
        "django_dramatiq.middleware.AdminMiddleware",
    ],
}
DRAMATIQ_TASKS_DATABASE = "default"

DbConnectionsMiddleware closes stale database connections around each message, which Celery's Django fixup does for Celery. Without it, long-lived workers hit "connection already closed" after database restarts. AdminMiddleware records task state for the Django admin; skip it for high-volume queues.

2. Define actors

import dramatiq

@dramatiq.actor(max_retries=5, time_limit=60_000, queue_name="emails")
def send_receipt(order_id):
    order = Order.objects.get(pk=order_id)
    ...

Call with send_receipt.send(order.pk), ideally inside transaction.on_commit() so the worker never sees an order that has not been committed yet. Time limits are in milliseconds.

3. Run workers

python manage.py rundramatiq --processes 2 --threads 4 --queues default emails

Each process runs several threads, so a worker uses processes × threads database connections at most. Under systemd:

[Service]
User=deploy
WorkingDirectory=/srv/app
EnvironmentFile=/etc/shop/env
ExecStart=/srv/app/.venv/bin/python manage.py rundramatiq --processes 2 --threads 4
KillSignal=SIGTERM
TimeoutStopSec=60
Restart=always

In containers, run the same command as a separate service from the web image.

4. Schedule periodic jobs

Dramatiq does not include a scheduler. Options:

• APScheduler in a small management command that enqueues actors on a cron schedule, run as exactly one process.
• periodiq, a scheduler built for Dramatiq.
• System cron or Kubernetes CronJobs calling a management command that sends messages.

Whatever you choose, run only one scheduler instance, or every job is enqueued several times.

5. Shut down safely

On SIGTERM, Dramatiq workers stop consuming, wait for running messages up to a timeout, then exit. Messages are acknowledged only after processing, so interrupted work is redelivered. Give the process manager a stop timeout longer than your typical task, and keep tasks short.

Explanation

Dramatiq encodes each actor call as a message, sends it to a broker queue, and workers consume messages with a pool of threads in each process. Middleware wraps every message, which is where retries, time limits, age limits, and Django database cleanup happen. Acknowledging after processing gives at-least-once delivery by default, which is safer than Celery's default but means duplicate executions are possible after crashes, so actors must be idempotent.

Edge cases / notes

• Threads and the GIL. CPU-bound actors gain little from threads; use more processes for them.
• Results. The Results middleware needs a backend (Redis or Memcached) and should only be enabled if you read results.
• Monitoring. Prometheus middleware exposes metrics; dramatiq-dashboard gives a simple UI for Redis brokers.

Next steps

• Compare options in Celery Alternatives for Django.
• Look at a lighter option: Run Huey as a Django Task Queue.
• Make retries safe with Make Celery Tasks Idempotent With Retries.

FAQ

Is Dramatiq a good replacement for Celery?

For many apps, yes. It covers queues, retries, and time limits with less configuration, but lacks a built-in scheduler and some advanced workflow features.

How do I run periodic tasks with Dramatiq?

Use APScheduler, periodiq, or cron to enqueue actors, with exactly one scheduler running.

Why do I get database connection errors in workers?

Add django_dramatiq.middleware.DbConnectionsMiddleware so connections are cleaned up around messages.

Does Dramatiq retry failed tasks automatically?

Yes, with exponential backoff, up to max_retries. Make actors idempotent.