Run Celery Workers on Heroku for a Django App

Problem statement

Your Django app on Heroku needs background jobs: emails, imports, webhooks, reports. Celery runs as its own process type on worker dynos, separate from web dynos. The Heroku-specific parts are the broker URL and its TLS settings, memory limits per dyno, dyno restarts every day, and running Celery beat exactly once.

1. Add a Redis add-on (Heroku Key-Value Store) and read its REDIS_URL for the broker and results.
2. Add worker: celery -A config worker -l info --concurrency 2 to the Procfile and scale it: heroku ps:scale worker=1.
3. For rediss:// URLs, set ssl_cert_reqs as the add-on requires (Heroku uses self-signed certificates).
4. Run beat as a single beat process type with one dyno, or replace it with Heroku Scheduler for simple jobs.
5. Make tasks idempotent and keep them short, because dynos restart at least daily and receive SIGTERM with a short grace period.

Step-by-step solution

1. Add Redis and configure Celery

heroku addons:create heroku-redis:mini
heroku config:get REDIS_URL

import os, ssl

REDIS_URL = os.environ["REDIS_URL"]
CELERY_BROKER_URL = REDIS_URL
CELERY_RESULT_BACKEND = REDIS_URL
if REDIS_URL.startswith("rediss://"):
    CELERY_BROKER_USE_SSL = {"ssl_cert_reqs": ssl.CERT_NONE}
    CELERY_REDIS_BACKEND_USE_SSL = {"ssl_cert_reqs": ssl.CERT_NONE}
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True
CELERY_TASK_ACKS_LATE = True
CELERY_WORKER_PREFETCH_MULTIPLIER = 1

Heroku documents disabling certificate verification for its Redis because it uses self-signed certificates; check the add-on's current guidance. Plan names change over time; heroku addons:plans heroku-redis lists current ones.

2. Add the worker process

# Procfile
release: python manage.py migrate --noinput
web: gunicorn config.wsgi --max-requests 1000 --max-requests-jitter 100
worker: celery -A config worker -l info --concurrency 2 --max-tasks-per-child 200

git push heroku main
heroku ps:scale worker=1
heroku logs --tail --dyno worker

Each worker dyno runs --concurrency child processes. On a 512 MB dyno, two children with a typical Django footprint fit; more cause R14 memory errors.

3. Schedule periodic tasks

Two options:

• Celery beat: add beat: celery -A config beat -l info and scale it to exactly one dyno. Use django-celery-beat's database scheduler so schedules survive restarts. Do not use --beat inside the worker command if you scale workers above one.
• Heroku Scheduler: a free add-on that runs commands every 10 minutes, hourly, or daily. A command such as python manage.py send_reminders can enqueue Celery tasks or do the work directly.

Scheduler's timing is best-effort; use beat when you need exact schedules or frequent runs.

4. Survive restarts

Heroku restarts dynos at least once every 24 hours and on every deploy, sending SIGTERM and then SIGKILL after about 30 seconds. Celery's warm shutdown stops taking new tasks and waits for running ones; long tasks get killed. Combine acks_late with idempotent tasks so a killed task is redelivered and safe to run again, and break long jobs into smaller tasks.

5. Watch memory and queues

Use --max-tasks-per-child to recycle child processes that grow, and monitor memory in Heroku Metrics. Track queue length with Flower on a separate dyno, or a metrics add-on, and scale workers when queues grow.

Explanation

On Heroku, every Procfile entry is a process type that runs in its own dynos with the same slug and config vars. Celery workers need nothing more than the broker URL and the same Django settings as the web app. Heroku's Redis add-on provides a TLS URL, which Celery requires you to configure explicitly. The platform's restart behavior and memory limits shape how you configure concurrency, acknowledgements, and task size.

Edge cases / notes

• Eco and Basic dynos. Eco dynos sleep when idle and are unsuitable for workers that must run continuously; use Basic or larger.
• Results. Store results only if you read them, and set CELERY_RESULT_EXPIRES so Redis memory does not fill up.
• Other brokers. CloudAMQP (RabbitMQ) works too; set its URL as the broker and keep Redis for caching.

Next steps

• Review the main deployment in Deploy Django on Heroku.
• Fix TLS errors with Fix Redis SSL CERTIFICATE_VERIFY_FAILED.
• Size dynos with Fix Heroku R14 Memory Quota Exceeded.

FAQ

Can I run Celery in the web dyno?

You can, with a process manager, but web and worker then share memory and restarts. A separate worker dyno is cleaner.

How do I run Celery beat on Heroku?

As its own process type scaled to exactly one dyno, or replace it with Heroku Scheduler for simple schedules.

Why does Celery fail to connect to Heroku Redis?

The URL uses rediss:// with a self-signed certificate. Configure ssl_cert_reqs as Heroku documents.

What concurrency should I use?

Start with 2 on a 512 MB dyno and adjust based on memory usage.