Run Celery Workers on Render, Railway, and Fly.io

Problem statement

Your Django web service already runs on Render, Railway, or Fly.io. Now you need Celery for background jobs. Each platform models non-HTTP processes differently: Render has background workers, Railway runs any service from the same repository with a different start command, and Fly.io uses process groups inside one app. The Celery setup is the same everywhere; the platform configuration is what differs.

1. Provision Redis on the platform (Render Key Value, Railway Redis, Upstash or a Fly-deployed Redis) and set REDIS_URL for web and worker.
2. Create the worker from the same repository or image with celery -A config worker -l info --concurrency 2.
3. Render: add a Background Worker service. Railway: add a second service with a custom start command. Fly.io: add a worker entry under [processes] in fly.toml.
4. Run beat as one instance only, or use the platform's cron feature for scheduled commands.
5. Share all environment variables between web and worker so settings match.

Step-by-step solution

1. Common Celery settings

CELERY_BROKER_URL = os.environ["REDIS_URL"]
CELERY_RESULT_BACKEND = os.environ.get("CELERY_RESULT_BACKEND", "")  # often unnecessary
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True
CELERY_TASK_ACKS_LATE = True
CELERY_WORKER_PREFETCH_MULTIPLIER = 1
CELERY_TASK_TIME_LIMIT = 600

acks_late with idempotent tasks makes restarts during deploys safe, because unfinished tasks are redelivered.

2. Render

Create a Background Worker service from the same repository and branch, with:

• Build command: same as web (pip install -r requirements.txt).
• Start command: celery -A config worker -l info --concurrency 2.
• Environment: link the same environment group as the web service.

In a render.yaml blueprint:

services:
  - type: web
    name: shop-web
    runtime: python
    startCommand: gunicorn config.wsgi
    envVars: [{ fromGroup: shop-env }]
  - type: worker
    name: shop-worker
    runtime: python
    startCommand: celery -A config worker -l info --concurrency 2
    envVars: [{ fromGroup: shop-env }]

For scheduled jobs, Render Cron Jobs run a command on a schedule, which can replace beat.

3. Railway

Add a new service from the same GitHub repository. In its settings, set the start command to celery -A config worker -l info --concurrency 2 and remove any public domain. Reference shared variables (or variables from the web service) so DATABASE_URL, REDIS_URL, and SECRET_KEY match. Add a Redis database from Railway's templates and reference its URL with Railway's variable references.

Railway services can also run on a cron schedule, which is a simple way to run management commands periodically without beat.

4. Fly.io

Define process groups in fly.toml; each group runs on its own Machines from the same image:

[processes]
  app = "gunicorn config.wsgi --bind 0.0.0.0:8000 --workers 2"
  worker = "celery -A config worker -l info --concurrency 2"

[[services]]
  processes = ["app"]
  internal_port = 8000
  protocol = "tcp"

Only the app group gets a service and public ports. Scale independently: fly scale count app=2 worker=1. Set secrets once with fly secrets set; all process groups receive them. For Redis, use Upstash through Fly's integration or run Redis as a separate Fly app.

5. Beat and shutdowns

Run beat as its own single-instance service or process group (beat = "celery -A config beat -l info" scaled to 1), never inside multiple worker instances. Each platform sends SIGTERM before stopping a worker; Celery finishes current tasks during the grace period, and longer tasks are redelivered with acks_late.

Explanation

Celery workers are long-running processes that connect to the broker and wait for messages; they need no HTTP port, only the same code and configuration as the web app. Every modern PaaS supports running such processes, under different names. Keeping one image and one shared set of environment variables guarantees that the worker registers the same tasks and connects to the same database and broker as the code that enqueues them.

Edge cases / notes

• Idle sleeping. Free or hobby tiers may stop idle services; workers must stay running to receive tasks.
• Concurrency and memory. Each Celery child process loads Django; size instance memory for the concurrency you choose.
• Results backend. Skip it unless code reads task results; it adds Redis memory and complexity.

Next steps

• Compare platforms in Render vs Railway vs Fly.io for Django.
• Configure the broker carefully with Configure Redis and Celery for Django in Production.
• On Heroku instead? See Run Celery Workers on Heroku.

FAQ

Do Celery workers need a public URL?

No. They connect to the broker. Remove public ports and domains from worker services.

Can I replace Celery beat with platform cron?

Yes, for jobs that run every few minutes or less often. Use beat for second-level precision or many schedules.

Why does the worker say a task is unregistered?

It runs different code or settings than the web service. Deploy both from the same commit with the same environment.

How many workers should I run?

Start with one worker at concurrency 2 and scale based on queue length and memory.