Run Celery Workers with systemd for Django

Problem statement

Running Celery workers manually in production is fragile. If you start a worker from a shell and disconnect, the process may stop. Even when it stays running, it often depends on shell-specific environment variables, the current working directory, or the active virtual environment.

Common production failures look like this:

• the worker dies after logout or reboot
• Celery starts with the wrong Django settings
• the broker URL is missing under systemd
• logs are split across shell history, files, and nowhere useful
• the worker does not restart after a crash
• deployments update code, but the worker keeps running old code

For a Django production setup, Celery should be managed like any other long-running service. On most Linux servers, that means systemd.

Quick answer

Use a dedicated systemd unit to run Celery as your app user, not as root. Point it at the correct release directory and virtualenv, load environment variables from a protected env file, and define restart behavior that matches your task runtime.

Then:

1. reload systemd
2. enable the service on boot
3. start it
4. verify it with systemctl status
5. queue a real test task
6. read logs with journalctl

For most hosts, use network-online.target instead of only network.target so the worker does not race broker connectivity during boot.

If a change breaks the worker, roll back by restoring the previous release or the previous unit file, then restart the service and verify task consumption.

Step-by-step solution

Step 1 - Confirm the Django and Celery production setup

Before creating a unit file, confirm the pieces Celery needs already work.

Verify the Celery app entrypoint

In most Django projects, Celery is started with an app path like:

yourproject.celery:app

A typical ExecStart uses:

celery -A yourproject worker --loglevel=INFO

Make sure yourproject matches the Python import path of your Django project.

Verify broker connectivity

Confirm Redis or RabbitMQ is reachable from the server. For example, if you use Redis:

redis-cli -u "$CELERY_BROKER_URL" ping

You should get:

PONG

If broker access is blocked by firewall rules, bad credentials, or the wrong hostname, systemd will not fix that.

Verify the runtime user and paths

Decide exactly what the service will use:

• app user: django
• app group: django
• project path: /srv/myapp/current
• virtualenv path: /srv/myapp/venv
• env file: /etc/myapp/myapp.env

If you use release directories, a common layout is:

/srv/myapp/
├── current -> /srv/myapp/releases/2026-04-24-120000
├── releases/
└── venv/

Verification check:

sudo -u django test -x /srv/myapp/venv/bin/celery && echo ok
sudo -u django test -d /srv/myapp/current && echo ok
sudo -u django test -r /etc/myapp/myapp.env && echo ok

Step 2 - Create a dedicated systemd service for the Celery worker

Choose the service user and group

Do not run Celery as root. Use the same non-root user that owns or operates the Django app.

Define the working directory and executable

Create /etc/systemd/system/celery.service:

[Unit]
Description=Celery Worker for Django app
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=django
Group=django
WorkingDirectory=/srv/myapp/current
EnvironmentFile=/etc/myapp/myapp.env
ExecStart=/srv/myapp/venv/bin/celery -A yourproject worker --loglevel=INFO
Restart=on-failure
RestartSec=5
TimeoutStopSec=300
KillMode=control-group
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

This assumes:

• Celery is installed in /srv/myapp/venv
• your Django code is under /srv/myapp/current
• the app import path is yourproject

If the broker runs locally on the same host, you can extend ordering, for example:

After=network-online.target redis.service

Set environment variables safely

Create the env file:

sudo install -d -m 0750 -o root -g django /etc/myapp
sudo touch /etc/myapp/myapp.env
sudo chown root:django /etc/myapp/myapp.env
sudo chmod 0640 /etc/myapp/myapp.env

Example /etc/myapp/myapp.env:

DJANGO_SETTINGS_MODULE=yourproject.settings.production
DATABASE_URL=postgres://myapp:password@127.0.0.1:5432/myapp
CELERY_BROKER_URL=redis://127.0.0.1:6379/0
CELERY_RESULT_BACKEND=redis://127.0.0.1:6379/1

Only include secrets the worker actually needs. Avoid exposing unrelated application secrets to the Celery service environment.

Do not hardcode secrets inside the unit file if that file is managed in git or copied broadly between hosts.

Configure restart behavior and limits

Good defaults for most Django Celery workers:

• Restart=on-failure to restart crashed workers
• RestartSec=5 to avoid tight restart loops
• TimeoutStopSec=300 to allow more time for graceful shutdown
• KillMode=control-group so related processes are stopped together

If your tasks can run for a long time, increase TimeoutStopSec further so workers can shut down cleanly during deploys.

Control startup ordering

network-online.target is a better default than network.target for workers that depend on a remote broker. It does not guarantee Redis or RabbitMQ is healthy, but it reduces boot-time startup races.

If the broker runs locally on the same host and has a named service, add ordering such as:

After=network-online.target redis.service

Step 3 - Reload systemd and start the worker

Reload unit files

sudo systemctl daemon-reload

Enable service at boot

sudo systemctl enable celery

Start and inspect service state

sudo systemctl start celery
sudo systemctl status celery

Expected status should include something like:

Active: active (running)

You can inspect the final installed definition with:

systemctl cat celery

Read logs with journalctl

sudo journalctl -u celery -f

Look for lines showing:

• Celery version
• broker connection established
• registered tasks loaded
• worker ready

If startup fails, journalctl is usually the fastest way to see import errors, missing env vars, or permission problems.

Step 4 - Verify task execution end to end

Starting the service is not enough. Confirm it actually consumes tasks.

Queue a test task from Django shell or app code

If you have a simple test task, open Django shell:

cd /srv/myapp/current
/srv/myapp/venv/bin/python manage.py shell

Then queue a task:

from yourapp.tasks import example_task
example_task.delay()

Confirm the worker consumes the task

In the worker logs:

sudo journalctl -u celery -f

You should see the task being received and finished.

Check logs for import errors, permission errors, and broker failures

Typical bad patterns:

• ModuleNotFoundError
• ImproperlyConfigured
• Permission denied
• broker connection refused or authentication errors

If it works in your shell but fails under systemd, the problem is usually one of:

• missing EnvironmentFile
• wrong WorkingDirectory
• wrong virtualenv path
• wrong service user permissions

Step 5 - Add optional Celery Beat under systemd

When Beat should be a separate service

Run Celery Beat as a separate service if you use scheduled tasks. Do not combine worker and beat into one systemd process.

Minimal service structure for Beat

Create /etc/systemd/system/celerybeat.service:

[Unit]
Description=Celery Beat for Django app
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=django
Group=django
WorkingDirectory=/srv/myapp/current
EnvironmentFile=/etc/myapp/myapp.env
ExecStart=/srv/myapp/venv/bin/celery -A yourproject beat --loglevel=INFO
Restart=on-failure
RestartSec=5
TimeoutStopSec=300
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl enable celerybeat
sudo systemctl start celerybeat

Avoid duplicate schedulers in multi-host setups

Only run one Beat scheduler for a given schedule unless you are using a setup specifically designed for distributed scheduling. Multiple Beat instances can enqueue duplicate jobs.

Step 6 - Make restarts and deployments safe

Restart the worker after code deploys

Celery workers keep old Python code in memory. After a deploy, restart them:

sudo systemctl restart celery

If you run Beat:

sudo systemctl restart celerybeat

Be careful with systemctl restart if tasks are long-running, because systemd will eventually force-stop the worker when TimeoutStopSec expires.

Handle in-flight tasks carefully

A normal stop is preferable to a forced kill:

sudo systemctl stop celery

TimeoutStopSec controls how long systemd waits before killing the service. Set it high enough for your shutdown pattern, then test real stop and restart behavior in logs. Celery workers commonly use prefork child processes, so validate that your worker exits cleanly and that child processes are not being killed too early during deploys.

Coordinate migrations and async tasks

If a deploy includes database schema changes, tasks may fail if workers run new code against old schema or old code against new schema. Plan migrations and worker restarts together, especially for tasks that read or write changed tables.

A safer pattern is:

1. deploy code that stays compatible with both old and new schema where possible
2. run migrations
3. restart workers
4. verify task execution
5. remove compatibility code in a later deploy if needed

Rollback note: if a deploy breaks task execution, switch current back to the previous release, restart the worker, and confirm tasks are being consumed again. Also check whether queued tasks contain payloads or assumptions tied to the failed release.

Explanation

This setup works because systemd gives Celery a consistent runtime environment:

• fixed user and group
• fixed working directory
• explicit environment variables
• controlled restart behavior
• boot-time startup
• centralized logs through the journal

That removes the common “works in shell, fails in production” problem.

For most Django deployments on Ubuntu or other Linux distributions, systemd is the simplest production process manager. Container-based deployments may use a different supervisor model, but on a standard VM or dedicated host, systemd is usually the right default.

When to automate this

If you repeat this setup across staging, production, and multiple projects, turn it into a reusable template. The first things worth scripting are unit file creation, env file permissions, daemon-reload, service enable/start, and post-deploy worker restarts. That reduces path mistakes and inconsistent restart policies.

Edge cases and production notes

• Keep env files readable only by root and the app group.
• Do not commit secrets in unit files or repository-managed config.
• Verify the service user can read the project and execute the virtualenv binaries.
• Confirm WorkingDirectory points to the correct release or current symlink.
• If you use symlinked deploys, restart workers after changing current, or they may keep using old code already loaded into memory.
• If you run multiple worker services, give them distinct service names and queue bindings so you can monitor and restart them independently.
• If you run more than one worker on the same host, set explicit worker names or separate service definitions so logs and Celery inspection output are easier to interpret.
• Under load, open file limits can become a real issue. If you see connection or descriptor errors, review systemd resource limits for the service instead of only tuning Celery settings.
• If tasks are long-running or critical, test stop, restart, and rollback behavior with real jobs before relying on the service in production.

Rollback and recovery

If a service change breaks production:

1. restore the previous unit file if it changed
2. run sudo systemctl daemon-reload
3. point current back to the previous release if needed
4. restart the worker:

   sudo systemctl restart celery
   

5. verify status and logs
6. check whether failed tasks are retrying, stuck in queue, or were published with payloads that only the failed release understands

After rollback, queue one known-safe test task and confirm it is consumed.

Common mistakes

• wrong app module path in -A yourproject
• missing DJANGO_SETTINGS_MODULE
• wrong WorkingDirectory
• wrong virtualenv path in ExecStart
• broker reachable from shell but not from the service environment
• using only network.target and hitting boot-time broker races
• TimeoutStopSec too low for task runtime
• Beat running on multiple hosts and creating duplicate scheduled jobs

Internal links

For the background model behind workers, queues, and brokers, see How Celery Works in Django Production.

If your broker is not stable yet, start with Configure Redis for Django and Celery in Production.

For the main web application process layout, see Deploy Django with Gunicorn and Nginx on Ubuntu.

If the service refuses to start, use Why Celery Workers Fail to Start in Production.

FAQ

How do I run Celery workers automatically after a server reboot?

Enable the service:

sudo systemctl enable celery

That creates the boot-time dependency so the worker starts automatically.

Should Celery Beat run in the same systemd service as the worker?

No. Run Beat as a separate service. It has a different role and should be restarted, monitored, and scheduled independently.

Why does Celery work in my shell but fail under systemd?

Usually because your shell provides environment variables, a different current directory, or an activated virtualenv that systemd does not have. Set WorkingDirectory, EnvironmentFile, and the full ExecStart path explicitly.

How do I restart Celery workers safely during a deploy?

Deploy the new code, then restart the worker with:

sudo systemctl restart celery

If tasks are long-running, review TimeoutStopSec and test shutdown behavior so in-flight work is not killed too quickly.

What is the best way to store Django and broker environment variables for a systemd service?

Use a dedicated env file outside the repository, referenced with EnvironmentFile=. Lock down permissions so only root and the app group can read it, and only include variables the worker actually needs.