Use AWS ElastiCache (Redis OSS or Valkey) With Django

Problem statement

Your Django app runs on AWS (ECS, EC2, EKS, or Elastic Beanstalk) and needs Redis for caching, sessions, Celery, or Channels. ElastiCache is the managed option, but it offers choices that affect Django directly: Valkey or Redis OSS engines, serverless or node-based clusters, cluster mode enabled or disabled, TLS and authentication options, and endpoints that behave differently during failover.

1. Create a Valkey (or Redis OSS) cache in the same VPC and subnets as the app, with cluster mode disabled if you use Celery.
2. Allow inbound 6379 on the cache's security group only from the app's security group.
3. Enable encryption in transit and use a rediss:// URL; add an AUTH token or IAM authentication.
4. Use the primary endpoint for writes (cache, sessions, Celery); never node endpoints.
5. Enable Multi-AZ with automatic failover, and configure clients to reconnect after failovers.

Step-by-step solution

1. Choose the deployment type

• Serverless: no node sizing, scales automatically, billed by data stored and requests. Exposes a single endpoint with TLS required. It behaves as a cluster-mode endpoint, so check that every client supports it.
• Node-based, cluster mode disabled: one primary and up to five replicas, a primary endpoint and a reader endpoint. Works with every Django, Celery, and Channels client.
• Node-based, cluster mode enabled: shards data across node groups; requires cluster-aware clients. Celery's Redis transport does not support Redis Cluster.

For a typical Django app with Celery, a node-based cluster with cluster mode disabled is the safe choice.

2. Network access

ElastiCache is only reachable inside the VPC. Put it in private subnets and create a security group that allows TCP 6379 from the security group of your ECS tasks, EC2 instances, or EKS nodes:

aws ec2 authorize-security-group-ingress --group-id sg-cache \
  --protocol tcp --port 6379 --source-group sg-app

Developers connect through a bastion, SSM port forwarding, or a VPN, not by opening the security group.

3. TLS and authentication

With encryption in transit enabled, use rediss://. Authentication options:

• AUTH token: a password set on the cluster. URL: rediss://:TOKEN@master.shop.abc123.euc1.cache.amazonaws.com:6379/0.
• RBAC users: username and password per user group.
• IAM authentication: short-lived tokens generated from IAM credentials, available for Valkey and Redis OSS 7+; requires generating tokens in the client.

REDIS_URL = os.environ["REDIS_URL"]  # rediss://:token@master....cache.amazonaws.com:6379
CACHES = {"default": {"BACKEND": "django.core.cache.backends.redis.RedisCache", "LOCATION": f"{REDIS_URL}/1"}}
CELERY_BROKER_URL = f"{REDIS_URL}/0?ssl_cert_reqs=CERT_REQUIRED"

ElastiCache certificates are signed by Amazon's public CA, so verification works with standard CA bundles.

4. Use the right endpoint

The primary endpoint DNS name always points at the current primary. The reader endpoint balances across replicas and is read-only. Node endpoints point at specific nodes and break after failover. Writes against a replica fail with READONLY You can't write against a read only replica.

5. Failover and maintenance

Enable Multi-AZ with automatic failover. During failover or maintenance, connections drop briefly. Configure retries and health checks in clients, and make Celery retry broker connections:

CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True
CELERY_BROKER_TRANSPORT_OPTIONS = {"visibility_timeout": 3600, "health_check_interval": 30}

Use django-redis's IGNORE_EXCEPTIONS for the cache so a failover slows pages instead of failing them.

Explanation

ElastiCache runs Valkey or Redis OSS nodes inside your VPC and manages replication, patching, and failover. Its endpoints are DNS names that AWS updates when roles change, which is why using the primary endpoint matters. Django, Celery, and Channels talk to it like any Redis server; the AWS-specific work is in networking (VPC and security groups), encryption, and authentication.

Edge cases / notes

• Eviction policy. Set maxmemory-policy in a parameter group: allkeys-lru for pure caches, noeviction for a Celery broker so queued tasks are never evicted.
• Valkey pricing. AWS prices the Valkey engine lower than Redis OSS for the same node types.
• Cost of serverless. Serverless minimums can exceed a small node's cost for low-traffic apps.

Next steps

• Configure the cache with Set Up Redis Cache for Django in Production.
• Fix write errors after failover with Fix Redis READONLY Replica Errors.
• Consider the Valkey engine with Use Valkey With Django.

FAQ

Can I connect to ElastiCache from my laptop?

Not directly. Use a bastion, SSM port forwarding, or a VPN into the VPC.

Does Celery work with ElastiCache cluster mode?

No. Use cluster mode disabled for Celery's Redis transport.

Should I choose Valkey or Redis OSS?

Valkey for new clusters in most cases: compatible clients and lower pricing on ElastiCache.

Which endpoint should Django use?

The primary endpoint for anything that writes, including cache, sessions, and Celery.