Fix Kubernetes ImagePullBackOff for a Django Deployment

Problem statement

A new release never starts. kubectl get pods shows:

NAME                   READY   STATUS             RESTARTS   AGE
web-6c9d7f8b5-x2k4p    0/1     ImagePullBackOff   0          4m

Kubernetes could not download the container image, so the Django container never ran. ErrImagePull is the first failure; ImagePullBackOff means the kubelet is now waiting longer between retries. The cause is in the events: a wrong name or tag, missing registry credentials, an expired token, a rate limit, or a network path to the registry that does not exist.

1. Read the reason: kubectl describe pod <pod> | sed -n '/Events/,$p'.
2. not found or manifest unknown: the tag does not exist; check what CI pushed.
3. unauthorized or 401: add an imagePullSecrets entry for a private registry.
4. toomanyrequests: Docker Hub rate limit; authenticate or mirror the base image.
5. no match for platform: build the image for the node's architecture.

Step-by-step solution

1. Read the pull error

kubectl describe pod web-6c9d7f8b5-x2k4p | sed -n '/Events/,$p'

Typical messages and their meaning:

Failed to pull image "ghcr.io/acme/web:3f2a1c9": ... not found
Failed to pull image "...": ... 401 Unauthorized
Failed to pull image "...": ... toomanyrequests: You have reached your pull rate limit
Failed to pull image "...": ... no match for platform in manifest

2. Fix the name and tag

Confirm the exact reference exists:

docker buildx imagetools inspect ghcr.io/acme/web:3f2a1c9

Common mistakes: CI pushed :latest but the manifest uses the commit SHA, the SHA is shortened differently (7 versus 8 characters), the repository name has uppercase letters (registries require lowercase), or the deploy ran before the push finished. Make the deploy job depend on the push job and use the same variable for the tag in both.

3. Add registry credentials

For GitHub Container Registry, create a token with read:packages and a pull secret:

kubectl create secret docker-registry ghcr-pull \
  --docker-server=ghcr.io \
  --docker-username=deploy-bot \
  --docker-password="$GHCR_TOKEN" \
  -n production

Reference it in the Deployment, or attach it to the namespace's default service account so every pod gets it:

spec:
  template:
    spec:
      imagePullSecrets:
        - name: ghcr-pull

kubectl patch serviceaccount default -n production \
  -p '{"imagePullSecrets": [{"name": "ghcr-pull"}]}'

The secret must be in the same namespace as the pod.

4. Handle cloud registries

• Amazon ECR on EKS: nodes pull with their IAM role. Attach AmazonEC2ContainerRegistryReadOnly to the node role; no secret needed. Outside EKS, ECR tokens expire after 12 hours, so a static secret stops working; use a credential helper or a CronJob that refreshes it.
• Google Artifact Registry on GKE: grant the node service account roles/artifactregistry.reader.
• Azure ACR on AKS: attach the registry with az aks update --attach-acr.

5. Avoid Docker Hub rate limits

Anonymous pulls from Docker Hub are rate limited per IP, and cluster nodes behind a NAT share one IP. Base images such as python, postgres, and redis can hit the limit during a rollout. Authenticate pulls with a Docker Hub account secret, or mirror images to your own registry and reference the mirror.

Explanation

The kubelet on each node pulls images through the container runtime (containerd). It authenticates with credentials from the pod's imagePullSecrets, the service account, or the node's cloud identity. Any failure is reported as ErrImagePull, and repeated failures put the container into a back-off loop, retrying with growing delays up to five minutes. Nothing about Django is involved yet; the image never reached the node.

Edge cases / notes

• imagePullPolicy. Always checks the registry on every start, so registry outages block restarts. IfNotPresent with immutable tags (commit SHAs) is safer.
• Private networks. Nodes without internet access need a VPC endpoint or private link to the registry.
• Mutable tags. Reusing :latest means nodes may run different images. Use unique tags per release.

Next steps

• If the image pulls but the container keeps crashing, see Fix Kubernetes CrashLoopBackOff for Django.
• Set up GHCR publishing with Deploy Django Docker Images From GHCR.
• For architecture problems, read Fix Docker exec format error.

FAQ

What is the difference between ErrImagePull and ImagePullBackOff?

ErrImagePull is a single failed attempt. ImagePullBackOff means repeated failures and Kubernetes is waiting before trying again.

Do I need to delete the pod after fixing the secret?

No. The kubelet keeps retrying and picks up the fix on the next attempt, though you can delete the pod to retry immediately.

Why does the pull work on my laptop?

Your Docker client is logged in to the registry. Cluster nodes need their own credentials.

Should I use in Kubernetes?

Avoid it. Unique tags make rollouts and rollbacks predictable and prevent nodes from running different code.