DevOps Tutorials, Guides & Best Practices

Your container is running. docker ps shows it’s up. But the status column says unhealthy – and you’re not sure what’s actually broken.

Docker’s health check system is a runtime verification mechanism: it doesn’t just check if the process is alive, it runs a command inside the container to verify the application is actually working. When that command fails, Docker marks the container unhealthy.

This guide covers diagnosis, common root causes, fix patterns, and how to prevent health check failures before they hit production.

What “Unhealthy” Actually Means

Docker health checks run independently of the container’s main process. Even if docker ps shows Up 10 minutes, the application inside might be broken – a port not listening, a database connection refused, a disk full.

Every health check produces an exit code:

– 0 – healthy

– 1 – unhealthy

– 2 (or any other non-zero) – reserved; Docker treats it as a warning, status unchanged

A container is marked unhealthy after the health check fails retries consecutive times (default: 3). Until then, it stays in starting or healthy.

This is fundamentally different from a crashed container. An unhealthy container is still running – Docker doesn’t kill it automatically. It’s up to your restart policy, orchestrator, or monitoring to react.

Quick Diagnosis: Is It Really Unhealthy?

Start with the container’s health log. This shows every health check run, its exit code, and any output:

docker inspect --format='{{json .State.Health}}' my-container | jqCode language: Bash (bash)

You’ll see something like:

{
  "Status": "unhealthy",
  "FailingStreak": 5,
  "Log": [
    {
      "Start": "2026-06-15T10:00:00Z",
      "End": "2026-06-15T10:00:03Z",
      "ExitCode": 1,
      "Output": "curl: (7) Failed to connect to localhost port 8080: Connection refused\n"
    }
  ]
}Code language: JSON / JSON with Comments (json)

Key fields to check:

– FailingStreak – how many consecutive failures (≥ retries = unhealthy)

– ExitCode – 1 means the health check command itself failed

– Output – the actual error message from the health check command

To filter only unhealthy containers across your system:

docker ps --filter "health=unhealthy"Code language: Bash (bash)

To watch health status changes in real time:

docker events --filter event=health_statusCode language: Bash (bash)

Common Root Causes and How to Fix Them

When a container shows unhealthy, the root cause usually falls into one of five categories. Here is how to diagnose and fix each one.

1. Application Not Listening on the Expected Port

Symptom: curl: (7) Failed to connect to localhost port 8080: Connection refused

The most common cause. Your health check tries to reach localhost:8080/health, but the application either hasn’t started yet or is listening on a different port.

Diagnose inside the container:

docker exec my-container sh -c 'curl -v http://localhost:8080/health'Code language: Bash (bash)

Or check what’s actually listening:

docker exec my-container netstat -tlnpCode language: Bash (bash)

Fix:

– Verify the app’s actual listening port matches the health check

– Check that the app binds to 0.0.0.0, not 127.0.0.1 (Docker’s port mapping requires 0.0.0.0)

– For slow-starting apps, increase --start-period

HEALTHCHECK --start-period=60s --retries=5 \
  CMD curl -f http://localhost:3000/health || exit 1Code language: Dockerfile (dockerfile)

2. Missing Dependencies

Symptom: psql: could not connect to server: Connection refused or redis-cli: Could not connect to Redis

Your app needs a database or cache, but the dependent service isn’t ready when the health check runs.

Quick check: run the health check command manually inside the container:

docker exec my-app sh -c 'pg_isready -h db -U postgres'Code language: Bash (bash)

Fix in Docker Compose:

services:
  app:
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 10s
      start_period: 30s
  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      retries: 5Code language: YAML (yaml)

The condition: service_healthy clause waits for the database to pass its own health check before starting the app container.

For complex dependency chains, a custom health check script can verify multiple services:

#!/bin/sh
pg_isready -h db -U postgres || exit 1
redis-cli -h cache ping || exit 1
curl -f http://localhost:8080/health || exit 1Code language: Bash (bash)

3. Resource Exhaustion

Symptom: Empty output, long timeouts, or curl: (28) Operation timed out

The container hit a resource limit – memory, CPU, or disk.

Diagnose:

docker stats my-container --no-streamCode language: Bash (bash)

Check disk inside the container:

docker exec my-container df -hCode language: Bash (bash)

For OOM (out of memory) kills, check dmesg on the host or the container’s exit code (137 = killed by OOM).

Fix:

– Increase memory limits: docker run -m 512m my-app

– Add log rotation to prevent disk fill

– Set appropriate --timeout on the health check (default 30s – too long for a sluggish container):

HEALTHCHECK --timeout=5s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1Code language: Dockerfile (dockerfile)

4. Misconfigured HEALTHCHECK Command

Symptom: exec: "healthcheck.sh": permission denied or curl: not found

Common mistakes in the Dockerfile:

Missing executable permission:

# Wrong – script isn't executable
COPY healthcheck.sh /usr/local/bin/
HEALTHCHECK CMD /usr/local/bin/healthcheck.sh

# Right
COPY healthcheck.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/healthcheck.sh
HEALTHCHECK CMD /usr/local/bin/healthcheck.shCode language: Dockerfile (dockerfile)

Missing curl in the image:

Minimal base images (Alpine, distroless) don’t include curl. Either install it:

FROM alpine:3.20
RUN apk add --no-cache curl
HEALTHCHECK CMD curl -f http://localhost:8080/health || exit 1Code language: Dockerfile (dockerfile)

Or use an alternative like wget:

HEALTHCHECK CMD wget --spider -q http://localhost:8080/health || exit 1Code language: Dockerfile (dockerfile)

Wrong shell in exec form. The exec form CMD ["curl", "-f", "..."] doesn’t use a shell – so || exit 1 won’t work:

# Wrong – pipe/OR syntax needs a shell
HEALTHCHECK CMD ["curl", "-f", "http://localhost:8080/health", "||", "exit", "1"]

# Right – use CMD-SHELL or CMD with a string
HEALTHCHECK CMD curl -f http://localhost:8080/health || exit 1Code language: Dockerfile (dockerfile)

5. Aggressive Health Check Timing

Symptom: Container flaps between healthy and unhealthy every few seconds.

The health check --timeout is too tight for the application’s actual response time under load, or --interval runs checks faster than the app can recover.

Fix – tune parameters for your workload:

Parameter	Default	Recommendation
--interval	30s	10–30s. Shorter for critical services, longer for batch jobs
--timeout	30s	2–5s for HTTP checks. Set to the app’s real p99 response time
--retries	3	3–5 for tolerance, 1–2 for fast failover in orchestrators
--start-period	0s	30–60s if your app has a slow bootstrap

Example for a Java app with a 40-second startup:

HEALTHCHECK --interval=15s --timeout=5s --start-period=60s --retries=3 \
  CMD curl -f http://localhost:8080/actuator/health || exit 1Code language: Dockerfile (dockerfile)

Fix Patterns That Work

Once you have identified the root cause, apply one of these three patterns to get your container back to healthy.

Pattern 1: Fix and Recreate

If the health check itself is wrong (wrong port, missing binary), fix the Dockerfile or Compose file and recreate:

docker compose up -d --force-recreate appCode language: Bash (bash)

Pattern 2: Restart Policy + Autoheal

Docker’s restart policy (--restart=always) doesn’t react to health status – it only restarts on process exit. To restart unhealthy containers automatically, you need a sidecar. The simplest option:

docker run -d --name autoheal \
  -e AUTOHEAL_CONTAINER_LABEL=all \
  -v /var/run/docker.sock:/var/run/docker.sock \
  willfarrell/autohealCode language: Bash (bash)

Or build self-healing into the health check itself – force an exit to trigger the restart policy:

HEALTHCHECK CMD curl -f http://localhost:8080/health || kill -s 15 1Code language: Dockerfile (dockerfile)

When curl fails, the health check kills PID 1 (the main process), the container exits, and Docker’s restart policy kicks in.

Pattern 3: Docker Compose with depends_on

For multi-service apps, chain health checks so services start in the right order:

services:
  api:
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
  postgres:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 3s
      retries: 10
  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]Code language: YAML (yaml)

Docker vs Kubernetes: The Liveness Probe Connection

If you work with both Docker and Kubernetes, the concepts map directly:

Docker	Kubernetes
HEALTHCHECK instruction	livenessProbe
--interval	periodSeconds
--timeout	timeoutSeconds
--retries	failureThreshold
--start-period	initialDelaySeconds
docker inspect .State.Health	kubectl describe pod

The key difference: Kubernetes kills and restarts the pod when the liveness probe fails. Docker only marks the container unhealthy – restarting is up to you (or your orchestrator).

If you’re designing health checks for an app that runs on both Docker Compose (local/dev) and Kubernetes (production), use the same endpoint and similar thresholds.

Prevention: Writing Health Checks That Don’t Lie

A bad health check is worse than no health check. It gives a false sense of safety while the app burns.

Do:

– Check the actual application endpoint, not just “port is open”

– Keep the check fast and lightweight (under 1 second normally)

– Use a dedicated /health or /ready endpoint that verifies critical dependencies (DB, cache, message queue)

– Test the health check locally: docker exec my-container

Don’t:

– Just curl localhost without --fail – HTTP 500 still returns exit 0

– Run expensive checks (database migrations, disk scans) inside the health check

– Use pg_isready alone for PostgreSQL – it only checks if the server accepts connections, not if the database is actually ready for queries

– Leave --timeout at the default 30s – a hung health check blocks status updates for 30 seconds

Good health check example for a web app:

HEALTHCHECK --interval=15s --timeout=3s --start-period=20s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1Code language: Dockerfile (dockerfile)

The /health endpoint itself verifies DB connectivity and returns 200 only when everything is ready.

Good health check for a worker (no HTTP):

HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
  CMD pgrep -f "sidekiq" || exit 1Code language: Dockerfile (dockerfile)

Summary

– Unhealthy ≠ crashed. The container is running, but the app inside isn’t working as expected

– Start with docker inspect – the health log tells you exactly what failed

– Run the health check command manually inside the container to reproduce the issue

– Most failures are wrong port, missing dependency, resource exhaustion, or misconfigured HEALTHCHECK

– For production, pair health checks with restart policies, Compose depends_on, or Kubernetes liveness probes

—

Internal links: Docker Compose Healthcheck · Docker Logs: View, Tail, and Grep · Docker Compose Restart Policy · Docker Compose Stop · Docker Compose Down