Table of Contents
Reliable Docker Compose Startup After a Reboot
A common and confusing failure mode: a Docker Compose stack that runs perfectly when you start it by hand starts misbehaving after the server reboots. One container pins a CPU core, the machine runs hot and the fans spin up — even though the overall system load is near zero. This page explains why it happens and shows a clean, general way to prevent it.
The trap: ''depends_on'' is not honored at reboot
Compose lets you declare start-up order and readiness gating:
services: aggregator: depends_on: backend: condition: service_healthy
This tells Compose: “do not start aggregator until backend reports healthy.” It works exactly as expected — but only when you run docker compose up.
The catch: after a host reboot, your containers are usually brought back not by Compose, but by the Docker daemon's restart policy (restart: always or restart: unless-stopped). The daemon restarts containers in an arbitrary order and ignores depends_on entirely.
Key fact:depends_on: condition: service_healthyis respected bydocker compose up— never by the Docker daemon when it restarts containers at boot.
Why this causes a runaway (hot fans, pinned core)
Many “aggregator”, “proxy” or “gateway” style services discover their backends once at start-up and never retry. If such a service starts before its backend is ready — exactly what the reboot race produces — it can end up:
- holding a dead or half-open connection,
- retrying in a tight loop with no back-off,
- pinning a full CPU core at ~100%.
The result: the box is essentially idle (load near zero) yet one core runs flat out, temperatures climb and the fans ramp up — with no obvious “busy” process at a casual glance.
Analogy: it is like a courier who memorises the delivery list once, at the door, before the addresses are posted — then drives in circles forever because the list came up blank.
The clean fix: let systemd own start-up
The reliable pattern is to stop relying on the daemon's restart policy for boot ordering, and hand that job to systemd — which does respect Compose's health gating, because it runs docker compose up itself.
1. Stop the daemon from race-starting containers
Change the restart policy in your docker-compose.yml from unless-stopped to on-failure:
services: backend: restart: on-failure aggregator: restart: on-failure
Why: the daemon auto-starts always and unless-stopped containers at boot (that is the race). It does not auto-start on-failure containers at boot — yet on-failure still restarts a container that crashes while running. So you keep crash recovery but remove the boot race.
2. Make systemd start the whole stack, in order
Create /etc/systemd/system/mystack.service:
[Unit] Description=My application stack (docker compose) Requires=docker.service After=docker.service network-online.target Wants=network-online.target [Service] Type=oneshot RemainAfterExit=yes WorkingDirectory=/opt/mystack ExecStart=/usr/bin/docker compose up -d --wait ExecStop=/usr/bin/docker compose stop [Install] WantedBy=multi-user.target
Enable it once:
sudo systemctl daemon-reload sudo systemctl enable --now mystack.service
Now systemd is the single thing that starts the stack at boot, and –wait blocks until every service is healthy — so the aggregator always comes up after its backends.
3. Handle cold-boot transients with a retry
On a cold boot, a backend that needs the network (DNS, a remote API) may crash-loop for a few seconds until networking is fully up. docker compose up –wait is intolerant of this: the moment any container reports unhealthy, it aborts — it will not wait for the container to recover on its own.
Make the unit retry, so the stack converges by itself:
[Unit] # ... StartLimitIntervalSec=900 StartLimitBurst=6 [Service] # ... Restart=on-failure RestartSec=20
If the first attempt fails because a backend was still settling, systemd waits and re-runs compose up –wait. By the retry the backend is healthy, the aggregator starts, and the unit goes active — with zero manual intervention. Seeing a handful of restarts in the boot log is normal, not a fault:
systemctl show mystack.service -p NRestarts --value
How to diagnose the runaway
If a box runs hot after a reboot but overall load is low, look for a single pinned container:
# per-container CPU — the culprit sits near a full core docker stats --no-stream # confirm it is a spin, not real work, then read its logs # for a tight retry / error loop (e.g. the same request repeating) docker logs --tail 50 <container>
A quick manual recovery (until the boot fix is in place) is simply to restart the affected container once its dependencies are up:
docker restart <container>
Optional: a runtime watchdog
The steps above fix boot ordering. A separate, rarer problem is a long-running proxy that holds a persistent session to a backend and has no working auto-reconnect: if the backend restarts during normal operation, the session dies and the proxy may spin again. If your software has this limitation, a small periodic health-check script (driven by a systemd timer or cron) that restarts the proxy when it detects the fault is a reasonable backstop. Guard it with a minimum interval between restarts so it can never storm.
Summary
depends_on: service_healthyorderscompose up, not the daemon at reboot.- Use
restart: on-failureso the daemon does not race-start containers at boot. - Let a systemd oneshot unit run
docker compose up -d –wait— it respects health ordering. - Add
Restart=on-failure+RestartSecso cold-boot transients self-heal. - Result: the stack comes up in the correct order every time, and a single misbehaving service can no longer pin a core and roast your fans.
