Docker Compose in production: 10 checks before you sleep well
- 1. Pin every image to an explicit version tag
- 2. Once a tag is tested, pin it by digest
- 3. Do not auto-update stateful services
- 4. Keep secrets out of the Compose file and out of git
- 5. Healthchecks must mean “ready for work”
- 6. Publish host ports only for the reverse proxy
- 7. Back up databases with a consistent logical dump
- 8. Rehearse the restore in a separate Compose project
- 9. Cap container logs and memory
- 10. Keep a written rollback — and do not make a bad night worse
A Compose file that “works on my machine” is not a production stack. The problems that wake people up at 3 a.m. are almost always the same: a :latest tag that moved, a secret that leaked into git, a dump pipeline that reported success while writing an empty file, or a rollback that turned into docker compose down -v.
This is a short, practical list of ten checks for day-2 operations of Docker Compose V2. Tick an item only when it is done and verified. Commands are examples; check them against the documentation for your Engine and Compose versions, and test in staging first.
1. Pin every image to an explicit version tag
Never use :latest in production. A later pull can install a new major version with no record of what was actually running.
# bad
image: postgres:latest
# good
image: postgres:16.6
2. Once a tag is tested, pin it by digest
A tag can be overwritten on the registry. A digest is immutable and is what you actually roll back to.
docker buildx imagetools inspect postgres:16.6 --format '{{json .Manifest.Digest}}'
# then in compose:
# image: postgres:16.6@sha256:<digest>
Before every update, record what is running:
docker compose images
docker inspect --format '{{.Name}} {{index .RepoDigests 0}} {{.Created}}' $(docker compose ps -q)
“It was 16.6” is not enough if the tag has moved.
3. Do not auto-update stateful services
Watchtower and similar tools have shipped breaking major upgrades into production databases. If you auto-update at all, restrict it to stateless, pinned, disposable services with an allowlist — never the data layer.
labels:
com.centurylinklabs.watchtower.enable: "false"
A notifier (for example Diun) is safer than an unattended apply.
4. Keep secrets out of the Compose file and out of git
Compose files get copied, pasted into tickets and pushed to Git. A password in environment: will leak. Prefer Compose secrets: (or a read-only bind mount) and let the application read /run/secrets/…. docker inspect and crash dumps expose Env.
.env
*.env
secrets/
!*.env.example
chmod 700 secrets && chmod 600 secrets/* .env
Give each service its own credentials, with least privilege, and a documented rotation path.
5. Healthchecks must mean “ready for work”
A TCP or PID check stays green while the application is still migrating the database or returning 503. Define a healthcheck for every service that other services or a proxy depend on, and set an explicit restart policy (unless-stopped, or on-failure with a cap). The Compose default is no.
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
interval: 10s
timeout: 5s
retries: 5
start_period: 40s
restart: unless-stopped
depends_on:
db:
condition: service_healthy
depends_on only covers the first boot. Clients still need retry logic.
6. Publish host ports only for the reverse proxy
ports: "5432:5432" on a public IP is a database on the internet, even if a proxy also exists. Application and database ports should stay unpublished. expose: is not a security boundary. Put the database on an internal network the proxy cannot reach if it does not need to.
networks:
frontend:
backend:
internal: true
# proxy only:
ports:
- "127.0.0.1:80:80"
- "127.0.0.1:443:443"
Audit after every change:
docker ps --format '{{.Names}}\t{{.Ports}}'
ss -tlnp
Drop network_mode: host and privileged: true unless you can explain a specific need.
7. Back up databases with a consistent logical dump
Do not tar a live data volume while the engine is running. That copy is often unrestorable. Use a logical dump (or a supported hot-backup tool), put set -o pipefail on the pipeline, and fail the job if the file is empty or smaller than a sanity threshold. pg_dump | gzip > file reports success when pg_dump fails but gzip does not.
set -euo pipefail
docker compose exec -T db pg_dump -U app --format=custom appdb \
> /backup/appdb-$(date +%F).dump
test -s /backup/appdb-$(date +%F).dump
Copy the dump off the host. Never treat a replica volume as the only copy — replicas also replicate DROP TABLE.
8. Rehearse the restore in a separate Compose project
A dump that has never been restored is a hope, not a backup. Restoring in-place on the production project is how you discover a bad dump the hard way.
docker compose -p restore-drill up -d db
# wait until healthy, then:
# cat /backup/appdb-YYYY-MM-DD.dump | docker compose -p restore-drill exec -T db \
# pg_restore -U app -d appdb --clean --if-exists
docker compose -p restore-drill down -v
9. Cap container logs and memory
The default json-file driver is unbounded. A chatty container will fill the disk and take the whole host down. Set memory limits so one leak does not invoke the host OOM killer on your database. Compose V2 honours deploy.resources.limits on docker compose up.
logging:
driver: json-file
options:
max-size: "10m"
max-file: "5"
deploy:
resources:
limits:
memory: 512M
Watch free space on /var/lib/docker (df -h, docker system df) before the filesystem is full.
10. Keep a written rollback — and do not make a bad night worse
Validate the project and take a fresh, verified backup before every upgrade (docker compose config --quiet). Apply the change first in a staging project with the same Compose files and separate volumes. Keep the previous digest written down, and know when not to roll back — for example after a one-way migration.
docker compose up -d --no-deps --force-recreate --pull never app
After a failed upgrade, do not run docker system prune -a or docker compose down -v, and do not “fix” data until you have a copy of the current volumes. Under pressure, those two commands turn a bad evening into a restore from last week — or from nothing.
The ten checks above are the short version of our Docker Compose production checklist, licensed CC BY 4.0 (attribution: Sats Sheets). The full checklist — a printable PDF plus an XLSX/Markdown tracker — is available in English, French, Italian, Polish, Hungarian, Dutch and Turkish, paid in sats: English edition. All of our IT checklists are listed in the Sats Shop catalog.
No warranty; test in staging first. Always have a tested backup before changing a production system.
Write a comment