Multi-Service Orchestration with Compose

Orchestrate multi-service local stacks with Docker Compose. Enforce startup sequences, manage shared state, isolate networks, and reset to a clean state on demand.

A stack of a database, cache, queue, API, and worker has to come up in the right order, share seed data deterministically, and tear back down to zero state — or it rots and stops matching CI. This guide gives platform engineers tactical steps for startup sequencing, shared state, network isolation, resource limits, and teardown. It is part of the broader containerized local environment patterns and builds on local network and port mapping for routing.

Prerequisites

  • Docker Compose v2 with BuildKit enabled (DOCKER_BUILDKIT=1).
  • jq for parsing docker compose ps --format json and docker network inspect.
  • A committed seed manifest checksum (.seed-manifest.sha256) for drift detection.

Service Dependency Graph and Startup Sequencing

Implicit depends_on guarantees container start order, not application readiness. Replace it with condition: service_healthy so dependent services block until their target is actually serving.

# docker-compose.yml
services:
  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d app_db"]
      interval: 3s
      timeout: 5s
      retries: 5
      start_period: 10s
  api:
    image: app/api:latest
    depends_on:
      db:
        condition: service_healthy

  1. Define explicit healthchecks for every infrastructure dependency (databases, caches, brokers).

  2. Replace bare depends_on with condition: service_healthy.

  3. For services lacking a native health endpoint, inject a lightweight readiness probe into the entrypoint.

  4. Validate boot order immediately after up:

    #!/usr/bin/env bash
set -euo pipefail
docker compose up -d --wait
docker compose ps --format '{{.Name}} {{.Status}}'

This maps directly onto the devcontainer configuration standards for IDE attachment ordering. When a service still attaches before its dependency is ready, the deep dive is resolving service startup order and healthcheck races.

Shared State and Seed Data Initialization

Deterministic environments need idempotent seed execution that survives restarts without manual intervention.

# docker-compose.yml
services:
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: app_db
      POSTGRES_PASSWORD: ${DB_PASSWORD:?set DB_PASSWORD in .env}
    volumes:
      - ./db/init:/docker-entrypoint-initdb.d:ro
      - db_data:/var/lib/postgresql/data
volumes:
  db_data:

  1. Author idempotent seed scripts (CREATE TABLE IF NOT EXISTS, INSERT ... ON CONFLICT DO NOTHING).

  2. Mount initialization directories read-only (:ro) to protect canonical manifests.

  3. Run seeds via the entrypoint init directory or a one-shot init container.

  4. Block compose up if local seed manifests diverge from the committed baseline:

    #!/usr/bin/env bash
set -euo pipefail
expected="$(cat .seed-manifest.sha256)"
actual="$(sha256sum db/init/*.sql | sha256sum | awk '{print $1}')"
if [[ "$expected" != "$actual" ]]; then
  echo "DRIFT DETECTED: seed manifests diverge from baseline" >&2
  exit 1
fi
echo "seed manifests match baseline"

Network Isolation and Inter-Service Discovery

Default bridge networks allocate IPs unpredictably and risk port collisions. Declare explicit topology so routing is reproducible.

# docker-compose.yml
networks:
  dev_overlay:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16
services:
  cache:
    image: redis:7-alpine
    networks:
      dev_overlay:
        aliases:
          - cache.internal
  api:
    image: app/api:latest
    networks:
      - dev_overlay

  1. Declare custom bridge networks with explicit IPAM subnets.

  2. Prefer DNS aliases over hardcoded IPs for inter-service calls.

  3. Validate cross-service resolution from an ephemeral debug container:

    #!/usr/bin/env bash
set -euo pipefail
docker run --rm --network "$(docker compose ps --format '{{.Name}}' | head -1 | sed 's/-.*//')_dev_overlay" \
  nicolaka/netshoot dig +short cache.internal

Custom-TLD resolution beyond container names is covered in configuring local DNS for microservice routing.

Resource Constraints and Local Performance Tuning

Unconstrained containers starve the host and hide production bottlenecks. Set quotas so throttling surfaces early.

# docker-compose.yml
services:
  worker:
    image: app/worker:latest
    deploy:
      resources:
        limits:
          cpus: "1.5"
          memory: 2G
  api:
    image: app/api:latest
    tmpfs:
      - /tmp:size=512m

  1. Enforce per-service CPU/memory limits to simulate production cgroup quotas.

  2. Use tmpfs for ephemeral logs and build scratch to bypass disk I/O.

  3. Add BuildKit cache mounts in Dockerfiles for iterative dependency resolution.

  4. Profile and watch for OOM kills:

    #!/usr/bin/env bash
set -euo pipefail
docker stats --no-stream --format 'table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}'
dmesg 2>/dev/null | grep -i 'oom' || echo "no OOM kills"

For file-sync latency that compounds slow dev loops, review volume mounting and hot-reload optimization. To trim the stack to only the services a task needs, see Compose profiles and targeted environments.

Automated Teardown and State Reset

Environment rot accumulates silently. Automate teardown so a fresh clone reproduces zero state.

# Makefile
.PHONY: reset
reset:
	docker compose down -v --remove-orphans
	docker system prune -f
	docker compose build --no-cache
  1. Add pre-push hooks that run compose down -v before state-altering operations.
  2. Provide Makefile targets to purge named volumes, dangling images, and orphaned networks.
  3. Validate zero-state reproducibility by cloning into a fresh directory and running make reset && docker compose up -d --wait.
  4. Add a CI step that runs make reset and asserts exit code 0.

Balance clean-state guarantees against cache loss with the tactics in optimizing Docker Compose for fast local rebuilds.

macOS (Docker Desktop): Healthcheck probes route through a Linux VM, adding ~200ms versus native Linux; widen interval/start_period accordingly. Prefer :cached source mounts. WSL2: Enable systemd in wsl.conf so pg_isready finds its socket, and run wsl --shutdown before docker system prune if a volume lock hangs. Keep the repo on the Linux filesystem to avoid 9p I/O penalties during bulk seeding. Apple Silicon (ARM64): Pull architecture-specific healthcheck binaries or use CMD-SHELL wrappers to avoid exec format error; only set platform: linux/amd64 for images lacking an arm64 manifest.

Rollback / recovery

If a stack wedges or volumes hold stale state, tear everything down, prune the project network, and rebuild from the committed configuration:

#!/usr/bin/env bash
set -euo pipefail
docker compose down -v --remove-orphans
docker network prune -f
git checkout HEAD -- docker-compose.yml
docker compose up -d --wait