Compose Profiles and Targeted Environments

A monolithic docker compose up that boots every database, worker, message broker, and observability sidecar wastes RAM and slows the dev loop when an engineer only needs the frontend. Docker Compose profiles let one manifest declare every service while launching a named subset per workflow — frontend-only, full-stack, or with-observability — so you stop maintaining divergent docker-compose.*.yml files. This pattern sits inside the broader Containerized Local Environments & Docker Compose Patterns baseline and pairs naturally with the service startup ordering and healthcheck work elsewhere in this section.

The alternative most teams reach for — a base docker-compose.yml plus docker-compose.override.yml, docker-compose.observability.yml, and a per-engineer docker-compose.local.yml glued together with -f flags — drifts almost immediately. Each file duplicates image tags, port mappings, and environment blocks, and a change to the db service has to be mirrored in four places before CI and every laptop agree. Profiles collapse that fan-out back into a single source of truth: every service is declared once, and a tag decides whether a given invocation starts it. This guide covers how the resolver decides which services run, how to name profiles after real workflows, how to make the default ergonomic, and how to catch the one failure mode profiles introduce — a dependency that silently sits in an unselected profile.

Prerequisites

• Docker Compose v2 (run docker compose version; profiles require the v2 plugin, not the legacy Python docker-compose binary, which ignores the profiles: key entirely and starts every service).
• A single docker-compose.yml that already defines all services. Profiles tag existing services; they do not create new ones, so start from a working full manifest and add tags to it.
• Healthchecks on stateful dependencies so subset launches still respect readiness — an untagged app that depends on a db must wait for pg_isready, or a fast frontend-only boot races the database. See the companion guide on service startup order and healthcheck races.
• Ports driven through .env defaults so subsets that overlap on a port do not collide. If two profiles can both bind 3000, resolve it the same way you would fix a 'port is already allocated' error — parameterize the host port.

Confirm the plugin is the v2 one before you rely on any of this. The legacy binary prints a version string that starts with docker-compose version 1.x; the plugin prints Docker Compose version v2.x. The profiles: key is a hard no-op on v1, and the failure is silent — every service starts as if the tags were not there.

#!/usr/bin/env bash
set -euo pipefail
# Fail loudly if the environment still has legacy compose on PATH
if docker compose version | grep -qE 'v2\.'; then
  echo "Compose v2 plugin present — profiles supported"
else
  echo "ERROR: profiles require Compose v2; found:" >&2
  docker compose version >&2
  exit 1
fi

Tagging services with profiles

A service without a profiles: key always runs. A service with one runs only when at least one of its profiles is requested on the command line or through COMPOSE_PROFILES. Use this asymmetry to keep core services (app, db) always-on while gating optional ones behind an explicit request. The mental model is an allowlist: an untagged service is unconditionally in every launch, and a tagged service opts out of the default launch until its profile is named.

# docker-compose.yml
services:
  app:
    build: .
    ports:
      - "${APP_PORT:-3000}:3000"
    depends_on:
      db:
        condition: service_healthy
  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_PASSWORD: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 3s
      timeout: 5s
      retries: 5
  worker:
    build: .
    command: ["npm", "run", "worker"]
    profiles: ["full-stack"]
    depends_on:
      db:
        condition: service_healthy
  grafana:
    image: grafana/grafana:11.1.0
    profiles: ["observability"]
    ports:
      - "${GRAFANA_PORT:-3001}:3000"

Steps:

1. Leave app and db untagged so every workflow gets them. These are the services no engineer can work without; making them unconditional means a bare docker compose up still produces a usable environment.
2. Tag worker with full-stack so it only starts when that profile is selected. A frontend engineer editing React components has no reason to run a background job processor.
3. Tag grafana with observability so the metrics stack is opt-in. Dashboards cost RAM and add a slow-starting container that most feature work never touches.
4. Verify which services a profile resolves to before launching, so you are never surprised by what a flag actually starts:

#!/usr/bin/env bash
# show what `full-stack` would start
set -euo pipefail
docker compose --profile full-stack config --services

The output lists app, db, and worker — the two untagged services plus the one that matched the requested profile. grafana is absent because its observability profile was not requested. That config --services command is the fastest way to reason about a manifest you did not write: it resolves the entire tag graph and prints the concrete set, so you review a flat list instead of tracing profiles: keys by eye.

One anti-pattern to avoid while tagging: do not push a service that everyone needs behind a profile "to be safe." Every profiled service is one more flag a teammate can forget, and one more candidate for the silent-omission failure covered below. The default launch should be genuinely usable on its own. A good test is to ask, for each tagged service, "would a new hire filing their first bug notice this is missing?" — if the answer is yes, the service probably belongs in the untagged core or in the committed default profile, not gated behind a flag most people never type.

Defining workflow-shaped profiles

Map profiles to how people actually work, not to individual services. A frontend engineer wants app plus a mocked API; a backend engineer wants the full data plane including queue workers and payment stubs; an SRE or platform engineer wants observability bolted on top. When a profile name matches a job-to-be-done, the launch command reads like an intent (--profile frontend) instead of a checklist of container names, and a new hire can pick the right one without understanding the dependency graph.

# docker-compose.yml (profile assignment excerpt)
services:
  mock-api:
    image: mockoon/cli:latest
    profiles: ["frontend"]
    command: ["--data", "/data/mocks.json", "--port", "3100"]
  payments:
    build: ./payments
    profiles: ["full-stack"]
  prometheus:
    image: prom/prometheus:v2.53.0
    profiles: ["observability"]

Steps:

1. Name profiles after intent (frontend, full-stack, observability), never after implementation detail like postgres-and-redis. The name is documentation; it should tell a reader which task the profile serves.
2. Allow a service to carry multiple profiles when several workflows need it: profiles: ["full-stack", "observability"] starts the service if either profile is requested. Profiles are additive, not mutually exclusive — a service belongs to the union of the profiles that name it.
3. Combine profiles at launch when an engineer needs more than one slice (covered in depth in running a subset of services with Compose profiles).

#!/usr/bin/env bash
set -euo pipefail
docker compose --profile frontend --profile observability up -d --wait

Because profiles are additive, avoid the temptation to build a minimal profile that tries to subtract services — Compose has no exclusion operator. If you find yourself wanting to turn a service off within a profile, the right move is to leave that service untagged only if it is truly universal, and otherwise give it its own opt-in tag. Keep the set of profile names small and intent-shaped: three or four (frontend, full-stack, observability, maybe e2e for browser tests) covers almost every team. A manifest with a dozen profiles has usually re-encoded the per-service problem that profiles were meant to remove.

How Compose resolves profiles at launch

Knowing the resolution order removes most profile surprises. When you run a command, Compose collects the requested profiles from three sources, unions the always-on services with every service whose tag matches, and then pulls in any additional service that a started service depends_on. That last step is the one that trips people up, so it is worth tracing the full pipeline once.

Steps:

1. Compose reads requested profiles from the --profile flags, the COMPOSE_PROFILES environment variable, and any profiles under a x- extension it has been told to use — all of them union together.
2. It selects every untagged service plus every tagged service matched by at least one requested profile.
3. It walks depends_on edges from the selected set. If a started service depends on a profiled service, that dependency is started too, even if its profile was not requested. This is deliberate — a broken dependency would be worse — but it means a profile boundary is not a hard wall.
4. Anything still unselected after those steps stays down. You can confirm the concrete result at any time with docker compose --profile <name> config --services.

The practical consequence of step 3: do not rely on a profile to keep a heavy service down if an always-on service depends_on it. If app depends on payments, then payments starts on every launch regardless of its full-stack tag. Keep depends_on edges from always-on services pointing only at other always-on services, and let optional services depend inward toward the core, never the reverse.

This asymmetry has a design implication worth stating plainly: the always-on core defines the floor of every launch, so keep it minimal. When you are tempted to add a depends_on from app to a new service, decide first whether that new service is truly universal. If it is optional, invert the relationship — have the optional service depend on app rather than the other way around — so the profile boundary stays meaningful. A core that only depends on db gives you the widest range of cheap targeted launches; a core that transitively depends on half the manifest collapses every profile back into the monolithic up you were trying to escape. Review the dependency direction each time you add a service, not just each time you add a profile.

Making the default selection ergonomic

Typing --profile on every command invites mistakes — someone forgets it, gets a half-started stack, and files a bug against a service that simply was not requested. Pin the team's default with COMPOSE_PROFILES in a committed .env and let individuals override it per shell. A committed default means a bare docker compose up produces the environment most engineers want, and the override path stays available for the minority who need a different slice.

# .env (committed defaults)
COMPOSE_PROFILES=full-stack
APP_PORT=3000
GRAFANA_PORT=3001

#!/usr/bin/env bash
# bin/up.sh — honor COMPOSE_PROFILES, allow per-run override
set -euo pipefail
: "${COMPOSE_PROFILES:=full-stack}"
echo "Starting profiles: ${COMPOSE_PROFILES}"
docker compose up -d --wait

Steps:

1. Commit a sane default in .env so docker compose up "just works" for most engineers. Full-stack is usually the safest default because it fails toward more running than a workflow needs, rather than a missing dependency.
2. Document the override (COMPOSE_PROFILES=frontend docker compose up) in the README so the escape hatch is discoverable. COMPOSE_PROFILES takes a comma-separated list, so COMPOSE_PROFILES=frontend,observability is the environment-variable equivalent of two --profile flags.
3. Wrap the common case in a make up target so newcomers do not need to learn the flag immediately, mirroring the one-command setup goal in reducing setup friction for junior engineers.

A short Makefile turns each profile into a named target, which is easier to remember than flag combinations and gives you a place to add pre-flight checks later:

# Makefile
.PHONY: up front full obs down
up: full
front:
	COMPOSE_PROFILES=frontend docker compose up -d --wait
full:
	COMPOSE_PROFILES=full-stack docker compose up -d --wait
obs:
	COMPOSE_PROFILES=full-stack,observability docker compose up -d --wait
down:
	docker compose --profile frontend --profile full-stack --profile observability down --remove-orphans

Note that make down names every profile explicitly. docker compose down without profiles only removes services that are currently active and untagged plus whichever profiles are in the ambient COMPOSE_PROFILES; a container started under a profile you are no longer requesting can linger. Naming all profiles in the teardown target guarantees a clean slate.

Drift diagnostics

The subtle failure with profiles is silent omission: a dependency lives in a profile the developer did not select, so the app starts but a feature is dead. There is no error — the container simply is not there — and the symptom surfaces later as a connection refused deep in a code path. Detect it before it confuses someone.

#!/usr/bin/env bash
# bin/profile-audit.sh — flag dependencies hidden behind unselected profiles
set -euo pipefail
active=$(docker compose config --services | sort)
echo "== Active services =="
echo "${active}"
echo "== Services declared but NOT active =="
comm -13 <(echo "${active}") <(docker compose --profile frontend --profile full-stack --profile observability config --services | sort)

# A dependency sitting in an unselected profile shows up here:
== Services declared but NOT active ==
payments

The comm -13 invocation prints lines unique to the second input — the full set of services across every profile minus the currently active set — so anything that could run but is not running shows up. Wire this into a make audit target and run it whenever a feature "works on CI but not locally"; the gap is almost always a service parked in a profile the local default does not include.

Cross-check that no always-on service depends_on a profiled one; that combination starts the dependency implicitly and surprises people in the opposite direction — a service you thought was gated turns out to run on every launch. Compose will pull in a profiled service if a started service depends on it, so audit depends_on edges when you add a profile. The following snippet greps the resolved config for services that carry a profile yet are reachable from the default launch:

#!/usr/bin/env bash
# bin/depends-audit.sh — warn when the default launch drags in a profiled service
set -euo pipefail
default=$(docker compose config --services | sort)
profiled=$(docker compose --profile frontend --profile full-stack --profile observability config \
  | grep -A1 'profiles:' >/dev/null 2>&1 && docker compose config --services | sort || true)
# A service present in the default launch that also declares a profile is an implicit pull-in.
for svc in ${default}; do
  if docker compose config | yq ".services.${svc}.profiles // \"\"" 2>/dev/null | grep -q .; then
    echo "WARNING: ${svc} is profiled but started by the default launch (depends_on pull-in)"
  fi
done

If yq is not installed, the same check reads cleanly by eye from docker compose config — scan for any service that has both a profiles: block and an inbound depends_on from an untagged service. The goal is the same either way: no surprise starts, and no surprise omissions.

Measuring the payoff

Profiles are worth the tagging effort because a targeted launch is measurably cheaper. The numbers below are representative of a mid-sized web stack on a developer laptop — a frontend-only launch skips the worker, payments service, and the two-container metrics stack, which is where most of the memory and cold-start time go. Measure your own with docker stats --no-stream after each launch and time docker compose up -d --wait for cold-start; the shape holds even when the absolute figures differ.

The cold-start difference tracks the same curve: a frontend launch reaches --wait readiness before the observability launch has finished pulling and health-checking Prometheus. On a constrained machine the RAM saving is what matters most — a 420 MB frontend stack leaves headroom for the IDE, a browser, and a language server, where a 1.4 GB everything-on launch pushes an 8 GB laptop into swap, at which point the whole machine gets slower, not just Docker.

To capture your own numbers reproducibly, wrap the measurement in a script so every engineer benchmarks the same way. Cold-start time is the wall-clock from an empty state to healthy, so remove any running stack first, then time the launch:

#!/usr/bin/env bash
# bin/bench.sh — cold-start and memory for a given profile set
set -euo pipefail
profiles="${1:-full-stack}"
docker compose --profile frontend --profile full-stack --profile observability down --remove-orphans >/dev/null 2>&1 || true
echo "Cold-start for COMPOSE_PROFILES=${profiles}:"
COMPOSE_PROFILES="${profiles}" bash -c 'time docker compose up -d --wait' 2>&1 | grep real
echo "Resident memory:"
docker stats --no-stream --format 'table {{.Name}}\t{{.MemUsage}}'

Run bin/bench.sh frontend and bin/bench.sh full-stack,observability back to back and the gap is obvious. The point of measuring is not the exact millisecond — it is giving the team a shared, defensible reason to keep the default profile small. Once "the full launch costs a gigabyte and forty seconds" is a number rather than a feeling, nobody argues for booting the metrics stack on every up.

Profiles in CI and end-to-end runs

Profiles are as useful in continuous integration as on a laptop, and for the same reason: a CI job should start only the services its stage exercises. A unit-test job needs app and db; an end-to-end job needs the browser-facing stack plus whatever it asserts against; a contract-test job might need only the mock API. Encode those as profiles and each CI stage requests exactly one, so the pipeline never pays to boot Grafana just to run Jest.

Add an e2e profile for services that only browser tests require — a headless test runner, a seeded fixtures container, or a mail catcher the tests assert against — and keep it out of the developer default so nobody runs it locally by accident.

# docker-compose.yml (CI-facing services)
services:
  playwright:
    image: mcr.microsoft.com/playwright:v1.45.0-jammy
    profiles: ["e2e"]
    depends_on:
      app:
        condition: service_healthy
  mailcatcher:
    image: dockage/mailcatcher:0.9.0
    profiles: ["e2e"]
    ports:
      - "${MAILCATCHER_PORT:-1080}:1080"

A CI step then requests the profile it needs and nothing more. Because COMPOSE_PROFILES is just an environment variable, most CI systems set it once at the job level and every docker compose call in that job inherits it:

#!/usr/bin/env bash
# ci/run-e2e.sh — start only the services end-to-end tests need
set -euo pipefail
export COMPOSE_PROFILES=full-stack,e2e
docker compose up -d --wait
docker compose run --rm playwright npx playwright test
code=$?
docker compose --profile full-stack --profile e2e down --remove-orphans
exit "${code}"

Steps:

1. Set COMPOSE_PROFILES once per CI job so every Compose invocation in that job resolves to the same service set — no repeated --profile flags to drift out of sync.
2. Use --wait so the job blocks until healthchecks pass, turning a flaky "service not ready" race into a deterministic gate before tests run.
3. Tear down with every profile named and --remove-orphans, and preserve the test exit code so a failing suite still fails the job even though teardown runs afterward.
4. Keep the CI-only services (playwright, mailcatcher) behind a profile the developer default never requests, so a local docker compose up stays lean.

This keeps one manifest authoritative for both environments. The service definitions CI runs against are the exact ones on every laptop — the only difference is which profiles each context requests — which is the parity that separate docker-compose.ci.yml files quietly erode. When a CI failure cannot be reproduced locally, the first check is whether the two contexts requested the same profiles, and docker compose config --services answers that in one line on both sides.

Platform caveats

macOS (Docker Desktop): every profiled service still runs inside the shared Linux VM, so a with-observability launch competes for the global RAM slider; raise it before adding Prometheus and Grafana, and expect the memory figures above to sit on top of the VM's own overhead. WSL2: COMPOSE_PROFILES set in Windows PowerShell does not propagate into the distro — export it inside WSL2 (export COMPOSE_PROFILES=full-stack in your shell profile) or put it in the committed .env, which the plugin reads from the project directory regardless of shell. Apple Silicon (ARM64): observability images (Grafana, Prometheus, OpenTelemetry collectors) usually ship arm64 manifests; for any that do not, pin platform: linux/amd64 only on that service rather than the whole stack, so the emulation penalty stays scoped to the one image and does not slow your always-on app and db.

Rollback and recovery

If a profile launch leaves a partial stack — a half-started full-stack that failed on --wait, or orphaned containers from a profile you no longer request — tear down everything it could have started. down without a profile only removes currently active services, so pass the profiles to clean up their containers too, then restore the manifest and env file to their committed state and bring the default stack back up.

#!/usr/bin/env bash
set -euo pipefail
docker compose --profile frontend --profile full-stack --profile observability down --remove-orphans
git checkout -- docker-compose.yml .env 2>/dev/null || true
docker compose up -d --wait

The --remove-orphans flag catches containers whose service definition or profile membership changed since they were started — exactly the state you land in after editing tags. If a named volume also needs resetting (a corrupted local database, say) add -v to the down line, but only when you intend to discard data; without it, down keeps named volumes so your local database survives the teardown.

Frequently Asked Questions

Related

• Run a subset of services with Compose profiles
• Resolve service startup order and healthcheck races
• Fix 'port is already allocated' when stacks overlap
• Reduce setup friction for junior engineers