Optimizing Docker Compose for Fast Local Rebuilds

A one-line source edit triggers a full multi-minute docker compose build because a COPY . . placed before dependency installation invalidates every downstream layer. This page eliminates the cache-invalidation cascade with correct layer ordering and BuildKit cache mounts; it extends multi-service orchestration with Compose within the broader containerized local environment patterns.

Diagnostic

The symptom is a rebuild time that does not match the size of the change: you touch a single component file and the terminal sits on RUN npm ci or RUN pip install for minutes. Before changing anything, confirm which layers are actually re-running. BuildKit annotates each step as CACHED or blank; a blank step below the first changed layer is the cost you are trying to remove.

Run a plain-progress build and look for steps that are NOT CACHED after a trivial change:

#!/usr/bin/env bash
set -euo pipefail
DOCKER_BUILDKIT=1 docker compose build --progress=plain 2>&1 \
  | grep -E '=> \[|CACHED'
docker system df -v | grep 'Build Cache'

Expected BAD output — the COPY . . and everything after it rebuild on a source-only change:

 => CACHED [1/5] FROM docker.io/library/node:20.11.0-alpine@sha256:abc123...
 => CACHED [2/5] WORKDIR /app
 => CACHED [3/5] COPY package*.json ./
 => [4/5] RUN npm ci --prefer-offline
 => [5/5] COPY . .
Build Cache: 12.4GB (used: 8.1GB, reclaimable: 4.3GB)

The [4/5] RUN npm ci line has no CACHED prefix even though no dependency changed — that is the wasted work. Confirm the heavy layer ordering with docker history, which lists layers newest-first with the bytes each one added:

#!/usr/bin/env bash
set -euo pipefail
docker history --no-trunc "$(docker compose images -q app)" \
  | awk '{print $1, $2, $3}' | head -n 6

A multi-hundred-megabyte layer created by COPY before npm ci is the smoking gun. If the largest layer sits below the dependency install in build order, every edit to your source discards that install and re-fetches the entire dependency tree.

To put a number on the waste, time two consecutive builds with only a whitespace change between them. A healthy setup shows the second build finishing in seconds because the install layer is reused; a broken one repeats the full install both times:

#!/usr/bin/env bash
set -euo pipefail
touch src/index.js
time docker compose build app >/dev/null 2>&1

If that real time is measured in minutes rather than seconds after a no-op edit, the ordering problem is confirmed and the resolution below applies directly.

Root Cause

Docker builds are content-addressed and evaluated top to bottom. Each instruction produces a layer keyed by the instruction text plus the checksum of its inputs; the moment one layer's key changes, the cache is invalidated for that layer and every instruction below it. This is deliberate — a later step may depend on an earlier one — but it makes ordering the single most important performance decision in a Dockerfile.

The failure mode here is placing high-churn inputs above low-churn work. COPY . . hashes the entire application source, so it changes on every edit. When it sits before npm ci or pip install, each edit changes the COPY layer's key, which invalidates the dependency install below it, which re-downloads and recompiles packages that did not change. The fix is to invert the frequency: copy the manifests (which change rarely) and install first, then copy the fast-changing source last.

Two secondary causes compound the cascade. First, context bloat: without a tight .dockerignore, the build context ships node_modules, .git, and build artifacts to the daemon, inflating the COPY layer, slowing the context upload, and increasing the probability of a spurious cache miss. A large context also makes the checksum BuildKit computes for COPY . . slower to calculate, so the daemon spends time hashing files that will never affect the image. Second, floating base tags: a bare node:20-alpine tag can be silently republished upstream, changing the FROM layer key and invalidating the entire image even when your code is untouched. Pinning to an immutable @sha256: digest removes that source of nondeterminism and makes the cache reproducible across machines.

Resolution

1. Order the Dockerfile dependency-first — copy manifests, install, then copy source. The --mount=type=cache line keeps the package manager's download cache warm across builds even when the install layer itself must re-run:

   # Dockerfile
   FROM node:20.11.0-alpine@sha256:8f31d000000000000000000000000000000000000000000000000000000000ab
   WORKDIR /app
   COPY package*.json ./
   RUN --mount=type=cache,target=/root/.npm npm ci --prefer-offline
   COPY . .
   CMD ["node", "server.js"]

2. Tighten .dockerignore to drop high-churn paths from the context so the final COPY layer stays small and deterministic:

   node_modules
   .git
   dist
   *.log
   .env

3. Add a persistent build cache to the Compose build config so a fresh checkout or a teammate can import the same layer cache instead of building cold:

   # docker-compose.yml
   services:
     app:
       build:
         context: .
         cache_from:
           - type=local,src=/tmp/.buildx-cache
         cache_to:
           - type=local,dest=/tmp/.buildx-cache,mode=max

4. Enable live sync so source edits sync into the running container without a rebuild at all, reserving rebuilds for the rare dependency change:

   # docker-compose.yml
   services:
     app:
       develop:
         watch:
           - path: ./src
             target: /app/src
             action: sync
           - path: ./package.json
             action: rebuild

   Run with docker compose watch. The sync action copies changed files straight into the container filesystem; only a change to package.json triggers the rebuild action, which re-runs the (now correctly ordered) build. If edits sync but the process never reloads, see fixing hot-reload not triggering on file changes.

Expected Output

After reordering, a source-only edit leaves the dependency layer cached and rebuilds only the final COPY:

 => CACHED [3/5] COPY package*.json ./
 => CACHED [4/5] RUN npm ci --prefer-offline
 => [5/5] COPY . .
 => exporting to image

The dependency install line reads CACHED, and total build time drops from minutes to seconds. Once docker compose watch is running, most edits never reach the build path at all: the file syncs into the container in well under a second and your process reloader picks it up. The measurable effect is a collapse across the whole feedback ladder — cold build, warm rebuild, and hot sync each an order of magnitude apart.

There are three distinct latencies worth tracking separately, because they respond to different fixes. The cold build (empty cache, first checkout) is bounded by network and CPU and is improved mainly by a shared cache import. The warm rebuild (a real dependency change) is bounded by the install step and is improved by ordering plus the download cache mount. The hot sync (an ordinary source edit) should not touch the build path at all once watch is configured, so it is bounded only by filesystem propagation. Optimizing the wrong one — for example chasing cold-build speed when your actual pain is the edit loop — spends effort where it does not help.

Prevention

A correct Dockerfile drifts back to a broken one the first time someone reorders a COPY in a hurry, so encode the invariant rather than relying on memory.

• Lint the Dockerfile in a pre-commit hook to block inefficient COPY ordering and unpinned base images before they merge: hadolint Dockerfile --ignore DL3008. hadolint flags a COPY of the full context above a package install, so a regression fails locally instead of surfacing as slow CI later.
• Pin base images to SHA digests so an upstream tag republish never silently busts the cache. A digest is content-addressed: node:20.11.0-alpine@sha256:... resolves to exactly one image forever, which makes every teammate's FROM layer identical and cacheable.
• Pre-populate and share the layer cache. Export the cache from CI with cache_to and import it on fresh clones with cache_from, so a new hire's first build imports warm layers instead of compiling from scratch. This builds on the cache-export tactics in environment sync and CI parity, and keeps the local and CI caches aligned so a green local build predicts a green pipeline.

The decision that governs day-to-day speed is simply rebuild or sync — and once the Dockerfile is ordered correctly, that choice is automatic. Treat the ordered Dockerfile, the pinned digest, and the shared cache as one unit: each one alone leaves a gap, but together they make a fresh clone and a hundredth edit equally fast for every engineer on the team.

Platform caveats

macOS (Docker Desktop): Use :cached source mounts and let compose watch sync rather than relying on VirtioFS to propagate every write; the build cache lives in the VM, so prune with docker builder prune, not host rm. WSL2: Keep the repo and /tmp/.buildx-cache on the Linux filesystem (~/, not /mnt/c) or cache writes crawl through the 9p bridge and negate the layer savings. Apple Silicon (ARM64): A cache built for amd64 will not satisfy an arm64 build — the layer keys embed the platform, so keep one cache per platform, or build --platform linux/arm64 consistently.

Rollback

#!/usr/bin/env bash
set -euo pipefail
docker compose down --volumes --remove-orphans
rm -rf /tmp/.buildx-cache
git checkout HEAD -- Dockerfile docker-compose.yml
docker compose build --no-cache

Frequently Asked Questions

Related

• Enforce startup order and clean teardown
• Fix hot-reload not triggering on file changes
• Reproduce CI-only failures locally with act