Optimizing Docker Compose for Fast Local Rebuilds

Eliminate Docker layer cache invalidation and slow rebuilds. Configure BuildKit cache mounts and align local rebuild cycles with CI for sub-second feedback loops.

A one-line source edit triggers a full multi-minute docker compose build because a COPY . . placed before dependency installation invalidates every downstream layer. This page eliminates the cache-invalidation cascade with correct layer ordering and BuildKit cache mounts; it extends multi-service orchestration with Compose within the broader containerized local environment patterns.

Diagnostic

Run a plain-progress build and look for steps that are NOT CACHED after a trivial change:

#!/usr/bin/env bash
set -euo pipefail
DOCKER_BUILDKIT=1 docker compose build --progress=plain 2>&1 \
  | grep -E '=> \[|CACHED'
docker system df -v | grep 'Build Cache'

Expected BAD output — the COPY . . and everything after it rebuild on a source-only change:

 => CACHED [1/5] FROM docker.io/library/node:20.11.0-alpine@sha256:abc123...
 => CACHED [2/5] WORKDIR /app
 => CACHED [3/5] COPY package*.json ./
 => [4/5] RUN npm ci --prefer-offline
 => [5/5] COPY . .
Build Cache: 12.4GB (used: 8.1GB, reclaimable: 4.3GB)

Confirm the heavy layer ordering with docker history:

#!/usr/bin/env bash
set -euo pipefail
docker history --no-trunc "$(docker compose images -q app)" \
  | awk '{print $1, $2, $3}' | head -n 6

A multi-hundred-megabyte layer created by COPY before npm ci is the smoking gun.

Root Cause

Docker caches layers top to bottom and invalidates everything after the first changed layer. When COPY . . (which changes on any source edit) sits before npm ci/pip install, every dependency layer downstream is discarded and re-run on each edit. Context bloat compounds it: without a tight .dockerignore, the build context ships node_modules, .git, and build artifacts, inflating the COPY layer and the cache-miss cost. Floating base tags add a third source of silent invalidation when upstream republishes the tag.

Resolution

  1. Order the Dockerfile dependency-first — copy manifests, install, then copy source:

    # Dockerfile
FROM node:20.11.0-alpine@sha256:8f31d000000000000000000000000000000000000000000000000000000000ab
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm npm ci --prefer-offline
COPY . .
CMD ["node", "server.js"]

  2. Tighten .dockerignore to drop high-churn paths from the context:

    node_modules
.git
dist
*.log
.env

  3. Add a persistent build cache to the Compose build config:

    # docker-compose.yml
services:
  app:
    build:
      context: .
      cache_from:
        - type=local,src=/tmp/.buildx-cache
      cache_to:
        - type=local,dest=/tmp/.buildx-cache,mode=max

  4. Enable live sync so source edits sync without a rebuild, keeping rebuilds for dependency changes only:

    # docker-compose.yml
services:
  app:
    develop:
      watch:
        - path: ./src
          target: /app/src
          action: sync
        - path: ./package.json
          action: rebuild

    Run with docker compose watch. If edits sync but the process never reloads, see fixing hot-reload not triggering on file changes.

Expected Output

After reordering, a source-only edit leaves the dependency layer cached and rebuilds only the final COPY:

 => CACHED [3/5] COPY package*.json ./
 => CACHED [4/5] RUN npm ci --prefer-offline
 => [5/5] COPY . .
 => exporting to image

The dependency install line reads CACHED, and total build time drops from minutes to seconds.

Prevention

  • Lint the Dockerfile in a pre-commit hook to block inefficient COPY ordering: hadolint Dockerfile --ignore DL3008.
  • Pin base images to SHA digests so upstream tag republishes never silently bust the cache.
  • Pre-populate the shared cache for new clones and align it with CI, building on the cache-export tactics in environment sync and CI parity.

macOS (Docker Desktop): Use :cached source mounts and let compose watch sync rather than relying on VirtioFS to propagate every write; the build cache lives in the VM, so prune with docker builder prune, not host rm. WSL2: Keep the repo and /tmp/.buildx-cache on the Linux filesystem (~/, not /mnt/c) or cache writes crawl through 9p. Apple Silicon (ARM64): A cache built for amd64 will not satisfy an arm64 build — keep one cache per platform, or build --platform linux/arm64 consistently.

Rollback

#!/usr/bin/env bash
set -euo pipefail
docker compose down --volumes --remove-orphans
rm -rf /tmp/.buildx-cache
git checkout HEAD -- Dockerfile docker-compose.yml
docker compose build --no-cache