Fixing Docker Layer Cache Invalidation on Every Build

Every docker compose build re-runs RUN npm ci (or pip install, or go mod download) even when you only touched a single application file, because a COPY . . placed above the dependency install rehashes on every edit and invalidates the layer beneath it. This guide shows how to prove that the copy layer is the culprit and how to reorder the Dockerfile so the dependency layer stays CACHED; it is part of Docker build cache optimization within the broader containerized local environment patterns.

Diagnostic

The tell-tale symptom is a rebuild whose duration is independent of the change: you edit one line, run the build, and the terminal parks on the dependency install step for the same minute-plus it took on a cold build. Before touching the Dockerfile, prove exactly which instruction lost its cache. BuildKit prefixes each already-cached step with CACHED; the first step without that prefix is where the cache broke, and everything below it re-runs regardless of whether it needed to.

Make a no-op change and run a plain-progress build so every step and its cache state print in order:

#!/usr/bin/env bash
set -euo pipefail
touch src/app.js
DOCKER_BUILDKIT=1 docker compose build --progress=plain app 2>&1 \
  | grep -E '=> (CACHED )?\['

The --progress=plain flag disables the collapsing TTY renderer so the full step list survives in the pipe. Expected BAD output — the manifest copy stays cached, but the install and everything after it rebuild on a source-only edit:

 => CACHED [1/6] FROM docker.io/library/node:20.11.1-alpine@sha256:8f31...
 => CACHED [2/6] WORKDIR /app
 => [3/6] COPY . .
 => [4/6] RUN npm ci --prefer-offline
 => [5/6] RUN npm run build
 => [6/6] EXPOSE 3000

The [3/6] COPY . . line has no CACHED prefix — that is the first miss, and [4/6] RUN npm ci inherits it. Docker did not decide the install was stale on its own merits; it re-ran because the layer above it changed and cache validity is strictly positional. To confirm the copy layer really carries your whole source tree, inspect the image history, which lists layers newest-first with the bytes each one added:

#!/usr/bin/env bash
set -euo pipefail
img="$(docker compose images -q app)"
docker history --no-trunc --format '{{.CreatedBy}}\t{{.Size}}' "$img" | head -n 8

If the largest layer is a COPY that sits below npm ci in build order (that is, earlier in the docker history listing, because history prints newest first), the ordering is inverted and every edit discards a heavy install. Put a number on the waste by timing two back-to-back builds separated by nothing but a whitespace edit:

#!/usr/bin/env bash
set -euo pipefail
docker compose build app >/dev/null 2>&1
printf '\n// touch %s\n' "$(date +%s)" >> src/app.js
time docker compose build app >/dev/null 2>&1

A healthy setup finishes the second build in seconds because the install layer is reused; a broken one repeats the full install and reports a real time in minutes. That gap is the entire cost this guide removes.

Root cause

A Docker image is a stack of content-addressed layers, and BuildKit evaluates instructions strictly top to bottom. Each instruction produces a layer whose cache key is derived from the instruction text plus a digest of its inputs. For a RUN, the inputs are the command string and the parent layer's identity; for a COPY or ADD, BuildKit additionally hashes the contents and metadata of every file being copied. When a key matches a previously built layer, that layer is reused; the instant one key differs, that layer and every layer below it are rebuilt, because a later step can legitimately depend on an earlier one and the daemon has no way to prove otherwise.

COPY . . hashes your entire application source, so its key changes on literally every edit. Placing it above RUN npm ci means each edit changes the copy layer's key, which invalidates the install below it, which re-downloads and recompiles dependencies that did not change. The order encodes an assumption — "the install depends on the copied source" — that is false: the install depends only on the manifests. The fix is to match layer order to change frequency. Copy the low-churn manifests (package.json, package-lock.json) first and install against them, then copy the high-churn source last. Because the manifest layer's key changes only when a dependency actually changes, the install stays CACHED across ordinary source edits.

Two factors quietly widen the blast radius. Context bloat is the first: without a tight .dockerignore, BuildKit ships node_modules, .git, and build artifacts to the daemon and folds them into the COPY hash, so an unrelated file (a fresh .git object, a rotated log) can flip the copy key even when your tracked source is unchanged. A stray edit inside node_modules should never cost a rebuild, yet with no ignore file it does. The second is a floating base tag: a bare node:20-alpine can be republished upstream, changing the FROM key and cascading a full rebuild through an otherwise untouched Dockerfile. Pinning to an immutable @sha256: digest removes that nondeterminism and makes the cache reproducible from one machine to the next.

Resolution

1. Reorder the Dockerfile so manifests and install sit above the source copy. The --mount=type=cache line additionally keeps the package manager's download cache warm across builds, so even a legitimate install re-run fetches locally rather than over the network:

   # Dockerfile
   FROM node:20.11.1-alpine@sha256:8f31d0000000000000000000000000000000000000000000000000000000ab12
   WORKDIR /app
   COPY package.json package-lock.json ./
   RUN --mount=type=cache,target=/root/.npm npm ci --prefer-offline
   COPY . .
   RUN npm run build
   EXPOSE 3000
   CMD ["node", "dist/server.js"]

2. Add a .dockerignore next to the Dockerfile so high-churn and secret paths never enter the build context or the COPY . . hash:

   .git
   node_modules
   dist
   coverage
   *.log
   .env
   .env.*

3. Point Compose at the Dockerfile and enable BuildKit so the reordered build and the cache mount take effect. BuildKit is the default in current Compose, but pin it explicitly for older daemons:

   # docker-compose.yml
   services:
     app:
       build:
         context: .
         dockerfile: Dockerfile
       environment:
         - NODE_ENV=development
       ports:
         - "3000:3000"

   Run the build with BuildKit forced on so the cache mount is honoured:

   #!/usr/bin/env bash
   set -euo pipefail
   export COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1
   docker compose build app

4. Verify the install layer now survives a source edit. Touch a source file, rebuild, and confirm the install prints CACHED. If it does not, the manifest hash changed — usually because the lockfile was regenerated or .dockerignore is still leaking node_modules into the context:

   #!/usr/bin/env bash
   set -euo pipefail
   touch src/app.js
   DOCKER_BUILDKIT=1 docker compose build --progress=plain app 2>&1 \
     | grep -E 'RUN npm ci'

Expected output

After reordering, a source-only edit leaves the base, manifest copy, and install layers cached and rebuilds only the layers that genuinely depend on the source:

 => CACHED [2/6] WORKDIR /app
 => CACHED [3/6] COPY package.json package-lock.json ./
 => CACHED [4/6] RUN npm ci --prefer-offline
 => [5/6] COPY . .
 => [6/6] RUN npm run build
 => exporting to image

The critical line is CACHED [4/6] RUN npm ci — the install is reused, so total build time collapses from minutes to the few seconds it takes to copy the source and re-run the compile step. Run the two-build timing loop from the Diagnostic again and the second real time should now read single-digit seconds. If your compile step (npm run build, webpack, tsc) still dominates that residual time, it is because it too depends on the full source and legitimately re-runs; that is expected and separate from the cache-invalidation bug this guide fixes.

Prevention

A correctly ordered Dockerfile silently degrades the first time someone drops a COPY . . near the top in a hurry, so encode the invariant instead of trusting review to catch it.

• Lint the Dockerfile in a pre-commit hook. hadolint rule DL3059 and its ordering heuristics flag a full-context copy above a package install and an unpinned FROM, failing the commit locally rather than surfacing as a slow pipeline later: hadolint Dockerfile. Wire it into .pre-commit-config.yaml so every contributor runs the same check.
• Pin base images to a SHA digest. A digest is content-addressed, so node:20.11.1-alpine@sha256:... resolves to exactly one image for every teammate and CI runner, keeping the FROM layer key — and therefore the whole cache — stable. Keep the human-readable tag in a comment for legibility.
• Assert the cache in CI. Build twice in the pipeline with an intervening touch of a source file and fail the job if the install step is not reported as cached. This is the same drift-detection instinct behind enforcing CI and local parity: a check that the layer order that is fast locally is also fast in the pipeline.

If your dependency install still re-runs after a rebuild despite correct ordering, the usual culprit is a build tool that rewrites the lockfile mid-build (a bare npm install instead of npm ci), or secrets leaking into the context and perturbing the copy hash — the latter overlaps with debugging env-variable leakage in multi-stage builds. Once ordering, a pinned base, and the two build tools (ci not install) are in place, the cache is deterministic and a hundredth edit costs the same as the second.

Platform caveats

macOS (Docker Desktop): The build cache lives inside the Linux VM, so reclaim it with docker builder prune, never a host rm. A large, unignored context is slower to hash through the VirtioFS boundary, so a tight .dockerignore matters more here than on native Linux.

WSL2: Keep the repository on the Linux filesystem (~/project, not /mnt/c/project). Building from a /mnt/c path forces BuildKit to hash the copy context across the 9p bridge, which is slow enough to look like a cache miss even when the layers are actually reused.

Apple Silicon (ARM64): Layer cache keys embed the target platform, so a cache warmed for linux/amd64 will not satisfy an arm64 build and vice versa. Build consistently for one platform locally, or keep a separate cache per --platform value; mixing them silently rebuilds from scratch.

Rollback

If the reorder introduces a regression (for example a build step that assumed the full source was already present), restore the previous Dockerfile and rebuild cleanly:

#!/usr/bin/env bash
set -euo pipefail
git checkout HEAD -- Dockerfile .dockerignore docker-compose.yml
docker compose build --no-cache app

Frequently Asked Questions

Related

• Order the Dockerfile and add cache mounts for fast rebuilds
• Trace env-variable leakage across multi-stage builds
• Fix hot-reload that stops triggering on file changes