How to Measure Developer Onboarding Time in Distributed Teams

Onboarding "feels slow" in the APAC office and "feels fine" in the US office, and without timestamps nobody can say whether the difference is the network, the toolchain, or the documentation. This walkthrough instruments setup latency deterministically so distributed teams can compare like for like and isolate where the friction actually lives. It is the field guide to time-to-first-PR metrics within onboarding architecture and friction mapping.

The core problem is that a single wall-clock number — "it took me three hours to get set up" — carries no structure. It cannot be compared between two engineers, it cannot be diffed across a week, and it cannot survive a timezone boundary. What you want instead is a series of attributable samples: each stamped with a trace id, a region, an architecture, and a set of phase boundaries so that the compute cost, the network cost, and the human-wait cost are recorded separately. Once each sample is structured, aggregate percentiles become meaningful and a regional regression stops being an anecdote and becomes a query.

Distributed teams add a second complication on top of the first: the same repository is exercised from wildly different network positions. An engineer twenty milliseconds from the registry mirror and one three hundred milliseconds away run identical scripts and produce durations that differ by an order of magnitude, and neither of them is doing anything wrong. If your measurement does not record where the sample came from, you will average those two populations together and conclude that onboarding is "sometimes slow" — a statement that is true, useless, and impossible to action. The whole design goal of the instrumentation below is to make every number carry enough context that it lands in the right bucket automatically.

Diagnostic

Capture bootstrap duration alongside registry latency to separate the two:

#!/usr/bin/env bash
set -euo pipefail
git log --format='%aI' --reverse | head -n 1
( time ./scripts/bootstrap.sh ) 2>&1 | tee setup.log
curl -s -o /dev/null -w 'registry: %{time_total}s\n' https://registry.npmjs.org

Expected BAD output — bootstrap runs long and registry latency spikes:

2024-03-15T09:12:04+00:00
real  18m42.108s
user  0m12.441s
sys   0m4.209s
registry: 2.842s

A real time over ~15m or registry latency over ~2.5s points at regional egress, not repository misconfiguration. The user and sys totals are the second half of the story: here they sum to under 17 seconds of actual CPU work against nearly nineteen minutes of wall time, which means the process spent almost all of its life blocked on I/O — downloading, waiting on DNS, or stalled on a proxy handshake. When user + sys is a tiny fraction of real, you are measuring the network, not the machine, and no amount of script tuning will move the number.

The curl probe is deliberately minimal — -o /dev/null discards the body and -w 'registry: %{time_total}s\n' prints only the total transfer time — so it adds negligible overhead to the diagnostic and can be run repeatedly to see whether the latency is steady or bursty. Run the block from the same shell that a new hire would use, not from a warmed-up CI runner, because a corporate proxy or a VPN split-tunnel changes the route and the whole point is to reproduce the contributor's actual conditions. If the first git log line — the earliest commit timestamp — is far in the past on a shallow clone, confirm you did not accidentally run against a cached checkout, since a stale working copy skips the clone phase entirely and understates the real cost.

Root Cause

Two distinct problems hide behind one symptom. First, unpinned base images and transitive resolution let node_modules resolve differently per region, especially when a corporate proxy redirects to a slow mirror. Second, raw wall-clock duration conflates network egress with local compute, so a slow ISP route looks identical to a heavy install. There is also a measurement trap underneath both: timestamps captured on hosts whose clocks have drifted — common on WSL2 after hibernate, or on ARM boards without a battery-backed RTC — produce deltas that are simply wrong, and a wrong delta is worse than no delta because it looks authoritative on a dashboard. Without isolating compute from network, and without trustworthy clocks, teams "fix" the wrong layer: they rewrite the bootstrap script when the real cost was a transatlantic registry round-trip, or they chase a phantom regression that was only a desynced clock. Correlating each sample against time-to-first-PR metrics keeps the local-setup number honest against the downstream review-and-merge number.

A useful way to think about it is that onboarding time is a sum of independent phases — clone, dependency install, image pull, service warm-up, and the human gaps between them — and each phase has a different owner. Network egress belongs to infrastructure; dependency resolution belongs to the lockfile; service warm-up belongs to the compose topology. If you only record the total, you have averaged five accountable owners into one unaccountable number. The instrumentation in the next section exists to un-average them, so that when the APAC p95 doubles you can point at the phase that moved instead of relitigating the whole setup.

Resolution

1. Separate network latency from compute with a path trace to the registry:

   #!/usr/bin/env bash
   set -euo pipefail
   mtr -n -r -c 100 registry.npmjs.org | tail -n +2

   High packet loss or TTL variance is an ISP routing problem; route those contributors through a regional mirror.
2. Pin runtimes and install strictly from the lockfile so resolution is identical everywhere:

   #!/usr/bin/env bash
   set -euo pipefail
   export npm_config_cache="$HOME/.npm-cache"
   npm ci --prefer-offline

3. Stamp every run with a trace id so deltas are attributable per contributor and region:

   #!/usr/bin/env bash
   set -euo pipefail
   TRACE_ID=$(uuidgen)
   START=$(date +%s)
   ./scripts/bootstrap.sh
   END=$(date +%s)
   curl -fsS -X POST "$METRICS_ENDPOINT" \
     -H 'Content-Type: application/json' \
     -d "{\"trace_id\":\"$TRACE_ID\",\"duration_s\":$((END-START)),\"region\":\"${AWS_REGION:-unknown}\"}"

4. Record per-phase boundaries, not just a single total, so an aggregator can attribute the regression to the phase that moved:

   #!/usr/bin/env bash
   set -euo pipefail
   TRACE_ID=${TRACE_ID:-$(uuidgen)}
   emit() { printf '{"trace_id":"%s","phase":"%s","t":%s}\n' "$TRACE_ID" "$1" "$(date +%s.%N)"; }
   { emit clone_start;   git submodule update --init --recursive; emit clone_end;
     emit install_start; npm ci --prefer-offline;                 emit install_end;
     emit warmup_start;  docker compose up -d --wait;             emit warmup_end;
   } | tee -a ~/.onboarding/phases.ndjson

The four steps build on each other: step one tells you whether the number is even worth chasing on the machine, step two removes resolution drift as a variable so two regions run the same install, and steps three and four turn each run into a durable, attributable record. Run them in order the first time — there is no point collecting per-phase telemetry from a bootstrap that still resolves a different dependency tree in every region, because the phases you record would not be comparable.

Use a UTC-normalized clock everywhere: date +%s reports seconds since the Unix epoch and is timezone-independent, which is exactly why it survives the trip from Sydney to San Francisco unchanged. The --wait flag on docker compose up blocks until healthchecks pass, so the warmup_end boundary reflects a genuinely ready stack rather than a container that has merely started. Standardizing the whole capture inside your devcontainer configuration standards means every contributor emits the same schema without thinking about it.

Expected Output

After pinning runtimes and routing through a regional mirror, bootstrap lands inside the target window with low registry latency:

real  3m41.902s
registry: 0.214s
{"trace_id":"a1b2c3d4-...","duration_s":221,"region":"ap-southeast-1"}

The per-phase stream shows where the remaining time is spent, which is the artifact your dashboard actually queries:

{"trace_id":"a1b2c3d4-...","phase":"clone_end","t":1710493200.14}
{"trace_id":"a1b2c3d4-...","phase":"install_end","t":1710493331.02}
{"trace_id":"a1b2c3d4-...","phase":"warmup_end","t":1710493421.77}

Subtracting adjacent boundaries gives install at 131 seconds and warm-up at 91 seconds — concrete, ownable numbers instead of a single opaque 221. Store the stream as newline-delimited JSON so it appends cheaply and parses with a one-liner: jq -s 'group_by(.trace_id)' ~/.onboarding/phases.ndjson reassembles every run's phases without a database. When you later aggregate across the team, each record already carries its own trace_id, so a slow warm-up on one contributor's arm64 laptop never contaminates the install percentile computed from everyone else.

The absolute epoch values also let you sanity-check the clock retroactively. If install_end minus install_start is negative, or a phase spans a suspiciously round number of hours, the host's clock stepped mid-run — usually an NTP correction or a laptop resuming from sleep — and that sample must be discarded rather than trusted. Building this rejection rule into the collector is cheaper than explaining a haunted dashboard spike three sprints later.

Prevention

1. Inject the trace id and timing into .devcontainer/postCreateCommand.sh so collection is automatic, not manual.
2. Validate each run against an SLO (< 15m first-run) and alert when regional p95 exceeds 25m.
3. Reject PRs whose commit metadata lacks an ONBOARDING_TRACE_ID, enforcing traceability across regions.

The SLO threshold is a policy choice, not a physical constant, so set it from your own distribution rather than a round number copied from a blog. Pull two weeks of samples, compute the current regional p95, and place the alert threshold a deliberate margin above it — tight enough to catch a genuine regression within a day, loose enough that a single slow coffee-shop wifi run does not page anyone. Because the collection lives in .devcontainer/postCreateCommand.sh, every contributor is measured the same way whether they open the repo in a cloud workspace or on a laptop, which is what makes the regional percentiles comparable in the first place. Treat the ONBOARDING_TRACE_ID gate as the enforcement backstop: without it a contributor can silently skip instrumentation, and one unmeasured region is enough to blind the whole dashboard.

Decide deliberately whether a slow sample is a network problem or a compute problem before anyone opens a ticket — the branch below is the same one your alert routing should encode, so a paged engineer is handed the owning team rather than a mystery.

Platform Caveats

WSL2: keep the repo on the Linux filesystem and confirm systemd-timesyncd is active so timestamps from Windows hosts are trustworthy. Apple Silicon (ARM64): record process.arch with each sample so arm64 laptops are not averaged together with amd64 CI runners. macOS (Docker Desktop): if measuring inside a container, mount /etc/localtime read-only to keep timezone consistent with the host.

Rollback

#!/usr/bin/env bash
set -euo pipefail
rm -f ~/.onboarding/telemetry.json
sed -i.bak '/ONBOARDING_START_TS/d' ~/.zshrc
unset ONBOARDING_TRACE_ID ONBOARDING_START_TS

This removes the local telemetry file, strips the timestamp exporter that the shell profile injected, and clears the two environment variables from the current session so a subsequent run starts clean. It intentionally leaves already-collected samples on the metrics endpoint untouched — rolling back the collector on one laptop should never delete a region's history — so if you need to purge server-side data, do it as a separate, deliberate operation against $METRICS_ENDPOINT. Because the sed edit writes a .bak alongside ~/.zshrc, you can restore the exporter by moving the backup back into place, which makes re-enabling collection a one-line reversal rather than a re-derivation of the original snippet.

Frequently Asked Questions

Related

• Define and report time-to-first-PR boundaries
• Fix time-to-first-PR regressions after a dependency upgrade
• Standardize onboarding capture in a devcontainer
• Automate runtime parity checks against staging