Measuring DORA in practice

Scope: the module overview defines the five DORA metrics and the throughput/stability split. This page goes one layer deeper — how you'd actually instrument the original four in a real pipeline, using this repo's own live collectors as the worked example, and the traps that make an instrumented metric lie.

TL;DR — in 30 seconds:

• No DORA number is directly observable — every real metric needs a proxy, and the gap between proxy and concept is most of the instrumentation work.
• This repo's four collectors map onto one timeline: commit → merge (Lead Time) → deploy (Deployment Frequency) → incident (Change Failure Rate) → recovery (Time to Restore).
• Three traps recur: treating the proxy as ground truth, a window mismatched to the signal's noise, and optimizing the collector's number instead of the underlying behavior.

1. A metric needs a proxy before it needs a dashboard

Ask ten engineers for their team's "lead time" and you'll get ten different numbers pulled from ten different tools — because none of the four keys are directly observable. There's no API that returns "lead time." Every real implementation picks a proxy: a concrete, queryable signal that stands in for the concept. Picking a good proxy, and being honest about where it diverges from the concept it approximates, is most of the instrumentation work.

This repo's Health Dashboard runs one collector per metric, each declaring its proxy explicitly:

Metric	Proxy this repo uses	Window	"Good" looks like
Deployment Frequency	Count of PRs merged to main	trailing 7 days	on-demand, several/week+
Lead Time for Changes	Median hours from PR open → merge	trailing 14 days	under a day
Change Failure Rate	% of merged PRs whose title/label marks them revert or rollback	trailing 30 days	low double digits or better
Time to Restore Service	Median hours from open → close for incident-labeled issues	trailing 30 days	under a day

Each proxy is a stand-in, not the real thing: "PRs merged to main" approximates "deployments" because merging is the release trigger here — a repo with a separate deploy step would need a different proxy entirely. That gap between proxy and concept is exactly what you have to be able to name for any DORA number you're handed. The "Good looks like" column is a loose read on the elite/high tier bands from the official DORA guide — useful as a compass, not a target to game.

2. Mapping a proxy to the timeline it actually measures

Each metric answers a question about a different segment of the same underlying path: a change is written, it merges, it ships, and — sometimes — it breaks and gets recovered from.

flowchart LR
    C["Commit / PR opened"] -->|"Lead Time for Changes"| M["PR merged to main"]
    M -->|"Deployment Frequency<br/>(how often M happens)"| D["Deployed to production"]
    D -->|"usually fine"| OK["Running normally"]
    D -->|"sometimes"| F["Incident / failure"]
    F -->|"Time to Restore Service"| R["Recovered"]
    M -.->|"Change Failure Rate =<br/>% of M's that lead to F"| F

Reading the diagram: Lead Time and Deployment Frequency describe the left half of the path (how fast and how often code reaches production); Change Failure Rate and Time to Restore describe what happens when the right half goes wrong. That's the throughput/stability split from the module overview, now anchored to where on the timeline each metric actually samples from.

3. What to avoid when you build the instrumentation

Three failure modes recur whenever a team wires up its own collectors, and each one shows up in this repo's collector configs as a deliberate mitigation:

• Treating the proxy as ground truth. This repo's collectors are explicitly tagged directional-v1 in their decision rules — the revert/rollback label on Change Failure Rate is a proxy for "shipped a change that needed a fix," not a certified defect count. Present a DORA number without its proxy caveat and a reader will over-trust it.
• A window mismatched to the signal's noise. Deployment Frequency uses a short 7-day window (it's a frequent event); Change Failure Rate and Time to Restore use 30 days (failures and incidents are rarer, so a short window would swing wildly on one data point). Choosing a window is a real design decision, not an afterthought.
• Optimizing the collector's number instead of the underlying behavior — the same "not a tradeoff" trap from the module overview, now at the instrumentation layer. A team that starts merging tiny no-op PRs to inflate Deployment Frequency has made the proxy go up while making nothing actually better.

4. How this connects

• This is the concrete form of the module overview's five metrics — the same four keys, now expressed as running collectors with explicit proxies, windows, and thresholds instead of abstract definitions.
• The capabilities page (next) is the other half. Measuring the four keys tells you where delivery is weak; the research-backed capabilities are why elite teams score well on all four at once.

Common gotchas

• Quoting a DORA number without its proxy caveat. Fix: always be ready to say what the proxy actually measures — "PRs merged to main" is not literally "deployments."
• Using a window that doesn't match how often the underlying event happens. Fix: frequent events (deploys) get short windows; rare, noisy events (incidents, failures) get longer windows, or one data point swings the number wildly.
• Optimizing the collector's number instead of the real behavior. Fix: watch for gaming — e.g. merging tiny no-op PRs to inflate Deployment Frequency makes the proxy go up while nothing actually improves.
• Assuming every team's proxy is interchangeable. Fix: "merge = release" only holds where merging is the deploy trigger — a repo with a separate deploy step needs a different proxy entirely.

Check yourself — what proxy does this repo use for Deployment Frequency, and why doesn't it directly measure 'deployments'?

The count of PRs merged to main in a trailing 7-day window. It's a proxy, not a direct measure, because merging is the release trigger in this repo — a repo with a separate deploy step would need a different proxy.

Check yourself — why does Change Failure Rate use a 30-day window while Deployment Frequency uses 7 days?

Failures and incidents are rarer than deploys, so a short window would swing wildly on a single data point. A longer window smooths out that noise for a rarer signal; a frequent signal like deploys is stable enough on a shorter one.

You can defend this when you can name the proxy each of the four metrics uses in a real collector, place each metric on the commit→deploy→incident timeline, and explain at least one way an instrumented metric can be gamed or can silently mislead if its proxy and window aren't made explicit.