Instrumenting the DORA Four: Lead Time, Deploy Frequency, MTTR and Change-Failure Rate

Your delivery health is already recorded, in full, in two systems you run every day: version control and CI. You do not need a survey to know how you are doing. You need a handful of queries and the discipline to read the answer honestly.

A quick disambiguation first, because the acronym is now overloaded for the regulated firms I work with. This post is about the other DORA: DevOps Research and Assessment, the Google research programme behind the annual State of DevOps reports and the book Accelerate. It has nothing to do with the Digital Operational Resilience Act. What it gives you is four metrics that, taken together, describe how quickly and how safely software changes reach production. The appeal is that all four can be computed from data you already hold, which also means they can be gamed, misdefined and turned into a vanity dashboard faster than almost any other engineering measure. This is how to stand them up so they are worth reading.

The four, in two honest pairs

The four metrics split cleanly into two that measure throughput and two that measure stability, and the split is the whole point. Throughput without stability is recklessness; stability without throughput is paralysis. You read them as a set, never in isolation.

• Deployment frequency — how often you successfully release to production. A throughput measure.
• Lead time for changes — how long a commit takes to reach production. The other throughput measure.
• Change-failure rate — the share of deployments that cause a failure in production. A stability measure.
• Mean time to restore (MTTR) — how long it takes to recover once a failure has occurred. The other stability measure. Recent DORA reports rename this failed-deployment recovery time, but the intent is unchanged.

DORA’s own reports cluster teams into performance bands, and the illustrative shape is worth holding in mind: elite performers restore service in well under an hour and keep change-failure rate in the low single-digit-to-teens percentage range, while lower performers measure recovery in days to weeks. Treat the exact band boundaries as indicative rather than a target to hit; they shift year to year, and chasing a band is itself a way to game the numbers.

The definitions people quietly fudge

Every argument about DORA metrics is really an argument about three definitions. Settle them explicitly, write them down, and apply them consistently, or the numbers mean nothing.

What counts as a deployment. Only a successful release to production counts. A staging deploy is not a deployment for this purpose; a rolled-back release is not a successful one. Google’s open-source Four Keys reference project settles this by keying off successful production deploy events from the CI system, and that is the right instinct: one row per successful production release, sourced from the pipeline, not from a spreadsheet someone maintains by hand.

What counts as a change. Lead time is measured from the commit, not from when a ticket was opened or when a pull request was raised. The clock starts at the commit timestamp and stops when the deployment carrying that commit reaches production. If you start the clock at ticket creation you are measuring your project-management process, which is a legitimate thing to measure but is not lead time for changes.

What counts as a failure. This is the one that gets fudged hardest, because it is the one people least want to be high. A change failure is a deployment that degrades service and requires remediation — a rollback, a hotfix, a patch. It is not every incident (some incidents have nothing to do with a deploy) and it is not zero because nobody logged the last three. If your change-failure rate is suspiciously low, the usual cause is not excellent engineering; it is under-recorded incidents.

The data you already have

Three sources give you all four metrics. Version control gives you commit SHAs and their timestamps. CI/CD gives you deployment events — what shipped to production, when, and with which commits. Incident tracking gives you failures and their open and resolve times. The engineering task is to land these three streams in one place with a consistent schema, then map each production deployment to the commits it carried and to any incident it caused. Once that mapping exists, the metrics are arithmetic. If your pipeline is not already emitting clean, gated events, that is the first thing to fix — the same discipline that makes a pre-commit and CI baseline actually enforceable is what makes these events trustworthy enough to measure from.

The query below assumes a modest normalised model in Postgres: a deploys table of production releases, a deploy_commits mapping to the commits each release carried, a commits table with author timestamps, and an incidents table linking each failure to the deploy that caused it. It computes all four metrics over a trailing four-week window.

-- Trailing 4-week DORA metrics from VCS + CI + incident data (PostgreSQL).
-- Lead time and MTTR use the median, not the mean: delivery data is skewed.
WITH window_bounds AS (
  SELECT now() - interval '28 days' AS since
),
prod_deploys AS (
  SELECT d.id, d.deployed_at
  FROM deploys d, window_bounds w
  WHERE d.environment = 'production'
    AND d.status = 'succeeded'
    AND d.deployed_at >= w.since
),
lead_times AS (
  -- One row per commit shipped; clock starts at the commit, stops at deploy.
  SELECT extract(epoch FROM (pd.deployed_at - c.committed_at)) AS secs
  FROM prod_deploys pd
  JOIN deploy_commits dc ON dc.deploy_id = pd.id
  JOIN commits c        ON c.sha = dc.commit_sha
),
failures AS (
  -- A change failure is a deploy that required remediation.
  SELECT i.deploy_id,
         extract(epoch FROM (i.resolved_at - i.opened_at)) AS restore_secs
  FROM incidents i
  JOIN prod_deploys pd ON pd.id = i.deploy_id
  WHERE i.resolved_at IS NOT NULL
)
SELECT
  (SELECT count(*) / 4.0 FROM prod_deploys)                       AS deploys_per_week,
  (SELECT percentile_cont(0.5) WITHIN GROUP (ORDER BY secs) / 3600
     FROM lead_times)                                             AS lead_time_hours_median,
  round(100.0 * (SELECT count(DISTINCT deploy_id) FROM failures)
              / nullif((SELECT count(*) FROM prod_deploys), 0), 1) AS change_failure_pct,
  (SELECT percentile_cont(0.5) WITHIN GROUP (ORDER BY restore_secs) / 3600
     FROM failures)                                               AS mttr_hours_median;

Two deliberate choices in that query matter. Lead time and MTTR use the median, because delivery data is long-tailed and a single overnight incident will wreck a mean. And change-failure rate counts distinct failing deploys over total deploys in the same window, so a deploy that causes three alerts still counts once.

Reading the numbers without gaming them

The moment these become targets, people optimise the measure instead of the thing. Deployment frequency rises because releases are sliced thinner while nothing ships faster. Lead time falls because commits are squashed at the last moment. Change-failure rate drops because incidents stop being logged. Every one of these is a local win and a global loss, and the two-pair structure is your defence: read throughput and stability together, and a team gaming one pair will give itself away on the other. A sudden fall in change-failure rate with no change in MTTR usually means failures are going unrecorded, not that quality improved.

The metrics are diagnostic, not evaluative. They belong in a retrospective, aimed at the system, never in an individual’s appraisal — the fastest way to poison the data is to pay people for it. Used well, they surface the boring, expensive truths: that lead time is dominated by review latency, that most failures trace to one under-tested service, that MTTR is long because rollback is manual rather than because incidents are rare. Those are engineering problems with engineering fixes, from safe-to-retry operations that shrink recovery time to progressive delivery that shrinks the blast radius of a bad change.

For regulated firms there is a second dividend. A supervisor asking about your change-management discipline is far better answered with instrumented evidence than with an assertion, and the same instinct that treats controls as code you can query applies here: a metric you can reproduce from source data is a control you can evidence. But that is a by-product. Instrument the four to improve, not to impress — the day the dashboard is built to be shown rather than to be read, it stops being worth reading.