Metrics That Matter (and the Ones That Don't)

The idea, in one line

A good metric changes a decision. A vanity metric just looks impressive. That single test tells you which numbers to keep and which to drop.

The classic trap: coverage

Coverage is the share of your code that runs during tests. The trap: a line can run without a single assertion — without anything actually checking the result. So it counts as 'covered' while testing nothing. Report coverage as a trend to watch, never as a target to hit. The moment it's a target, people game it.

Metrics that actually matter

The industry-standard set is DORA's four keys — the numbers that connect engineering to real delivery:

• Deployment frequency — how often you ship.
• Lead time for changes — how long from writing code to it being live.
• Change failure rate — the share of deploys that need an urgent fix.
• Time to restore (MTTR) — how fast you recover when something breaks.

For QA specifically, the sharpest number is escaped defects — bugs caught in production versus bugs caught before it. It answers the only question that matters: is our net actually catching things? Pair it with flake rate, because a flaky suite quietly poisons trust in every other number you report.

See it work

None of this needs fancy math. Say you shipped 200 deploys and 24 needed a hotfix — that's a change failure rate of 24 / 200 = 12%. Over a release, 3 bugs slipped to production and 147 were caught first, so your escaped-defect rate is 3 / 150 = 2%. And if tests failed intermittently 9 times across 500 runs, your flake rate is 9 / 500 = 1.8% — just under the 2% line where people stop trusting the suite.

Notice what's missing from all of that: '92% coverage.' It didn't help answer a single decision here.

Advanced — the discipline of saying no

The lead-level move is subtraction, not addition. Pick the three metrics that would genuinely change what you or the team do next, and refuse to report the rest. A short dashboard everyone acts on beats a long one everyone ignores.

Grounded in the DORA 'four keys' software delivery metrics (dora.dev) and State of DevOps benchmarks