qualitylab

Feedback latency

Every change runs the suite before a human looks at it

unmeasured/cost to adopt: medium/active

Running the suite automatically on every proposed change, before review, shortens feedback latency from days to minutes and moves defect discovery off the reviewer, who is the most expensive and least reliable detector available.

Do this firstOne command builds and tests the project from a clean checkout

The control is not “have tests.” It is that the tests run without anyone choosing to run them, on the change itself, before it reaches a person.

Two properties do the work, and both are commonly missing from setups that otherwise look like this one:

It runs on the proposed change, not on the branch after merge. A suite that runs post-merge tells you the trunk is broken, which is true but late. The interesting question is whether this change breaks it, and that question stops being cheaply answerable the moment the change is in.

It fails loudly and specifically. A red build whose message is “job failed” costs a context switch to interpret and trains people to re-run it first and read it second. The exit status is a claim about processes; it is never by itself a claim about your code.

The honest version of the cost: for a suite that is slow or flaky, this control is not medium-cost, it is expensive, and installing it before fixing that will make people route around it. If your suite is flaky, that is a determinism problem and it comes first — a gate people have learned to re-run is worse than no gate, because it also consumes the alarm.

The decoy

A nightly build. It runs the same tests and produces the same pass/fail, and it moves your feedback latency from minutes to a day while letting you say you have continuous integration. The batch is the problem, not the tests: one red nightly over twelve merged changes tells you the truth about none of them.

What would settle it

A controlled comparison of defect escape rate and review throughput on comparable teams before and after introducing per-change automated runs. The DORA programme measures adjacent things at the organisational level, but the specific per-change claim is inferred from those results rather than tested by them, and this entry will stay at tier IV until something tests it directly.

Seen in the wild

Anonymised field observation. Illustration, not evidence — these carry no tier and cannot raise one.

Last reviewed 2026-08-19.