Feedback latency · Adversarial exposure
Fuzz continuously, and read the bug counts sceptically
Coverage-guided fuzzing against a continuously built target finds crash-class defects quickly and cheaply once wired in — median five days to detection — but the count of bugs found overstates the count of vulnerabilities prevented by roughly two orders of magnitude.
Do this firstRun the suite under sanitizers, not just under the compiler
Point a generator at a build, keep it running, triage what falls out.
Two numbers set expectations correctly. Detection and repair are fast: a median of five days to find a regression and about five days to fix it. And the yield is mostly not security: the same corpus produced 23,907 bugs and 98 CVEs. Most of what fuzzing finds is availability-class — crashes, hangs, memory exhaustion — which is worth fixing and is not the thing the practice is usually sold on.
The triage cost is real and under-reported. Thirteen per cent of the bugs in that corpus were flaky, and flaky ones are overwhelmingly never fixed — 86% remained open, against 12% for the rest.
The decoy
A bug count. It is the number every fuzzing programme reports and it is a poor proxy for the thing you care about — of 23,907 bugs found across 316 projects in the largest public corpus, 98 received a CVE. Under half a percent.
Evidence
- Evaluating Fuzz Testing — IEvery one of 32 surveyed evaluations had methodological problems. In the reproduction, a statistically significant increase in unique crashing inputs (p=10^-10) corresponded to no significant increase in real distinct bugs (p=0.066) — the standard deduplication proxy overstates the difference between fuzzers.
- An Empirical Study of OSS-Fuzz Bugs — IIDetection and repair are fast — a median of five days to detect a regression and 5.3 days to fix. And the yield is mostly not security: of 23,907 bugs, 98 received a CVE. Flaky bugs, 13% of the corpus, are overwhelmingly never fixed at 86% unfixed against 12% for the rest.
Last reviewed 2026-08-19.