Data Integrity: What You Read Is What You Wrote (SRE Book, ch. 26)
https://sre.google/sre-book/data-integrity/
Method
none stated
Population
none stated
What it does not show
Reports successes, which is a plausible source of survivorship bias — no incident is described where rehearsed procedures still failed. An internally cited study of 19 recovery efforts is referenced but not reproduced, so its methodology cannot be checked. Gives no cross-organisation rate for how often untested backups fail, and attempts no controlled comparison between drilled and undrilled recovery.
Raymond Blum, Rhandeev Singh
Two recoveries of large user-data losses, both credited by the write-up to prior rehearsal. One restored 99%+ of data within hours of the initial estimate because similar situations had been simulated many times. The other recovered 436,223 of about 600,000 wrongly deleted audio references — 161,000 were unrecoverable because they were deleted before ever being backed up — requiring 5,337 tapes recalled by truck, reaching 74% by day two, 99.95% by day three and seven days in total, with the tooling built and tested weeks earlier during a scheduled disaster exercise.
Tier III: Single-organisation retrospective containing two detailed real incidents with hard numbers. No comparison group, and it is the organisation’s own account of its own recoveries.