Gandalf: An Intelligent, End-To-End Analytics Service for Safe Deployment in Large-Scale Cloud Infrastructure
https://www.usenix.org/system/files/nsdi20spring_li_prepub.pdf
Method
"Gandalf monitors and analyzes various fault signals. It will correlate each signal against all the ongoing rollouts using a spatial and temporal correlation algorithm."
Population
"Gandalf has been running in production in Microsoft Azure for more than 18 months, serving both data-plane and control-plane components."
What it does not show
Does not isolate canarying from the analytics stack around it — the ablations vary internal algorithm components, not canary versus no canary. Single proprietary system; the precision figures are against its own labelled incident set and may not transfer.
Ze Li, Qian Cheng, Ken Hsieh, Yingnong Dang, Peng Huang, Pankaj Singh, Xinsheng Yang, Qingwei Lin, Youjiang Wu, Sebastien Levy, Murali Chintalapati
Correlating fault telemetry against in-flight staged rollouts caught 155 critical failures over eight months at 92.4% precision and 100% recall for data-plane rollouts, with the paper reporting that 99.2% of suspicious rollouts were blocked during canary or pilot stages rather than in production.
Tier III: Single-organisation engineering report with precision and recall over defined production windows, but no randomised comparison against a no-correlation baseline; the accuracy figures are computed against the system’s own incident labels.