Fully Automated DORA Metrics Measurement for Continuous Improvement
https://homepages.ecs.vuw.ac.nz/~craig/publications/icssp2024-ruegger.pdf
Method
"We evaluated the developed solution in an industrial case study consisting of 37 microservices over a four-week period."
Population
"The presented case-study in this paper is a collaboration with a Zurich-based software company... At the time of the case study, the company employed twelve developers... organized as a single cross-functional Scrum team."
What it does not show
One company, one twelve-person team, four weeks. Demonstrates measurement feasibility and self-report distortion; does not test whether telemetry-derived metrics predict organisational outcomes the way the survey-based models claim.
Janick Rüegger, Martin Kropp, Sebastian Graf, Craig Anslow
Deriving the same delivery metrics from version control, CI and telemetry rather than from a survey exposes wide variation between individual services that a team-level self-report hides: “team performance has limited representational capabilities for individual microservices performance.” Also catalogues concrete weaknesses of survey measurement — subjective responses, coarse Likert scales, recall error, and poor scalability.
Tier III: Single-organisation industrial case study using real telemetry rather than survey responses, but one team and a four-week window with no comparison group.