Dependabot and security pull requests: large empirical study
https://doi.org/10.1007/s10664-024-10523-y
Method
"We performed a large empirical study on dependency updates and security pull requests to understand: (1) the degree and reasons of Dependabot's popularity...(3) the management of security pull requests (PRs), the threat lifetime, and the fix delay"
Population
"we collected a dataset of 9,916,318 pull request-related issues in 1,743,035 projects on GitHub for more than 10 different programming languages"
What it does not show
Detects known vulnerabilities present in the manifest, not confirmed reachability at runtime. Pull request outcomes are not verification that a merged fix was deployed or that closed ones left exploitable code in production. Survey response rates of 13% and 14% risk selection bias.
Hocine Rebatchi, Tegawende F. Bissyande, Naouel Moha
Known-vulnerable dependency versions sit undetected for a mean of 512 days, median 419. Bot-created security pull requests are merged only 25% of the time — 44% are auto-closed, mostly superseded by a newer one, and 31% remain open — while manually created security pull requests are merged 74% of the time. Where merges happen they are fast: median under a day, and 78% of auto-merges complete in minutes.
Tier II: Large-scale mining of real repository activity plus a supplementary developer survey. Observational with no control group; the survey portions are self-reported and had low response rates.