← 研踪 PaperTrailarXiv:2601.16809 · v2
Will It Survive? Deciphering the Fate of AI Agent-Authored Code in Open Source
首次提交 2026-01-23 23:00
版本更新 2026-11-12 23:26
雷达观测 2026-11-13 14:15
摘要
AI coding agents such as GitHub Copilot, OpenAI Codex and Devin now open pull requests (PRs) that are merged into open-source projects. Industry reports suggest that AI-assisted code is revised or reverted soon after it is written, which raises the concern that agent code is "disposable". However, little is known about what happens to agent code after it is merged. Knowing how long this code survives, and why it is changed, can help maintainers decide how much review it needs. We performed a survival analysis of the 648,959 lines that 2,794 merged PRs added outside vendored and generated paths in 215 open-source repositories from the AIDev dataset, comparing agent-attributed and human code within the same repositories. Agent-attributed code is not modified sooner than human code. When each PR is weighted equally, agent-attributed lines are modified later (HR = 0.83, 95% CI [0.75, 0.93]). This difference comes from the supervised assistants, mainly Copilot and Codex (HR = 0.76). Code from Devin does not differ from human code over the whole period (HR = 0.95), but among PRs merged after mid-May 2025, code from both the supervised assistants and Devin is modified later. Without PR weighting, the difference is not statistically significant. When agent-attributed code is modified, the intent of the change is similar to that of human code (Cramer's V = 0.080). A file's size and recent changes predict whether agent-attributed code will be modified better than its code tokens do (AUC-ROC 0.78 vs. 0.70), and process features predict when it will be modified only weakly (Macro F1 of 0.31 against 0.25 for a random guess). Our findings suggest that agent code is not disposable, but whether it survives longer than human code depends on the agent, on when the code was merged and on how the comparison is computed.
作者备注：This paper has been submitted to a conference and is currently under review
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-12 23:26 · 最新观测版本
Will It Survive? Deciphering the Fate of AI Agent-Authored Code in Open Source
AI coding agents such as GitHub Copilot, OpenAI Codex and Devin now open pull requests (PRs) that are merged into open-source projects. Industry reports suggest that AI-assisted code is revised or reverted soon after it is written, which raises the concern that agent code is "disposable". However, little is known about what happens to agent code after it is merged. Knowing how long this code survives, and why it is changed, can help maintainers decide how much review it needs. We performed a survival analysis of the 648,959 lines that 2,794 merged PRs added outside vendored and generated paths in 215 open-source repositories from the AIDev dataset, comparing agent-attributed and human code within the same repositories. Agent-attributed code is not modified sooner than human code. When each PR is weighted equally, agent-attributed lines are modified later (HR = 0.83, 95% CI [0.75, 0.93]). This difference comes from the supervised assistants, mainly Copilot and Codex (HR = 0.76). Code from Devin does not differ from human code over the whole period (HR = 0.95), but among PRs merged after mid-May 2025, code from both the supervised assistants and Devin is modified later. Without PR weighting, the difference is not statistically significant. When agent-attributed code is modified, the intent of the change is similar to that of human code (Cramer's V = 0.080). A file's size and recent changes predict whether agent-attributed code will be modified better than its code tokens do (AUC-ROC 0.78 vs. 0.70), and process features predict when it will be modified only weakly (Macro F1 of 0.31 against 0.25 for a random guess). Our findings suggest that agent code is not disposable, but whether it survives longer than human code depends on the agent, on when the code was merged and on how the comparison is computed.
查看此版本解读 →查看此版本原文 ↗