跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.10098 · v1

Autonomous Code Review: Hidden-Test Results on GitHub Issues

首次提交 2026-11-07 01:11
版本更新 2026-11-07 01:11
雷达观测 2026-11-10 10:11
arXiv ↗PDF ↗

摘要

Can an autonomous cross-provider review step change whether code written for a real issue passes hidden tests? We report a retrospective benchmark of 12 historical GitHub issues (five C#, five JavaScript, and two Python) from five repositories. Two directed author--reviewer pairs, OpenAI GPT-6.1-Sol and Anthropic Claude Sonnet 5.5, each ran twice per issue, yielding 48 planned workflows. A bounded terminal agent produced an initial patch; a cross-provider model reviewed it; the author model adjudicated findings and could revise the patch. The same hidden evaluator assessed initial and final patches. Forty workflows completed, while eight terminated before a complete evaluator outcome. Four completed workflows passed, and all four already passed before review. No initial-to-final pass/fail transitions occurred among completed workflows.

作者备注:8 pages, 1 figure, 5 tables

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-07 01:11 · 最新观测版本

Autonomous Code Review: Hidden-Test Results on GitHub Issues

Can an autonomous cross-provider review step change whether code written for a real issue passes hidden tests? We report a retrospective benchmark of 12 historical GitHub issues (five C#, five JavaScript, and two Python) from five repositories. Two directed author--reviewer pairs, OpenAI GPT-6.1-Sol and Anthropic Claude Sonnet 5.5, each ran twice per issue, yielding 48 planned workflows. A bounded terminal agent produced an initial patch; a cross-provider model reviewed it; the author model adjudicated findings and could revise the patch. The same hidden evaluator assessed initial and final patches. Forty workflows completed, while eight terminated before a complete evaluator outcome. Four completed workflows passed, and all four already passed before review. No initial-to-final pass/fail transitions occurred among completed workflows.

查看此版本解读 →查看此版本原文 ↗