跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文,不再错过。

覆盖计算机全学科,以 AI 为快捷入口。关注方向、作者和关键词,把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+2471新增论文
+2025其中 AI 相关
+163已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描:2026-11-10 22:42 · 全站最近入库:2026-11-10 15:22各信源进度 →

统计口径与数据新鲜度

统计 2026-11-09 23:40 — 2026-11-10 23:40(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-10 22:42最近发现新增版本:2026-11-10 15:22库内论文最新更新时间:2026-11-10 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

20,643 已收录论文17,531 AI 相关5,643 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 1 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-10 22:38(北京时间)
版本更新 · v2arXiv:2605.29253

OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories

Yibing Liu, Yangze Liu, Xiaolong Yin, Bin Wang, Chong Zhang, Hao Yin, Zhongyi Han

原文摘要

Task success can hide process anomalies in real-world agent executions. An agent may pass the final task oracle while still accumulating unresolved ambiguity, unsafe external writes, ignored errors, weakly grounded commitments, or capability-boundary overcommitment. We study this mismatch as the Outcome-Process Gap and introduce OpenClawBench, a large-scale dataset for measuring and supervising process-side anomalies in real agent execution processes. OpenClawBench is built from BFCL-driven OpenClaw sessions produced by 6 source models and contains 31,264 annotated trajectories. FullTax converts the aligned trajectories into structured anomaly supervision: binary labels, supporting evidence, onset/span localization, severity, recoverability, and a 5-class anomaly taxonomy. Using OpenClawBench, we make the Outcome-Process Gap measurable. Among 31,135 oracle-passing executions, 2,904 (9.33%) are still labeled process-anomalous under FullTax. Within oracle-passing executions that contain high-risk process evidence, 1,765 of 1,904 (92.70%) are labeled anomalous. On Gold-300, a test subset labeled independently by two human annotators, silver labels match the final human binary label on 93.3% of trajectories. A LoRA-fine-tuned Gemma 3 12B detector trained on the high-confidence FullTax supervised pool reaches binary F1=0.729 on the cleaner-labels silver test split (n=2,646). It outperforms the GPT-5.4 frontier reference by +0.302 absolute, the no-fine-tuning base by +0.357, and wins on all six source-model agent slices. The gain comes from calibration rather than higher recall: zero-shot detectors over-flag at 42-50% against a 14.8% label rate, while the fine-tuned detector predicts anomalies 18.2% of the time. Together, OpenClawBench turns real agent execution logs into auditable and reusable supervision for studying, diagnosing, and operationally monitoring runtime agent reliability.

cs.AI