← 研踪 PaperTrailarXiv:2610.02405 · v2
When Terminal-Agent Training Stalls: Demystifying Data Generation and Verification Challenge
首次提交 2026-10-02 03:32
版本更新 2026-11-17 03:59
雷达观测 2026-11-18 19:44
摘要
Automated terminal-agent training relies on generated environments and verifiers to turn open-ended interaction into reward. Yet a runnable Docker image and executable test suite do not guarantee that the resulting reward is faithful, informative, or even attributable to agent behavior. We present a pipeline that generates containerized terminal tasks and their executable verifiers, and use it to demystify failure modes in task generation and live-environment evaluation. Our analysis separates four requirements that are often conflated: structural validity, verifier faithfulness, policy solvability, and infrastructure reliability. In a GRPO training of Qwen3.5-9B, it overfits on a relatively easy dataset within 20 steps. Conversely, in a controlled Qwen3.5-9B comparison, adding hard tasks lowers mean rollout pass@2 from 81.3% to 20.6%, restoring informative failures without changing the training configuration. We further identify near-duplicate task generation, incomplete anti-shortcut safeguards, Dockerfile failures, asynchronous pass@k bias, and concurrency-induced environment errors as distinct threats to measurement. These findings should motivate the community for a terminal agent specific evaluation harness that pairs structural checks and rollout-derived solvability with extensive verifier audits.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-17 03:59 · 最新观测版本
When Terminal-Agent Training Stalls: Demystifying Data Generation and Verification Challenge
Automated terminal-agent training relies on generated environments and verifiers to turn open-ended interaction into reward. Yet a runnable Docker image and executable test suite do not guarantee that the resulting reward is faithful, informative, or even attributable to agent behavior. We present a pipeline that generates containerized terminal tasks and their executable verifiers, and use it to demystify failure modes in task generation and live-environment evaluation. Our analysis separates four requirements that are often conflated: structural validity, verifier faithfulness, policy solvability, and infrastructure reliability. In a GRPO training of Qwen3.5-9B, it overfits on a relatively easy dataset within 20 steps. Conversely, in a controlled Qwen3.5-9B comparison, adding hard tasks lowers mean rollout pass@2 from 81.3% to 20.6%, restoring informative failures without changing the training configuration. We further identify near-duplicate task generation, incomplete anti-shortcut safeguards, Dockerfile failures, asynchronous pass@k bias, and concurrency-induced environment errors as distinct threats to measurement. These findings should motivate the community for a terminal agent specific evaluation harness that pairs structural checks and rollout-derived solvability with extensive verifier audits.
查看此版本解读 →查看此版本原文 ↗