arXiv:2611.25745 · v1
Verification Behavior Across Five Qwen2.5-Instruct Checkpoints
摘要
We evaluate five Qwen2.5-Instruct checkpoints (0.5B to 32B) under four prompting conditions using three fixed tasks per information condition and five generations per task. Two AI raters scored 600 trajectories. Under the coordinated instruction, both raters recorded strict declarations of insufficiency in 11/15 cases at 7B, 0/15 at 14B, and 5/15 at 32B. The runner sent a standard follow-up when a request detector fired. The three checkpoints received at least one follow-up in 15, 12, and 15 cases, respectively. On information-sufficient tasks at 3B through 32B, scored format markers appeared in 60/60 protocol trajectories and 0/60 baseline trajectories. After adjudication, 1/60 in each condition met the diagnostic-execution criterion (H3). Across all checkpoints and prompts, 7/300 information-sufficient trajectories met H3 after adjudication, all on the same task. The prespecified upward transition did not occur in this run. The results separate visible structure, diagnostic execution, and persistence after a request for evidence.
作者备注:10 pages, 1 figure. Includes English translations of the experiment prompts and the behavioral rating rubric. AI language models were used for behavioral ratings and to assist manuscript preparation