跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+479新增论文
+361其中 AI 相关
+64已有论文修订
+93新增 AI 导读

arXiv 最近完成扫描:2026-11-16 01:34 · 全站最近入库:2026-11-16 21:56各信源进度 →

统计口径与数据新鲜度

统计 2026-11-15 23:52 — 2026-11-16 23:52(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-16 01:34最近发现新增版本:2026-11-16 21:56库内论文最新更新时间:2026-11-14 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

23,827 已收录论文19,941 AI 相关6,922 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 1 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-16 01:33(北京时间)
版本更新 · v2arXiv:2610.06439

Better Call Reward: Reward Hacking as Strategic Abstention in Legal Reasoning Models

Subramanyam Sahoo, Justin Shenk

原文摘要

What happens when a legal AI model learns to look like a lawyer instead of reasoning like one? We fine tune Qwen3-8B with Group Relative Policy Optimisation (GRPO) against a proxy built from three surface features: citation count, legalese density, and response length. The model does not learn to reason more effectively. It stops committing to an answer. Across 16 yes or no legal reasoning tasks from LegalBench (N=320), overall accuracy collapses from 0.500 (chance) to 0.072 (McNemar p < 10^-36), driven entirely by the rate of properly formatted answers falling from 0.900 to 0.109. When the model does commit, accuracy is 0.657 against 0.556 before training, on only about 35 answers, suggesting the collapse is not mainly a loss of capability. The mechanism is visible in the training logs: the proxy keeps paying for length beyond what fits in the 256 token generation limit, completions grow until they fill that budget, and the required answer line is crowded out. We term this the Saul Goodman effect, a policy that becomes more lawyerly in form while becoming noncommittal, and prove formally that abstention is the optimal response to any surface feature proxy that attaches no penalty to it. A citation audit shows that structural plausibility checks cannot catch this failure: every case name we extracted from the trained model's held out completions passes our structural check, so citations must be verified against a case database. To detect this failure before deployment, we introduce three diagnostic tools: the Confidence Theater Score (CTS), the Citation Plausibility Rate (CPR), and the Regret Gap (RG), and report where each one falls short. In a domain where a confidently wrong answer can constitute malpractice, the broader lesson is direct: a reward function that measures how legal a response looks will produce a model that is maximally photogenic and minimally useful.

cs.LG · cs.AI · cs.CL · cs.CY