跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2605.05166 · v2

The First Token Knows: Single-Decode Confidence for Hallucination Detection

首次提交 2026-05-07 01:34
版本更新 2026-11-14 20:06
雷达观测 2026-11-17 12:06
arXiv ↗PDF ↗

摘要

Sampling-based uncertainty methods detect hallucinations by generating multiple answers and measuring agreement, often with additional semantic clustering. We study whether uncertainty is already visible in the first content-bearing answer token of a single greedy decode. We define first-token confidence, phi_first, as one minus the normalized entropy of the top-K first-token distribution. Across PopQA and TriviaQA with three 7-8B instruction-tuned models, phi_first matches or modestly exceeds semantic self-consistency while requiring only one decode instead of one greedy decode plus ten sampled generations and NLI clustering. A subsumption analysis shows that phi_first is moderately to strongly correlated with semantic agreement, and combining both signals yields only small additional AUROC gains. We position phi_first as a cheap deployment-oriented baseline for factual QA uncertainty estimation, while noting its limits for high-confidence falsehoods and long-form generation.

作者备注:9 pages, 1 figure, 6 tables. Published in the Proceedings of the Second European Workshop on Trustworthy AI (TRUST-AI 2026), CEUR Workshop Proceedings, Vol. 4254, pp. 55-63

期刊信息:Proceedings of the Second European Workshop on Trustworthy AI (TRUST-AI 2026), CEUR Workshop Proceedings, Vol. 4254, pp. 55-63, 2026

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-14 20:06 · 最新观测版本

The First Token Knows: Single-Decode Confidence for Hallucination Detection

Sampling-based uncertainty methods detect hallucinations by generating multiple answers and measuring agreement, often with additional semantic clustering. We study whether uncertainty is already visible in the first content-bearing answer token of a single greedy decode. We define first-token confidence, phi_first, as one minus the normalized entropy of the top-K first-token distribution. Across PopQA and TriviaQA with three 7-8B instruction-tuned models, phi_first matches or modestly exceeds semantic self-consistency while requiring only one decode instead of one greedy decode plus ten sampled generations and NLI clustering. A subsumption analysis shows that phi_first is moderately to strongly correlated with semantic agreement, and combining both signals yields only small additional AUROC gains. We position phi_first as a cheap deployment-oriented baseline for factual QA uncertainty estimation, while noting its limits for high-confidence falsehoods and long-form generation.

查看此版本解读 →查看此版本原文 ↗