The First Token Knows: Single-Decode Confidence for Hallucination Detection
Mina Gabriel, Pei Wang
原文摘要
Sampling-based uncertainty methods detect hallucinations by generating multiple answers and measuring agreement, often with additional semantic clustering. We study whether uncertainty is already visible in the first content-bearing answer token of a single greedy decode. We define first-token confidence, phi_first, as one minus the normalized entropy of the top-K first-token distribution. Across PopQA and TriviaQA with three 7-8B instruction-tuned models, phi_first matches or modestly exceeds semantic self-consistency while requiring only one decode instead of one greedy decode plus ten sampled generations and NLI clustering. A subsumption analysis shows that phi_first is moderately to strongly correlated with semantic agreement, and combining both signals yields only small additional AUROC gains. We position phi_first as a cheap deployment-oriented baseline for factual QA uncertainty estimation, while noting its limits for high-confidence falsehoods and long-form generation.
cs.CL · cs.AI