← 研踪 PaperTrailarXiv:2602.11908 · v4
When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation
首次提交 2026-02-12 21:06
版本更新 2026-11-01 19:42
雷达观测 2026-11-04 23:43
摘要
LLMs are widely used, yet they remain prone to factual errors that erode trust and limit adoption in high-risk settings. One approach to mitigate this risk is to equip models with uncertainty estimation mechanisms that abstain when confidence is low. However, this binary "all-or-nothing" approach is excessively restrictive in long-form settings, often discarding valuable information. We introduce Selective Abstraction (SA), a framework that enables LLMs to trade specificity for reliability by selectively reducing the detail of uncertain content. We formalize SA through selective risk and coverage, and propose Atom-wise Selective Abstraction, a claim-level instantiation that decomposes responses into atomic claims and replaces uncertain atoms with higher-confidence, less specific abstractions. We develop a novel end-to-end pipeline for evaluating long-form generation that instantiates risk as factual correctness and measures coverage as retained information. Our comprehensive empirical evaluation across five LLMs and two long-form factual benchmarks shows that atom-wise SA consistently improves the risk-coverage trade-off, outperforming seven baselines, including claim removal, self-revision, sampling-based aggregation, and chain-of-verification. It improves AURC by up to 35.61% over claim removal, demonstrating that reducing specificity boosts reliability while preserving substantial information. We further provide a risk-guided procedure for selecting a confidence threshold that satisfies a target risk.
作者备注：Accepted as poster to NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v4 · 2026-11-01 19:42 · 最新观测版本
When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation
LLMs are widely used, yet they remain prone to factual errors that erode trust and limit adoption in high-risk settings. One approach to mitigate this risk is to equip models with uncertainty estimation mechanisms that abstain when confidence is low. However, this binary "all-or-nothing" approach is excessively restrictive in long-form settings, often discarding valuable information. We introduce Selective Abstraction (SA), a framework that enables LLMs to trade specificity for reliability by selectively reducing the detail of uncertain content. We formalize SA through selective risk and coverage, and propose Atom-wise Selective Abstraction, a claim-level instantiation that decomposes responses into atomic claims and replaces uncertain atoms with higher-confidence, less specific abstractions. We develop a novel end-to-end pipeline for evaluating long-form generation that instantiates risk as factual correctness and measures coverage as retained information. Our comprehensive empirical evaluation across five LLMs and two long-form factual benchmarks shows that atom-wise SA consistently improves the risk-coverage trade-off, outperforming seven baselines, including claim removal, self-revision, sampling-based aggregation, and chain-of-verification. It improves AURC by up to 35.61% over claim removal, demonstrating that reducing specificity boosts reliability while preserving substantial information. We further provide a risk-guided procedure for selecting a confidence threshold that satisfies a target risk.
查看此版本解读 →查看此版本原文 ↗