← 研踪 PaperTrailarXiv:2608.25005 · v2
The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure
首次提交 2026-08-26 02:00
版本更新 2026-11-08 07:13
雷达观测 2026-11-10 12:16
摘要
Recent work uses an NLI benchmark of the imperfective paradox to argue that large language models exhibit a Teleological Bias and Calibration Crisis. We identify three validity concerns in benchmark construction, including aspectual reduction, semantic bias, and relation bias. Under a strict NLI standard, 76% of the Group A instances do not explicitly rule out culmination; in native-speaker annotation, 38% of Group A and 29% of Group C examples permit an alternative interpretation. To better isolate the semantic contrast, we construct Lexically Matched Minimal Pairs. Moreover, by formulating event-semantic NLI as a multi-step reasoning problem, we evaluate intermediate semantic judgments and use oracle-guided settings. We find that incorrect NLI entailment does not fully support teleological bias. Prompting often changes final NLI decisions without reliably improving the underlying semantic judgments, offering a possible explanation for the Calibration Crisis.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-08 07:13 · 最新观测版本
The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure
Recent work uses an NLI benchmark of the imperfective paradox to argue that large language models exhibit a Teleological Bias and Calibration Crisis. We identify three validity concerns in benchmark construction, including aspectual reduction, semantic bias, and relation bias. Under a strict NLI standard, 76% of the Group A instances do not explicitly rule out culmination; in native-speaker annotation, 38% of Group A and 29% of Group C examples permit an alternative interpretation. To better isolate the semantic contrast, we construct Lexically Matched Minimal Pairs. Moreover, by formulating event-semantic NLI as a multi-step reasoning problem, we evaluate intermediate semantic judgments and use oracle-guided settings. We find that incorrect NLI entailment does not fully support teleological bias. Prompting often changes final NLI decisions without reliably improving the underlying semantic judgments, offering a possible explanation for the Calibration Crisis.
查看此版本解读 →查看此版本原文 ↗