跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.05306 · v1

You don't need long context

Peter BelcakSaurav MuralidharanJindong JiangYonggan FuWonmin ByeonHanrong YeRyan AngillyAli TaghibakhshiShih-Yang LiuSiru OuyangGreg HeinrichPavlo Molchanov
cs.AI
首次提交 2026-11-04 02:17
版本更新 2026-11-04 02:17
雷达观测 2026-11-04 23:39
arXiv ↗PDF ↗

摘要

Later tokens are costlier to generate than earlier tokens. On frontier open weight models, the disparity in cost may rise to as much as 7x. Through frequent compaction, we show across a broad range of long-horizon and long-input tasks that frontier language models sacrifice little to no task accuracy when maintaining short contexts. We further demonstrate that LLM performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context, and based on that we outline the concept of semantic state space models (SSSMs). Our findings question the need for long-context language models.

作者备注：Technical report. 31 pages, 8 figures

查看 arXiv 原文 ↗阅读 PDF ↗

AI 论文解读

基于摘要 · 非全文审阅

作者报告通过频繁压缩上下文，前沿语言模型在长程与长输入任务上几乎不损失准确率，并据此质疑长上下文模型的必要性。

正在读取账号与研究记录…

版本差异

目前只观测到一个版本，无法比较变化。之后收录新版本时，会在这里对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-04 02:17 · 最新观测版本

You don't need long context

Later tokens are costlier to generate than earlier tokens. On frontier open weight models, the disparity in cost may rise to as much as 7x. Through frequent compaction, we show across a broad range of long-horizon and long-input tasks that frontier language models sacrifice little to no task accuracy when maintaining short contexts. We further demonstrate that LLM performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context, and based on that we outline the concept of semantic state space models (SSSMs). Our findings question the need for long-context language models.

查看此版本解读 →查看此版本原文 ↗