跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.19649 · v1

LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

首次提交 2026-11-15 06:35
版本更新 2026-11-15 06:35
雷达观测 2026-11-17 12:06
arXiv ↗PDF ↗

摘要

Looped Transformers improve parameter efficiency by repeatedly applying the same Transformer layers. During autoregressive decoding, however, generating each token requires several sequential passes through these layers, leading to high decoding latency. Pipelined self-speculative methods use shallow-loop predictions to process tokens at different loops in the same batch, but each token must reach the drafting depth before the next can enter the pipeline. We introduce LoopFill, a training-free speculative decoding algorithm for looped LLM that combines multi-token suffix drafts with shallow-loop predictions in a single draft tree. Both drafters contribute to one tree, so a suffix lookup can immediately extend a shallow prediction and several future tokens can enter the pipeline together. The shared block computes the highest-scoring draft candidates at different loop depths in one batch, and full-depth verification preserves the model's autoregressive output distribution. Across seven benchmarks, LoopFill's average speedup over greedy autoregressive decoding is 5.85×5.85\times on Ouro-2.6B and 5.13×5.13\times on Huginn-3.5B. The speedups exceed those of LoopAhead, the strongest baseline, by 40%40\% on Ouro-2.6B and 25%25\% on Huginn-3.5B.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-15 06:35 · 最新观测版本

LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

Looped Transformers improve parameter efficiency by repeatedly applying the same Transformer layers. During autoregressive decoding, however, generating each token requires several sequential passes through these layers, leading to high decoding latency. Pipelined self-speculative methods use shallow-loop predictions to process tokens at different loops in the same batch, but each token must reach the drafting depth before the next can enter the pipeline. We introduce LoopFill, a training-free speculative decoding algorithm for looped LLM that combines multi-token suffix drafts with shallow-loop predictions in a single draft tree. Both drafters contribute to one tree, so a suffix lookup can immediately extend a shallow prediction and several future tokens can enter the pipeline together. The shared block computes the highest-scoring draft candidates at different loop depths in one batch, and full-depth verification preserves the model's autoregressive output distribution. Across seven benchmarks, LoopFill's average speedup over greedy autoregressive decoding is 5.85×5.85\times on Ouro-2.6B and 5.13×5.13\times on Huginn-3.5B. The speedups exceed those of LoopAhead, the strongest baseline, by 40%40\% on Ouro-2.6B and 25%25\% on Huginn-3.5B.

查看此版本解读 →查看此版本原文 ↗