← 研踪 PaperTrailarXiv:2610.34639 · v1
Cross-Loop Query Handoff: How Later Loops Redirect Two-Hop Retrieval in the Ouro Looped Language Models
首次提交 2026-10-26 07:46
版本更新 2026-10-26 07:46
雷达观测 2026-10-29 00:36
摘要
How do later loops in a pretrained looped language model repair two-hop retrieval? We study Ouro-1.4B and Ouro-2.6B by transplanting attention queries between loops at the same layer, head and final prompt position. On synthetic in-context questions, injecting the layer-indexed collection of loop-3 queries throughout loop 1 rescues 90% and 97% of questions answered incorrectly at loop 1 and correctly at loop 3, versus 24% and 21% for shuffled donors. The rescue also holds under greedy generation in Ouro-2.6B; in Ouro-1.4B it is limited to four-candidate ranking. Crossed donor-recipient swaps favor retrieval through the donor's bridge entity over copying its answer, although a substantial positional component remains in Ouro-2.6B. Late keys and values rescue far fewer targets. Freezing queries at their loop-1 values also damages two-hop accuracy more than a matched question that supplies the bridge. On natural questions, rescue is smaller and replicates only in Ouro-2.6B, and crossed swaps do not establish bridge redirection. These results identify a transferable collection of late-loop queries that supports bridge-conditioned retrieval on synthetic scenes; they do not locate where the bridge is computed or establish a single-vector handoff.
作者备注：Code and reproducibility: https://github.com/Muuzzziiy/cql-handoff 16 pages, 1 figure, 7 tables
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-10-26 07:46 · 最新观测版本
Cross-Loop Query Handoff: How Later Loops Redirect Two-Hop Retrieval in the Ouro Looped Language Models
How do later loops in a pretrained looped language model repair two-hop retrieval? We study Ouro-1.4B and Ouro-2.6B by transplanting attention queries between loops at the same layer, head and final prompt position. On synthetic in-context questions, injecting the layer-indexed collection of loop-3 queries throughout loop 1 rescues 90% and 97% of questions answered incorrectly at loop 1 and correctly at loop 3, versus 24% and 21% for shuffled donors. The rescue also holds under greedy generation in Ouro-2.6B; in Ouro-1.4B it is limited to four-candidate ranking. Crossed donor-recipient swaps favor retrieval through the donor's bridge entity over copying its answer, although a substantial positional component remains in Ouro-2.6B. Late keys and values rescue far fewer targets. Freezing queries at their loop-1 values also damages two-hop accuracy more than a matched question that supplies the bridge. On natural questions, rescue is smaller and replicates only in Ouro-2.6B, and crossed swaps do not establish bridge redirection. These results identify a transferable collection of late-loop queries that supports bridge-conditioned retrieval on synthetic scenes; they do not locate where the bridge is computed or establish a single-vector handoff.
查看此版本解读 →查看此版本原文 ↗