← 研踪 PaperTrailarXiv:2609.39082 · v3
Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence
首次提交 2026-09-30 14:21
版本更新 2026-11-15 22:44
雷达观测 2026-11-17 19:15
摘要
Fixed-state spectral recurrences offer constant-memory inference and strong long-range retention through heterogeneous complex modes, but their static transitions cannot selectively filter distractors or adapt to evolving dynamics. Existing input-dependent recurrences introduce selectivity, yet real-valued gates lack adaptive rotation, while fine-grained complex transitions incur substantial projection and data-movement overhead and can expose persistent modes to cumulative drift. To address these limitations, we introduce Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence (SPARC), which modulates a layer's heterogeneous complex memory through two input-dependent scalars. Each mode retains its learned baseline timescale, frequency, and write response, while a retention-coupled clock damps phase corrections in persistent modes. The same shared signals regulate incoming content through modal gates and retention-dependent normalization. The resulting diagonal affine recurrence supports parallel associative scans for full-sequence backpropagation through time (BPTT) and exact structured Real-Time Recurrent Learning (RTRL), with sensitivity storage and updates linear in the recurrent layer's parameter count. We evaluate SPARC on partially observable continuous control, POPGym memory benchmarks, sequence classification, and language modeling. SPARC places among the top two means across all ten reinforcement-learning tasks. Compared with RG-LRU, it improves permuted-MNIST and sequential-CIFAR-10 accuracy by and percentage points, respectively, reduces held-out language-model perplexity by , and accelerates mixer forward--backward computation by up to at width . Code is available at https://github.com/Botwwt/sparc.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-15 22:44 · 最新观测版本
Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence
Fixed-state spectral recurrences offer constant-memory inference and strong long-range retention through heterogeneous complex modes, but their static transitions cannot selectively filter distractors or adapt to evolving dynamics. Existing input-dependent recurrences introduce selectivity, yet real-valued gates lack adaptive rotation, while fine-grained complex transitions incur substantial projection and data-movement overhead and can expose persistent modes to cumulative drift. To address these limitations, we introduce Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence (SPARC), which modulates a layer's heterogeneous complex memory through two input-dependent scalars. Each mode retains its learned baseline timescale, frequency, and write response, while a retention-coupled clock damps phase corrections in persistent modes. The same shared signals regulate incoming content through modal gates and retention-dependent normalization. The resulting diagonal affine recurrence supports parallel associative scans for full-sequence backpropagation through time (BPTT) and exact structured Real-Time Recurrent Learning (RTRL), with sensitivity storage and updates linear in the recurrent layer's parameter count. We evaluate SPARC on partially observable continuous control, POPGym memory benchmarks, sequence classification, and language modeling. SPARC places among the top two means across all ten reinforcement-learning tasks. Compared with RG-LRU, it improves permuted-MNIST and sequential-CIFAR-10 accuracy by and percentage points, respectively, reduces held-out language-model perplexity by , and accelerates mixer forward--backward computation by up to at width . Code is available at https://github.com/Botwwt/sparc.
查看此版本解读 →查看此版本原文 ↗