Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence
Wentao Wang, Hengyu Zhong, Yunhan Jiang, Jialiang An, Meng Lu
原文摘要
Fixed-state spectral recurrences offer constant-memory inference and strong long-range retention through heterogeneous complex modes, but their static transitions cannot selectively filter distractors or adapt to evolving dynamics. Existing input-dependent recurrences introduce selectivity, yet real-valued gates lack adaptive rotation, while fine-grained complex transitions incur substantial projection and data-movement overhead and can expose persistent modes to cumulative drift. To address these limitations, we introduce Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence (SPARC), which modulates a layer's heterogeneous complex memory through two input-dependent scalars. Each mode retains its learned baseline timescale, frequency, and write response, while a retention-coupled clock damps phase corrections in persistent modes. The same shared signals regulate incoming content through modal gates and retention-dependent normalization. The resulting diagonal affine recurrence supports parallel associative scans for full-sequence backpropagation through time (BPTT) and exact structured Real-Time Recurrent Learning (RTRL), with sensitivity storage and updates linear in the recurrent layer's parameter count. We evaluate SPARC on partially observable continuous control, POPGym memory benchmarks, sequence classification, and language modeling. SPARC places among the top two means across all ten reinforcement-learning tasks. Compared with RG-LRU, it improves permuted-MNIST and sequential-CIFAR-10 accuracy by $10.13$ and $10.66$ percentage points, respectively, reduces held-out language-model perplexity by $2.00\%$, and accelerates mixer forward--backward computation by up to $1.69\times$ at width $D=2048$. Code is available at https://github.com/Botwwt/sparc.
cs.LG