← 研踪 PaperTrailarXiv:2610.19730 · v2
Design and Analysis of Temporal Representations in Neural Networks
首次提交 2026-10-14 15:51
版本更新 2026-11-09 23:17
雷达观测 2026-11-10 15:22
摘要
Neural networks, spiking or not, compute with amplitudes: activation values, spike counts, firing rates, never event times. No measure exists of how much a network uses time. We propose one, the degree of temporal representation: the fraction of a network's four functions (encoding, activation, decoding, learning) whose variables or parameters are temporal, and hypothesise that it predicts performance on temporal tasks. At fixed architecture, accuracy and energy efficiency increase with the degree: a delay network of maximal degree reaches 98.94\% on Google Speech Commands with 175k parameters, above transformers. To test whether networks of different degree really differ, we convert the delay network into artificial and spiking networks with the same inputs and outputs. From this common start, the artificial one learns differently and temporal gradients depend on event times. Hence we obtain a design rule: for temporal data, make event times variables of a network's functions. Spiking networks, converted after training, keep the delay network's accuracy, unreachable by spiking networks trained from scratch.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-09 23:17 · 最新观测版本
Design and Analysis of Temporal Representations in Neural Networks
Neural networks, spiking or not, compute with amplitudes: activation values, spike counts, firing rates, never event times. No measure exists of how much a network uses time. We propose one, the degree of temporal representation: the fraction of a network's four functions (encoding, activation, decoding, learning) whose variables or parameters are temporal, and hypothesise that it predicts performance on temporal tasks. At fixed architecture, accuracy and energy efficiency increase with the degree: a delay network of maximal degree reaches 98.94\% on Google Speech Commands with 175k parameters, above transformers. To test whether networks of different degree really differ, we convert the delay network into artificial and spiking networks with the same inputs and outputs. From this common start, the artificial one learns differently and temporal gradients depend on event times. Hence we obtain a design rule: for temporal data, make event times variables of a network's functions. Spiking networks, converted after training, keep the delay network's accuracy, unreachable by spiking networks trained from scratch.
查看此版本解读 →查看此版本原文 ↗