← 研踪 PaperTrailarXiv:2604.15742 · v2
Collective Kernel EFT for Pre-activation ResNets
首次提交 2026-04-17 14:31
版本更新 2026-10-28 08:08
雷达观测 2026-10-30 10:23
摘要
In finite-width deep neural networks, the empirical kernel $G$ evolves stochastically across layers. We develop a collective kernel effective field theory (EFT) for a simplified pre-activation residual model. Exact conditional Gaussianity of the residual increment gives finite-width discrete identities for $G$ and its sigma-kernel source. A projected $G$-only closure and a continuous-depth approximation define reference equations for the mean kernel $K_0$, the kernel covariance $V_4$, and the next-to-leading mean correction $K_1$. Numerical tests show that this closure captures the leading mean-kernel evolution, whereas its covariance and next-to-leading accuracy depend on sigma-kernel modes omitted from the $G$-only state. For the scalar ($N=1$) model with $\tanh$ activation, a Hermite--Galerkin extension agrees with the extrapolated $K_1$ value and reduces the $V_4$ endpoint error by $85$--$93\%$ relative to the $G$-only reference. These results clarify the range of validity of the equal-time Markov $G$-only description and identify the additional collective information required beyond it.
作者备注：45 pages. v2: major improvements. Published version
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-10-28 08:08 · 最新观测版本
Collective Kernel EFT for Pre-activation ResNets
In finite-width deep neural networks, the empirical kernel $G$ evolves stochastically across layers. We develop a collective kernel effective field theory (EFT) for a simplified pre-activation residual model. Exact conditional Gaussianity of the residual increment gives finite-width discrete identities for $G$ and its sigma-kernel source. A projected $G$-only closure and a continuous-depth approximation define reference equations for the mean kernel $K_0$, the kernel covariance $V_4$, and the next-to-leading mean correction $K_1$. Numerical tests show that this closure captures the leading mean-kernel evolution, whereas its covariance and next-to-leading accuracy depend on sigma-kernel modes omitted from the $G$-only state. For the scalar ($N=1$) model with $\tanh$ activation, a Hermite--Galerkin extension agrees with the extrapolated $K_1$ value and reduces the $V_4$ endpoint error by $85$--$93\%$ relative to the $G$-only reference. These results clarify the range of validity of the equal-time Markov $G$-only description and identify the additional collective information required beyond it.
查看此版本解读 →查看此版本原文 ↗