arXiv:2610.37388 · v1

Para-Gaussian Approximations of Random Deep Neural Networks

首次提交 2026-10-27 23:12

版本更新 2026-10-27 23:12

雷达观测 2026-10-29 00:34

摘要

It is known that randomly initialized neural networks have Gaussian behaviour in the $n\to \infty$ limit. Unfortunately, this approximation can be poor for finite $n$ and gets worse as one goes deeper into the network. In this work, we develop a more accurate approximation that is \emph{slightly} non-Gaussian, which we call the para-Gaussian distribution and is constructed from the Hermite polynomials. We show how this para-Gaussian approximation can lead to a practical recursive method to approximate the deeper layers of the network. Finally, we apply this theory to linear neural networks (where things can be solved exactly) and to ReLU neural networks (where things can be approximated).

作者备注：34 oages, 9 figures