跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2604.20219 · v3

Layer-wise Geometric Approximation Rates for Deep Networks

首次提交 2026-04-22 14:09
版本更新 2026-11-16 23:54
雷达观测 2026-11-17 16:10
arXiv ↗PDF ↗

摘要

Depth is widely viewed as a central contributor to the success of deep neural networks, yet standard approximation results typically give error bounds only for the final output of a network. Inspired by classical multiscale expansions and multigrade deep learning, we give depth a scale-dependent interpretation by constructing a single network whose intermediate readouts approximate the target at successively finer scales. For any f∈Lp([0,1]d)f\in L^p([0,1]^d) with 1≤p<∞1\le p <\infty and any integer N≥2N\ge2, we construct a single network of width 3dN+2d+43dN+2d+4 with the same activation pattern in every hidden layer, using sine in the first two neurons and ReLU\mathtt{ReLU} in the rest. For every ℓ≥0\ell\ge0, the LpL^p approximation error of the readout ΦℓΦ_\ell at layer ℓ+4\ell+4 is bounded by (2d+1)ωf,p(N−ℓ)(2d+1)ω_{f,p}(N^{-\ell}), where ωf,pω_{f,p} denotes the LpL^p translation modulus of ff. Under the Hölder condition ωf,p(t)≤Ctαω_{f,p}(t)\le Ct^α with C>0C>0 and 0<α≤10< α\le 1, this estimate yields the geometric bound (2d+1)CN−αℓ(2d+1)C N^{-α\ell}. More generally, if ωf,p(t)ω_{f,p}(t) has a known upper bound that tends to zero as t↓0t\downarrow0, we obtain a stopping rule for adaptive depth selection. We can stop at the first depth where the error bound meets the desired tolerance and later append layers to meet a tighter tolerance while keeping all earlier maps and readouts fixed.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v3,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v3 · 2026-11-16 23:54 · 最新观测版本

Layer-wise Geometric Approximation Rates for Deep Networks

Depth is widely viewed as a central contributor to the success of deep neural networks, yet standard approximation results typically give error bounds only for the final output of a network. Inspired by classical multiscale expansions and multigrade deep learning, we give depth a scale-dependent interpretation by constructing a single network whose intermediate readouts approximate the target at successively finer scales. For any f∈Lp([0,1]d)f\in L^p([0,1]^d) with 1≤p<∞1\le p <\infty and any integer N≥2N\ge2, we construct a single network of width 3dN+2d+43dN+2d+4 with the same activation pattern in every hidden layer, using sine in the first two neurons and ReLU\mathtt{ReLU} in the rest. For every ℓ≥0\ell\ge0, the LpL^p approximation error of the readout ΦℓΦ_\ell at layer ℓ+4\ell+4 is bounded by (2d+1)ωf,p(N−ℓ)(2d+1)ω_{f,p}(N^{-\ell}), where ωf,pω_{f,p} denotes the LpL^p translation modulus of ff. Under the Hölder condition ωf,p(t)≤Ctαω_{f,p}(t)\le Ct^α with C>0C>0 and 0<α≤10< α\le 1, this estimate yields the geometric bound (2d+1)CN−αℓ(2d+1)C N^{-α\ell}. More generally, if ωf,p(t)ω_{f,p}(t) has a known upper bound that tends to zero as t↓0t\downarrow0, we obtain a stopping rule for adaptive depth selection. We can stop at the first depth where the error bound meets the desired tolerance and later append layers to meet a tighter tolerance while keeping all earlier maps and readouts fixed.

查看此版本解读 →查看此版本原文 ↗