跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2603.09589 · v2

Memorization capacity of deep ReLU neural networks characterized by width and depth

首次提交 2026-03-10 20:38
版本更新 2026-11-17 19:11
雷达观测 2026-11-18 11:33
arXiv ↗PDF ↗

摘要

This paper studies the memorization capacity of deep neural networks with ReLU activation. Specifically, we investigate the minimal size of such networks to memorize any NN data points in the unit ball with pairwise separation distance δδ and labels from a finite set of size CC. Most prior studies characterize the memorization capacity by the number of parameters or neurons. We extend these results by constructing ReLU networks with adjustable width WW and depth LL satisfying W2L2log⁡W=O(N(log⁡(δ−1)+log⁡C))W^2L^2\log W=\mathcal{O}(N(\log(δ^{-1})+\log C)) that can memorize any such dataset. We also establish width-depth lower bounds that depend explicitly on δδ and CC for networks capable of memorizing all such datasets. When log⁡(δ−1)≍log⁡N\log(δ^{-1})\asymp\log N, our construction achieves W2L2≍NW^2L^2\asymp N for suitable choices of width and depth. This order is optimal if log⁡C≍log⁡N\log C\asymp\log N, and optimal up to logarithmic factors if log⁡C=o(log⁡N)\log C=o(\log N).

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-17 19:11 · 最新观测版本

Memorization capacity of deep ReLU neural networks characterized by width and depth

This paper studies the memorization capacity of deep neural networks with ReLU activation. Specifically, we investigate the minimal size of such networks to memorize any NN data points in the unit ball with pairwise separation distance δδ and labels from a finite set of size CC. Most prior studies characterize the memorization capacity by the number of parameters or neurons. We extend these results by constructing ReLU networks with adjustable width WW and depth LL satisfying W2L2log⁡W=O(N(log⁡(δ−1)+log⁡C))W^2L^2\log W=\mathcal{O}(N(\log(δ^{-1})+\log C)) that can memorize any such dataset. We also establish width-depth lower bounds that depend explicitly on δδ and CC for networks capable of memorizing all such datasets. When log⁡(δ−1)≍log⁡N\log(δ^{-1})\asymp\log N, our construction achieves W2L2≍NW^2L^2\asymp N for suitable choices of width and depth. This order is optimal if log⁡C≍log⁡N\log C\asymp\log N, and optimal up to logarithmic factors if log⁡C=o(log⁡N)\log C=o(\log N).

查看此版本解读 →查看此版本原文 ↗