← 研踪 PaperTrailarXiv:2603.09589 · v2
Memorization capacity of deep ReLU neural networks characterized by width and depth
首次提交 2026-03-10 20:38
版本更新 2026-11-17 19:11
雷达观测 2026-11-18 11:33
摘要
This paper studies the memorization capacity of deep neural networks with ReLU activation. Specifically, we investigate the minimal size of such networks to memorize any data points in the unit ball with pairwise separation distance and labels from a finite set of size . Most prior studies characterize the memorization capacity by the number of parameters or neurons. We extend these results by constructing ReLU networks with adjustable width and depth satisfying that can memorize any such dataset. We also establish width-depth lower bounds that depend explicitly on and for networks capable of memorizing all such datasets. When , our construction achieves for suitable choices of width and depth. This order is optimal if , and optimal up to logarithmic factors if .
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-17 19:11 · 最新观测版本
Memorization capacity of deep ReLU neural networks characterized by width and depth
This paper studies the memorization capacity of deep neural networks with ReLU activation. Specifically, we investigate the minimal size of such networks to memorize any data points in the unit ball with pairwise separation distance and labels from a finite set of size . Most prior studies characterize the memorization capacity by the number of parameters or neurons. We extend these results by constructing ReLU networks with adjustable width and depth satisfying that can memorize any such dataset. We also establish width-depth lower bounds that depend explicitly on and for networks capable of memorizing all such datasets. When , our construction achieves for suitable choices of width and depth. This order is optimal if , and optimal up to logarithmic factors if .
查看此版本解读 →查看此版本原文 ↗