← 研踪 PaperTrailarXiv:2611.24562 · v1
Taming the Tails of Stochastic Gradient Descent in the Interpolation Regime
首次提交 2026-11-18 23:00
版本更新 2026-11-18 23:00
雷达观测 2026-11-19 15:04
摘要
In this paper, we study stochastic gradient descent in the smooth interpolation regime, where the population objective shares a common minimizer with the sample losses. In contrast to prior work, which has largely focused on in-expectation analysis, we obtain high-probability convergence guarantees under the same assumptions. In the convex case, we prove a fast rate of order after iterations, with failure probability at most . In the strongly convex case, we derive a family of moment-based bounds of order for , and complement them with a matching lower bound showing that the corresponding deterioration in the exponential rate is essentially unavoidable. In both settings, our theoretical results are unified across a broad class of step-size schedules. Building on this analysis, we extend our results to a low-noise regime and to nonconvex objectives satisfying the Polyak-Lojasiewicz condition.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-18 23:00 · 最新观测版本
Taming the Tails of Stochastic Gradient Descent in the Interpolation Regime
In this paper, we study stochastic gradient descent in the smooth interpolation regime, where the population objective shares a common minimizer with the sample losses. In contrast to prior work, which has largely focused on in-expectation analysis, we obtain high-probability convergence guarantees under the same assumptions. In the convex case, we prove a fast rate of order after iterations, with failure probability at most . In the strongly convex case, we derive a family of moment-based bounds of order for , and complement them with a matching lower bound showing that the corresponding deterioration in the exponential rate is essentially unavoidable. In both settings, our theoretical results are unified across a broad class of step-size schedules. Building on this analysis, we extend our results to a low-noise regime and to nonconvex objectives satisfying the Polyak-Lojasiewicz condition.
查看此版本解读 →查看此版本原文 ↗