跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.24562 · v1

Taming the Tails of Stochastic Gradient Descent in the Interpolation Regime

首次提交 2026-11-18 23:00
版本更新 2026-11-18 23:00
雷达观测 2026-11-19 15:04
arXiv ↗PDF ↗

摘要

In this paper, we study stochastic gradient descent in the smooth interpolation regime, where the population objective shares a common minimizer with the sample losses. In contrast to prior work, which has largely focused on in-expectation analysis, we obtain high-probability convergence guarantees under the same assumptions. In the convex case, we prove a fast rate of order log⁡(β−1)⋅n−1\log(β^{-1}) \cdot n^{-1} after nn iterations, with failure probability at most ββ. In the strongly convex case, we derive a family of moment-based bounds of order β−p⋅exp⁡(−O(p⋅n))β^{-p} \cdot \exp(-\mathcal{O}(p \cdot n)) for p∈(0,1]p \in (0,1], and complement them with a matching lower bound showing that the corresponding deterioration in the exponential rate is essentially unavoidable. In both settings, our theoretical results are unified across a broad class of step-size schedules. Building on this analysis, we extend our results to a low-noise regime and to nonconvex objectives satisfying the Polyak-Lojasiewicz condition.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-18 23:00 · 最新观测版本

Taming the Tails of Stochastic Gradient Descent in the Interpolation Regime

In this paper, we study stochastic gradient descent in the smooth interpolation regime, where the population objective shares a common minimizer with the sample losses. In contrast to prior work, which has largely focused on in-expectation analysis, we obtain high-probability convergence guarantees under the same assumptions. In the convex case, we prove a fast rate of order log⁡(β−1)⋅n−1\log(β^{-1}) \cdot n^{-1} after nn iterations, with failure probability at most ββ. In the strongly convex case, we derive a family of moment-based bounds of order β−p⋅exp⁡(−O(p⋅n))β^{-p} \cdot \exp(-\mathcal{O}(p \cdot n)) for p∈(0,1]p \in (0,1], and complement them with a matching lower bound showing that the corresponding deterioration in the exponential rate is essentially unavoidable. In both settings, our theoretical results are unified across a broad class of step-size schedules. Building on this analysis, we extend our results to a low-noise regime and to nonconvex objectives satisfying the Polyak-Lojasiewicz condition.

查看此版本解读 →查看此版本原文 ↗