← 研踪 PaperTrailarXiv:2611.14811 · v1
Improved last-iterate bounds for SGD with and without replacement
首次提交 2026-11-11 06:54
版本更新 2026-11-11 06:54
雷达观测 2026-11-13 10:09
摘要
In this work, we study last-iterate convergence bounds for stochastic gradient descent (SGD) on convex and -smooth loss functions and improve the bound from $\cO((\log T)^3/T)$ to $\cO((\log T)^3/(T\log\log T))$ in the interpolation regime. The improvement comes from applying a horizon-dependent multiscale estimate of gradients to the comparison of trajectories. The bound applies to both sampling with replacement and sampling without replacement within one pass.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-11 06:54 · 最新观测版本
Improved last-iterate bounds for SGD with and without replacement
In this work, we study last-iterate convergence bounds for stochastic gradient descent (SGD) on convex and -smooth loss functions and improve the bound from $\cO((\log T)^3/T)$ to $\cO((\log T)^3/(T\log\log T))$ in the interpolation regime. The improvement comes from applying a horizon-dependent multiscale estimate of gradients to the comparison of trajectories. The bound applies to both sampling with replacement and sampling without replacement within one pass.
查看此版本解读 →查看此版本原文 ↗