Acceleration of stochastic gradient descent with momentum by averaging:
finite-sample rates and asymptotic normality

v1v2 (latest)

Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality

28 May 2023

ArXiv (abs)PDF HTML

Papers citing "Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality"

9 / 9 papers shown

Title
On the fast convergence of minibatch heavy ball momentum Raghu Bollapragada Tyler Chen Rachel A. Ward 85 19 0 15 Jun 2022
On the Convergence of mSGD and AdaGrad for Stochastic Optimization Ruinan Jin Yu Xing Xingkang He 51 11 0 26 Jan 2022
Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimization Vien V. Mai M. Johansson 79 56 0 13 Feb 2020
Online Covariance Matrix Estimation in Stochastic Gradient Descent Wanrong Zhu Xi Chen Wei Biao Wu 108 57 0 10 Feb 2020
Understanding the Role of Momentum in Stochastic Gradient Methods Igor Gitman Hunter Lang Pengchuan Zhang Lin Xiao 71 95 0 30 Oct 2019
A Unified Analysis of Stochastic Momentum Methods for Deep Learning Yan Yan Tianbao Yang Zhe Li Qihang Lin Yi Yang 45 120 0 30 Aug 2018
On the insufficiency of existing momentum schemes for Stochastic Optimization Rahul Kidambi Praneeth Netrapalli Prateek Jain Sham Kakade ODL 90 120 0 15 Mar 2018
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods Nicolas Loizou Peter Richtárik 80 203 0 27 Dec 2017
Statistical Inference for Model Parameters in Stochastic Gradient Descent Xi Chen Jason D. Lee Xin T. Tong Yichen Zhang 79 139 0 27 Oct 2016

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.