Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization

1 October 2020

Papers citing "Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization"

14 / 14 papers shown

Title
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball Othmane Sebbouh Robert Mansel Gower Aaron Defazio 34 22 0 14 Jun 2020
Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimization Vien V. Mai M. Johansson 26 55 0 13 Feb 2020
Momentum Improves Normalized SGD Ashok Cutkosky Harsh Mehta ODL 55 122 0 09 Feb 2020
On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization Hao Yu Rong Jin Sen Yang FedML 61 381 0 09 May 2019
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances Bugra Can Mert Gurbuzbalaban Lingjiong Zhu 46 44 0 22 Jan 2019
On the Curved Geometry of Accelerated Optimization Aaron Defazio 85 25 0 11 Dec 2018
A Unified Analysis of Stochastic Momentum Methods for Deep Learning Yan Yan Tianbao Yang Zhe Li Qihang Lin Yi Yang 20 119 0 30 Aug 2018
On the insufficiency of existing momentum schemes for Stochastic Optimization Rahul Kidambi Praneeth Netrapalli Prateek Jain Sham Kakade ODL 63 118 0 15 Mar 2018
Stochastic Heavy Ball S. Gadat Fabien Panloup Sofiane Saadane 80 102 0 14 Sep 2016
Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization Tianbao Yang Qihang Lin Zhe Li 44 121 0 12 Apr 2016
On the Influence of Momentum Acceleration on Online Learning Kun Yuan Bicheng Ying Ali H. Sayed 49 58 0 14 Mar 2016
From Averaging to Acceleration, There is Only a Step-size Nicolas Flammarion Francis R. Bach 72 138 0 07 Apr 2015
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.0K 39,383 0 01 Sep 2014
Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithm Deanna Needell Nathan Srebro Rachel A. Ward 94 551 0 21 Oct 2013