Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
1604.03257
Cited By
v1
v2 (latest)
Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization
12 April 2016
Tianbao Yang
Qihang Lin
Zhe Li
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization"
49 / 49 papers shown
Title
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum
H. Cai
Sulaiman A. Alghunaim
Ali H.Sayed
115
1
0
18 Jun 2024
Almost sure convergence rates of stochastic gradient methods under gradient domination
Simon Weissmann
Sara Klein
Waïss Azizian
Leif Döring
93
3
0
22 May 2024
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Hao Sun
Li Shen
Qihuang Zhong
Liang Ding
Shi-Yong Chen
Jingwei Sun
Jing Li
Guangzhong Sun
Dacheng Tao
98
34
0
01 Mar 2023
On Almost Sure Convergence Rates of Stochastic Gradient Methods
Jun Liu
Ye Yuan
97
38
0
09 Feb 2022
Accelerated Gradient Flow: Risk, Stability, and Implicit Regularization
Yue Sheng
Alnur Ali
122
2
0
20 Jan 2022
A Novel Convergence Analysis for Algorithms of the Adam Family
Zhishuai Guo
Yi Tian Xu
W. Yin
Rong Jin
Tianbao Yang
88
49
0
07 Dec 2021
Training Generative Adversarial Networks with Adaptive Composite Gradient
Huiqing Qi
Fang Li
Shengli Tan
Xiangyun Zhang
GAN
51
4
0
10 Nov 2021
EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
Ilyas Fatkhullin
Igor Sokolov
Eduard A. Gorbunov
Zhize Li
Peter Richtárik
129
0
0
07 Oct 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Guojing Cong
Tianyi Liu
110
0
0
01 Oct 2021
Momentum Accelerates the Convergence of Stochastic AUPRC Maximization
Guanghui Wang
Minghao Yang
Lijun Zhang
Tianbao Yang
85
22
0
02 Jul 2021
Escaping Saddle Points Faster with Stochastic Momentum
Jun-Kun Wang
Chi-Heng Lin
Jacob D. Abernethy
ODL
77
22
0
05 Jun 2021
Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems
Stefano Sarao Mannelli
Pierfrancesco Urbani
71
10
0
23 Feb 2021
The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods
Wei Tao
Sheng Long
Gao-wei Wu
Qing Tao
43
14
0
15 Feb 2021
Accelerating Training of Batch Normalization: A Manifold Perspective
Mingyang Yi
49
3
0
08 Jan 2021
Adaptive Gradient Quantization for Data-Parallel SGD
Fartash Faghri
Iman Tabrizian
I. Markov
Dan Alistarh
Daniel M. Roy
Ali Ramezani-Kebrya
MQ
63
83
0
23 Oct 2020
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization
Jun-Kun Wang
Jacob D. Abernethy
110
7
0
04 Oct 2020
Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization
Aaron Defazio
100
23
0
01 Oct 2020
Federated Learning with Nesterov Accelerated Gradient
Zhengjie Yang
Wei Bao
Dong Yuan
Nguyen H. Tran
Albert Y. Zomaya
FedML
68
32
0
18 Sep 2020
Online Algorithms for Estimating Change Rates of Web Pages
Konstantin Avrachenkov
Kishor P. Patil
Gugan Thoppe
56
16
0
17 Sep 2020
Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Data
Qianqian Tong
Guannan Liang
J. Bi
FedML
108
28
0
14 Sep 2020
Understanding and Detecting Convergence for Stochastic Gradient Descent with Momentum
Jerry Chee
Ping Li
45
12
0
27 Aug 2020
Differentially Private Accelerated Optimization Algorithms
Nurdan Kuru
cS. .Ilker Birbil
Mert Gurbuzbalaban
S. Yıldırım
76
23
0
05 Aug 2020
A High Probability Analysis of Adaptive SGD with Momentum
Xiaoyun Li
Francesco Orabona
142
69
0
28 Jul 2020
A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning
Samuel Horváth
Peter Richtárik
79
60
0
19 Jun 2020
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball
Othmane Sebbouh
Robert Mansel Gower
Aaron Defazio
67
23
0
14 Jun 2020
An Analysis of the Adaptation Speed of Causal Models
Rémi Le Priol
Reza Babanezhad Harikandeh
Yoshua Bengio
Simon Lacoste-Julien
CML
58
14
0
18 May 2020
MixML: A Unified Analysis of Weakly Consistent Parallel Learning
Yucheng Lu
J. Nash
Christopher De Sa
FedML
91
12
0
14 May 2020
A Simple Convergence Proof of Adam and Adagrad
Alexandre Défossez
Léon Bottou
Francis R. Bach
Nicolas Usunier
140
159
0
05 Mar 2020
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings
Mahmoud Assran
Michael G. Rabbat
78
59
0
27 Feb 2020
Faster On-Device Training Using New Federated Momentum Algorithm
Zhouyuan Huo
Qian Yang
Bin Gu
Heng-Chiao Huang
FedML
195
47
0
06 Feb 2020
A Rule for Gradient Estimator Selection, with an Application to Variational Inference
Tomas Geffner
Justin Domke
66
6
0
05 Nov 2019
Understanding the Role of Momentum in Stochastic Gradient Methods
Igor Gitman
Hunter Lang
Pengchuan Zhang
Lin Xiao
77
95
0
30 Oct 2019
On the Acceleration of Deep Learning Model Parallelism with Staleness
An Xu
Zhouyuan Huo
Heng-Chiao Huang
65
38
0
05 Sep 2019
The Role of Memory in Stochastic Optimization
Antonio Orvieto
Jonas Köhler
Aurelien Lucchi
94
31
0
02 Jul 2019
Continuous Time Analysis of Momentum Methods
Nikola B. Kovachki
Andrew M. Stuart
131
36
0
10 Jun 2019
Provable Smoothness Guarantees for Black-Box Variational Inference
Justin Domke
74
36
0
24 Jan 2019
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances
Bugra Can
Mert Gurbuzbalaban
Lingjiong Zhu
102
45
0
22 Jan 2019
Stagewise Training Accelerates Convergence of Testing Error Over SGD
Zhuoning Yuan
Yan Yan
Rong Jin
Tianbao Yang
111
11
0
10 Dec 2018
Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learning
Hassan Rafique
Mingrui Liu
Qihang Lin
Tianbao Yang
126
111
0
04 Oct 2018
On the Generalization of Stochastic Gradient Descent with Momentum
Ali Ramezani-Kebrya
Kimon Antonakopoulos
Volkan Cevher
Ashish Khisti
Ben Liang
MLT
73
26
0
12 Sep 2018
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
Dongruo Zhou
Yiqi Tang
Yuan Cao
Ziyan Yang
Quanquan Gu
100
150
0
16 Aug 2018
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
Xiangyi Chen
Sijia Liu
Ruoyu Sun
Mingyi Hong
101
324
0
08 Aug 2018
Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration
Soham De
Anirbit Mukherjee
Enayat Ullah
79
101
0
18 Jul 2018
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods
Nicolas Loizou
Peter Richtárik
80
204
0
27 Dec 2017
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization
Yi Tian Xu
Rong Jin
Tianbao Yang
97
26
0
04 Dec 2017
Linearly convergent stochastic heavy ball method for minimizing generalization error
Nicolas Loizou
Peter Richtárik
131
45
0
30 Oct 2017
Convergence Analysis of Optimization Algorithms
Hyoungseok Kim
Jihoon Kang
Woo-Myoung Park
SukHyun Ko
Yoon-Ho Choi
Daesung Yu
YoungSook Song
JungWon Choi
22
8
0
06 Jul 2017
Stochastic Heavy Ball
S. Gadat
Fabien Panloup
Sofiane Saadane
132
105
0
14 Sep 2016
RSG: Beating Subgradient Method without Smoothness and Strong Convexity
Tianbao Yang
Qihang Lin
172
85
0
09 Dec 2015
1