ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 1604.03257
  4. Cited By
Unified Convergence Analysis of Stochastic Momentum Methods for Convex
  and Non-convex Optimization
v1v2 (latest)

Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization

12 April 2016
Tianbao Yang
Qihang Lin
Zhe Li
ArXiv (abs)PDFHTML

Papers citing "Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization"

49 / 49 papers shown
Title
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum
H. Cai
Sulaiman A. Alghunaim
Ali H.Sayed
115
1
0
18 Jun 2024
Almost sure convergence rates of stochastic gradient methods under gradient domination
Almost sure convergence rates of stochastic gradient methods under gradient domination
Simon Weissmann
Sara Klein
Waïss Azizian
Leif Döring
93
3
0
22 May 2024
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning
  Rate and Momentum for Training Deep Neural Networks
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Hao Sun
Li Shen
Qihuang Zhong
Liang Ding
Shi-Yong Chen
Jingwei Sun
Jing Li
Guangzhong Sun
Dacheng Tao
98
34
0
01 Mar 2023
On Almost Sure Convergence Rates of Stochastic Gradient Methods
On Almost Sure Convergence Rates of Stochastic Gradient Methods
Jun Liu
Ye Yuan
97
38
0
09 Feb 2022
Accelerated Gradient Flow: Risk, Stability, and Implicit Regularization
Accelerated Gradient Flow: Risk, Stability, and Implicit Regularization
Yue Sheng
Alnur Ali
122
2
0
20 Jan 2022
A Novel Convergence Analysis for Algorithms of the Adam Family
A Novel Convergence Analysis for Algorithms of the Adam Family
Zhishuai Guo
Yi Tian Xu
W. Yin
Rong Jin
Tianbao Yang
88
49
0
07 Dec 2021
Training Generative Adversarial Networks with Adaptive Composite
  Gradient
Training Generative Adversarial Networks with Adaptive Composite Gradient
Huiqing Qi
Fang Li
Shengli Tan
Xiangyun Zhang
GAN
51
4
0
10 Nov 2021
EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
Ilyas Fatkhullin
Igor Sokolov
Eduard A. Gorbunov
Zhize Li
Peter Richtárik
129
0
0
07 Oct 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization
  with Momentum
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Guojing Cong
Tianyi Liu
110
0
0
01 Oct 2021
Momentum Accelerates the Convergence of Stochastic AUPRC Maximization
Momentum Accelerates the Convergence of Stochastic AUPRC Maximization
Guanghui Wang
Minghao Yang
Lijun Zhang
Tianbao Yang
85
22
0
02 Jul 2021
Escaping Saddle Points Faster with Stochastic Momentum
Escaping Saddle Points Faster with Stochastic Momentum
Jun-Kun Wang
Chi-Heng Lin
Jacob D. Abernethy
ODL
77
22
0
05 Jun 2021
Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic
  High-Dimensional Non-Convex Problems
Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems
Stefano Sarao Mannelli
Pierfrancesco Urbani
71
10
0
23 Feb 2021
The Role of Momentum Parameters in the Optimal Convergence of Adaptive
  Polyak's Heavy-ball Methods
The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods
Wei Tao
Sheng Long
Gao-wei Wu
Qing Tao
43
14
0
15 Feb 2021
Accelerating Training of Batch Normalization: A Manifold Perspective
Accelerating Training of Batch Normalization: A Manifold Perspective
Mingyang Yi
49
3
0
08 Jan 2021
Adaptive Gradient Quantization for Data-Parallel SGD
Adaptive Gradient Quantization for Data-Parallel SGD
Fartash Faghri
Iman Tabrizian
I. Markov
Dan Alistarh
Daniel M. Roy
Ali Ramezani-Kebrya
MQ
63
83
0
23 Oct 2020
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex
  Optimization
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization
Jun-Kun Wang
Jacob D. Abernethy
110
7
0
04 Oct 2020
Momentum via Primal Averaging: Theoretical Insights and Learning Rate
  Schedules for Non-Convex Optimization
Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization
Aaron Defazio
100
23
0
01 Oct 2020
Federated Learning with Nesterov Accelerated Gradient
Federated Learning with Nesterov Accelerated Gradient
Zhengjie Yang
Wei Bao
Dong Yuan
Nguyen H. Tran
Albert Y. Zomaya
FedML
68
32
0
18 Sep 2020
Online Algorithms for Estimating Change Rates of Web Pages
Online Algorithms for Estimating Change Rates of Web Pages
Konstantin Avrachenkov
Kishor P. Patil
Gugan Thoppe
56
16
0
17 Sep 2020
Effective Federated Adaptive Gradient Methods with Non-IID Decentralized
  Data
Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Data
Qianqian Tong
Guannan Liang
J. Bi
FedML
108
28
0
14 Sep 2020
Understanding and Detecting Convergence for Stochastic Gradient Descent
  with Momentum
Understanding and Detecting Convergence for Stochastic Gradient Descent with Momentum
Jerry Chee
Ping Li
45
12
0
27 Aug 2020
Differentially Private Accelerated Optimization Algorithms
Differentially Private Accelerated Optimization Algorithms
Nurdan Kuru
cS. .Ilker Birbil
Mert Gurbuzbalaban
S. Yıldırım
76
23
0
05 Aug 2020
A High Probability Analysis of Adaptive SGD with Momentum
A High Probability Analysis of Adaptive SGD with Momentum
Xiaoyun Li
Francesco Orabona
142
69
0
28 Jul 2020
A Better Alternative to Error Feedback for Communication-Efficient
  Distributed Learning
A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning
Samuel Horváth
Peter Richtárik
79
60
0
19 Jun 2020
Almost sure convergence rates for Stochastic Gradient Descent and
  Stochastic Heavy Ball
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball
Othmane Sebbouh
Robert Mansel Gower
Aaron Defazio
67
23
0
14 Jun 2020
An Analysis of the Adaptation Speed of Causal Models
An Analysis of the Adaptation Speed of Causal Models
Rémi Le Priol
Reza Babanezhad Harikandeh
Yoshua Bengio
Simon Lacoste-Julien
CML
58
14
0
18 May 2020
MixML: A Unified Analysis of Weakly Consistent Parallel Learning
MixML: A Unified Analysis of Weakly Consistent Parallel Learning
Yucheng Lu
J. Nash
Christopher De Sa
FedML
91
12
0
14 May 2020
A Simple Convergence Proof of Adam and Adagrad
A Simple Convergence Proof of Adam and Adagrad
Alexandre Défossez
Léon Bottou
Francis R. Bach
Nicolas Usunier
140
159
0
05 Mar 2020
On the Convergence of Nesterov's Accelerated Gradient Method in
  Stochastic Settings
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings
Mahmoud Assran
Michael G. Rabbat
78
59
0
27 Feb 2020
Faster On-Device Training Using New Federated Momentum Algorithm
Faster On-Device Training Using New Federated Momentum Algorithm
Zhouyuan Huo
Qian Yang
Bin Gu
Heng-Chiao Huang
FedML
195
47
0
06 Feb 2020
A Rule for Gradient Estimator Selection, with an Application to
  Variational Inference
A Rule for Gradient Estimator Selection, with an Application to Variational Inference
Tomas Geffner
Justin Domke
66
6
0
05 Nov 2019
Understanding the Role of Momentum in Stochastic Gradient Methods
Understanding the Role of Momentum in Stochastic Gradient Methods
Igor Gitman
Hunter Lang
Pengchuan Zhang
Lin Xiao
77
95
0
30 Oct 2019
On the Acceleration of Deep Learning Model Parallelism with Staleness
On the Acceleration of Deep Learning Model Parallelism with Staleness
An Xu
Zhouyuan Huo
Heng-Chiao Huang
65
38
0
05 Sep 2019
The Role of Memory in Stochastic Optimization
The Role of Memory in Stochastic Optimization
Antonio Orvieto
Jonas Köhler
Aurelien Lucchi
94
31
0
02 Jul 2019
Continuous Time Analysis of Momentum Methods
Continuous Time Analysis of Momentum Methods
Nikola B. Kovachki
Andrew M. Stuart
131
36
0
10 Jun 2019
Provable Smoothness Guarantees for Black-Box Variational Inference
Provable Smoothness Guarantees for Black-Box Variational Inference
Justin Domke
74
36
0
24 Jan 2019
Accelerated Linear Convergence of Stochastic Momentum Methods in
  Wasserstein Distances
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances
Bugra Can
Mert Gurbuzbalaban
Lingjiong Zhu
102
45
0
22 Jan 2019
Stagewise Training Accelerates Convergence of Testing Error Over SGD
Stagewise Training Accelerates Convergence of Testing Error Over SGD
Zhuoning Yuan
Yan Yan
Rong Jin
Tianbao Yang
111
11
0
10 Dec 2018
Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and
  Applications in Machine Learning
Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learning
Hassan Rafique
Mingrui Liu
Qihang Lin
Tianbao Yang
126
111
0
04 Oct 2018
On the Generalization of Stochastic Gradient Descent with Momentum
On the Generalization of Stochastic Gradient Descent with Momentum
Ali Ramezani-Kebrya
Kimon Antonakopoulos
Volkan Cevher
Ashish Khisti
Ben Liang
MLT
73
26
0
12 Sep 2018
On the Convergence of Adaptive Gradient Methods for Nonconvex
  Optimization
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
Dongruo Zhou
Yiqi Tang
Yuan Cao
Ziyan Yang
Quanquan Gu
100
150
0
16 Aug 2018
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex
  Optimization
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
Xiangyi Chen
Sijia Liu
Ruoyu Sun
Mingyi Hong
101
324
0
08 Aug 2018
Convergence guarantees for RMSProp and ADAM in non-convex optimization
  and an empirical comparison to Nesterov acceleration
Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration
Soham De
Anirbit Mukherjee
Enayat Ullah
79
101
0
18 Jul 2018
Momentum and Stochastic Momentum for Stochastic Gradient, Newton,
  Proximal Point and Subspace Descent Methods
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods
Nicolas Loizou
Peter Richtárik
80
204
0
27 Dec 2017
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature
  for Non-Convex Optimization
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization
Yi Tian Xu
Rong Jin
Tianbao Yang
97
26
0
04 Dec 2017
Linearly convergent stochastic heavy ball method for minimizing
  generalization error
Linearly convergent stochastic heavy ball method for minimizing generalization error
Nicolas Loizou
Peter Richtárik
131
45
0
30 Oct 2017
Convergence Analysis of Optimization Algorithms
Convergence Analysis of Optimization Algorithms
Hyoungseok Kim
Jihoon Kang
Woo-Myoung Park
SukHyun Ko
Yoon-Ho Choi
Daesung Yu
YoungSook Song
JungWon Choi
22
8
0
06 Jul 2017
Stochastic Heavy Ball
Stochastic Heavy Ball
S. Gadat
Fabien Panloup
Sofiane Saadane
132
105
0
14 Sep 2016
RSG: Beating Subgradient Method without Smoothness and Strong Convexity
RSG: Beating Subgradient Method without Smoothness and Strong Convexity
Tianbao Yang
Qihang Lin
172
85
0
09 Dec 2015
1