Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization

12 April 2016

Papers citing "Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization"

32 / 32 papers shown

Title
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum H. Cai Sulaiman A. Alghunaim Ali H.Sayed 52 1 0 18 Jun 2024
Almost sure convergence rates of stochastic gradient methods under gradient domination Simon Weissmann Sara Klein Waïss Azizian Leif Döring 39 3 0 22 May 2024
From Optimization to Control: Quasi Policy Iteration Mohammad Amin Sharifi Kolarijani Peyman Mohajerin Esfahani 32 2 0 18 Nov 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 26 0 0 19 Oct 2023
Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality Kejie Tang Weidong Liu Yichen Zhang Xi Chen 21 2 0 28 May 2023
Differentiable Arbitrating in Zero-sum Markov Games Jing Wang Meichen Song Feng Gao Boyi Liu Zhaoran Wang Yi Wu 43 2 0 20 Feb 2023
On Almost Sure Convergence Rates of Stochastic Gradient Methods Jun Liu Ye Yuan 21 36 0 09 Feb 2022
A Novel Convergence Analysis for Algorithms of the Adam Family Zhishuai Guo Yi Tian Xu W. Yin Rong Jin Tianbao Yang 39 47 0 07 Dec 2021
Training Generative Adversarial Networks with Adaptive Composite Gradient Huiqing Qi Fang Li Shengli Tan Xiangyun Zhang GAN 29 3 0 10 Nov 2021
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 46 46 0 07 Oct 2021
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum Guojing Cong Tianyi Liu 16 0 0 01 Oct 2021
Momentum Accelerates the Convergence of Stochastic AUPRC Maximization Guanghui Wang Minghao Yang Lijun Zhang Tianbao Yang 41 22 0 02 Jul 2021
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization Jun-Kun Wang Jacob D. Abernethy 11 7 0 04 Oct 2020
Federated Learning with Nesterov Accelerated Gradient Zhengjie Yang Wei Bao Dong Yuan Nguyen H. Tran Albert Y. Zomaya FedML 19 29 0 18 Sep 2020
Online Algorithms for Estimating Change Rates of Web Pages Konstantin Avrachenkov Kishor P. Patil Gugan Thoppe 6 16 0 17 Sep 2020
Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Data Qianqian Tong Guannan Liang J. Bi FedML 41 27 0 14 Sep 2020
A High Probability Analysis of Adaptive SGD with Momentum Xiaoyun Li Francesco Orabona 92 66 0 28 Jul 2020
Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball Othmane Sebbouh Robert Mansel Gower Aaron Defazio 8 22 0 14 Jun 2020
MixML: A Unified Analysis of Weakly Consistent Parallel Learning Yucheng Lu J. Nash Christopher De Sa FedML 32 12 0 14 May 2020
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings Mahmoud Assran Michael G. Rabbat 14 59 0 27 Feb 2020
Faster On-Device Training Using New Federated Momentum Algorithm Zhouyuan Huo Qian Yang Bin Gu Heng-Chiao Huang FedML 22 47 0 06 Feb 2020
Understanding the Role of Momentum in Stochastic Gradient Methods Igor Gitman Hunter Lang Pengchuan Zhang Lin Xiao 33 94 0 30 Oct 2019
Continuous Time Analysis of Momentum Methods Nikola B. Kovachki Andrew M. Stuart 23 33 0 10 Jun 2019
Rapidly Adapting Moment Estimation Guoqiang Zhang Kenta Niwa W. Kleijn ODL 8 0 0 24 Feb 2019
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances Bugra Can Mert Gurbuzbalaban Lingjiong Zhu 11 45 0 22 Jan 2019
Stagewise Training Accelerates Convergence of Testing Error Over SGD Zhuoning Yuan Yan Yan Rong Jin Tianbao Yang 60 11 0 10 Dec 2018
Weakly-Convex Concave Min-Max Optimization: Provable Algorithms and Applications in Machine Learning Hassan Rafique Mingrui Liu Qihang Lin Tianbao Yang 15 107 0 04 Oct 2018
On the Generalization of Stochastic Gradient Descent with Momentum Ali Ramezani-Kebrya Kimon Antonakopoulos V. Cevher Ashish Khisti Ben Liang MLT 14 23 0 12 Sep 2018
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods Nicolas Loizou Peter Richtárik 19 200 0 27 Dec 2017
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization Yi Tian Xu Rong Jin Tianbao Yang 35 25 0 04 Dec 2017
Stochastic Heavy Ball S. Gadat Fabien Panloup Sofiane Saadane 18 103 0 14 Sep 2016
RSG: Beating Subgradient Method without Smoothness and Strong Convexity Tianbao Yang Qihang Lin 32 84 0 09 Dec 2015