v1v2v3v4 (latest)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization

16 August 2018

Quanquan Gu

Papers citing "On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization"

31 / 31 papers shown

Title
Attribute Inference Attacks for Federated Regression Tasks Francesco Diana Othmane Marfoq Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML 576 1 0 19 Nov 2024
Conjugate-Gradient-like Based Adaptive Moment Estimation Optimization Algorithm for Deep Learning Jiawu Tian Liwei Xu Xiaowei Zhang Yongqi Li ODL 87 0 0 02 Apr 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 101 13 0 06 Feb 2024
A High Probability Analysis of Adaptive SGD with Momentum Xiaoyun Li Francesco Orabona 123 67 0 28 Jul 2020
A new regret analysis for Adam-type algorithms Ahmet Alacaoglu Yura Malitsky P. Mertikopoulos Volkan Cevher ODL 62 42 0 21 Mar 2020
Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets Mingrui Liu Youssef Mroueh Jerret Ross Wei Zhang Xiaodong Cui Payel Das Tianbao Yang ODL 61 63 0 26 Dec 2019
Simple and optimal high-probability bounds for strongly-convex stochastic gradient descent Nicholas J. A. Harvey Christopher Liaw Sikander Randhawa 39 37 0 02 Sep 2019
On the Convergence of Adam and Beyond Sashank J. Reddi Satyen Kale Surinder Kumar 99 2,504 0 19 Apr 2019
A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm Chi Jin Praneeth Netrapalli Rong Ge Sham Kakade Michael I. Jordan 100 150 0 11 Feb 2019
Tight Analyses for Non-Smooth Stochastic Gradient Descent Nicholas J. A. Harvey Christopher Liaw Y. Plan Sikander Randhawa 55 138 0 13 Dec 2018
A Sufficient Condition for Convergences of Adam and RMSProp Fangyu Zou Li Shen Zequn Jie Weizhong Zhang Wei Liu 61 371 0 23 Nov 2018
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization Xiangyi Chen Sijia Liu Ruoyu Sun Mingyi Hong 58 324 0 08 Aug 2018
Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration Soham De Anirbit Mukherjee Enayat Ullah 64 101 0 18 Jul 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 68 147 0 20 Jun 2018
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks Jinghui Chen Dongruo Zhou Yiqi Tang Ziyan Yang Yuan Cao Quanquan Gu ODL 79 193 0 18 Jun 2018
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes Rachel A. Ward Xiaoxia Wu Léon Bottou ODL 69 367 0 05 Jun 2018
On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes Xiaoyun Li Francesco Orabona 69 297 0 21 May 2018
How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD Zeyuan Allen-Zhu ODL 73 171 0 08 Jan 2018
Natasha 2: Faster Non-Convex Optimization Than SGD Zeyuan Allen-Zhu ODL 79 246 0 29 Aug 2017
Variants of RMSProp and Adagrad with Logarithmic Regret Bounds Mahesh Chandra Mukkamala Matthias Hein ODL 54 258 0 17 Jun 2017
The Marginal Value of Adaptive Gradient Methods in Machine Learning Ashia Wilson Rebecca Roelofs Mitchell Stern Nathan Srebro Benjamin Recht ODL 68 1,032 0 23 May 2017
Natasha: Faster Non-Convex Stochastic Optimization Via Strongly Non-Convex Parameter Zeyuan Allen-Zhu 62 80 0 02 Feb 2017
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 775 36,861 0 25 Aug 2016
Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization Tianbao Yang Qihang Lin Zhe Li 61 122 0 12 Apr 2016
Stochastic Variance Reduction for Nonconvex Optimization Sashank J. Reddi Ahmed S. Hefny S. Sra Barnabás Póczós Alex Smola 101 602 0 19 Mar 2016
Variance Reduction for Faster Non-Convex Optimization Zeyuan Allen-Zhu Elad Hazan ODL 118 392 0 17 Mar 2016
Identity Mappings in Deep Residual Networks Kaiming He Xinming Zhang Shaoqing Ren Jian Sun 354 10,192 0 16 Mar 2016
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.9K 150,260 0 22 Dec 2014
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 122 1,555 0 22 Sep 2013
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 155 6,625 0 22 Dec 2012
Adaptive Bound Optimization for Online Convex Optimization H. B. McMahan Matthew J. Streeter ODL 101 390 0 26 Feb 2010