On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization

8 August 2018

Papers citing "On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization"

23 / 73 papers shown

Title
A General Family of Stochastic Proximal Gradient Methods for Deep Learning Jihun Yun A. Lozano Eunho Yang 24 12 0 15 Jul 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 50 162 0 03 Jul 2020
Taming neural networks with TUSLA: Non-convex learning via adaptive stochastic gradient Langevin algorithms A. Lovas Iosif Lytras Miklós Rásonyi Sotirios Sabanis 25 25 0 25 Jun 2020
Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Bounds Yingxue Zhou Xiangyi Chen Mingyi Hong Zhiwei Steven Wu A. Banerjee 32 25 0 24 Jun 2020
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients Chenfei Zhu Yu Cheng Zhe Gan Furong Huang Jingjing Liu Tom Goldstein ODL 35 2 0 21 Jun 2020
A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning Sijia Liu Pin-Yu Chen B. Kailkhura Gaoyuan Zhang A. Hero III P. Varshney 26 224 0 11 Jun 2020
Bayesian Neural Network via Stochastic Gradient Descent Abhinav Sagar UQCV BDL 18 2 0 04 Jun 2020
Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimization Yangyang Xu Yibo Xu 38 23 0 31 May 2020
MixML: A Unified Analysis of Weakly Consistent Parallel Learning Yucheng Lu J. Nash Christopher De Sa FedML 37 12 0 14 May 2020
Stopping Criteria for, and Strong Convergence of, Stochastic Gradient Descent on Bottou-Curtis-Nocedal Functions V. Patel 23 23 0 01 Apr 2020
A new regret analysis for Adam-type algorithms Ahmet Alacaoglu Yura Malitsky P. Mertikopoulos V. Cevher ODL 48 42 0 21 Mar 2020
LaProp: Separating Momentum and Adaptivity in Adam Liu Ziyin Zhikang T.Wang Masahito Ueda ODL 13 18 0 12 Feb 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 40 168 0 19 Dec 2019
ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization Xiangyi Chen Sijia Liu Kaidi Xu Xingguo Li Xue Lin Mingyi Hong David Cox ODL 19 105 0 15 Oct 2019
On Empirical Comparisons of Optimizers for Deep Learning Dami Choi Christopher J. Shallue Zachary Nado Jaehoon Lee Chris J. Maddison George E. Dahl 41 256 0 11 Oct 2019
DEAM: Adaptive Momentum with Discriminative Weight for Stochastic Optimization Jiyang Bai Yuxiang Ren Jiawei Zhang ODL 23 1 0 25 Jul 2019
Why gradient clipping accelerates training: A theoretical justification for adaptivity J.N. Zhang Tianxing He S. Sra Ali Jadbabaie 30 446 0 28 May 2019
Stochastic Gradient Methods with Block Diagonal Matrix Adaptation Jihun Yun A. Lozano Eunho Yang ODL 17 5 0 26 May 2019
Rapidly Adapting Moment Estimation Guoqiang Zhang Kenta Niwa W. Kleijn ODL 8 0 0 24 Feb 2019
Escaping Saddle Points with Adaptive Gradient Methods Matthew Staib Sashank J. Reddi Satyen Kale Sanjiv Kumar S. Sra ODL 24 73 0 26 Jan 2019
DADAM: A Consensus-based Distributed Adaptive Gradient Method for Online Optimization Parvin Nazari Davoud Ataee Tarzanagh George Michailidis ODL 29 67 0 25 Jan 2019
A Sufficient Condition for Convergences of Adam and RMSProp Fangyu Zou Li Shen Zequn Jie Weizhong Zhang Wei Liu 33 366 0 23 Nov 2018
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks Jinghui Chen Dongruo Zhou Yiqi Tang Ziyan Yang Yuan Cao Quanquan Gu ODL 19 193 0 18 Jun 2018