Escaping Saddle Points with Adaptive Gradient Methods

v1v2 (latest)

Escaping Saddle Points with Adaptive Gradient Methods

26 January 2019

Sashank J. Reddi

Sanjiv Kumar

ArXiv (abs)PDF HTML

Papers citing "Escaping Saddle Points with Adaptive Gradient Methods"

11 / 11 papers shown

Title
Particle Semi-Implicit Variational Inference Jen Ning Lim A. M. Johansen 93 7 0 30 Jun 2024
Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks Matteo Tucat Anirbit Mukherjee Procheta Sen Mingfei Sun Omar Rivasplata MLT 68 1 0 12 Apr 2024
Sharp Analysis for Nonconvex SGD Escaping from Saddle Points Cong Fang Zhouchen Lin Tong Zhang 71 104 0 01 Feb 2019
A Sufficient Condition for Convergences of Adam and RMSProp Fangyu Zou Li Shen Zequn Jie Weizhong Zhang Wei Liu 59 371 0 23 Nov 2018
Escaping Saddle Points in Constrained Optimization Aryan Mokhtari Asuman Ozdaglar Ali Jadbabaie 58 53 0 06 Sep 2018
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization Dongruo Zhou Yiqi Tang Yuan Cao Ziyan Yang Quanquan Gu 57 151 0 16 Aug 2018
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization Xiangyi Chen Sijia Liu Ruoyu Sun Mingyi Hong 58 323 0 08 Aug 2018
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 78 1,048 0 11 Apr 2018
Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients Lukas Balles Philipp Hennig 75 169 0 22 May 2017
Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition Rong Ge Furong Huang Chi Jin Yang Yuan 140 1,058 0 06 Mar 2015
An Introduction to Matrix Concentration Inequalities J. Tropp 168 1,154 0 07 Jan 2015