Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent

28 November 2017

Papers citing "Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent"

43 / 43 papers shown

Title
Langevin Multiplicative Weights Update with Applications in Polynomial Portfolio Management Yi-Hu Feng Tianlin Li Tian Xie 62 0 0 26 Feb 2025
Grams: Gradient Descent with Adaptive Momentum Scaling Yang Cao Xiaoyu Li Zhao-quan Song ODL 98 2 0 22 Dec 2024
Cautious Optimizers: Improving Training with One Line of Code Kaizhao Liang Lizhang Chen B. Liu Qiang Liu ODL 108 5 0 25 Nov 2024
Comparisons Are All You Need for Optimizing Smooth Functions Chenyi Zhang Tongyang Li AAML 37 1 0 19 May 2024
Beyond first-order methods for non-convex non-concave min-max optimization Abhijeet Vyas Brian Bullins 23 1 0 17 Apr 2023
A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization Chuan He Heng Huang Zhaosong Lu 16 1 0 10 Jan 2023
Escaping From Saddle Points Using Asynchronous Coordinate Gradient Descent Marco Bornstein Jin-Peng Liu Jingling Li Furong Huang 21 0 0 17 Nov 2022
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients Hualin Zhang Huan Xiong Bin Gu 32 7 0 04 Oct 2022
On the fast convergence of minibatch heavy ball momentum Raghu Bollapragada Tyler Chen Rachel A. Ward 26 17 0 15 Jun 2022
An Adaptive Gradient Method with Energy and Momentum Hailiang Liu Xuping Tian ODL 18 9 0 23 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 28 8 0 18 Feb 2022
Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization Canzhe Zhao Yanjie Ze Jing Dong Baoxiang Wang Shuai Li 47 4 0 25 Jan 2022
Adaptive Gaussian Process based Stochastic Trajectory Optimization for Motion Planning Yichang Feng Haiyun Zhang Jin Wang Guodong Lu 33 30 0 30 Dec 2021
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 25 15 0 28 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 38 1 0 25 Oct 2021
On the Convergence of Projected Alternating Maximization for Equitable and Optimal Transport Minhui Huang Shiqian Ma Lifeng Lai 29 3 0 29 Sep 2021
Majorization Minimization Methods for Distributed Pose Graph Optimization Taosha Fan Todd D. Murphey 34 18 0 30 Jul 2021
The loss landscape of deep linear neural networks: a second-order analysis E. M. Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 24 9 0 28 Jul 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 16 4 0 21 May 2021
AEGD: Adaptive Gradient Descent with Energy Hailiang Liu Xuping Tian ODL 27 11 0 10 Oct 2020
Learning explanations that are hard to vary Giambattista Parascandolo Alexander Neitz Antonio Orvieto Luigi Gresele Bernhard Schölkopf FAtt 21 178 0 01 Sep 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
Replica Exchange for Non-Convex Optimization Jing-rong Dong Xin T. Tong 19 21 0 23 Jan 2020
Second-Order Guarantees of Stochastic Gradient Descent in Non-Convex Optimization Stefan Vlaski Ali H. Sayed ODL 20 21 0 19 Aug 2019
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies Kaipeng Zhang Alec Koppel Haoqi Zhu Tamer Basar 38 186 0 19 Jun 2019
Why gradient clipping accelerates training: A theoretical justification for adaptivity Junzhe Zhang Tianxing He S. Sra Ali Jadbabaie 30 442 0 28 May 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 21 57 0 28 May 2019
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization Rong Ge Zhize Li Weiyao Wang Xiang Wang 19 33 0 01 May 2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources Yanghua Peng Hang Zhang Yifei Ma Tong He Zhi-Li Zhang Sheng Zha Mu Li 22 23 0 26 Apr 2019
A Deterministic Gradient-Based Approach to Avoid Saddle Points L. Kreusser Stanley J. Osher Bao Wang ODL 26 3 0 21 Jan 2019
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 41 570 0 04 Jul 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 25 146 0 20 Jun 2018
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning Dong Yin Yudong Chen Kannan Ramchandran Peter L. Bartlett FedML 29 97 0 14 Jun 2018
Escaping Saddles with Stochastic Gradients Hadi Daneshmand Jonas Köhler Aurelien Lucchi Thomas Hofmann 21 161 0 15 Mar 2018
Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form Srinadh Bhojanapalli Nicolas Boumal Prateek Jain Praneeth Netrapalli 23 42 0 01 Mar 2018
signSGD: Compressed Optimisation for Non-Convex Problems Jeremy Bernstein Yu-Xiang Wang Kamyar Azizzadenesheli Anima Anandkumar FedML ODL 44 1,019 0 13 Feb 2018
On Symplectic Optimization M. Betancourt Michael I. Jordan Ashia Wilson 16 90 0 10 Feb 2018
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization Yi Tian Xu R. L. Jin Tianbao Yang 32 25 0 04 Dec 2017
Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization Pan Xu Jinghui Chen Difan Zou Quanquan Gu 31 200 0 20 Jul 2017
Stochastic Heavy Ball S. Gadat Fabien Panloup Sofiane Saadane 15 103 0 14 Sep 2016
Quasi-stationary Monte Carlo and the ScaLE Algorithm M. Pollock Paul Fearnhead A. M. Johansen Gareth O. Roberts 31 18 0 12 Sep 2016
A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights Weijie Su Stephen P. Boyd Emmanuel J. Candes 108 1,154 0 04 Mar 2015
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014