How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
Learning a Single Neuron with Gradient Methods Gilad Yehudai Ohad Shamir MLT 27 63 0 15 Jan 2020
Proximal methods avoid active strict saddles of weakly convex functions Damek Davis Dmitriy Drusvyatskiy 15 3 0 16 Dec 2019
Analysis of the Optimization Landscapes for Overcomplete Representation Learning Qing Qu Yuexiang Zhai Xiao Li Yuqian Zhang Zhihui Zhu 22 9 0 05 Dec 2019
Stationary Points of Shallow Neural Networks with Quadratic Activation Function D. Gamarnik Eren C. Kizildag Ilias Zadik 19 13 0 03 Dec 2019
Error bound of critical points and KL property of exponent $1/2$ for squared F-norm regularized factorization Ting Tao S. Pan Shujun Bi 25 4 0 11 Nov 2019
Optimal Analysis of Subset-Selection Based L_p Low Rank Approximation Chen Dan Hong Wang Hongyang R. Zhang Yuchen Zhou Pradeep Ravikumar 27 16 0 30 Oct 2019
Efficiently avoiding saddle points with zero order methods: No gradients required Lampros Flokas Emmanouil-Vasileios Vlatakis-Gkaragkounis Georgios Piliouras 28 32 0 29 Oct 2019
Online Stochastic Gradient Descent with Arbitrary Initialization Solves Non-smooth, Non-convex Phase Retrieval Yan Shuo Tan Roman Vershynin 22 35 0 28 Oct 2019
Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature Representations Cong Fang Hanze Dong Tong Zhang 28 18 0 25 Oct 2019
Improving the convergence of SGD through adaptive batch sizes Scott Sievert Zachary B. Charles ODL 25 8 0 18 Oct 2019
Splitting Steepest Descent for Growing Neural Architectures Qiang Liu Lemeng Wu Dilin Wang 16 61 0 06 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai Jason D. Lee 24 116 0 03 Oct 2019
Generalization Bounds for Convolutional Neural Networks Shan Lin Jingwei Zhang MLT 30 34 0 03 Oct 2019
Escaping Saddle Points for Zeroth-order Nonconvex Optimization using Estimated Gradient Descent Qinbo Bai Mridul Agarwal Vaneet Aggarwal 21 7 0 03 Oct 2019
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension Yuandong Tian MLT 22 8 0 30 Sep 2019
Mildly Overparametrized Neural Nets can Memorize Training Data Efficiently Rong Ge Runzhe Wang Haoyu Zhao TDI 18 20 0 26 Sep 2019
Dynamics of Deep Neural Networks and Neural Tangent Hierarchy Jiaoyang Huang H. Yau 33 147 0 18 Sep 2019
Quantum algorithm for finding the negative curvature direction in non-convex optimization Kaining Zhang Min-hsiu Hsieh Liu Liu Dacheng Tao 18 3 0 17 Sep 2019
Meta-Learning with Implicit Gradients Aravind Rajeswaran Chelsea Finn Sham Kakade Sergey Levine 51 844 0 10 Sep 2019
Towards Understanding the Importance of Noise in Training Neural Networks Mo Zhou Tianyi Liu Yan Li Dachao Lin Enlu Zhou T. Zhao MLT 26 26 0 07 Sep 2019
Short-and-Sparse Deconvolution -- A Geometric Approach Yenson Lau Qing Qu Han-Wen Kuo Pengcheng Zhou Yuqian Zhang John N. Wright 33 29 0 28 Aug 2019
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization T. Poggio Andrzej Banburski Q. Liao ODL 40 161 0 25 Aug 2019
$KL property of exponent $1/2$ of $\ell_{2,0}$-norm and DC regularized factorizations for low-rank matrix recovery$ KL property of exponent $1/2$ of $\ell_{2,0}$ -norm and DC regularized factorizations for low-rank matrix recovery Shujun Bi Ting Tao S. Pan 23 1 0 24 Aug 2019
Second-Order Guarantees of Stochastic Gradient Descent in Non-Convex Optimization Stefan Vlaski Ali H. Sayed ODL 37 21 0 19 Aug 2019
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 30 57 0 12 Aug 2019
Extending the step-size restriction for gradient descent to avoid strict saddle points Hayden Schaeffer S. McCalla 24 4 0 05 Aug 2019
On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift Alekh Agarwal Sham Kakade Jason D. Lee G. Mahajan 13 316 0 01 Aug 2019
Heavy-ball Algorithms Always Escape Saddle Points Tao Sun Dongsheng Li Zhe Quan Hao Jiang Shengguo Li Y. Dou ODL 20 21 0 23 Jul 2019
Hyperlink Regression via Bregman Divergence Akifumi Okuno Hidetoshi Shimodaira 36 6 0 22 Jul 2019
Distributed Inexact Successive Convex Approximation ADMM: Analysis-Part I Sandeep Kumar K. Rajawat Daniel P. Palomar 32 4 0 21 Jul 2019
Distributed Global Optimization by Annealing Brian Swenson S. Kar H. Vincent Poor José M. F. Moura 25 3 0 20 Jul 2019
Towards Understanding Generalization in Gradient-Based Meta-Learning Simon Guiroy Vikas Verma C. Pal 15 21 0 16 Jul 2019
The Landscape of Non-convex Empirical Risk with Degenerate Population Risk Shuang Li Gongguo Tang M. Wakin MedIm 31 7 0 11 Jul 2019
SNAP: Finding Approximate Second-Order Stationary Solutions Efficiently for Non-convex Linearly Constrained Problems Songtao Lu Meisam Razaviyayn Bo Yang Kejun Huang Mingyi Hong 27 11 0 09 Jul 2019
Learning One-hidden-layer neural networks via Provable Gradient Descent with Random Initialization Shuhao Xia Yuanming Shi ODL MLT 14 0 0 04 Jul 2019
Distributed Learning in Non-Convex Environments -- Part II: Polynomial Escape from Saddle-Points Stefan Vlaski Ali H. Sayed 27 53 0 03 Jul 2019
Distributed Learning in Non-Convex Environments -- Part I: Agreement at a Linear Rate Stefan Vlaski Ali H. Sayed 17 69 0 03 Jul 2019
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies Kaipeng Zhang Alec Koppel Haoqi Zhu Tamer Basar 44 186 0 19 Jun 2019
On the Noisy Gradient Descent that Generalizes as SGD Jingfeng Wu Wenqing Hu Haoyi Xiong Jun Huan Vladimir Braverman Zhanxing Zhu MLT 24 10 0 18 Jun 2019
Escaping from saddle points on Riemannian manifolds Yue Sun Nicolas Flammarion Maryam Fazel 31 71 0 18 Jun 2019
Critical Point Finding with Newton-MR by Analogy to Computing Square Roots Charles G. Frye 14 1 0 12 Jun 2019
Efficiently escaping saddle points on manifolds Christopher Criscitiello Nicolas Boumal 25 62 0 10 Jun 2019
On the Convergence of SARAH and Beyond Bingcong Li Meng Ma G. Giannakis 33 26 0 05 Jun 2019
Global Optimality Guarantees For Policy Gradient Methods Jalaj Bhandari Daniel Russo 39 186 0 05 Jun 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 21 57 0 28 May 2019
Concavifiability and convergence: necessary and sufficient conditions for gradient descent analysis Thulasi Tholeti Sheetal Kalyani MLT 14 0 0 28 May 2019
Robustness of accelerated first-order algorithms for strongly convex optimization problems Hesameddin Mohammadi Meisam Razaviyayn M. Jovanović 17 41 0 27 May 2019
Linear Range in Gradient Descent Angxiu Ni Chaitanya Talnikar ODL 14 2 0 11 May 2019
Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization Baojian Zhou F. Chen Yiming Ying 34 7 0 09 May 2019
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization Rong Ge Zhize Li Weiyao Wang Xiang Wang 19 34 0 01 May 2019