How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
Deep Neural Networks with Multi-Branch Architectures Are Less Non-Convex Hongyang R. Zhang Junru Shao Ruslan Salakhutdinov 39 14 0 06 Jun 2018
Structured Local Optima in Sparse Blind Deconvolution Yuqian Zhang Han-Wen Kuo John N. Wright 26 56 0 01 Jun 2018
Understanding Generalization and Optimization Performance of Deep CNNs Pan Zhou Jiashi Feng MLT 33 48 0 28 May 2018
Multichannel Sparse Blind Deconvolution on the Sphere Yanjun Li Y. Bresler 17 16 0 26 May 2018
How Much Restricted Isometry is Needed In Nonconvex Matrix Recovery? Richard Y. Zhang C. Josz Somayeh Sojoudi Javad Lavaei 22 42 0 25 May 2018
Adaptive Stochastic Gradient Langevin Dynamics: Taming Convergence and Saddle Point Escape Time Hejian Sang Jia-Wei Liu ODL 21 1 0 23 May 2018
Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps S. Du Surbhi Goel MLT 30 17 0 20 May 2018
The Global Optimization Geometry of Shallow Linear Neural Networks Zhihui Zhu Daniel Soudry Yonina C. Eldar M. Wakin ODL 23 36 0 13 May 2018
Are ResNets Provably Better than Linear Predictors? Ohad Shamir 19 54 0 18 Apr 2018
On Gradient-Based Learning in Continuous Games Eric Mazumdar Lillian J. Ratliff S. Shankar Sastry 27 134 0 16 Apr 2018
Stability and Convergence Trade-off of Iterative Optimization Algorithms Yuansi Chen Chi Jin Bin Yu 8 93 0 04 Apr 2018
Tight Query Complexity Lower Bounds for PCA via Finite Sample Deformed Wigner Law Max Simchowitz A. Alaoui Benjamin Recht 35 39 0 04 Apr 2018
Non-Convex Matrix Completion Against a Semi-Random Adversary Yu Cheng Rong Ge AAML 36 24 0 28 Mar 2018
On the Local Minima of the Empirical Risk Chi Jin Lydia T. Liu Rong Ge Michael I. Jordan FedML 16 56 0 25 Mar 2018
Bayesian Optimization with Expensive Integrands Saul Toscano-Palmerin P. Frazier 27 49 0 23 Mar 2018
Gradient Descent with Random Initialization: Fast Global Convergence for Nonconvex Phase Retrieval Yuxin Chen Yuejie Chi Jianqing Fan Cong Ma 6 235 0 21 Mar 2018
Escaping Saddles with Stochastic Gradients Hadi Daneshmand Jonas Köhler Aurelien Lucchi Thomas Hofmann 29 162 0 15 Mar 2018
How to Start Training: The Effect of Initialization and Architecture Boris Hanin David Rolnick 19 253 0 05 Mar 2018
Fast and Sample Efficient Inductive Matrix Completion via Multi-Phase Procrustes Flow Xiao Zhang S. Du Quanquan Gu 26 24 0 03 Mar 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation S. Du Jason D. Lee 27 267 0 03 Mar 2018
The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects Zhanxing Zhu Jingfeng Wu Ting Yu Lei Wu Jin Ma 11 40 0 01 Mar 2018
Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form Srinadh Bhojanapalli Nicolas Boumal Prateek Jain Praneeth Netrapalli 29 44 0 01 Mar 2018
On the Sublinear Convergence of Randomly Perturbed Alternating Gradient Descent to Second Order Stationary Solutions Songtao Lu Mingyi Hong Zhengdao Wang 13 4 0 28 Feb 2018
Harnessing Structures in Big Data via Guaranteed Low-Rank Matrix Estimation Yudong Chen Yuejie Chi 18 171 0 23 Feb 2018
Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization Zhe Wang Yi Zhou Yingbin Liang Guanghui Lan 35 47 0 20 Feb 2018
An Alternative View: When Does SGD Escape Local Minima? Robert D. Kleinberg Yuanzhi Li Yang Yuan MLT 14 314 0 17 Feb 2018
Stochastic Variance-Reduced Cubic Regularized Newton Method Dongruo Zhou Pan Xu Quanquan Gu ODL 16 45 0 13 Feb 2018
Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator Maryam Fazel Rong Ge Sham Kakade M. Mesbahi 35 597 0 15 Jan 2018
Generalization Error Bounds for Noisy, Iterative Algorithms Ankit Pensia Varun Jog Po-Ling Loh 29 109 0 12 Jan 2018
Non-convex Optimization for Machine Learning Prateek Jain Purushottam Kar 33 479 0 21 Dec 2017
Third-order Smoothness Helps: Even Faster Stochastic Optimization Algorithms for Finding Local Minima Yaodong Yu Pan Xu Quanquan Gu 11 3 0 18 Dec 2017
Saving Gradient and Negative Curvature Computations: Finding Local Minima More Efficiently Yaodong Yu Difan Zou Quanquan Gu 21 10 0 11 Dec 2017
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization Yi Tian Xu Rong Jin Tianbao Yang 35 25 0 04 Dec 2017
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima S. Du Jason D. Lee Yuandong Tian Barnabás Póczós Aarti Singh MLT 32 235 0 03 Dec 2017
Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent Chi Jin Praneeth Netrapalli Michael I. Jordan ODL 37 261 0 28 Nov 2017
Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms Yazhen Wang 16 17 0 27 Nov 2017
Run-and-Inspect Method for Nonconvex Optimization and Global Optimality Bounds for R-Local Minimizers Yifan Chen Yuejiao Sun W. Yin 30 5 0 22 Nov 2017
Neon2: Finding Local Minima via First-Order Oracles Zeyuan Allen-Zhu Yuanzhi Li 21 130 0 17 Nov 2017
Stochastic Cubic Regularization for Fast Nonconvex Optimization Nilesh Tripuraneni Mitchell Stern Chi Jin Jeffrey Regier Michael I. Jordan 27 173 0 08 Nov 2017
Model-free Nonconvex Matrix Completion: Local Minima Analysis and Applications in Memory-efficient Kernel PCA Ji Chen Xiaodong Li 48 27 0 06 Nov 2017
Conditional Gradient Method for Stochastic Submodular Maximization: Closing the Gap Aryan Mokhtari S. Hassani Amin Karbasi 27 70 0 05 Nov 2017
First-order Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Time Yi Tian Xu Rong Jin Tianbao Yang ODL 27 116 0 03 Nov 2017
Learning One-hidden-layer Neural Networks with Landscape Design Rong Ge Jason D. Lee Tengyu Ma MLT 29 260 0 01 Nov 2017
Gradient Sparsification for Communication-Efficient Distributed Optimization Jianqiao Wangni Jialei Wang Ji Liu Tong Zhang 15 522 0 26 Oct 2017
First-order Methods Almost Always Avoid Saddle Points Jason D. Lee Ioannis Panageas Georgios Piliouras Max Simchowitz Michael I. Jordan Benjamin Recht ODL 95 83 0 20 Oct 2017
The Scaling Limit of High-Dimensional Online Independent Component Analysis Chuang Wang Yue M. Lu 6 13 0 15 Oct 2017
On Noisy Negative Curvature Descent: Competing with Gradient Descent for Faster Non-convex Optimization Mingrui Liu Tianbao Yang 38 23 0 25 Sep 2017
Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descent Yuanxin Li Yuejie Chi Huishuai Zhang Yingbin Liang 24 29 0 23 Sep 2017
When is a Convolutional Filter Easy To Learn? S. Du Jason D. Lee Yuandong Tian MLT 15 130 0 18 Sep 2017
The Impact of Local Geometry and Batch Size on Stochastic Gradient Descent for Nonconvex Problems V. Patel MLT 28 8 0 14 Sep 2017