How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamics Xi Chen S. Du Xin T. Tong 34 33 0 30 Apr 2019
Provable Bregman-divergence based Methods for Nonconvex and Non-Lipschitz Problems Qiuwei Li Zhihui Zhu Gongguo Tang M. Wakin 17 26 0 22 Apr 2019
SSRGD: Simple Stochastic Recursive Gradient Descent for Escaping Saddle Points Zhize Li 15 38 0 19 Apr 2019
Convergence rates for the stochastic gradient descent method for non-convex objective functions Benjamin J. Fehrman Benjamin Gess Arnulf Jentzen 19 101 0 02 Apr 2019
Annealing for Distributed Global Optimization Brian Swenson S. Kar H. Vincent Poor J. M. F. Moura 35 30 0 18 Mar 2019
Theory III: Dynamics and Generalization in Deep Networks Andrzej Banburski Q. Liao Brando Miranda Lorenzo Rosasco Fernanda De La Torre Jack Hidary T. Poggio AI4CE 35 3 0 12 Mar 2019
Online Meta-Learning Chelsea Finn Aravind Rajeswaran Sham Kakade Sergey Levine CLL 33 249 0 22 Feb 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 25 22 0 21 Feb 2019
On Nonconvex Optimization for Machine Learning: Gradients, Stochasticity, and Saddle Points Chi Jin Praneeth Netrapalli Rong Ge Sham Kakade Michael I. Jordan 30 61 0 13 Feb 2019
The Complexity of Making the Gradient Small in Stochastic Convex Optimization Dylan J. Foster Ayush Sekhari Ohad Shamir Nathan Srebro Karthik Sridharan Blake E. Woodworth 16 51 0 13 Feb 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 28 148 0 02 Feb 2019
What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? Chi Jin Praneeth Netrapalli Michael I. Jordan 21 82 0 02 Feb 2019
Sharp Analysis for Nonconvex SGD Escaping from Saddle Points Cong Fang Zhouchen Lin Tong Zhang 23 104 0 01 Feb 2019
Stochastic Recursive Variance-Reduced Cubic Regularization Methods Dongruo Zhou Quanquan Gu 19 26 0 31 Jan 2019
Numerically Recovering the Critical Points of a Deep Linear Autoencoder Charles G. Frye Neha S. Wadia M. DeWeese K. Bouchard 30 6 0 29 Jan 2019
Escaping Saddle Points with Adaptive Gradient Methods Matthew Staib Sashank J. Reddi Satyen Kale Sanjiv Kumar S. Sra ODL 14 73 0 26 Jan 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 57 961 0 24 Jan 2019
Perturbed Proximal Descent to Escape Saddle Points for Non-convex and Non-smooth Objective Functions Zhishen Huang Stephen Becker 14 8 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 26 94 0 24 Jan 2019
Trajectory Normalized Gradients for Distributed Optimization Jianqiao Wangni Ke Li Jianbo Shi Jitendra Malik 27 2 0 24 Jan 2019
A Deterministic Gradient-Based Approach to Avoid Saddle Points L. Kreusser Stanley J. Osher Bao Wang ODL 32 3 0 21 Jan 2019
Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent Wenqing Hu Zhanxing Zhu Haoyi Xiong Jun Huan MLT 11 9 0 18 Jan 2019
Sharp Restricted Isometry Bounds for the Inexistence of Spurious Local Minima in Nonconvex Matrix Recovery Richard Y. Zhang Somayeh Sojoudi Javad Lavaei 11 51 0 07 Jan 2019
SGD Converges to Global Minimum in Deep Learning via Star-convex Path Yi Zhou Junjie Yang Huishuai Zhang Yingbin Liang Vahid Tarokh 22 71 0 02 Jan 2019
On the Benefit of Width for Neural Networks: Disappearance of Bad Basins Dawei Li Tian Ding Ruoyu Sun 34 38 0 28 Dec 2018
Sample Efficient Stochastic Variance-Reduced Cubic Regularization Method Dongruo Zhou Pan Xu Quanquan Gu 11 4 0 29 Nov 2018
Blind Over-the-Air Computation and Data Fusion via Provable Wirtinger Flow Jialin Dong Yuanming Shi Z. Ding 17 59 0 12 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 44 1,126 0 09 Nov 2018
A Geometric Approach of Gradient Descent Algorithms in Linear Neural Networks S. Mahabadi Zhenyu Liao Romain Couillet 15 13 0 08 Nov 2018
Global Optimality in Distributed Low-rank Matrix Factorization Zhihui Zhu Qiuwei Li Xinshuo Yang Gongguo Tang Michael B. Wakin 19 4 0 07 Nov 2018
Understanding the Acceleration Phenomenon via High-Resolution Differential Equations Bin Shi S. Du Michael I. Jordan Weijie J. Su 17 254 0 21 Oct 2018
Fault Tolerance in Iterative-Convergent Machine Learning Aurick Qiao Bryon Aragam Bingjing Zhang Eric Xing 26 41 0 17 Oct 2018
Cubic Regularization with Momentum for Nonconvex Optimization Zhe Wang Yi Zhou Yingbin Liang Guanghui Lan 14 26 0 09 Oct 2018
Stein Neural Sampler Tianyang Hu Zixiang Chen Hanxi Sun Jincheng Bai Mao Ye Guang Cheng SyDa GAN 22 34 0 08 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 65 1,252 0 04 Oct 2018
Directional Analysis of Stochastic Gradient Descent via von Mises-Fisher Distributions in Deep learning Cheolhyoung Lee Kyunghyun Cho Wanmo Kang 17 8 0 29 Sep 2018
Stochastic Second-order Methods for Non-convex Optimization with Inexact Hessian and Gradient Liu Liu Xuanqing Liu Cho-Jui Hsieh Dacheng Tao ODL 19 10 0 26 Sep 2018
Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview Yuejie Chi Yue M. Lu Yuxin Chen 39 416 0 25 Sep 2018
Second-order Guarantees of Distributed Gradient Algorithms Amir Daneshmand G. Scutari Vyacheslav Kungurtsev 24 59 0 23 Sep 2018
Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization Bryan Wilder B. Dilkina Milind Tambe OffRL AI4CE 19 292 0 14 Sep 2018
Escaping Saddle Points in Constrained Optimization Aryan Mokhtari Asuman Ozdaglar Ali Jadbabaie 11 53 0 06 Sep 2018
Collapse of Deep and Narrow Neural Nets Lu Lu Yanhui Su George Karniadakis ODL 27 153 0 15 Aug 2018
On the Analysis of Trajectories of Gradient Descent in the Optimization of Deep Neural Networks Adepu Ravi Sankar Vishwak Srinivasan V. Balasubramanian 11 1 0 21 Jul 2018
Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile P. Mertikopoulos Bruno Lecouat Houssam Zenati Chuan-Sheng Foo V. Chandrasekhar Georgios Piliouras 43 292 0 07 Jul 2018
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 50 571 0 04 Jul 2018
On the Implicit Bias of Dropout Poorya Mianjy R. Arora René Vidal 27 66 0 26 Jun 2018
Finding Local Minima via Stochastic Nested Variance Reduction Dongruo Zhou Pan Xu Quanquan Gu 23 23 0 22 Jun 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 25 146 0 20 Jun 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 33 134 0 20 Jun 2018
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning Dong Yin Yudong Chen Kannan Ramchandran Peter L. Bartlett FedML 32 98 0 14 Jun 2018