How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
The Global Landscape of Neural Networks: An Overview Ruoyu Sun Dawei Li Shiyu Liang Tian Ding R. Srikant 22 84 0 02 Jul 2020
Tilted Empirical Risk Minimization Tian Li Ahmad Beirami Maziar Sanjabi Virginia Smith 22 128 0 02 Jul 2020
Optimization Landscape of Tucker Decomposition Abraham Frandsen Rong Ge 25 14 0 29 Jun 2020
Extracting Latent State Representations with Linear Dynamics from Rich Observations Abraham Frandsen Rong Ge 19 2 0 29 Jun 2020
Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum Zeke Xie Xinrui Wang Huishuai Zhang Issei Sato Masashi Sugiyama ODL 37 46 0 29 Jun 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 90 53 0 24 Jun 2020
Greedy Adversarial Equilibrium: An Efficient Alternative to Nonconvex-Nonconcave Min-Max Optimization Oren Mangoubi Nisheeth K. Vishnoi 24 7 0 22 Jun 2020
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems P. Mertikopoulos Nadav Hallak Ali Kavis V. Cevher 30 85 0 19 Jun 2020
Optimization and Generalization of Regularization-Based Continual Learning: a Loss Approximation Viewpoint Dong Yin Mehrdad Farajtabar Ang Li Nir Levine Alex Mott CLL 24 21 0 19 Jun 2020
An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias Lu Yu Krishnakumar Balasubramanian S. Volgushev Murat A. Erdogdu 42 50 0 14 Jun 2020
Evading Curse of Dimensionality in Unconstrained Private GLMs via Private Gradient Descent Shuang Song Thomas Steinke Om Thakkar Abhradeep Thakurta 35 50 0 11 Jun 2020
Recht-Ré Noncommutative Arithmetic-Geometric Mean Conjecture is False Zehua Lai Lek-Heng Lim 12 19 0 02 Jun 2020
Exit Time Analysis for Approximations of Gradient Descent Trajectories Around Saddle Points Rishabh Dixit Mert Gurbuzbalaban W. Bajwa 12 3 0 01 Jun 2020
The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks Itay Safran Gilad Yehudai Ohad Shamir 103 34 0 01 Jun 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 39 275 0 01 Jun 2020
Online non-convex learning for river pollution source identification Wenjie Huang Jing Jiang Xiao Liu 17 3 0 22 May 2020
Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent Tian Tong Cong Ma Yuejie Chi 31 115 0 18 May 2020
Escaping Saddle Points Efficiently with Occupation-Time-Adapted Perturbations Xin Guo Jiequn Han Mahan Tajrobehkar Wenpin Tang 27 2 0 09 May 2020
The critical locus of overparameterized neural networks Y. Cooper UQCV 21 10 0 08 May 2020
Frugal Optimization for Cost-related Hyperparameters Qingyun Wu Chi Wang Silu Huang 16 1 0 04 May 2020
Climate Adaptation: Reliably Predicting from Imbalanced Satellite Data Ruchit Rawal Prabhu Pradhan 28 1 0 26 Apr 2020
Learning Constrained Adaptive Differentiable Predictive Control Policies With Guarantees Ján Drgoňa Aaron Tuor D. Vrabie 14 18 0 23 Apr 2020
Inference by Stochastic Optimization: A Free-Lunch Bootstrap Jean-Jacques Forneron Serena Ng 14 5 0 20 Apr 2020
On Learning Rates and Schrödinger Operators Bin Shi Weijie J. Su Michael I. Jordan 34 60 0 15 Apr 2020
Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model Z. Fan Yi Sun Tianhao Wang Yihong Wu 30 18 0 31 Mar 2020
Second-Order Guarantees in Centralized, Federated and Decentralized Nonconvex Optimization Stefan Vlaski Ali H. Sayed 26 5 0 31 Mar 2020
Nonconvex Matrix Completion with Linearly Parameterized Factors Ji Chen Xiaodong Li Zongming Ma 16 3 0 29 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
Efficient Clustering for Stretched Mixtures: Landscape and Optimality Kaizheng Wang Yuling Yan Mateo Díaz 14 13 0 22 Mar 2020
A Hybrid Model-based and Data-driven Approach to Spectrum Sharing in mmWave Cellular Networks H. S. Ghadikolaei H. Ghauch Gábor Fodor Mikael Skoglund Carlo Fischione 9 14 0 19 Mar 2020
Online Tensor-Based Learning for Multi-Way Data Ali Anaissi Basem Suleiman S. M. Zandavi OOD 49 0 0 10 Mar 2020
Columnwise Element Selection for Computationally Efficient Nonnegative Coupled Matrix Tensor Factorization Thirunavukarasu Balasubramaniam R. Nayak Chau Yuen 16 7 0 07 Mar 2020
Asynchronous and Parallel Distributed Pose Graph Optimization Yulun Tian Alec Koppel Amrit Singh Bedi Jonathan P. How 47 37 0 06 Mar 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 58 1,395 0 29 Feb 2020
First Order Methods take Exponential Time to Converge to Global Minimizers of Non-Convex Functions Krishna Reddy Kesari Jean Honorio 22 1 0 28 Feb 2020
Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions? Ohad Shamir 9 17 0 27 Feb 2020
The Landscape of Matrix Factorization Revisited Hossein Valavi Sulin Liu Peter J. Ramadge 17 5 0 27 Feb 2020
Provable Meta-Learning of Linear Representations Nilesh Tripuraneni Chi Jin Michael I. Jordan OOD 19 188 0 26 Feb 2020
Convergence to Second-Order Stationarity for Non-negative Matrix Factorization: Provably and Concurrently Ioannis Panageas Stratis Skoulakis Antonios Varvitsiotis Tianlin Li 8 2 0 26 Feb 2020
Few-Shot Learning via Learning the Representation, Provably S. Du Wei Hu Sham Kakade Jason D. Lee Qi Lei SSL 12 258 0 21 Feb 2020
Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function Lingkai Kong Molei Tao 20 22 0 14 Feb 2020
Fast Convergence for Langevin Diffusion with Manifold Structure Ankur Moitra Andrej Risteski 27 7 0 13 Feb 2020
A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance Xiaoyun Li Zhenxun Zhuang Francesco Orabona 35 18 0 12 Feb 2020
Understanding Global Loss Landscape of One-hidden-layer ReLU Networks, Part 1: Theory Bo Liu FAtt MLT 29 1 0 12 Feb 2020
Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions J.N. Zhang Hongzhou Lin Stefanie Jegelka Ali Jadbabaie S. Sra 12 44 0 10 Feb 2020
Ill-Posedness and Optimization Geometry for Nonlinear Neural Network Training Thomas O'Leary-Roseberry Omar Ghattas 11 5 0 07 Feb 2020
Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization Thomas O'Leary-Roseberry Nick Alger Omar Ghattas ODL 42 9 0 07 Feb 2020
On the Sample Complexity and Optimization Landscape for Quadratic Feasibility Problems Parth Thaker Gautam Dasarathy Angelia Nedić 24 5 0 04 Feb 2020
Replica Exchange for Non-Convex Optimization Jing-rong Dong Xin T. Tong 29 21 0 23 Jan 2020
Intermittent Pulling with Local Compensation for Communication-Efficient Federated Learning Yining Qi Zhihao Qu Song Guo Xin Gao Ruixuan Li Baoliu Ye FedML 18 8 0 22 Jan 2020