How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
On the Stability of Nonlinear Receding Horizon Control: A Geometric Perspective T. Westenbroek Max Simchowitz Michael I. Jordan S. Shankar Sastry 17 9 0 27 Mar 2021
Escaping Saddle Points in Distributed Newton's Method with Communication Efficiency and Byzantine Resilience Avishek Ghosh R. Maity A. Mazumdar Kannan Ramchandran FedML 22 5 0 17 Mar 2021
Escaping Saddle Points with Stochastically Controlled Stochastic Gradient Methods Guannan Liang Qianqian Tong Chunjiang Zhu J. Bi 33 3 0 07 Mar 2021
Acceleration via Fractal Learning Rate Schedules Naman Agarwal Surbhi Goel Cyril Zhang 24 18 0 01 Mar 2021
Noisy Truncated SGD: Optimization and Generalization Yingxue Zhou Xinyan Li A. Banerjee 19 3 0 26 Feb 2021
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li Sadhika Malladi Sanjeev Arora 49 78 0 24 Feb 2021
Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization Tianyi Liu Yan Li S. Wei Enlu Zhou T. Zhao 21 13 0 24 Feb 2021
WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points Albert No Taeho Yoon Sehyun Kwon Ernest K. Ryu GAN 27 2 0 15 Feb 2021
Stochastic Gradient Langevin Dynamics with Variance Reduction Zhishen Huang Stephen Becker 17 7 0 12 Feb 2021
Lazy OCO: Online Convex Optimization on a Switching Budget Uri Sherman Tomer Koren 24 15 0 07 Feb 2021
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning Tomoya Murata Taiji Suzuki FedML 27 50 0 05 Feb 2021
Sign-RIP: A Robust Restricted Isometry Property for Low-rank Matrix Recovery Jianhao Ma S. Fattahi 18 12 0 05 Feb 2021
Escaping Saddle Points for Nonsmooth Weakly Convex Functions via Perturbed Proximal Algorithms Minhui Huang 24 6 0 04 Feb 2021
Simulated annealing from continuum to discretization: a convergence analysis via the Eyring--Kramers law Wenpin Tang X. Zhou 28 9 0 03 Feb 2021
On the Differentially Private Nature of Perturbed Gradient Descent Thulasi Tholeti Sheetal Kalyani 19 1 0 18 Jan 2021
Efficient Semi-Implicit Variational Inference Vincent Moens Hang Ren A. Maraval Rasul Tutunov Jun Wang H. Ammar 85 6 0 15 Jan 2021
The Nonconvex Geometry of Linear Inverse Problems Armin Eftekhari Peyman Mohajerin Esfahani 31 1 0 07 Jan 2021
Boundary Conditions for Linear Exit Time Gradient Trajectories Around Saddle Points: Analysis and Algorithm Rishabh Dixit Mert Gurbuzbalaban W. Bajwa 16 1 0 07 Jan 2021
Fast Global Convergence for Low-rank Matrix Recovery via Riemannian Gradient Descent with Random Initialization T. Hou Zhenzhen Li Ziyun Zhang 34 18 0 31 Dec 2020
Stochastic Approximation for Online Tensorial Independent Component Analysis C. J. Li Michael I. Jordan 30 2 0 28 Dec 2020
Byzantine-Resilient Non-Convex Stochastic Gradient Descent Zeyuan Allen-Zhu Faeze Ebrahimian Jingkai Li Dan Alistarh FedML 22 68 0 28 Dec 2020
Mathematical Models of Overparameterized Neural Networks Cong Fang Hanze Dong Tong Zhang 40 22 0 27 Dec 2020
Regularization in network optimization via trimmed stochastic gradient descent with noisy label Kensuke Nakamura Bong-Soo Sohn Kyoung-Jae Won Byung-Woo Hong NoLa 15 0 0 21 Dec 2020
On Duality Gap as a Measure for Monitoring GAN Training Sahil Sidheekh Aroof Aimen Vineet Madan N. C. Krishnan 6 5 0 12 Dec 2020
Recent Theoretical Advances in Non-Convex Optimization Marina Danilova Pavel Dvurechensky Alexander Gasnikov Eduard A. Gorbunov Sergey Guminov Dmitry Kamzolov Innokentiy Shibaev 38 77 0 11 Dec 2020
Notes on Deep Learning Theory Eugene Golikov VLM AI4CE 16 2 0 10 Dec 2020
Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance Anas Barakat Pascal Bianchi W. Hachem S. Schechtman 39 13 0 07 Dec 2020
Learning Graph Neural Networks with Approximate Gradient Descent Qunwei Li Shaofeng Zou Leon Wenliang Zhong GNN 37 1 0 07 Dec 2020
Characterization of Excess Risk for Locally Strongly Convex Population Risk Mingyang Yi Ruoyu Wang Zhi-Ming Ma 22 2 0 04 Dec 2020
Sample Complexity of Policy Gradient Finding Second-Order Stationary Points Long Yang Qian Zheng Gang Pan 33 21 0 02 Dec 2020
Adam $^+$ : A Stochastic Method with Adaptive Variance Reduction Mingrui Liu Wei Zhang Francesco Orabona Tianbao Yang 19 27 0 24 Nov 2020
SALR: Sharpness-aware Learning Rate Scheduler for Improved Generalization Xubo Yue Maher Nouiehed Raed Al Kontar ODL 22 4 0 10 Nov 2020
Escape saddle points faster on manifolds via perturbed Riemannian stochastic recursive gradient Andi Han Junbin Gao 22 5 0 23 Oct 2020
Deep Neural Networks Are Congestion Games: From Loss Landscape to Wardrop Equilibrium and Beyond Nina Vesseron I. Redko Charlotte Laclau 33 5 0 21 Oct 2020
Towards Understanding the Dynamics of the First-Order Adversaries Zhun Deng Hangfeng He Jiaoyang Huang Weijie J. Su AAML 25 11 0 20 Oct 2020
The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers Preetum Nakkiran Behnam Neyshabur Hanie Sedghi OffRL 29 11 0 16 Oct 2020
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization Jun-Kun Wang Jacob D. Abernethy 24 7 0 04 Oct 2020
BAMSProd: A Step towards Generalizing the Adaptive Optimization Methods to Deep Binary Model Junjie Liu Dongchao Wen Deyu Wang Wei Tao Tse-Wei Chen Kinya Osa Masami Kato MQ 29 1 0 29 Sep 2020
Escaping Saddle-Points Faster under Interpolation-like Conditions Abhishek Roy Krishnakumar Balasubramanian Saeed Ghadimi P. Mohapatra 17 1 0 28 Sep 2020
The Complexity of Constrained Min-Max Optimization C. Daskalakis Stratis Skoulakis Manolis Zampetakis 22 136 0 21 Sep 2020
Alternating Direction Method of Multipliers for Quantization Tianjian Huang Prajwal Singhania Maziar Sanjabi Pabitra Mitra Meisam Razaviyayn MQ 30 10 0 08 Sep 2020
S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima Wonyong Sung Iksoo Choi Jinhwan Park Seokhyun Choi Sungho Shin ODL 30 7 0 05 Sep 2020
$Column $\ell_{2,0}$-norm regularized factorization model of low-rank matrix recovery and its computation$ Column $\ell_{2,0}$ -norm regularized factorization model of low-rank matrix recovery and its computation Ting Tao Yitian Qian S. Pan 40 2 0 24 Aug 2020
Notes on Worst-case Inefficiency of Gradient Descent Even in R^2 Shiliang Zuo 14 0 0 17 Aug 2020
Distributed Gradient Flow: Nonsmoothness, Nonconvexity, and Saddle Point Evasion Brian Swenson Ryan W. Murray H. Vincent Poor S. Kar 22 16 0 12 Aug 2020
Binary Search and First Order Gradient Based Method for Stochastic Optimization V. Pandey ODL 11 0 0 27 Jul 2020
Quantum algorithms for escaping from saddle points Chenyi Zhang Jiaqi Leng Tongyang Li 19 19 0 20 Jul 2020
From Symmetry to Geometry: Tractable Nonconvex Problems Yuqian Zhang Qing Qu John N. Wright 34 43 0 14 Jul 2020
Regularized linear autoencoders recover the principal components, eventually Xuchan Bao James Lucas Sushant Sachdeva Roger C. Grosse 47 29 0 13 Jul 2020
Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH) Yuqing Li Yaoyu Zhang N. Yip 13 5 0 07 Jul 2020