How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
Gromov-Wasserstein Discrepancy with Local Differential Privacy for Distributed Structural Graphs Hongwei Jin Xun Chen 31 9 0 01 Feb 2022
$Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the $O(ε^{-7/4})$ Complexity$ Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the $O(ε^{-7/4})$ Complexity Huan Li Zhouchen Lin 42 21 0 27 Jan 2022
Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimization Canzhe Zhao Yanjie Ze Jing Dong Baoxiang Wang Shuai Li 54 4 0 25 Jan 2022
Learning to Predict Gradients for Semi-Supervised Continual Learning Yan Luo Yongkang Wong Mohan S. Kankanhalli Qi Zhao SSL CLL 49 6 0 23 Jan 2022
Learning to Minimize the Remainder in Supervised Learning Yan Luo Yongkang Wong Mohan S. Kankanhalli Qi Zhao 57 1 0 23 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 36 8 0 09 Jan 2022
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 24 1 0 25 Dec 2021
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 31 15 0 28 Nov 2021
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications Khaled B. Letaief Yuanming Shi Jianmin Lu Jianhua Lu 48 417 0 24 Nov 2021
Learning equilibria with personalized incentives in a class of nonmonotone games F. Fabiani Andrea Simonetto Paul Goulart 22 11 0 06 Nov 2021
On the Optimization Landscape of Maximum Mean Discrepancy A. Itai Amir Globerson A. Wiesel 22 1 0 26 Oct 2021
On the Second-order Convergence Properties of Random Search Methods Aurelien Lucchi Antonio Orvieto Adamos Solomou 24 8 0 25 Oct 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 43 1 0 25 Oct 2021
Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization Francis R. Bach Lénaïc Chizat MLT 31 23 0 15 Oct 2021
A Cubic Regularization Approach for Finding Local Minimax Points in Nonconvex Minimax Optimization Ziyi Chen Zhengyang Hu Qunwei Li Zhe Wang Yi Zhou 47 7 0 14 Oct 2021
Finding Second-Order Stationary Points in Nonconvex-Strongly-Concave Minimax Optimization Luo Luo Yujun Li Cheng Chen 14 14 0 10 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 48 11 0 06 Oct 2021
On the Estimation Bias in Double Q-Learning Zhizhou Ren Guangxiang Zhu Haotian Hu Beining Han Jian-Hai Chen Chongjie Zhang 24 17 0 29 Sep 2021
Convergence of a Human-in-the-Loop Policy-Gradient Algorithm With Eligibility Trace Under Reward, Policy, and Advantage Feedback Ishaan Shah D. Halpern Kavosh Asadi Michael L. Littman 25 0 0 15 Sep 2021
Concave Utility Reinforcement Learning with Zero-Constraint Violations Mridul Agarwal Qinbo Bai Vaneet Aggarwal 38 12 0 12 Sep 2021
Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility Liyun Tu Austin Talbot Neil Gallagher David Carlson DRL 34 2 0 09 Sep 2021
Coordinate Descent Methods for DC Minimization: Optimality Conditions and Global Convergence Ganzhao Yuan 33 3 0 09 Sep 2021
Constants of Motion: The Antidote to Chaos in Optimization and Game Dynamics Georgios Piliouras Xiao Wang 39 0 0 08 Sep 2021
The staircase property: How hierarchical structure can guide deep learning Emmanuel Abbe Enric Boix-Adserà Matthew Brennan Guy Bresler Dheeraj M. Nagaraj 25 48 0 24 Aug 2021
Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization Yuetian Luo Xudong Li Anru R. Zhang 33 9 0 03 Aug 2021
The loss landscape of deep linear neural networks: a second-order analysis El Mehdi Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 26 9 0 28 Jul 2021
SGD with a Constant Large Learning Rate Can Converge to Local Maxima Liu Ziyin Botao Li James B. Simon Masakuni Ueda 29 8 0 25 Jul 2021
Taxonomizing local versus global structure in neural network loss landscapes Yaoqing Yang Liam Hodgkinson Ryan Theisen Joe Zou Joseph E. Gonzalez Kannan Ramchandran Michael W. Mahoney 42 37 0 23 Jul 2021
Distributed stochastic optimization with large delays Zhengyuan Zhou P. Mertikopoulos Nicholas Bambos Peter Glynn Yinyu Ye 28 9 0 06 Jul 2021
Provable Convergence of Nesterov's Accelerated Gradient Method for Over-Parameterized Neural Networks Xin Liu Zhisong Pan Wei Tao 22 8 0 05 Jul 2021
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity Arthur Jacot François Ged Berfin cSimcsek Clément Hongler Franck Gabriel 35 52 0 30 Jun 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 42 75 0 28 Jun 2021
Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization Tian-Chun Ye S. Du 23 46 0 27 Jun 2021
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients Miao Zhang Steven W. Su Shirui Pan Xiaojun Chang Ehsan Abbasnejad Reza Haffari 31 68 0 21 Jun 2021
A Survey on Fault-tolerance in Distributed Optimization and Machine Learning Shuo Liu AI4CE OOD 58 13 0 16 Jun 2021
Unique sparse decomposition of low rank matrices Dian Jin Xin Bing Yuqian Zhang 33 4 0 14 Jun 2021
Fast Federated Learning in the Presence of Arbitrary Device Unavailability Xinran Gu Kaixuan Huang Jingzhao Zhang Longbo Huang FedML 35 96 0 08 Jun 2021
Escaping Saddle Points Faster with Stochastic Momentum Jun-Kun Wang Chi-Heng Lin Jacob D. Abernethy ODL 24 22 0 05 Jun 2021
Smooth Bilevel Programming for Sparse Regularization C. Poon Gabriel Peyré 19 18 0 02 Jun 2021
Overparameterization of deep ResNet: zero loss and mean-field analysis Zhiyan Ding Shi Chen Qin Li S. Wright ODL 33 25 0 30 May 2021
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances Berfin cSimcsek François Ged Arthur Jacot Francesco Spadaro Clément Hongler W. Gerstner Johanni Brea AI4CE 41 92 0 25 May 2021
Practical Schemes for Finding Near-Stationary Points of Convex Finite-Sums Kaiwen Zhou Lai Tian Anthony Man-Cho So James Cheng 28 10 0 25 May 2021
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly Yuchen Jin Dinesh Manocha Liangyu Zhao Yibo Zhu Chuanxiong Guo Marco Canini Arvind Krishnamurthy 39 18 0 22 May 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 22 4 0 21 May 2021
Sharp Restricted Isometry Property Bounds for Low-rank Matrix Recovery Problems with Corrupted Measurements Ziye Ma Yingjie Bi Javad Lavaei Somayeh Sojoudi 29 14 0 18 May 2021
Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis Zezhong Zhang Guangxu Zhu Rui Wang Vincent K. N. Lau Kaibin Huang 38 31 0 20 Apr 2021
Noether: The More Things Change, the More Stay the Same Grzegorz Gluch R. Urbanke 22 17 0 12 Apr 2021
Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing Mohammad Mahdi Kamani R. Forsati Jianmin Wang M. Mahdavi FaML 13 11 0 04 Apr 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 32 29 0 31 Mar 2021