How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points R. Tappenden Martin Takáč 18 0 0 02 Nov 2022
Local Model Reconstruction Attacks in Federated Learning and their Uses Ilias Driouich Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML FedML 38 2 0 28 Oct 2022
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 170 68 0 27 Oct 2022
Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies Bin Hu Kaipeng Zhang Na Li M. Mesbahi Maryam Fazel Tamer Bacsar 87 27 0 10 Oct 2022
Differentially Private Deep Learning with ModelMix Hanshen Xiao Jun Wan S. Devadas 29 3 0 07 Oct 2022
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients Hualin Zhang Huan Xiong Bin Gu 35 7 0 04 Oct 2022
spred: Solving $L_1$ Penalty with SGD Liu Ziyin Zihao Wang 51 14 0 03 Oct 2022
Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition Jianhao Ma Li-Zhen Guo S. Fattahi 46 4 0 01 Oct 2022
Nonconvex Matrix Factorization is Geodesically Convex: Global Landscape Analysis for Fixed-rank Matrix Optimization From a Riemannian Perspective Yuetian Luo Nicolas García Trillos 24 6 0 29 Sep 2022
On Quantum Speedups for Nonconvex Optimization via Quantum Tunneling Walks Yizhou Liu Weijie J. Su Tongyang Li 41 18 0 29 Sep 2022
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold Can Yaras Peng Wang Zhihui Zhu Laura Balzano Qing Qu 30 42 0 19 Sep 2022
Versatile Single-Loop Method for Gradient Estimator: First and Second Order Optimality, and its Application to Federated Learning Kazusato Oko Shunta Akiyama Tomoya Murata Taiji Suzuki FedML 45 0 0 01 Sep 2022
Accelerating SGD for Highly Ill-Conditioned Huge-Scale Online Matrix Completion G. Zhang Hong-Ming Chiu Richard Y. Zhang 27 10 0 24 Aug 2022
Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization Zhize Li Jian Li 43 6 0 22 Aug 2022
CoShNet: A Hybrid Complex Valued Neural Network using Shearlets Manny Ko Ujjawal K. Panchal Héctor Andrade-Loarca Andres Mendez-Vazquez 33 1 0 14 Aug 2022
Quantized Adaptive Subgradient Algorithms and Their Applications Ke Xu Jianqiao Wangni Yifan Zhang Deheng Ye Jiaxiang Wu P. Zhao 36 0 0 11 Aug 2022
Agnostic Learning of General ReLU Activation Using Gradient Descent Pranjal Awasthi Alex K. Tang Aravindan Vijayaraghavan MLT 15 7 0 04 Aug 2022
Gradient descent provably escapes saddle points in the training of shallow ReLU networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 36 5 0 03 Aug 2022
A Gradient Smoothed Functional Algorithm with Truncated Cauchy Random Perturbations for Stochastic Optimization Akash Mondal A. PrashanthL. S. Bhatnagar 30 2 0 30 Jul 2022
Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time Jun-Kun Wang Andre Wibisono 32 9 0 05 Jul 2022
Improved Global Guarantees for the Nonconvex Burer--Monteiro Factorization via Rank Overparameterization Richard Y. Zhang 30 24 0 05 Jul 2022
A Theoretical Analysis of the Learning Dynamics under Class Imbalance Emanuele Francazi Marco Baity-Jesi Aurelien Lucchi 30 16 0 01 Jul 2022
AFAFed -- Protocol analysis E. Baccarelli M. Scarpiniti Alireza Momenzadeh S. S. Ahrabi FedML 8 0 0 29 Jun 2022
Explicit Regularization in Overparametrized Models via Noise Injection Antonio Orvieto Anant Raj Hans Kersting Francis R. Bach 23 26 0 09 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 29 10 0 08 Jun 2022
Preconditioned Gradient Descent for Overparameterized Nonconvex Burer--Monteiro Factorization with Global Optimality Certification G. Zhang S. Fattahi Richard Y. Zhang 62 23 0 07 Jun 2022
On the Convergence of Optimizing Persistent-Homology-Based Losses Yikai Zhang Jiacheng Yao Yusu Wang Chao Chen 18 1 0 06 Jun 2022
Subspace Phase Retrieval Meng Xu Dekuan Dong J. Wang 24 2 0 06 Jun 2022
First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces Michael I. Jordan Tianyi Lin Emmanouil-Vasileios Vlatakis-Gkaragkounis 34 19 0 04 Jun 2022
Understanding Deep Learning via Decision Boundary Shiye Lei Fengxiang He Yancheng Yuan Dacheng Tao 25 13 0 03 Jun 2022
Non-convex online learning via algorithmic equivalence Udaya Ghai Zhou Lu Elad Hazan 16 8 0 30 May 2022
HOUDINI: Escaping from Moderately Constrained Saddles Dmitrii Avdiukhin G. Yaroslavtsev 26 0 0 27 May 2022
Uniform Generalization Bound on Time and Inverse Temperature for Gradient Descent Algorithm and its Application to Analysis of Simulated Annealing Keisuke Suzuki AI4CE 33 0 0 25 May 2022
Weak Convergence of Approximate reflection coupling and its Application to Non-convex Optimization Keisuke Suzuki 36 5 0 24 May 2022
Estimation and Inference by Stochastic Optimization Jean-Jacques Forneron 38 5 0 06 May 2022
Byzantine Fault Tolerance in Distributed Machine Learning : a Survey Djamila Bouhata Hamouma Moumen Moumen Hamouma Ahcène Bounceur AI4CE 31 7 0 05 May 2022
Gradient Descent, Stochastic Optimization, and Other Tales Jun Lu 22 8 0 02 May 2022
Accelerating nuclear-norm regularized low-rank matrix optimization through Burer-Monteiro decomposition Ching-pei Lee Ling Liang Tianyun Tang Kim-Chuan Toh 37 11 0 29 Apr 2022
BEINIT: Avoiding Barren Plateaus in Variational Quantum Algorithms Ankit Kulshrestha Ilya Safro 18 37 0 28 Apr 2022
Accelerated Multiplicative Weights Update Avoids Saddle Points almost always Yi-Hu Feng Ioannis Panageas Tianlin Li 16 2 0 25 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 23 20 0 30 Mar 2022
Personalized incentives as feedback design in generalized Nash equilibrium problems F. Fabiani Andrea Simonetto Paul Goulart 21 4 0 24 Mar 2022
Noisy Low-rank Matrix Optimization: Geometry of Local Minima and Convergence Rate Ziye Ma Somayeh Sojoudi 43 6 0 08 Mar 2022
Whiplash Gradient Descent Dynamics Subhransu S. Bhattacharjee I. Petersen 14 0 0 04 Mar 2022
Second-order Symmetric Non-negative Latent Factor Analysis L. Li Xin Luo 25 3 0 04 Mar 2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases Nikunj Saunshi Jordan T. Ash Surbhi Goel Dipendra Kumar Misra Cyril Zhang Sanjeev Arora Sham Kakade A. Krishnamurthy SSL 29 109 0 28 Feb 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 31 8 0 18 Feb 2022
Trace norm regularization for multi-task learning with scarce data Etienne Boursier Mikhail Konobeev Nicolas Flammarion 19 11 0 14 Feb 2022
Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learning Tomoya Murata Taiji Suzuki FedML 6 3 0 12 Feb 2022
Efficiently Escaping Saddle Points in Bilevel Optimization Minhui Huang Xuxing Chen Kaiyi Ji Shiqian Ma Lifeng Lai 34 21 0 08 Feb 2022