How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
On the Second-Order Convergence of Biased Policy Gradient Algorithms Siqiao Mu Diego Klabjan 50 2 0 05 Nov 2023
Online Non-convex Optimization with Long-term Non-convex Constraints Shijie Pan Wenjie Huang 26 0 0 04 Nov 2023
Riemannian stochastic optimization methods avoid strict saddle points Ya-Ping Hsieh Mohammad Reza Karimi Andreas Krause P. Mertikopoulos 43 5 0 04 Nov 2023
Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compression Sijin Chen Zhize Li Yuejie Chi FedML 46 5 0 29 Oct 2023
Optimization Landscape of Policy Gradient Methods for Discrete-time Static Output Feedback Jingliang Duan Jie Li Xuyang Chen Kai Zhao Shengbo Eben Li Lin Zhao 24 5 0 29 Oct 2023
A randomized algorithm for nonconvex minimization with inexact evaluations and complexity guarantees Shuyao Li Stephen J. Wright 30 3 0 28 Oct 2023
Optimal Transport for Treatment Effect Estimation Hao Wang Zhichao Chen Jiajun Fan Haoxuan Li Tianqiao Liu Weiming Liu Quanyu Dai Yichao Wang Zhenhua Dong Ruiming Tang OT CML 30 35 0 27 Oct 2023
Neural Collapse in Multi-label Learning with Pick-all-label Loss Pengyu Li Xiao Li Yutong Wang Qing Qu 35 8 0 24 Oct 2023
Series of Hessian-Vector Products for Tractable Saddle-Free Newton Optimisation of Neural Networks E. T. Oldewage Ross M. Clarke José Miguel Hernández-Lobato ODL 30 1 0 23 Oct 2023
Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function Liu Liu Xuanqing Liu Cho-Jui Hsieh Dacheng Tao 26 3 0 18 Oct 2023
On Unsupervised Image-to-image translation and GAN stability BahaaEddin AlAila Zahra Jandaghi Abolfazl Farahani M. Al-Saad GAN 27 1 0 18 Oct 2023
Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression Adam Block Dylan J. Foster Akshay Krishnamurthy Max Simchowitz Cyril Zhang 35 4 0 17 Oct 2023
Over-the-Air Federated Learning and Optimization Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Wei Chen Khaled B. Letaief FedML 28 11 0 16 Oct 2023
Provably Accelerating Ill-Conditioned Low-rank Estimation via Scaled Gradient Descent, Even with Overparameterization Cong Ma Xingyu Xu Tian Tong Yuejie Chi 20 9 0 09 Oct 2023
CoNO: Complex Neural Operator for Continuous Dynamical Systems Karn Tiwari N. M. A. Krishnan P. PrathoshA 30 1 0 03 Oct 2023
How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization Nuoya Xiong Lijun Ding Simon S. Du 48 12 0 03 Oct 2023
Are Graph Neural Networks Optimal Approximation Algorithms? Morris Yau Eric Lu Nikolaos Karalias Jessica Xu Stefanie Jegelka 35 1 0 01 Oct 2023
On the different regimes of Stochastic Gradient Descent Antonio Sclocchi Matthieu Wyart 33 18 0 19 Sep 2023
A Gentle Introduction to Gradient-Based Optimization and Variational Inequalities for Machine Learning Neha S. Wadia Yatin Dandi Michael I. Jordan AI4CE 14 0 0 09 Sep 2023
Max-affine regression via first-order methods Seonho Kim Kiryung Lee 33 2 0 15 Aug 2023
On Neural Network approximation of ideal adversarial attack and convergence of adversarial training Rajdeep Haldar Qifan Song AAML 31 0 0 30 Jul 2023
Minimizing robust density power-based divergences for general parametric density models Akifumi Okuno 34 2 0 11 Jul 2023
Accelerating Inexact HyperGradient Descent for Bilevel Optimization Hai-Long Yang Luo Luo C. J. Li Michael I. Jordan 36 13 0 30 Jun 2023
Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds Yong Cai Michael I. Jordan Tianyi Lin Argyris Oikonomou Emmanouil-Vasileios Vlatakis-Gkaragkounis 38 4 0 29 Jun 2023
Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles Le‐Yu Chen Yaohua Ma J.N. Zhang 88 2 0 26 Jun 2023
Bootstrapped Representations in Reinforcement Learning Charline Le Lan Stephen Tu Mark Rowland Anna Harutyunyan Rishabh Agarwal Marc G. Bellemare Will Dabney OffRL OOD SSL 79 10 0 16 Jun 2023
Symmetry & Critical Points for Symmetric Tensor Decomposition Problems Yossi Arjevani Gal Vinograd 39 5 0 13 Jun 2023
Understanding Predictive Coding as an Adaptive Trust-Region Method Francesco Innocenti Ryan Singh Christopher L. Buckley 25 0 0 29 May 2023
Accelerated Methods for Riemannian Min-Max Optimization Ensuring Bounded Geometric Penalties David Martínez-Rubio Christophe Roux Christopher Criscitiello Sebastian Pokutta 24 6 0 25 May 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 36 6 0 25 May 2023
Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays Adrian Redder Arunselvan Ramaswamy Holger Karl 25 1 0 11 May 2023
Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learning Yulai Zhao Zhuoran Yang Zhaoran Wang Jason D. Lee 48 3 0 08 May 2023
A Cubic-regularized Policy Newton Algorithm for Reinforcement Learning Mizhaan Prajit Maniyar Akash Mondal Prashanth L.A. S. Bhatnagar 51 0 0 21 Apr 2023
Simulated Annealing in Early Layers Leads to Better Generalization Amirm. Sarfi Zahra Karimpour Muawiz Chaudhary N. Khalid Mirco Ravanelli Sudhir Mudur Eugene Belilovsky AI4CE CLL 23 7 0 10 Apr 2023
Learning Rate Schedules in the Presence of Distribution Shift Matthew Fahrbach Adel Javanmard Vahab Mirrokni Pratik Worah 29 6 0 27 Mar 2023
Greedy Pruning with Group Lasso Provably Generalizes for Matrix Sensing Nived Rajaraman Devvrit Aryan Mokhtari Kannan Ramchandran 29 0 0 20 Mar 2023
Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators Tian-cheng Li Fan Chen Huajie Chen Zaiwen Wen 18 4 0 19 Mar 2023
Can Learning Be Explained By Local Optimality In Robust Low-rank Matrix Recovery? Jianhao Ma S. Fattahi 36 1 0 21 Feb 2023
Private (Stochastic) Non-Convex Optimization Revisited: Second-Order Stationary Points and Excess Risks Arun Ganesh Daogao Liu Sewoong Oh Abhradeep Thakurta ODL 34 13 0 20 Feb 2023
Almost Sure Saddle Avoidance of Stochastic Gradient Methods without the Bounded Gradient Assumption Jun Liu Ye Yuan ODL 19 1 0 15 Feb 2023
Efficient displacement convex optimization with particle gradient descent Hadi Daneshmand Jason D. Lee Chi Jin 29 5 0 09 Feb 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 32 1 0 28 Jan 2023
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurelien Lucchi 35 13 0 19 Jan 2023
A Newton-CG based augmented Lagrangian method for finding a second-order stationary point of nonconvex equality constrained optimization with complexity guarantees Chuan He Zhaosong Lu Ting Kei Pong 11 7 0 09 Jan 2023
A Dynamics Theory of Implicit Regularization in Deep Low-Rank Matrix Factorization JIAN-PENG Cao Chao Qian Yihui Huang Dicheng Chen Yuncheng Gao Jiyang Dong D. Guo X. Qu 26 1 0 29 Dec 2022
Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data Harsh Rangwani Sumukh K Aithal Mayank Mishra R. Venkatesh Babu 33 29 0 28 Dec 2022
Plateau-reduced Differentiable Path Tracing Michael Fischer Tobias Ritschel 26 9 0 30 Nov 2022
Escaping From Saddle Points Using Asynchronous Coordinate Gradient Descent Marco Bornstein Jin-Peng Liu Jingling Li Furong Huang 23 0 0 17 Nov 2022
Neural Langevin Dynamics: towards interpretable Neural Stochastic Differential Equations Simon Koop M. Peletier J. Portegies Vlado Menkovski DiffM 35 1 0 17 Nov 2022
Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models Shujian Zhang Chengyue Gong Xingchao Liu RALM 53 6 0 02 Nov 2022