How to Escape Saddle Points Efficiently

2 March 2017

Papers citing "How to Escape Saddle Points Efficiently"

50 / 468 papers shown

Title
Second-Order Convergence in Private Stochastic Non-Convex Optimization Youming Tao Zuyuan Zhang Dongxiao Yu Xiuzhen Cheng Falko Dressler Di Wang 12 0 0 21 May 2025
Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization G. Zhang S. Fattahi Richard Y. Zhang 49 36 0 13 Apr 2025
Exploring Energy Landscapes for Minimal Counterfactual Explanations: Applications in Cybersecurity and Beyond Spyridon Evangelatos Eleni Veroni Vasilis Efthymiou Christos Nikolopoulos Georgios Th. Papadopoulos Panagiotis G. Sarigiannidis AAML 39 0 0 23 Mar 2025
A Near Complete Nonasymptotic Generalization Theory For Multilayer Neural Networks: Beyond the Bias-Variance Tradeoff Hao Yu Xiangyang Ji AI4CE 60 0 0 03 Mar 2025
Langevin Multiplicative Weights Update with Applications in Polynomial Portfolio Management Yi-Hu Feng Tianlin Li Tian Xie 62 0 0 26 Feb 2025
$k$ -SVD with Gradient Descent Emily Gan Yassir Jedra Devavrat Shah 68 0 0 01 Feb 2025
SSE-SAM: Balancing Head and Tail Classes Gradually through Stage-Wise SAM Xingyu Lyu Qianqian Xu Zhiyong Yang Shaojie Lyu Qingming Huang 89 0 0 18 Dec 2024
Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem Hesameddin Mohammadi Mohammad Tinati Stephen Tu Mahdi Soltanolkotabi M. Jovanović 78 0 0 24 Nov 2024
A new Input Convex Neural Network with application to options pricing Vincent Lemaire Gilles Pagès Christian Yeo 74 0 0 19 Nov 2024
Attribute Inference Attacks for Federated Regression Tasks Francesco Diana Othmane Marfoq Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML 273 1 0 19 Nov 2024
Complexity-Aware Training of Deep Neural Networks for Optimal Structure Discovery Valentin Frank Ingmar Guenter Athanasios Sideris CVBM 26 0 0 14 Nov 2024
SPGD: Steepest Perturbed Gradient Descent Optimization Amir M. Vahedi Horea T. Ilies 35 1 0 07 Nov 2024
MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning Suning Huang Zheyu Zhang Tianhai Liang Yihan Xu Zhehao Kou Chenhao Lu Guowei Xu Zhengrong Xue Huazhe Xu MoE 49 2 0 19 Oct 2024
Loss Landscape Characterization of Neural Networks without Over-Parametrization Rustem Islamov Niccolò Ajroldi Antonio Orvieto Aurelien Lucchi 45 4 0 16 Oct 2024
On-the-fly Modulation for Balanced Multimodal Learning Yake Wei D. Hu Henghui Du Zhicheng Dou 34 7 0 15 Oct 2024
Learning to Optimize for Mixed-Integer Non-linear Programming Bo Tang Elias Boutros Khalil Ján Drgoňa 42 2 0 14 Oct 2024
Adaptive Batch Size for Privately Finding Second-Order Stationary Points Daogao Liu Kunal Talwar 215 0 0 10 Oct 2024
Noise is All You Need: Private Second-Order Convergence of Noisy SGD Dmitrii Avdiukhin Michael Dinitz Chenglin Fan G. Yaroslavtsev 39 1 0 09 Oct 2024
Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models Zeman Li Xinwei Zhang Peilin Zhong Yuan Deng Meisam Razaviyayn Vahab Mirrokni 27 2 0 09 Oct 2024
$$\ell_1$-norm rank-one symmetric matrix factorization has no spurious second-order stationary points$ $\ell_1$ -norm rank-one symmetric matrix factorization has no spurious second-order stationary points Jiewen Guan Anthony Man-Cho So 44 2 0 07 Oct 2024
Super Level Sets and Exponential Decay: A Synergistic Approach to Stable Neural Network Training J. Chaudhary Dipak Nidhi J. Heikkonen H. Merisaari R. Kanth 31 0 0 25 Sep 2024
On the Hardness of Meaningful Local Guarantees in Nonsmooth Nonconvex Optimization Guy Kornowski Swati Padmanabhan Ohad Shamir 229 0 0 16 Sep 2024
A Sample Efficient Alternating Minimization-based Algorithm For Robust Phase Retrieval Adarsh Barik Anand Krishna Vincent Y. F. Tan 18 0 0 07 Sep 2024
Smoothed Robust Phase Retrieval Zhong Zheng Lingzhou Xue 37 2 0 03 Sep 2024
Automatically Adaptive Conformal Risk Control Vincent Blot Anastasios Nikolas Angelopoulos Michael I Jordan Nicolas Brunel AI4CE 46 2 0 25 Jun 2024
Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes Dan Qiao Kaiqi Zhang Esha Singh Daniel Soudry Yu-Xiang Wang NoLa 41 3 0 10 Jun 2024
Concurrent Training and Layer Pruning of Deep Neural Networks Valentin Frank Ingmar Guenter Athanasios Sideris 3DPC 45 3 0 06 Jun 2024
A Global Geometric Analysis of Maximal Coding Rate Reduction Peng Wang Huikang Liu Druv Pai Yaodong Yu Zhihui Zhu Q. Qu Yi Ma 44 7 0 04 Jun 2024
Prospects of Privacy Advantage in Quantum Machine Learning Jamie Heredge Niraj Kumar Dylan Herman Shouvanik Chakrabarti Romina Yalovetzky Shree Hari Sureshbabu Changhao Li Marco Pistoia 39 4 0 14 May 2024
Improving Trainability of Variational Quantum Circuits via Regularization Strategies Jun Zhuang Jack Cunningham Chaowen Guan 43 4 0 02 May 2024
Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization Chris Junchi Li 59 0 0 01 May 2024
Efficient algorithms for regularized Poisson Non-negative Matrix Factorization Nathanael Perraudin Adrien Teutrie Cécile Hébert G. Obozinski 35 1 0 25 Apr 2024
cuFastTuckerPlus: A Stochastic Parallel Sparse FastTucker Decomposition Using GPU Tensor Cores Zixuan Li Mingxing Duan Huizhang Luo Wangdong Yang KenLi Li Keqin Li 34 0 0 15 Apr 2024
3D Gaussian Splatting as Markov Chain Monte Carlo Shakiba Kheradmand Daniel Rebain Gopal Sharma Weiwei Sun Jeff Tseng Hossam N. Isack Abhishek Kar Andrea Tagliasacchi Kwang Moo Yi 3DGS 55 49 0 15 Apr 2024
Preventing Model Collapse in Gaussian Process Latent Variable Models Ying Li Zhidi Lin Feng Yin Michael Minyi Zhang VLM 37 1 0 02 Apr 2024
Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensing Shuyao Li Yu Cheng Ilias Diakonikolas Jelena Diakonikolas Rong Ge Stephen J. Wright 49 2 0 12 Mar 2024
Beyond Single-Model Views for Deep Learning: Optimization versus Generalizability of Stochastic Optimization Algorithms Toki Tahmid Inan Mingrui Liu Amarda Shehu 36 0 0 01 Mar 2024
Training-set-free two-stage deep learning for spectroscopic data de-noising Dongchen Huang Junde Liu Tian Qian Hongming Weng 36 0 0 29 Feb 2024
Escaping Local Optima in Global Placement Ke Xue Xi Lin Yunqi Shi Shixiong Kai Siyuan Xu Chao Qian 18 2 0 28 Feb 2024
LoRA Training in the NTK Regime has No Spurious Local Minima Uijeong Jang Jason D. Lee Ernest K. Ryu 44 14 0 19 Feb 2024
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 53 0 0 08 Feb 2024
Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape Juno Kim Taiji Suzuki 28 19 0 02 Feb 2024
Accelerating Distributed Stochastic Optimization via Self-Repellent Random Walks Jie Hu Vishwaraj Doshi Do Young Eun 55 2 0 18 Jan 2024
Avoiding strict saddle points of nonconvex regularized problems Luwei Bai Yaohua Hu Hao Wang Xiaoqi Yang 13 0 0 17 Jan 2024
Guaranteed Nonconvex Factorization Approach for Tensor Train Recovery Zhen Qin M. Wakin Zhihui Zhu 45 5 0 05 Jan 2024
Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity Yiyue Chen H. Vikalo C. Wang FedML 49 5 0 20 Dec 2023
Adam-like Algorithm with Smooth Clipping Attains Global Minima: Analysis Based on Ergodicity of Functional SDEs Keisuke Suzuki 29 0 0 29 Nov 2023
The Local Landscape of Phase Retrieval Under Limited Samples Kaizhao Liu Zihao Wang Lei Wu 22 2 0 26 Nov 2023
High Probability Guarantees for Random Reshuffling Hengxu Yu Xiao Li 47 2 0 20 Nov 2023
Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization with Optimal Noise Scheduling Naoki Sato Hideaki Iiduka 30 3 0 15 Nov 2023