Escaping Saddles with Stochastic Gradients

15 March 2018

Papers citing "Escaping Saddles with Stochastic Gradients"

31 / 31 papers shown

Title
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 45 0 0 08 Feb 2024
Score-Aware Policy-Gradient Methods and Performance Guarantees using Local Lyapunov Conditions: Applications to Product-Form Stochastic Networks and Queueing Systems Céline Comte Matthieu Jonckheere J. Sanders Albert Senen-Cerda 30 0 0 05 Dec 2023
How to escape sharp minima with random perturbations Kwangjun Ahn Ali Jadbabaie S. Sra ODL 34 6 0 25 May 2023
Almost Sure Saddle Avoidance of Stochastic Gradient Methods without the Bounded Gradient Assumption Jun Liu Ye Yuan ODL 16 1 0 15 Feb 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 29 1 0 28 Jan 2023
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurelien Lucchi 23 13 0 19 Jan 2023
Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data Harsh Rangwani Sumukh K Aithal Mayank Mishra R. Venkatesh Babu 31 28 0 28 Dec 2022
Decentralized Nonconvex Optimization with Guaranteed Privacy and Accuracy Yongqiang Wang Tamer Basar 21 21 0 14 Dec 2022
On the Overlooked Structure of Stochastic Gradients Zeke Xie Qian-Yuan Tang Mingming Sun P. Li 31 6 0 05 Dec 2022
Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models Shujian Zhang Chengyue Gong Xingchao Liu RALM 49 6 0 02 Nov 2022
Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition Jianhao Ma Li-Zhen Guo S. Fattahi 38 4 0 01 Oct 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 28 8 0 18 Feb 2022
Non-Asymptotic Analysis of Online Multiplicative Stochastic Gradient Descent Riddhiman Bhattacharya Tiefeng Jiang 16 0 0 14 Dec 2021
Exponential escape efficiency of SGD from sharp minima in non-stationary regime Hikaru Ibayashi Masaaki Imaizumi 34 4 0 07 Nov 2021
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima Zixiang Chen Dongruo Zhou Quanquan Gu 40 1 0 25 Oct 2021
The loss landscape of deep linear neural networks: a second-order analysis E. M. Achour Franccois Malgouyres Sébastien Gerchinovitz ODL 24 9 0 28 Jul 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 27 29 0 31 Mar 2021
Provable Super-Convergence with a Large Cyclical Learning Rate Samet Oymak 33 12 0 22 Feb 2021
Learning explanations that are hard to vary Giambattista Parascandolo Alexander Neitz Antonio Orvieto Luigi Gresele Bernhard Schölkopf FAtt 21 178 0 01 Sep 2020
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems P. Mertikopoulos Nadav Hallak Ali Kavis V. Cevher 21 85 0 19 Jun 2020
Replica Exchange for Non-Convex Optimization Jing-rong Dong Xin T. Tong 22 21 0 23 Jan 2020
Shadowing Properties of Optimization Algorithms Antonio Orvieto Aurelien Lucchi 30 18 0 12 Nov 2019
Second-Order Guarantees of Stochastic Gradient Descent in Non-Convex Optimization Stefan Vlaski Ali H. Sayed ODL 26 21 0 19 Aug 2019
Distributed Learning in Non-Convex Environments -- Part II: Polynomial Escape from Saddle-Points Stefan Vlaski Ali H. Sayed 21 53 0 03 Jul 2019
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies Kaipeng Zhang Alec Koppel Haoqi Zhu Tamer Basar 41 186 0 19 Jun 2019
On the Noisy Gradient Descent that Generalizes as SGD Jingfeng Wu Wenqing Hu Haoyi Xiong Jun Huan Vladimir Braverman Zhanxing Zhu MLT 24 10 0 18 Jun 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 20 237 0 18 Jan 2019
SGD Converges to Global Minimum in Deep Learning via Star-convex Path Yi Zhou Junjie Yang Huishuai Zhang Yingbin Liang Vahid Tarokh 14 71 0 02 Jan 2019
Continuous-time Models for Stochastic Optimization Algorithms Antonio Orvieto Aurelien Lucchi 16 31 0 05 Oct 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 25 146 0 20 Jun 2018
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014