Convergence, Sticking and Escape: Stochastic Dynamics Near Critical Points in SGD

24 May 2025

Papers citing "Convergence, Sticking and Escape: Stochastic Dynamics Near Critical Points in SGD"

23 / 23 papers shown

Title
A weak convergence approach to large deviations for stochastic approximations H. Hult Adam Lindhe Pierre Nyquist Guo-Jhen Wu 83 2 0 04 Feb 2025
Revisiting Step-Size Assumptions in Stochastic Approximation Caio Kalil Lauand Sean P. Meyn 61 2 0 28 May 2024
Stochastic Gradient Flow Dynamics of Test Risk and its Exact Solution for Weak Features Rodrigo Veiga Anastasia Remizova Nicolas Macris 66 1 0 12 Feb 2024
Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent Liu Ziyin Botao Li Tomer Galanti Masakuni Ueda 55 7 0 23 Mar 2023
Large deviations rates for stochastic gradient descent with strongly convex functions Dragana Bajović D. Jakovetić S. Kar 51 6 0 02 Nov 2022
Incremental Learning in Diagonal Linear Networks Raphael Berthier CLL AI4CE 50 17 0 31 Aug 2022
On uniform-in-time diffusion approximation for stochastic gradient descent Lei Li Yuliang Wang 76 4 0 11 Jul 2022
The effective noise of Stochastic Gradient Descent Francesca Mignacco Pierfrancesco Urbani 25 38 0 20 Dec 2021
SGD with a Constant Large Learning Rate Can Converge to Local Maxima Liu Ziyin Botao Li James B. Simon Masakuni Ueda 46 9 0 25 Jul 2021
On Proximal Policy Optimization's Heavy-tailed Gradients Saurabh Garg Joshua Zhanson Emilio Parisotto Adarsh Prasad J. Zico Kolter Zachary Chase Lipton Sivaraman Balakrishnan Ruslan Salakhutdinov Pradeep Ravikumar 64 13 0 20 Feb 2021
Eliminating Sharp Minima from SGD with Truncated Heavy-tailed Noise Xingyu Wang Sewoong Oh C. Rhee 35 16 0 08 Feb 2021
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification Francesca Mignacco Florent Krzakala Pierfrancesco Urbani Lenka Zdeborová MLT 52 68 0 10 Jun 2020
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks Umut Simsekli Mert Gurbuzbalaban T. H. Nguyen G. Richard Levent Sagun 68 58 0 29 Nov 2019
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise T. H. Nguyen Umut Simsekli Mert Gurbuzbalaban G. Richard 53 64 0 21 Jun 2019
Accelerating Minibatch Stochastic Gradient Descent using Typicality Sampling Xinyu Peng Li Li Feiyue Wang BDL 112 59 0 11 Mar 2019
Traditional and Heavy-Tailed Self Regularization in Neural Network Models Charles H. Martin Michael W. Mahoney 66 124 0 24 Jan 2019
Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization T. H. Nguyen Umut Simsekli G. Richard 58 28 0 22 Jan 2019
Quasi-potential as an implicit regularizer for the loss function in the stochastic gradient descent Wenqing Hu Zhanxing Zhu Haoyi Xiong Jun Huan MLT 31 10 0 18 Jan 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 93 249 0 18 Jan 2019
Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations Qianxiao Li Cheng Tai E. Weinan 95 150 0 05 Nov 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 243 1,890 0 28 Dec 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 424 2,937 0 15 Sep 2016
Stochastic modified equations and adaptive stochastic gradient algorithms Qianxiao Li Cheng Tai E. Weinan 59 284 0 19 Nov 2015