Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions

Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions

27 January 2023

Mert Gurbuzbalaban

Papers citing "Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions"

13 / 13 papers shown

Title
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 47 0 0 11 Feb 2025
Limit Theorems for Stochastic Gradient Descent with Infinite Variance Jose H. Blanchet Aleksandar Mijatović Wenhao Yang 26 0 0 21 Oct 2024
Crafting Heavy-Tails in Weight Matrix Spectrum without Gradient Noise Vignesh Kothapalli Tianyu Pang Shenyang Deng Zongmin Liu Yaoqing Yang 37 3 0 07 Jun 2024
Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping Umut Simsekli Mert Gurbuzbalaban S. Yıldırım Lingjiong Zhu 38 2 0 04 Mar 2024
From Mutual Information to Expected Dynamics: New Generalization Bounds for Heavy-Tailed SGD Benjamin Dupuis Paul Viallard 18 3 0 01 Dec 2023
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training Yefan Zhou Tianyu Pang Keqin Liu Charles H. Martin Michael W. Mahoney Yaoqing Yang 39 7 0 01 Dec 2023
Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descent Krunoslav Lehman Pavasovic Alain Durmus Umut Simsekli OffRL 18 2 0 27 Oct 2023
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent Lingjiong Zhu Mert Gurbuzbalaban Anant Raj Umut Simsekli 31 6 0 20 May 2023
Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models Anant Raj Umut Simsekli Alessandro Rudi DiffM 31 1 0 30 Mar 2023
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD than Constant Stepsize Mert Gurbuzbalaban Yuanhan Hu Umut Simsekli Lingjiong Zhu LRM 23 1 0 10 Feb 2023
$Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$-Covers$ Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$ -Covers Sejun Park Umut Simsekli Murat A. Erdogdu 43 9 0 19 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 31 4 0 06 Sep 2022
Chaotic Regularization and Heavy-Tailed Limits for Deterministic Gradient Descent S. H. Lim Yijun Wan Umut cSimcsekli 34 12 0 23 May 2022