Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers

2 August 2021

Papers citing "Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers"

21 / 21 papers shown

Title
Generalization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior Romain Chor Abdellatif Zaidi Piotr Krasnowski 49 0 0 25 Apr 2025
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors Romain Chor Milad Sefidgaran Piotr Krasnowski 96 1 0 21 Feb 2025
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 49 0 0 11 Feb 2025
Model Balancing Helps Low-data Training and Fine-tuning Zihang Liu Yihan Hu Tianyu Pang Yefan Zhou Pu Ren Yaoqing Yang 42 2 0 16 Oct 2024
Crafting Heavy-Tails in Weight Matrix Spectrum without Gradient Noise Vignesh Kothapalli Tianyu Pang Shenyang Deng Zongmin Liu Yaoqing Yang 42 3 0 07 Jun 2024
Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets Benjamin Dupuis Paul Viallard George Deligiannidis Umut Simsekli 48 2 0 26 Apr 2024
Tighter Generalisation Bounds via Interpolation Paul Viallard Maxime Haddouche Umut Simsekli Benjamin Guedj 19 3 0 07 Feb 2024
From Mutual Information to Expected Dynamics: New Generalization Bounds for Heavy-Tailed SGD Benjamin Dupuis Paul Viallard 18 3 0 01 Dec 2023
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training Yefan Zhou Tianyu Pang Keqin Liu Charles H. Martin Michael W. Mahoney Yaoqing Yang 42 7 0 01 Dec 2023
Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descent Krunoslav Lehman Pavasovic Alain Durmus Umut Simsekli OffRL 26 2 0 27 Oct 2023
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression Xuxing Chen Krishnakumar Balasubramanian Promit Ghosal Bhavya Agrawalla 38 7 0 02 Oct 2023
Generalization Guarantees via Algorithm-dependent Rademacher Complexity Sarah Sachs T. Erven Liam Hodgkinson Rajiv Khanna Umut Simsekli 25 3 0 04 Jul 2023
A unified framework for information-theoretic generalization bounds Y.-C. Chu Maxim Raginsky 30 14 0 18 May 2023
Generalization Bounds with Data-dependent Fractal Dimensions Benjamin Dupuis George Deligiannidis Umut cSimcsekli AI4CE 39 12 0 06 Feb 2023
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares Anant Raj Melih Barsbey Mert Gurbuzbalaban Lingjiong Zhu Umut Simsekli 19 9 0 02 Jun 2022
Generalization Bounds for Gradient Methods via Discrete and Continuous Prior Jun Yu Li Xu Luo Jian Li 27 4 0 27 May 2022
Learning continuous models for continuous physics Aditi S. Krishnapriyan A. Queiruga N. Benjamin Erichson Michael W. Mahoney AI4CE 32 33 0 17 Feb 2022
Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization Stanislaw Jastrzebski Devansh Arpit Oliver Åstrand Giancarlo Kerg Huan Wang Caiming Xiong R. Socher Kyunghyun Cho Krzysztof J. Geras AI4CE 184 66 0 28 Dec 2020
Failures of model-dependent generalization bounds for least-norm interpolation Peter L. Bartlett Philip M. Long 93 29 0 16 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,532 0 23 Jan 2020
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates Jeffrey Negrea Mahdi Haghifam Gintare Karolina Dziugaite Ashish Khisti Daniel M. Roy FedML 115 147 0 06 Nov 2019