Information-Theoretic Generalization Bounds for Stochastic Gradient Descent

1 February 2021

Gergely Neu

Gintare Karolina Dziugaite

Mahdi Haghifam

Daniel M. Roy

ArXiv PDF HTML

Papers citing "Information-Theoretic Generalization Bounds for Stochastic Gradient Descent"

36 / 36 papers shown

Title
Generalization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior Romain Chor Abdellatif Zaidi Piotr Krasnowski 49 0 0 25 Apr 2025
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors Romain Chor Milad Sefidgaran Piotr Krasnowski 91 1 0 21 Feb 2025
Stability-based Generalization Bounds for Variational Inference Yadi Wei R. Khardon BDL 49 0 0 17 Feb 2025
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 47 0 0 11 Feb 2025
Understanding Generalization of Federated Learning: the Trade-off between Model Stability and Optimization Dun Zeng Zheshun Wu Shiyu Liu Yu Pan Xiaoying Tang Zenglin Xu MLT FedML 89 1 0 25 Nov 2024
Deep Companion Learning: Enhancing Generalization Through Historical Consistency Ruizhao Zhu Venkatesh Saligrama FedML 40 0 0 26 Jul 2024
Enhancing Domain Adaptation through Prompt Gradient Alignment Hoang Phan Lam C. Tran Quyen Tran Trung Le 52 0 0 13 Jun 2024
Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets Benjamin Dupuis Paul Viallard George Deligiannidis Umut Simsekli 48 2 0 26 Apr 2024
Information-Theoretic Generalization Bounds for Deep Neural Networks Haiyun He Christina Lee Yu 35 4 0 04 Apr 2024
A PAC-Bayesian Link Between Generalisation and Flat Minima Maxime Haddouche Paul Viallard Umut Simsekli Benjamin Guedj 45 3 0 13 Feb 2024
Enhancing selectivity using Wasserstein distance based reweighing Pratik Worah OOD 56 0 0 21 Jan 2024
Convex SGD: Generalization Without Early Stopping Julien Hendrickx A. Olshevsky MLT LRM 25 1 0 08 Jan 2024
A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons and Adaptable Structure Yanjie Li Weijun Li Lina Yu Min Wu Jinyi Liu ... Xin Ning Yugui Zhang Baoli Lu Jian Xu Shuang Li 18 0 0 03 Jan 2024
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications Huayi Tang Yong Liu 62 1 0 08 Nov 2023
Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective Yuxin Dong Tieliang Gong Hao Chen Chen Li 23 4 0 02 May 2023
Learning Trajectories are Generalization Indicators Jingwen Fu Zhizheng Zhang Dacheng Yin Yan Lu Nanning Zheng AI4CE 33 3 0 25 Apr 2023
Information Theoretic Lower Bounds for Information Theoretic Upper Bounds Roi Livni 11 14 0 09 Feb 2023
Tighter Information-Theoretic Generalization Bounds from Supersamples Ziqiao Wang Yongyi Mao 32 17 0 05 Feb 2023
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions Anant Raj Lingjiong Zhu Mert Gurbuzbalaban Umut Simsekli 34 15 0 27 Jan 2023
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 30 10 0 19 Nov 2022
Information-Theoretic Analysis of Unsupervised Domain Adaptation Ziqiao Wang Yongyi Mao 38 11 0 03 Oct 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 324 48 0 29 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 31 4 0 06 Sep 2022
On Leave-One-Out Conditional Mutual Information For Generalization Mohamad Rida Rammal Alessandro Achille Aditya Golatkar Suhas Diggavi Stefano Soatto VLM 28 5 0 01 Jul 2022
Benefits of Additive Noise in Composing Classes with Bounded Capacity A. F. Pour H. Ashtiani 33 3 0 14 Jun 2022
Towards Understanding Sharpness-Aware Minimization Maksym Andriushchenko Nicolas Flammarion AAML 35 134 0 13 Jun 2022
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD Konstantinos E. Nikolakakis Farzin Haddadpour Amin Karbasi Dionysios S. Kalogerias 43 17 0 26 Apr 2022
Benign Underfitting of Stochastic Gradient Descent Tomer Koren Roi Livni Yishay Mansour Uri Sherman MLT 20 13 0 27 Feb 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 34 8 0 09 Jan 2022
Towards a Unified Information-Theoretic Framework for Generalization Mahdi Haghifam Gintare Karolina Dziugaite Shay Moran Daniel M. Roy 24 34 0 09 Nov 2021
User-friendly introduction to PAC-Bayes bounds Pierre Alquier FedML 60 197 0 21 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 37 22 0 07 Oct 2021
Speedy Performance Estimation for Neural Architecture Search Binxin Ru Clare Lyle Lisa Schut M. Fil Mark van der Wilk Y. Gal 18 36 0 08 Jun 2020
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates Jeffrey Negrea Mahdi Haghifam Gintare Karolina Dziugaite Ashish Khisti Daniel M. Roy FedML 113 147 0 06 Nov 2019
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 127 577 0 27 Feb 2015