Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates

6 November 2019

Jeffrey Negrea

Mahdi Haghifam

Gintare Karolina Dziugaite

Ashish Khisti

Daniel M. Roy

FedML

ArXiv PDF HTML

Papers citing "Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates"

41 / 41 papers shown

Title
Generalization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior Milad Sefidgaran Abdellatif Zaidi Piotr Krasnowski 44 0 0 25 Apr 2025
Randomized Pairwise Learning with Adaptive Sampling: A PAC-Bayes Analysis Sijia Zhou Yunwen Lei Ata Kabán 29 0 0 03 Apr 2025
Generalization in Federated Learning: A Conditional Mutual Information Framework Ziqiao Wang Cheng Long Yongyi Mao FedML 52 0 0 06 Mar 2025
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors Milad Sefidgaran A. Zaidi Piotr Krasnowski 88 1 0 21 Feb 2025
Stability-based Generalization Bounds for Variational Inference Yadi Wei R. Khardon BDL 44 0 0 17 Feb 2025
Understanding the Generalization Error of Markov algorithms through Poissonization Benjamin Dupuis Maxime Haddouche George Deligiannidis Umut Simsekli 44 0 0 11 Feb 2025
Towards Exact Computation of Inductive Bias Akhilan Boopathy William Yue Jaedong Hwang Abhiram Iyer Ila Fiete 32 0 0 22 Jun 2024
Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets Benjamin Dupuis Paul Viallard George Deligiannidis Umut Simsekli 40 2 0 26 Apr 2024
A PAC-Bayesian Link Between Generalisation and Flat Minima Maxime Haddouche Paul Viallard Umut Simsekli Benjamin Guedj 37 3 0 13 Feb 2024
Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shift Renchunzi Xie Ambroise Odonnat Vasilii Feofanov I. Redko Jianfeng Zhang Bo An UQCV 72 1 0 17 Jan 2024
Information-Theoretic Generalization Bounds for Transductive Learning and its Applications Huayi Tang Yong Liu 57 1 0 08 Nov 2023
Generalization Bounds for Label Noise Stochastic Gradient Descent Jung Eun Huh Patrick Rebeschini 13 1 0 01 Nov 2023
Learning Trajectories are Generalization Indicators Jingwen Fu Zhizheng Zhang Dacheng Yin Yan Lu Nanning Zheng AI4CE 28 3 0 25 Apr 2023
Randomized Adversarial Training via Taylor Expansion Gao Jin Xinping Yi Dengyu Wu Ronghui Mu Xiaowei Huang AAML 36 34 0 19 Mar 2023
Generalization Bounds with Data-dependent Fractal Dimensions Benjamin Dupuis George Deligiannidis Umut cSimcsekli AI4CE 33 12 0 06 Feb 2023
Tighter Information-Theoretic Generalization Bounds from Supersamples Ziqiao Wang Yongyi Mao 21 17 0 05 Feb 2023
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 18 10 0 19 Nov 2022
On the Algorithmic Stability and Generalization of Adaptive Optimization Methods Han Nguyen Hai Pham Sashank J. Reddi Barnabás Póczos ODL AI4CE 15 2 0 08 Nov 2022
The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima Peter L. Bartlett Philip M. Long Olivier Bousquet 70 34 0 04 Oct 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 26 4 0 06 Sep 2022
Understanding Generalization via Leave-One-Out Conditional Mutual Information Mahdi Haghifam Shay Moran Daniel M. Roy Gintare Karolina Dziugaite 20 14 0 29 Jun 2022
A General framework for PAC-Bayes Bounds for Meta-Learning A. Rezazadeh AI4CE 21 4 0 11 Jun 2022
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD Konstantinos E. Nikolakakis Farzin Haddadpour Amin Karbasi Dionysios S. Kalogerias 34 17 0 26 Apr 2022
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning Wei Huang Chunrui Liu Yilan Chen Tianyu Liu R. Xu BDL MLT 19 2 0 04 Feb 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 23 8 0 09 Jan 2022
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning Soufiane Hayou Bo He Gintare Karolina Dziugaite 17 2 0 22 Oct 2021
User-friendly introduction to PAC-Bayes bounds Pierre Alquier FedML 42 196 0 21 Oct 2021
Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learning Haiyun He Hanshu Yan Vincent Y. F. Tan 23 11 0 03 Oct 2021
PAC-Bayes, MAC-Bayes and Conditional Mutual Information: Fast rate bounds that handle general VC classes Peter Grünwald Thomas Steinke Lydia Zakynthinou 21 29 0 17 Jun 2021
Individually Conditional Individual Mutual Information Bound on Generalization Error Ruida Zhou C. Tian Tie Liu 11 31 0 17 Dec 2020
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning Yiding Jiang Pierre Foret Scott Yak Daniel M. Roy H. Mobahi Gintare Karolina Dziugaite Samy Bengio Suriya Gunasekar Isabelle M Guyon Behnam Neyshabur Google Research OOD 16 55 0 14 Dec 2020
Transfer Meta-Learning: Information-Theoretic Bounds and Information Meta-Risk Minimization Sharu Theresa Jose Osvaldo Simeone G. Durisi 19 17 0 04 Nov 2020
A Bayesian Perspective on Training Speed and Model Selection Clare Lyle Lisa Schut Binxin Ru Y. Gal Mark van der Wilk 34 23 0 27 Oct 2020
On Random Subset Generalization Error Bounds and the Stochastic Gradient Langevin Dynamics Algorithm Borja Rodríguez Gálvez Germán Bassi Ragnar Thobaben Mikael Skoglund 13 32 0 21 Oct 2020
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks Umut Simsekli Ozan Sener George Deligiannidis Murat A. Erdogdu 38 55 0 16 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 23 93 0 15 Jun 2020
Speedy Performance Estimation for Neural Architecture Search Binxin Ru Clare Lyle Lisa Schut M. Fil Mark van der Wilk Y. Gal 15 35 0 08 Jun 2020
Generalization Bounds via Information Density and Conditional Information Density Fredrik Hellström G. Durisi 13 65 0 16 May 2020
Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms Mahdi Haghifam Jeffrey Negrea Ashish Khisti Daniel M. Roy Gintare Karolina Dziugaite 19 104 0 27 Apr 2020
Reasoning About Generalization via Conditional Mutual Information Thomas Steinke Lydia Zakynthinou 13 159 0 24 Jan 2020
Pac-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning O. Catoni 142 453 0 03 Dec 2007