v1v2 (latest)

Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization

14 February 2024

Idan Attias

Gintare Karolina Dziugaite

Papers citing "Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization"

35 / 35 papers shown

Title
Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds Ziqiao Wang Yongyi Mao 85 7 0 31 Oct 2023
Generalization Bounds: Perspectives from Information Theory and PAC-Bayes Fredrik Hellström G. Durisi Benjamin Guedj Maxim Raginsky 51 38 0 08 Sep 2023
Optimal Learners for Realizable Regression: PAC Learning and Online Learning Idan Attias Steve Hanneke Alkis Kalavasis Amin Karbasi Grigoris Velegkas 87 21 0 07 Jul 2023
Information Theoretic Lower Bounds for Information Theoretic Upper Bounds Roi Livni 87 16 0 09 Feb 2023
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization Mahdi Haghifam Borja Rodríguez Gálvez Ragnar Thobaben Mikael Skoglund Daniel M. Roy Gintare Karolina Dziugaite 76 19 0 27 Dec 2022
Evaluated CMI Bounds for Meta Learning: Tightness and Expressiveness Fredrik Hellström G. Durisi 63 13 0 12 Oct 2022
Reconstructing Training Data from Trained Neural Networks Niv Haim Gal Vardi Gilad Yehudai Ohad Shamir Michal Irani 89 141 0 15 Jun 2022
Strong Memory Lower Bounds for Learning Natural Models Gavin Brown Mark Bun Adam D. Smith 78 12 0 09 Jun 2022
A Characterization of Multiclass Learnability Nataly Brukhim Dan Carmon Irit Dinur Shay Moran Amir Yehudayoff 67 56 0 03 Mar 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 124 630 0 15 Feb 2022
Membership Inference Attacks From First Principles Nicholas Carlini Steve Chien Milad Nasr Shuang Song Andreas Terzis Florian Tramèr MIACV MIALM 87 709 0 07 Dec 2021
Towards a Unified Information-Theoretic Framework for Generalization Mahdi Haghifam Gintare Karolina Dziugaite Shay Moran Daniel M. Roy 142 34 0 09 Nov 2021
Information-theoretic generalization bounds for black-box learning algorithms Hrayr Harutyunyan Maxim Raginsky Greg Ver Steeg Aram Galstyan 120 44 0 04 Oct 2021
Adversarially Robust Learning with Unknown Perturbation Sets Omar Montasser Steve Hanneke Nathan Srebro AAML 74 28 0 03 Feb 2021
SGD Generalizes Better Than GD (And Regularization Doesn't Help) I Zaghloul Amir Tomer Koren Roi Livni 52 46 0 01 Feb 2021
Individually Conditional Individual Mutual Information Bound on Generalization Error Ruida Zhou C. Tian Tie Liu 73 32 0 17 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown Basel Alomair Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 512 1,953 0 14 Dec 2020
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning? Gavin Brown Mark Bun Vitaly Feldman Adam D. Smith Kunal Talwar 311 100 0 11 Dec 2020
Reducing Adversarially Robust Learning to Non-Robust PAC Learning Omar Montasser Steve Hanneke Nathan Srebro 88 33 0 22 Oct 2020
On Random Subset Generalization Error Bounds and the Stochastic Gradient Langevin Dynamics Algorithm Borja Rodríguez Gálvez Germán Bassi Ragnar Thobaben Mikael Skoglund 86 32 0 21 Oct 2020
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation Vitaly Feldman Chiyuan Zhang TDI 219 470 0 09 Aug 2020
A Limitation of the PAC-Bayes Framework Roi Livni Shay Moran 65 25 0 24 Jun 2020
Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms Mahdi Haghifam Jeffrey Negrea Ashish Khisti Daniel M. Roy Gintare Karolina Dziugaite 182 108 0 27 Apr 2020
Reasoning About Generalization via Conditional Mutual Information Thomas Steinke Lydia Zakynthinou 156 166 0 24 Jan 2020
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates Jeffrey Negrea Mahdi Haghifam Gintare Karolina Dziugaite Ashish Khisti Daniel M. Roy FedML 191 153 0 06 Nov 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 142 500 0 12 Jun 2019
VC Classes are Adversarially Robustly Learnable, but Only Improperly Omar Montasser Steve Hanneke Nathan Srebro 47 141 0 12 Feb 2019
A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm Chi Jin Praneeth Netrapalli Rong Ge Sham Kakade Michael I. Jordan 115 150 0 11 Feb 2019
Tightening Mutual Information Based Bounds on Generalization Error Yuheng Bu Shaofeng Zou Venugopal V. Veeravalli 62 177 0 15 Jan 2019
Privately Learning High-Dimensional Distributions Gautam Kamath Jerry Li Vikrant Singhal Jonathan R. Ullman FedML 97 150 0 01 May 2018
Information-theoretic analysis of generalization capability of learning algorithms Aolin Xu Maxim Raginsky 191 448 0 22 May 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 351 4,636 0 10 Nov 2016
Membership Inference Attacks against Machine Learning Models Reza Shokri M. Stronati Congzheng Song Vitaly Shmatikov SLR MIALM MIACV 280 4,160 0 18 Oct 2016
Sample compression schemes for VC classes Shay Moran Amir Yehudayoff 94 93 0 24 Mar 2015
Fingerprinting Codes and the Price of Approximate Differential Privacy Mark Bun Jonathan R. Ullman Salil P. Vadhan FedML 91 212 0 13 Nov 2013