Generalization bounds via distillation

12 April 2021

Papers citing "Generalization bounds via distillation"

27 / 27 papers shown

Title
Disentangling Locality and Entropy in Ranking Distillation Andrew Parry Debasis Ganguly Sean MacAvaney 65 0 0 27 May 2025
Generalization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior Romain Chor Abdellatif Zaidi Piotr Krasnowski 101 1 0 25 Apr 2025
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors Romain Chor Milad Sefidgaran Piotr Krasnowski 292 2 0 21 Feb 2025
How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning Arthur Jacot Seok Hoan Choi Yuxiao Wen AI4CE 147 2 0 08 Jul 2024
Slicing Mutual Information Generalization Bounds for Neural Networks Kimia Nadjahi Kristjan Greenewald Rickard Brüel-Gabrielsson Justin Solomon 99 4 0 06 Jun 2024
FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillation Yuting Ma Lechao Cheng Yaxiong Wang Zhun Zhong Xiaohua Xu Meng Wang FedML 82 4 0 27 May 2024
Exploring Dark Knowledge under Various Teacher Capacities and Addressing Capacity Mismatch Wen-Shu Fan Xin-Chun Li Bowen Tao 84 2 0 21 May 2024
Minimum Description Length and Generalization Guarantees for Representation Learning Romain Chor Abdellatif Zaidi Piotr Krasnowski 138 10 0 05 Feb 2024
Towards the Fundamental Limits of Knowledge Transfer over Finite Domains Qingyue Zhao Banghua Zhu 100 4 0 11 Oct 2023
Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learning B. Le Simon S. Woo AAML 83 29 0 12 Sep 2023
Cluster-aware Semi-supervised Learning: Relational Knowledge Distillation Provably Learns Clustering Yijun Dong Kevin Miller Qiuyu Lei Rachel A. Ward 74 4 0 20 Jul 2023
Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD Yijun Wan Melih Barsbey Milad Sefidgaran Umut Simsekli 82 1 0 13 Jun 2023
Do Not Blindly Imitate the Teacher: Using Perturbed Loss for Knowledge Distillation Rongzhi Zhang Jiaming Shen Tianqi Liu Jia-Ling Liu Michael Bendersky Marc Najork Chao Zhang 104 20 0 08 May 2023
Learning From Biased Soft Labels Hua Yuan Ning Xu Yuge Shi Xin Geng Yong Rui FedML 87 6 0 16 Feb 2023
PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees Chulin Xie De-An Huang Wen-Hsuan Chu Daguang Xu Chaowei Xiao Yue Liu Anima Anandkumar FedML 98 12 0 13 Feb 2023
Asymmetric Temperature Scaling Makes Larger Networks Teach Well Again Xin-Chun Li Wenxuan Fan Shaoming Song Yinchuan Li Bingshuai Li Yunfeng Shao De-Chuan Zhan 123 31 0 10 Oct 2022
Effectiveness of Function Matching in Driving Scene Recognition Shingo Yashima 48 1 0 20 Aug 2022
Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning Romain Chor Abdellatif Zaidi Milad Sefidgaran FedML 87 15 0 06 Jun 2022
Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms Romain Chor A. Gohari Gaël Richard Umut Simsekli 109 24 0 04 Mar 2022
The Sample Complexity of One-Hidden-Layer Neural Networks Gal Vardi Ohad Shamir Nathan Srebro 69 10 0 13 Feb 2022
On Predicting Generalization using GANs Yi Zhang Arushi Gupta Nikunj Saunshi Sanjeev Arora AI4CE 172 6 0 28 Nov 2021
Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks Tolga Birdal Aaron Lou Leonidas Guibas Umut cSimcsekli 87 65 0 25 Nov 2021
MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps Muhammad Awais Fengwei Zhou Chuanlong Xie Jiawei Li Sung-Ho Bae Zhenguo Li AAML 87 18 0 09 Nov 2021
Self-training Converts Weak Learners to Strong Learners in Mixture Models Spencer Frei Difan Zou Zixiang Chen Quanquan Gu 95 17 0 25 Jun 2021
Compression Implies Generalization Allan Grønlund M. Hogsgaard Lior Kamma Kasper Green Larsen MLT AI4CE 23 0 0 15 Jun 2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms A. Camuto George Deligiannidis Murat A. Erdogdu Mert Gurbuzbalaban Umut cSimcsekli Lingjiong Zhu 83 29 0 09 Jun 2021
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks Melih Barsbey Romain Chor Murat A. Erdogdu Gaël Richard Umut Simsekli 73 41 0 07 Jun 2021