Hydra: Preserving Ensemble Diversity for Model Distillation

14 January 2020

Papers citing "Hydra: Preserving Ensemble Diversity for Model Distillation"

22 / 22 papers shown

Title
Improving Uncertainty Quantification of Variance Networks by Tree-Structured Learning Wenxuan Ma Xing Yan Kun Zhang UQCV 30 0 0 24 Dec 2022
Weighted Ensemble Self-Supervised Learning Yangjun Ruan Saurabh Singh Warren Morningstar Alexander A. Alemi Sergey Ioffe Ian S. Fischer Joshua V. Dillon FedML 29 15 0 18 Nov 2022
Label driven Knowledge Distillation for Federated Learning with non-IID Data Minh-Duong Nguyen Viet Quoc Pham D. Hoang Long Tran-Thanh Diep N. Nguyen W. Hwang 28 2 0 29 Sep 2022
Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation G. Nam Hyungi Lee Byeongho Heo Juho Lee UQCV FedML 31 7 0 30 Jun 2022
Functional Ensemble Distillation Coby Penso Idan Achituve Ethan Fetaya FedML 37 2 0 05 Jun 2022
Simple Regularisation for Uncertainty-Aware Knowledge Distillation Martin Ferianc Miguel R. D. Rodrigues UQCV 49 0 0 19 May 2022
Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning Yae Jee Cho Andre Manoel Gauri Joshi Robert Sim Dimitrios Dimitriadis FedML 32 129 0 27 Apr 2022
Improving robustness and calibration in ensembles with diversity regularization H. A. Mehrtens Camila González Anirban Mukhopadhyay UQCV 19 7 0 26 Jan 2022
Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems Yoshitomo Matsubara Luca Soldaini Eric Lind Alessandro Moschitti 34 6 0 15 Jan 2022
A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications Xinlei Zhou Han Liu Farhad Pourpanah T. Zeng Xizhao Wang UQCV UD 29 58 0 03 Nov 2021
Diversity Matters When Learning From Ensembles G. Nam Jongmin Yoon Yoonho Lee Juho Lee UQCV FedML VLM 43 36 0 27 Oct 2021
Learning Rich Nearest Neighbor Representations from Self-supervised Ensembles Bram Wallace Devansh Arpit Huan Wang Caiming Xiong SSL OOD 32 0 0 19 Oct 2021
Deep Classifiers with Label Noise Modeling and Distance Awareness Vincent Fortuin Mark Collier F. Wenzel J. Allingham J. Liu Dustin Tran Balaji Lakshminarayanan Jesse Berent Rodolphe Jenatton E. Kokiopoulou UQCV 36 11 0 06 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 69 697 0 04 Sep 2021
Twin Neural Network Regression S. J. Wetzel Kevin Ryczko R. Melko Isaac Tamblyn UQCV 28 11 0 29 Dec 2020
Training independent subnetworks for robust prediction Marton Havasi Rodolphe Jenatton Stanislav Fort Jeremiah Zhe Liu Jasper Snoek Balaji Lakshminarayanan Andrew M. Dai Dustin Tran UQCV OOD 41 208 0 13 Oct 2020
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge Chaoyang He M. Annavaram A. Avestimehr FedML 28 23 0 28 Jul 2020
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning Arsenii Ashukha Alexander Lyzhov Dmitry Molchanov Dmitry Vetrov UQCV FedML 38 314 0 15 Feb 2020
Self-Distillation Amplifies Regularization in Hilbert Space H. Mobahi Mehrdad Farajtabar Peter L. Bartlett 35 229 0 13 Feb 2020
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 212 474 0 12 Jun 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 276 5,695 0 05 Dec 2016