Learning Student-Friendly Teacher Networks for Knowledge Distillation

Learning Student-Friendly Teacher Networks for Knowledge Distillation

12 February 2021

Papers citing "Learning Student-Friendly Teacher Networks for Knowledge Distillation"

17 / 17 papers shown

Title
Rethinking Knowledge in Distillation: An In-context Sample Retrieval Perspective Jinjing Zhu Songze Li Lin Wang 47 0 0 13 Jan 2025
Quantifying Knowledge Distillation Using Partial Information Decomposition Pasan Dissanayake Faisal Hamman Barproda Halder Ilia Sucholutsky Qiuyi Zhang Sanghamitra Dutta 36 0 0 12 Nov 2024
Collaborative Learning for Enhanced Unsupervised Domain Adaptation Minhee Cho Hyesong Choi Hayeon Jo Dongbo Min 27 1 0 04 Sep 2024
Self-supervised Video Object Segmentation with Distillation Learning of Deformable Attention Quang-Trung Truong Duc Thanh Nguyen Binh-Son Hua Sai-Kit Yeung VOS 34 1 0 25 Jan 2024
Teacher-Student Architecture for Knowledge Distillation: A Survey Chengming Hu Xuan Li Danyang Liu Haolun Wu Xi Chen Ju Wang Xue Liu 21 16 0 08 Aug 2023
Knowledge Diffusion for Distillation Tao Huang Yuan Zhang Mingkai Zheng Shan You Fei Wang Chao Qian Chang Xu 37 50 0 25 May 2023
Lifting the Curse of Capacity Gap in Distilling Language Models Chen Zhang Yang Yang Jiahao Liu Jingang Wang Yunsen Xian Benyou Wang Dawei Song MoE 32 19 0 20 May 2023
Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation Yuxin Ren Zi-Qi Zhong Xingjian Shi Yi Zhu Chun Yuan Mu Li 21 7 0 16 May 2023
Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket Nianhui Guo Joseph Bethge Christoph Meinel Haojin Yang MQ 28 19 0 23 Nov 2022
Teacher-Student Architecture for Knowledge Learning: A Survey Chengming Hu Xuan Li Dan Liu Xi Chen Ju Wang Xue Liu 20 35 0 28 Oct 2022
Multi-Level Branched Regularization for Federated Learning Jinkyu Kim Geeho Kim Bohyung Han FedML 10 53 0 14 Jul 2022
Parameter-Efficient and Student-Friendly Knowledge Distillation Jun Rao Xv Meng Liang Ding Shuhan Qi Dacheng Tao 37 46 0 28 May 2022
Learn From the Past: Experience Ensemble Knowledge Distillation Chaofei Wang Shaowei Zhang S. Song Gao Huang 25 4 0 25 Feb 2022
It's All in the Head: Representation Knowledge Distillation through Classifier Sharing Emanuel Ben-Baruch M. Karklinsky Yossi Biton Avi Ben-Cohen Hussam Lawen Nadav Zamir 18 11 0 18 Jan 2022
Prune Your Model Before Distill It Jinhyuk Park Albert No VLM 38 27 0 30 Sep 2021
Collaborative Distillation for Ultra-Resolution Universal Style Transfer Huan Wang Yijun Li Yuehai Wang Haoji Hu Ming-Hsuan Yang 107 103 0 18 Mar 2020
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 192 473 0 12 Jun 2018