A Comprehensive Overhaul of Feature Distillation

3 April 2019

Papers citing "A Comprehensive Overhaul of Feature Distillation"

50 / 125 papers shown

Title
DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer Haiduo Huang Jiangcheng Song Yadong Zhang Pengju Ren 18 0 0 21 May 2025
FiGKD: Fine-Grained Knowledge Distillation via High-Frequency Detail Transfer Seonghak Kim 19 0 0 17 May 2025
Swapped Logit Distillation via Bi-level Teacher Alignment Stephen Ekaputra Limantoro Jhe-Hao Lin Chih-Yu Wang Yi-Lung Tsai Hong-Han Shuai Ching-Chun Huang Wen-Huang Cheng 59 0 0 27 Apr 2025
Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data Yuting He Yiqiang Chen Xiaodong Yang H. Yu Yi-Hua Huang Yang Gu FedML 69 21 0 20 Apr 2025
GranQ: Granular Zero-Shot Quantization with Channel-Wise Activation Scaling in QAT Inpyo Hong Youngwan Jo Hyojeong Lee Sunghyun Ahn Sanghyun Park MQ 67 0 0 24 Mar 2025
Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Models Y. Cai Ziqi Zhang Ding Li Yao Guo Xiangqun Chen 60 0 0 13 Mar 2025
FEDS: Feature and Entropy-Based Distillation Strategy for Efficient Learned Image Compression H. Fu Jie Liang Zhenman Fang Jingning Han 49 0 0 09 Mar 2025
VRM: Knowledge Distillation via Virtual Relation Matching W. Zhang Fei Xie Weidong Cai Chao Ma 78 0 0 28 Feb 2025
Contrastive Representation Distillation via Multi-Scale Feature Decoupling Cuipeng Wang Tieyuan Chen Haipeng Wang 54 0 0 09 Feb 2025
Variational Bayesian Adaptive Learning of Deep Latent Variables for Acoustic Knowledge Transfer Hu Hu Sabato Marco Siniscalchi Chao-Han Huck Yang Chin-Hui Lee 85 0 0 28 Jan 2025
Rethinking Knowledge in Distillation: An In-context Sample Retrieval Perspective Jinjing Zhu Songze Li Lin Wang 57 0 0 13 Jan 2025
Knowledge Distillation with Adapted Weight Sirong Wu Xi Luo Junjie Liu Yuhui Deng 53 0 0 06 Jan 2025
GazeGen: Gaze-Driven User Interaction for Visual Content Generation He-Yen Hsieh Ziyun Li Sai Qian Zhang W. Ting Kao-Den Chang B. D. Salvo Chiao Liu H. T. Kung VGen 37 0 0 07 Nov 2024
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 60 1 0 18 Oct 2024
PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillation Mike Ranzinger Jon Barker Greg Heinrich Pavlo Molchanov Bryan Catanzaro Andrew Tao 51 5 0 02 Oct 2024
Harmonizing knowledge Transfer in Neural Network with Unified Distillation Yaomin Huang Zaomin Yan Yaxin Peng Faming Fang Guixu Zhang 34 0 0 27 Sep 2024
Towards Model-Agnostic Dataset Condensation by Heterogeneous Models Jun-Yeong Moon Jung Uk Kim Gyeong-Moon Park DD 37 1 0 22 Sep 2024
Revisiting Prompt Pretraining of Vision-Language Models Zhenyuan Chen Lingfeng Yang Shuo Chen Zhaowei Chen Jiajun Liang Xiang Li MLLM VPVLM VLM 48 1 0 10 Sep 2024
Relational Representation Distillation Nikolaos Giakoumoglou Tania Stathaki 45 0 0 16 Jul 2024
HDKD: Hybrid Data-Efficient Knowledge Distillation Network for Medical Image Classification Omar S. El-Assiouti Ghada Hamed Dina Khattab H. M. Ebied 52 1 0 10 Jul 2024
FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning Seongyoon Kim Minchan Jeong Sungnyun Kim Sungwoo Cho Sumyeong Ahn Se-Young Yun FedML 55 0 0 04 Jun 2024
Aligning in a Compact Space: Contrastive Knowledge Distillation between Heterogeneous Architectures Hongjun Wu Li Xiao Xingkuo Zhang Yining Miao 48 1 0 28 May 2024
OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All Yuanhuiyi Lyu Xueye Zheng Dahun Kim Lin Wang 59 14 0 25 May 2024
CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective Wencheng Zhu Xin Zhou Pengfei Zhu Yu Wang Qinghua Hu VLM 69 1 0 22 Apr 2024
On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models Sean Farhat Deming Chen 47 0 0 04 Apr 2024
Attention-guided Feature Distillation for Semantic Segmentation Amir M. Mansourian Arya Jalali Rozhan Ahmadi S. Kasaei 41 0 0 08 Mar 2024
Distilling Privileged Multimodal Information for Expression Recognition using Optimal Transport Haseeb Aslam Muhammad Osama Zeeshan Soufiane Belharbi M. Pedersoli A. L. Koerich Simon L Bacon Eric Granger 28 9 0 27 Jan 2024
TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation Taeyang Yun Hyunkuk Lim Jeong-Hoon Lee Min Song 34 11 0 16 Jan 2024
AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One Michael Ranzinger Greg Heinrich Jan Kautz Pavlo Molchanov VLM 49 42 0 10 Dec 2023
Choosing Wisely and Learning Deeply: Selective Cross-Modality Distillation via CLIP for Domain Generalization Jixuan Leng Yijiang Li Haohan Wang VLM 37 0 0 26 Nov 2023
Maximizing Discrimination Capability of Knowledge Distillation with Energy Function Seonghak Kim Gyeongdo Ham Suin Lee Donggon Jang Daeshik Kim 39 4 0 24 Nov 2023
Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning Seonghak Kim Gyeongdo Ham Yucheol Cho Daeshik Kim 32 3 0 23 Nov 2023
Using Early Readouts to Mediate Featural Bias in Distillation Rishabh Tiwari D. Sivasubramanian Anmol Reddy Mekala Ganesh Ramakrishnan Pradeep Shenoy 26 5 0 28 Oct 2023
Understanding the Effects of Projectors in Knowledge Distillation Yudong Chen Sen Wang Jiajun Liu Xuwei Xu Frank de Hoog Brano Kusy Zi Huang 28 0 0 26 Oct 2023
Bidirectional Knowledge Reconfiguration for Lightweight Point Cloud Analysis Peipei Li Xing Cui Yibo Hu Man Zhang Ting Yao Tao Mei 38 0 0 08 Oct 2023
Review helps learn better: Temporal Supervised Knowledge Distillation Dongwei Wang Zhi Han Yanmei Wang Xi’ai Chen Baichen Liu Yandong Tang 60 1 0 03 Jul 2023
CrossKD: Cross-Head Knowledge Distillation for Object Detection Jiabao Wang Yuming Chen Zhaohui Zheng Xiang Li Ming-Ming Cheng Qibin Hou 51 33 0 20 Jun 2023
Improving Knowledge Distillation via Regularizing Feature Norm and Direction Yuzhu Wang Lechao Cheng Manni Duan Yongheng Wang Zunlei Feng Shu Kong 52 20 0 26 May 2023
Knowledge Diffusion for Distillation Tao Huang Yuan Zhang Mingkai Zheng Shan You Fei Wang Chao Qian Chang Xu 42 51 0 25 May 2023
Decoupled Kullback-Leibler Divergence Loss Jiequan Cui Zhuotao Tian Zhisheng Zhong Xiaojuan Qi Bei Yu Hanwang Zhang 41 38 0 23 May 2023
Student-friendly Knowledge Distillation Mengyang Yuan Bo Lang Fengnan Quan 22 17 0 18 May 2023
Function-Consistent Feature Distillation Dongyang Liu Meina Kan Shiguang Shan Xilin Chen 59 18 0 24 Apr 2023
Knowledge Distillation Under Ideal Joint Classifier Assumption Huayu Li Xiwen Chen G. Ditzler Janet Roveda Ao Li 18 1 0 19 Apr 2023
DIME-FM: DIstilling Multimodal and Efficient Foundation Models Ximeng Sun Pengchuan Zhang Peizhao Zhang Hardik Shah Kate Saenko Xide Xia VLM 30 20 0 31 Mar 2023
Decomposed Cross-modal Distillation for RGB-based Temporal Action Detection Pilhyeon Lee Taeoh Kim Minho Shim Dongyoon Wee H. Byun 41 11 0 30 Mar 2023
Decoupled Multimodal Distilling for Emotion Recognition Yong Li Yuan-Zheng Wang Zhen Cui 21 73 0 24 Mar 2023
From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels Zhendong Yang Ailing Zeng Zhe Li Tianke Zhang Chun Yuan Yu Li 36 74 0 23 Mar 2023
Audio Representation Learning by Distilling Video as Privileged Information Amirhossein Hajavi Ali Etemad 21 4 0 06 Feb 2023
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models Sucheng Ren Fangyun Wei Zheng-Wei Zhang Han Hu 44 35 0 03 Jan 2023
Image Classification with Small Datasets: Overview and Benchmark Lorenzo Brigato Björn Barz Luca Iocchi Joachim Denzler VLM 30 17 0 23 Dec 2022