Decoupled Knowledge Distillation

16 March 2022

Jiajun Liang

Papers citing "Decoupled Knowledge Distillation"

50 / 78 papers shown

Title
Uncertainty-Aware Multi-Expert Knowledge Distillation for Imbalanced Disease Grading Shuo Tong Shangde Gao Ke Liu Zihang Huang Hongxia Xu Haochao Ying Jian Wu 24 0 0 01 May 2025
Head-Tail-Aware KL Divergence in Knowledge Distillation for Spiking Neural Networks Tianqing Zhang Zixin Zhu Kairong Yu Hongwei Wang 151 0 0 29 Apr 2025
Swapped Logit Distillation via Bi-level Teacher Alignment Stephen Ekaputra Limantoro Jhe-Hao Lin Chih-Yu Wang Yi-Lung Tsai Hong-Han Shuai Ching-Chun Huang Wen-Huang Cheng 54 0 0 27 Apr 2025
Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understanding Jinlong Li Cristiano Saltori Fabio Poiesi N. Sebe 168 0 0 20 Mar 2025
VRM: Knowledge Distillation via Virtual Relation Matching W. Zhang Fei Xie Weidong Cai Chao Ma 76 0 0 28 Feb 2025
TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models Makoto Shing Kou Misaki Han Bao Sho Yokoi Takuya Akiba VLM 57 1 0 28 Jan 2025
Rethinking Knowledge in Distillation: An In-context Sample Retrieval Perspective Jinjing Zhu Songze Li Lin Wang 47 0 0 13 Jan 2025
Data Pruning Can Do More: A Comprehensive Data Pruning Approach for Object Re-identification Zi Yang Haojin Yang Soumajit Majumder Jorge M. Cardoso Guillermo Gallego MoMe VLM 100 1 0 13 Dec 2024
Linear Projections of Teacher Embeddings for Few-Class Distillation Noel Loo Fotis Iliopoulos Wei Hu Erik Vee 30 0 0 30 Sep 2024
Harmonizing knowledge Transfer in Neural Network with Unified Distillation Yaomin Huang Zaomin Yan Chaomin Shen Faming Fang Guixu Zhang 29 0 0 27 Sep 2024
Towards Model-Agnostic Dataset Condensation by Heterogeneous Models Jun-Yeong Moon Jung Uk Kim Gyeong-Moon Park DD 33 1 0 22 Sep 2024
Deep Companion Learning: Enhancing Generalization Through Historical Consistency Ruizhao Zhu Venkatesh Saligrama FedML 29 0 0 26 Jul 2024
Continual Distillation Learning: Knowledge Distillation in Prompt-based Continual Learning Qifan Zhang Yunhui Guo Yu Xiang VLM CLL 56 0 0 18 Jul 2024
Instance Temperature Knowledge Distillation Zhengbo Zhang Yuxi Zhou Jia Gong Jun Liu Zhigang Tu 28 2 0 27 Jun 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 50 2 0 12 Jun 2024
ReDistill: Residual Encoded Distillation for Peak Memory Reduction of CNNs Fang Chen Gourav Datta Mujahid Al Rafi Hyeran Jeon Meng Tang 93 1 0 06 Jun 2024
FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning Seongyoon Kim Minchan Jeong Sungnyun Kim Sungwoo Cho Sumyeong Ahn Se-Young Yun FedML 47 0 0 04 Jun 2024
GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost Xinyi Shang Peng Sun Tao Lin 50 2 0 23 May 2024
CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective Wencheng Zhu Xin Zhou Pengfei Zhu Yu Wang Qinghua Hu VLM 61 1 0 22 Apr 2024
MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities Kunxi Li Tianyu Zhan Kairui Fu Shengyu Zhang Kun Kuang Jiwei Li Zhou Zhao Fei Wu MoMe 24 0 0 20 Apr 2024
FedDistill: Global Model Distillation for Local Model De-Biasing in Non-IID Federated Learning Changlin Song Divya Saxena Jiannong Cao Yuqing Zhao FedML 42 3 0 14 Apr 2024
On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models Sean Farhat Deming Chen 42 0 0 04 Apr 2024
TSCM: A Teacher-Student Model for Vision Place Recognition Using Cross-Metric Knowledge Distillation Yehui Shen Mingmin Liu Huimin Lu Xieyuanli Chen 39 1 0 02 Apr 2024
Learning to Project for Cross-Task Knowledge Distillation Dylan Auty Roy Miles Benedikt Kolbeinsson K. Mikolajczyk 40 0 0 21 Mar 2024
LIX: Implicitly Infusing Spatial Geometric Prior Knowledge into Visual Semantic Segmentation for Autonomous Driving Sicen Guo Zhiyuan Wu Qijun Chen Ioannis Pitas Rui Fan Rui Fan 37 1 0 13 Mar 2024
GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation Ayan Banerjee Sanket Biswas Josep Lladós Umapada Pal 38 1 0 17 Feb 2024
Data-efficient Large Vision Models through Sequential Autoregression Jianyuan Guo Zhiwei Hao Chengcheng Wang Yehui Tang Han Wu Han Hu Kai Han Chang Xu VLM 38 10 0 07 Feb 2024
Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF Banghua Zhu Michael I. Jordan Jiantao Jiao 31 25 0 29 Jan 2024
Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion Cunhang Fan Yujie Chen Jun Xue Yonghui Kong Jianhua Tao Zhao Lv 22 2 0 19 Jan 2024
Revisiting Knowledge Distillation under Distribution Shift Songming Zhang Ziyu Lyu Xiaofeng Chen 29 1 0 25 Dec 2023
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything Yunyang Xiong Bala Varadarajan Lemeng Wu Xiaoyu Xiang Fanyi Xiao ... Dilin Wang Fei Sun Forrest N. Iandola Raghuraman Krishnamoorthi Vikas Chandra VLM 40 139 0 01 Dec 2023
Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning Seonghak Kim Gyeongdo Ham Yucheol Cho Daeshik Kim 22 2 0 23 Nov 2023
Understanding the Effects of Projectors in Knowledge Distillation Yudong Chen Sen Wang Jiajun Liu Xuwei Xu Frank de Hoog Brano Kusy Zi Huang 26 0 0 26 Oct 2023
Weight Averaging Improves Knowledge Distillation under Domain Shift Valeriy Berezovskiy Nikita Morozov MoMe 31 1 0 20 Sep 2023
Multi-Label Knowledge Distillation Penghui Yang Ming-Kun Xie Chen-Chen Zong Lei Feng Gang Niu Masashi Sugiyama Sheng-Jun Huang 33 10 0 12 Aug 2023
Teacher-Student Architecture for Knowledge Distillation: A Survey Chengming Hu Xuan Li Danyang Liu Haolun Wu Xi Chen Ju Wang Xue Liu 21 16 0 08 Aug 2023
Make A Long Image Short: Adaptive Token Length for Vision Transformers Yuqin Zhu Yichen Zhu ViT 69 17 0 05 Jul 2023
Review helps learn better: Temporal Supervised Knowledge Distillation Dongwei Wang Zhi Han Yanmei Wang Xi’ai Chen Baichen Liu Yandong Tang 57 1 0 03 Jul 2023
CrossKD: Cross-Head Knowledge Distillation for Object Detection Jiabao Wang Yuming Chen Zhaohui Zheng Xiang Li Ming-Ming Cheng Qibin Hou 40 32 0 20 Jun 2023
VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning Challenges Robert-Jan Bruintjes A. Lengyel Marcos Baptista-Rios O. Kayhan Davide Zambrano Nergis Tomen J. C. V. Gemert 25 9 0 31 May 2023
Improving Knowledge Distillation via Regularizing Feature Norm and Direction Yuzhu Wang Lechao Cheng Manni Duan Yongheng Wang Zunlei Feng Shu Kong 29 19 0 26 May 2023
Knowledge Diffusion for Distillation Tao Huang Yuan Zhang Mingkai Zheng Shan You Fei Wang Chao Qian Chang Xu 37 50 0 25 May 2023
Decoupled Kullback-Leibler Divergence Loss Jiequan Cui Zhuotao Tian Zhisheng Zhong Xiaojuan Qi Bei Yu Hanwang Zhang 39 38 0 23 May 2023
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation? Zheng Li Yuxuan Li Penghai Zhao Renjie Song Xiang Li Jian Yang 34 19 0 22 May 2023
Lifting the Curse of Capacity Gap in Distilling Language Models Chen Zhang Yang Yang Jiahao Liu Jingang Wang Yunsen Xian Benyou Wang Dawei Song MoE 32 19 0 20 May 2023
Student-friendly Knowledge Distillation Mengyang Yuan Bo Lang Fengnan Quan 20 17 0 18 May 2023
Do Not Blindly Imitate the Teacher: Using Perturbed Loss for Knowledge Distillation Rongzhi Zhang Jiaming Shen Tianqi Liu Jia-Ling Liu Michael Bendersky Marc Najork Chao Zhang 48 18 0 08 May 2023
Function-Consistent Feature Distillation Dongyang Liu Meina Kan Shiguang Shan Xilin Chen 44 18 0 24 Apr 2023
Knowledge Distillation Under Ideal Joint Classifier Assumption Huayu Li Xiwen Chen G. Ditzler Janet Roveda Ao Li 18 1 0 19 Apr 2023
Grouped Knowledge Distillation for Deep Face Recognition Weisong Zhao Xiangyu Zhu Kaiwen Guo Xiaoyu Zhang Zhen Lei CVBM 15 6 0 10 Apr 2023