Exploring Inter-Channel Correlation for Diversity-preserved KnowledgeDistillation

8 February 2022

Xiaojun Chang

ArXiv (abs)PDF HTML Github (17★)

Papers citing "Exploring Inter-Channel Correlation for Diversity-preserved KnowledgeDistillation"

29 / 29 papers shown

Title
VRM: Knowledge Distillation via Virtual Relation Matching W. Zhang Fei Xie Weidong Cai Chao Ma 198 0 0 28 Feb 2025
I2CKD : Intra- and Inter-Class Knowledge Distillation for Semantic Segmentation Ayoub Karine Thibault Napoléon M. Jridi VLM 279 0 0 24 Feb 2025
CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective Wencheng Zhu Xin Zhou Pengfei Zhu Yu Wang Qinghua Hu VLM 123 1 0 22 Apr 2024
Attention-guided Feature Distillation for Semantic Segmentation Amir M. Mansourian Arya Jalali Rozhan Ahmadi S. Kasaei 188 0 0 08 Mar 2024
Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching Mingi Ji Byeongho Heo Sungrae Park 130 148 0 05 Feb 2021
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 99 299 0 06 Dec 2020
GhostNet: More Features from Cheap Operations Kai Han Yunhe Wang Qi Tian Jianyuan Guo Chunjing Xu Chang Xu 99 2,690 0 27 Nov 2019
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 176 1,054 0 23 Oct 2019
On the Efficacy of Knowledge Distillation Ligang He Rui Mao 98 619 0 03 Oct 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 142 843 0 25 Aug 2019
Similarity-Preserving Knowledge Distillation Frederick Tung Greg Mori 126 981 0 23 Jul 2019
Distilling Object Detectors with Fine-grained Feature Imitation Tao Wang Li-xin Yuan Xiaopeng Zhang Jiashi Feng ObjD 68 384 0 09 Jun 2019
Relational Knowledge Distillation Wonpyo Park Dongju Kim Yan Lu Minsu Cho 89 1,425 0 10 Apr 2019
A Comprehensive Overhaul of Feature Distillation Byeongho Heo Jeesoo Kim Sangdoo Yun Hyojin Park Nojun Kwak J. Choi 89 585 0 03 Apr 2019
Correlation Congruence for Knowledge Distillation Baoyun Peng Xiao Jin Jiaheng Liu Shunfeng Zhou Yichao Wu Yu Liu Dongsheng Li Zhaoning Zhang 94 513 0 03 Apr 2019
Knowledge Adaptation for Efficient Semantic Segmentation Tong He Chunhua Shen Zhi Tian Dong Gong Changming Sun Youliang Yan SSeg 69 225 0 12 Mar 2019
Structured Knowledge Distillation for Dense Prediction Yifan Liu Chris Liu Jingdong Wang Zhenbo Luo 98 584 0 11 Mar 2019
Self-supervised Knowledge Distillation Using Singular Value Decomposition Seunghyun Lee D. Kim B. Song 55 140 0 18 Jul 2018
Learning Deep Representations with Probabilistic Knowledge Transfer Nikolaos Passalis Anastasios Tefas 66 413 0 28 Mar 2018
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation Liang-Chieh Chen Yukun Zhu George Papandreou Florian Schroff Hartwig Adam SSeg 490 13,194 0 07 Feb 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 218 19,353 0 13 Jan 2018
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices Xiangyu Zhang Xinyu Zhou Mengxiao Lin Jian Sun AI4TS 152 6,886 0 04 Jul 2017
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer Sergey Zagoruyko N. Komodakis 147 2,586 0 12 Dec 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 358 8,002 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,510 0 10 Dec 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 367 19,745 0 09 Mar 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 469 43,347 0 11 Feb 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 328 3,906 0 19 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,529 0 04 Sep 2014