NormKD: Normalized Logits for Knowledge Distillation

1 August 2023

Papers citing "NormKD: Normalized Logits for Knowledge Distillation"

30 / 30 papers shown

Title
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao Wenyuan Xu Shibiao Xu ViT 403 0 0 06 May 2025
Respecting Transfer Gap in Knowledge Distillation Yulei Niu Long Chen Chan Zhou Hanwang Zhang 62 24 0 23 Oct 2022
Knowledge Distillation via the Target-aware Transformer Sihao Lin Hongwei Xie Bing Wang Kaicheng Yu Xiaojun Chang Xiaodan Liang G. Wang ViT 38 104 0 22 May 2022
Masked Generative Distillation Zhendong Yang Zhe Li Mingqi Shao Dachuan Shi Zehuan Yuan Chun Yuan FedML 66 171 0 03 May 2022
Cross-Image Relational Knowledge Distillation for Semantic Segmentation Chuanguang Yang Helong Zhou Zhulin An Xue Jiang Yong Xu Qian Zhang 91 170 0 14 Apr 2022
Knowledge Distillation with the Reused Teacher Classifier Defang Chen Jianhan Mei Hailin Zhang C. Wang Yan Feng Chun-Yen Chen 54 168 0 26 Mar 2022
Decoupled Knowledge Distillation Borui Zhao Quan Cui Renjie Song Yiyu Qiu Jiajun Liang 56 536 0 16 Mar 2022
Distilling Holistic Knowledge with Graph Neural Networks Sheng Zhou Yucheng Wang Defang Chen Jiawei Chen Xin Eric Wang Can Wang Jiajun Bu 39 54 0 12 Aug 2021
Distilling Knowledge via Knowledge Review Pengguang Chen Shu Liu Hengshuang Zhao Jiaya Jia 181 429 0 19 Apr 2021
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 319 42,038 0 03 Dec 2019
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 133 1,042 0 23 Oct 2019
On the Efficacy of Knowledge Distillation Ligang He Rui Mao 88 603 0 03 Oct 2019
Relational Knowledge Distillation Wonpyo Park Dongju Kim Yan Lu Minsu Cho 63 1,396 0 10 Apr 2019
A Comprehensive Overhaul of Feature Distillation Byeongho Heo Jeesoo Kim Sangdoo Yun Hyojin Park Nojun Kwak J. Choi 76 571 0 03 Apr 2019
Snapshot Distillation: Teacher-Student Optimization in One Generation Chenglin Yang Lingxi Xie Chi Su Alan Yuille 50 193 0 01 Dec 2018
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma Xiangyu Zhang Haitao Zheng Jian Sun 145 4,957 0 30 Jul 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 167 19,124 0 13 Jan 2018
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 364 26,241 0 05 Sep 2017
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices Xiangyu Zhang Xinyu Zhou Mengxiao Lin Jian Sun AI4TS 130 6,830 0 04 Jul 2017
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer Sergey Zagoruyko N. Komodakis 108 2,561 0 12 Dec 2016
Feature Pyramid Networks for Object Detection Nayeon Lee Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 429 21,951 0 09 Dec 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 285 7,951 0 23 May 2016
Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer Jonathan Long Trevor Darrell VOS SSeg 440 37,704 0 20 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.6K 192,638 0 10 Dec 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 432 61,900 0 04 Jun 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 292 19,523 0 09 Mar 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 260 3,862 0 19 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.2K 99,991 0 04 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.3K 39,383 0 01 Sep 2014
Do Deep Nets Really Need to be Deep? Lei Jimmy Ba R. Caruana 158 2,114 0 21 Dec 2013