Improved Knowledge Distillation via Teacher Assistant

Improved Knowledge Distillation via Teacher Assistant

9 February 2019

Seyed Iman Mirzadeh

Mehrdad Farajtabar

Akihiro Matsukawa

Papers citing "Improved Knowledge Distillation via Teacher Assistant"

18 / 168 papers shown

Title
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 33 199 0 08 Mar 2021
Localization Distillation for Dense Object Detection Zhaohui Zheng Rongguang Ye Ping Wang Dongwei Ren W. Zuo Qibin Hou Ming-Ming Cheng ObjD 104 115 0 24 Feb 2021
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks Yoonho Boo Sungho Shin Jungwook Choi Wonyong Sung MQ 30 29 0 30 Sep 2020
Densely Guided Knowledge Distillation using Multiple Teacher Assistants Wonchul Son Jaemin Na Junyong Choi Wonjun Hwang 25 111 0 18 Sep 2020
DualDE: Dually Distilling Knowledge Graph Embedding for Faster and Cheaper Reasoning Yushan Zhu Wen Zhang Mingyang Chen Hui Chen Xu-Xin Cheng Wei Zhang Huajun Chen Zhejiang University 22 15 0 13 Sep 2020
Learning with Privileged Information for Efficient Image Super-Resolution Wonkyung Lee Junghyup Lee Dohyung Kim Bumsub Ham 33 134 0 15 Jul 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,843 0 09 Jun 2020
ResKD: Residual-Guided Knowledge Distillation Xuewei Li Songyuan Li Bourahla Omar Fei Wu Xi Li 21 47 0 08 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
Residual Knowledge Distillation Mengya Gao Yujun Shen Quanquan Li Chen Change Loy 14 28 0 21 Feb 2020
Understanding and Improving Knowledge Distillation Jiaxi Tang Rakesh Shivanna Zhe Zhao Dong Lin Anima Singh Ed H. Chi Sagar Jain 27 129 0 10 Feb 2020
Towards Oracle Knowledge Distillation with Neural Architecture Search Minsoo Kang Jonghwan Mun Bohyung Han FedML 30 43 0 29 Nov 2019
QKD: Quantization-aware Knowledge Distillation Jangho Kim Yash Bhalgat Jinwon Lee Chirag I. Patel Nojun Kwak MQ 21 63 0 28 Nov 2019
Preparing Lessons: Improve Knowledge Distillation with Better Supervision Tiancheng Wen Shenqi Lai Xueming Qian 25 67 0 18 Nov 2019
Revisiting Knowledge Distillation via Label Smoothing Regularization Li-xin Yuan Francis E. H. Tay Guilin Li Tao Wang Jiashi Feng 22 90 0 25 Sep 2019
Knowledge Transfer Graph for Deep Collaborative Learning Soma Minami Tsubasa Hirakawa Takayoshi Yamashita H. Fujiyoshi 30 9 0 10 Sep 2019
Extreme Low Resolution Activity Recognition with Confident Spatial-Temporal Attention Transfer Yucai Bai Qinglong Zou Xieyuanli Chen Lingxi Li Zhengming Ding Long Chen 18 3 0 09 Sep 2019
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018