v1v2v3v4v5v6v7 (latest)

Knowledge Distillation: A Survey

9 June 2020

Papers citing "Knowledge Distillation: A Survey"

50 / 328 papers shown

Title
Self-supervised Label Augmentation via Input Transformations Hankook Lee Sung Ju Hwang Jinwoo Shin SSL 47 62 0 14 Oct 2019
VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition Mengjia Yan Mengao Zhao Zining Xu Qian Zhang Guoli Wang Zhizhong Su CVBM 69 92 0 11 Oct 2019
Cross-modal knowledge distillation for action recognition Fida Mohammad Thoker Juergen Gall 60 84 0 10 Oct 2019
Knowledge Distillation from Internal Representations Gustavo Aguilar Yuan Ling Yu Zhang Benjamin Yao Xing Fan Edward Guo 80 181 0 08 Oct 2019
Graph Few-shot Learning via Knowledge Transfer Huaxiu Yao Chuxu Zhang Ying Wei Meng Jiang Suhang Wang Junzhou Huang Nitesh Chawla Z. Li 112 168 0 07 Oct 2019
On the Efficacy of Knowledge Distillation Ligang He Rui Mao 98 618 0 03 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 255 7,547 0 02 Oct 2019
Training convolutional neural networks with cheap convolutions and online distillation Jiao Xie Shaohui Lin Yichen Zhang Linkai Luo 42 12 0 28 Sep 2019
Two-stage Image Classification Supervised by a Single Teacher Single Student Model Jianhang Zhou Shaoning Zeng Bob Zhang 29 4 0 26 Sep 2019
Compact Trilinear Interaction for Visual Question Answering Tuong Khanh Long Do Thanh-Toan Do Huy Tran Erman Tjiputra Quang-Dieu Tran 57 59 0 26 Sep 2019
Revisiting Knowledge Distillation via Label Smoothing Regularization Li-xin Yuan Francis E. H. Tay Guilin Li Tao Wang Jiashi Feng 53 91 0 25 Sep 2019
FEED: Feature-level Ensemble for Knowledge Distillation Seonguk Park Nojun Kwak FedML 61 42 0 24 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 113 1,869 0 23 Sep 2019
Ensemble Knowledge Distillation for Learning Improved and Efficient Networks Umar Asif Jianbin Tang S. Harrer FedML 61 75 0 17 Sep 2019
Knowledge Transfer Graph for Deep Collaborative Learning Soma Minami Tsubasa Hirakawa Takayoshi Yamashita H. Fujiyoshi 55 11 0 10 Sep 2019
Knowledge distillation for optimization of quantized deep neural networks Sungho Shin Yoonho Boo Wonyong Sung MQ 44 6 0 04 Sep 2019
Knowledge Distillation for End-to-End Person Search Bharti Munjal Fabio Galasso S. Amin FedML 90 16 0 03 Sep 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 142 843 0 25 Aug 2019
Dynamic Kernel Distillation for Efficient Pose Estimation in Videos Xuecheng Nie Yuncheng Li Linjie Luo Ning Zhang Jiashi Feng 3DH 61 65 0 24 Aug 2019
Well-Read Students Learn Better: On the Importance of Pre-training Compact Models Iulia Turc Ming-Wei Chang Kenton Lee Kristina Toutanova 65 225 0 23 Aug 2019
Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation Chengchao Shen Mengqi Xue Xinchao Wang Mingli Song Li Sun Xiuming Zhang MoMe 50 58 0 20 Aug 2019
Adaptive Regularization of Labels Qianggang Ding Sifan Wu Hao Sun Jiadong Guo Shutao Xia ODL 51 29 0 15 Aug 2019
UM-Adapt: Unsupervised Multi-Task Adaptation Using Adversarial Cross-Task Distillation Jogendra Nath Kundu Nishank Lakkakula R. Venkatesh Babu 72 59 0 11 Aug 2019
Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection Xuanyi Dong Yezhou Yang CVBM 105 67 0 06 Aug 2019
Self-Knowledge Distillation in Natural Language Processing Sangchul Hahn Heeyoul Choi 84 112 0 02 Aug 2019
Distilling Knowledge From a Deep Pose Regressor Network Muhamad Risqi U. Saputra Pedro Porto Buarque de Gusmão Yasin Almalioglu Andrew Markham A. Trigoni 81 103 0 02 Aug 2019
Learning Lightweight Lane Detection CNNs by Self Attention Distillation Yuenan Hou Zheng Ma Chunxiao Liu Chen Change Loy 68 557 0 02 Aug 2019
Lifelong GAN: Continual Learning for Conditional Image Generation Mengyao Zhai Lei Chen Frederick Tung Jiawei He Megha Nawhal Greg Mori CLL 85 183 0 23 Jul 2019
Similarity-Preserving Knowledge Distillation Frederick Tung Greg Mori 126 981 0 23 Jul 2019
Highlight Every Step: Knowledge Distillation via Collaborative Teaching Haoran Zhao Xin Sun Junyu Dong Changrui Chen Zihe Dong 71 59 0 23 Jul 2019
Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition Ye Bai Jiangyan Yi J. Tao Zhengkun Tian Zhengqi Wen KELM 66 38 0 13 Jul 2019
BAM! Born-Again Multi-Task Networks for Natural Language Understanding Kevin Clark Minh-Thang Luong Urvashi Khandelwal Christopher D. Manning Quoc V. Le 66 230 0 10 Jul 2019
Graph-based Knowledge Distillation by Multi-head Attention Network Seunghyun Lee B. Song 60 77 0 04 Jul 2019
Compression of Acoustic Event Detection Models With Quantized Distillation Bowen Shi Ming Sun Chieh-Chi Kao Viktor Rozgic Spyros Matsoukas Chao Wang 53 14 0 01 Jul 2019
Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning Sihui Luo Xinchao Wang Gongfan Fang Yao Hu Dapeng Tao Xiuming Zhang MoMe 49 48 0 24 Jun 2019
GAN-Knowledge Distillation for one-stage Object Detection Wanwei Wang Jin ke Yu Fan Zong ObjD 26 29 0 20 Jun 2019
Distilling Object Detectors with Fine-grained Feature Imitation Tao Wang Li-xin Yuan Xiaopeng Zhang Jiashi Feng ObjD 60 383 0 09 Jun 2019
When Does Label Smoothing Help? Rafael Müller Simon Kornblith Geoffrey E. Hinton UQCV 207 1,953 0 06 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 161 18,179 0 28 May 2019
Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation Hanyang Kong Jian-jun Zhao X. Tu Junliang Xing Shengmei Shen Jiashi Feng SupR CVBM 29 22 0 26 May 2019
ShrinkTeaNet: Million-scale Lightweight Face Recognition via Shrinking Teacher-Student Networks C. Duong Khoa Luu Kha Gia Quach Ngan Le CVBM 55 39 0 25 May 2019
Zero-shot Knowledge Transfer via Adversarial Belief Matching P. Micaelli Amos Storkey 54 230 0 23 May 2019
Adversarially Robust Distillation Micah Goldblum Liam H. Fowl Soheil Feizi Tom Goldstein AAML 65 210 0 23 May 2019
Zero-Shot Knowledge Distillation in Deep Networks Gaurav Kumar Nayak Konda Reddy Mopuri Vaisakh Shaj R. Venkatesh Babu Anirban Chakraborty 75 245 0 20 May 2019
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation Linfeng Zhang Jiebo Song Anni Gao Jingwei Chen Chenglong Bao Kaisheng Ma FedML 76 865 0 17 May 2019
Learning What and Where to Transfer Yunhun Jang Hankook Lee Sung Ju Hwang Jinwoo Shin 70 151 0 15 May 2019
Semi-supervised Acoustic Event Detection based on tri-training Bowen Shi Ming Sun Chieh-Chi Kao Viktor Rozgic Spyros Matsoukas Chao Wang 63 19 0 29 Apr 2019
Conditional Teacher-Student Learning Zhong Meng Jinyu Li Yong Zhao Jiawei Liu 60 91 0 28 Apr 2019
TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial Networks Md. Akmal Haidar Mehdi Rezagholizadeh 78 52 0 23 Apr 2019
Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More Jingwen Ye Yixin Ji Xinchao Wang Kairi Ou Dapeng Tao Xiuming Zhang MoMe 107 77 0 23 Apr 2019