v1v2v3v4v5v6v7 (latest)

Knowledge Distillation: A Survey

9 June 2020

Papers citing "Knowledge Distillation: A Survey"

50 / 328 papers shown

Title
KTAN: Knowledge Transfer Adversarial Network Peiye Liu Wu Liu Huadong Ma Tao Mei Mingoo Seok GAN 74 28 0 18 Oct 2018
Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Setting Mennatullah Siam Chen Jiang S. Lu Laura Petrich Mahmoud Gamal Mohamed Elhoseiny Martin Jägersand VOS 143 72 0 17 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System Jiaxi Tang Ke Wang 64 189 0 19 Sep 2018
Attention-Guided Answer Distillation for Machine Reading Comprehension Minghao Hu Yuxing Peng Furu Wei Zhen Huang Dongsheng Li Nan Yang M. Zhou FaML 69 75 0 23 Aug 2018
Emotion Recognition in Speech using Cross-Modal Transfer in the Wild Samuel Albanie Arsha Nagrani Andrea Vedaldi Andrew Zisserman CVBM 66 271 0 16 Aug 2018
Teacher Guided Architecture Search P. Bashivan Mark Tensen J. DiCarlo 3DV 66 27 0 04 Aug 2018
MnasNet: Platform-Aware Neural Architecture Search for Mobile Mingxing Tan Bo Chen Ruoming Pang Vijay Vasudevan Mark Sandler Andrew G. Howard Quoc V. Le MQ 128 3,015 0 31 Jul 2018
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma Xiangyu Zhang Haitao Zheng Jian Sun 181 5,006 0 30 Jul 2018
Self-supervised Knowledge Distillation Using Singular Value Decomposition Seunghyun Lee D. Kim B. Song 52 140 0 18 Jul 2018
Modality Distillation with Multiple Stream Networks for Action Recognition Nuno C. Garcia Pietro Morerio Vittorio Murino 71 184 0 19 Jun 2018
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 293 480 0 12 Jun 2018
Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students Chenglin Yang Lingxi Xie Siyuan Qiao Alan Yuille 70 136 0 15 May 2018
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary Byeongho Heo Minsik Lee Sangdoo Yun J. Choi AAML 92 146 0 15 May 2018
Born Again Neural Networks Tommaso Furlanello Zachary Chase Lipton Michael Tschannen Laurent Itti Anima Anandkumar 80 1,034 0 12 May 2018
Label Refinery: Improving ImageNet Classification through Label Progression Hessam Bagherinezhad Maxwell Horton Mohammad Rastegari Ali Farhadi 62 190 0 07 May 2018
Quantization Mimic: Towards Very Tiny CNN for Object Detection Yi Wei Xinyu Pan Hongwei Qin Wanli Ouyang Junjie Yan ObjD 116 89 0 06 May 2018
Boosting Self-Supervised Learning via Knowledge Transfer M. Noroozi Ananth Vinjimoor Paolo Favaro Hamed Pirsiavash SSL 287 296 0 01 May 2018
Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification Chenrui Zhang Yuxin Peng SSL 71 56 0 26 Apr 2018
An Information-Theoretic View for Deep Learning Jingwei Zhang Tongliang Liu Dacheng Tao MLT FAtt 168 25 0 24 Apr 2018
Neural Compatibility Modeling with Attentive Knowledge Distillation Xuemeng Song Fuli Feng Xianjing Han Xin Yang Wen Liu Liqiang Nie 99 145 0 17 Apr 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 320 408 0 09 Apr 2018
Learning Deep Representations with Probabilistic Knowledge Transfer Nikolaos Passalis Anastasios Tefas 63 412 0 28 Mar 2018
Adversarial Network Compression Vasileios Belagiannis Azade Farshad Fabio Galasso GAN AAML 48 58 0 28 Mar 2018
Knowledge Transfer with Jacobian Matching Suraj Srinivas François Fleuret 53 171 0 01 Mar 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 105 488 0 19 Feb 2018
Model compression via distillation and quantization A. Polino Razvan Pascanu Dan Alistarh MQ 86 732 0 15 Feb 2018
Paraphrasing Complex Network: Network Compression via Factor Transfer Jangho Kim Seonguk Park Nojun Kwak 77 551 0 14 Feb 2018
Few-shot learning of neural networks from scratch by pseudo example optimization Akisato Kimura Zoubin Ghahramani Koh Takeuchi Tomoharu Iwata N. Ueda 64 52 0 08 Feb 2018
Deep Net Triage: Analyzing the Importance of Network Layers via Structural Compression Theodore S. Nowak Jason J. Corso FAtt 30 3 0 15 Jan 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 204 19,333 0 13 Jan 2018
Data Distillation: Towards Omni-Supervised Learning Ilija Radosavovic Piotr Dollár Ross B. Girshick Georgia Gkioxari Kaiming He 87 419 0 12 Dec 2017
Graph Distillation for Action Detection with Privileged Modalities Zelun Luo Jun-Ting Hsieh Lu Jiang Juan Carlos Niebles Li Fei-Fei 84 104 0 30 Nov 2017
Parallel WaveNet: Fast High-Fidelity Speech Synthesis Aaron van den Oord Yazhe Li Igor Babuschkin Karen Simonyan Oriol Vinyals ... Alex Graves Helen King T. Walters Dan Belov Demis Hassabis 221 859 0 28 Nov 2017
Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients A. Ross Finale Doshi-Velez AAML 154 683 0 26 Nov 2017
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy Asit K. Mishra Debbie Marr FedML 65 331 0 15 Nov 2017
Moonshine: Distilling with Cheap Convolutions Elliot J. Crowley Gavia Gray Amos Storkey 59 121 0 07 Nov 2017
Data-Free Knowledge Distillation for Deep Neural Networks Raphael Gontijo-Lopes Stefano Fenu Thad Starner 60 273 0 19 Oct 2017
N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning A. Ashok Nicholas Rhinehart Fares N. Beainy Kris Kitani 69 171 0 18 Sep 2017
Model Distillation with Knowledge Transfer from Face Classification to Alignment and Verification Chong-Jun Wang Xipeng Lan Yang Zhang CVBM 63 26 0 09 Sep 2017
Incremental Learning of Object Detectors without Catastrophic Forgetting K. Shmelkov Cordelia Schmid Alahari Karteek ObjD 82 520 0 23 Aug 2017
Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Net Guorui Zhou Ying Fan Runpeng Cui Weijie Bian Xiaoqiang Zhu Kun Gai 71 116 0 14 Aug 2017
DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer Yuntao Chen Naiyan Wang Zhaoxiang Zhang FedML 102 223 0 05 Jul 2017
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices Xiangyu Zhang Xinyu Zhou Mengxiao Lin Jian Sun AI4TS 147 6,884 0 04 Jul 2017
Deep Mutual Learning Ying Zhang Tao Xiang Timothy M. Hospedales Huchuan Lu FedML 153 1,654 0 01 Jun 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.2K 20,892 0 17 Apr 2017
Knowledge Adaptation: Teaching to Adapt Sebastian Ruder Parsa Ghaffari J. Breslin CLL TTA 56 53 0 07 Feb 2017
Ensemble Distillation for Neural Machine Translation Markus Freitag Yaser Al-Onaizan B. Sankaran FedML 50 111 0 06 Feb 2017
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer Sergey Zagoruyko N. Komodakis 147 2,586 0 12 Dec 2016
In Teacher We Trust: Learning Compressed Models for Pedestrian Detection Jonathan Shen Noranart Vesdapunt Vishnu Boddeti Kris Kitani 39 29 0 01 Dec 2016