Follow Your Path: a Progressive Method for Knowledge Distillation

20 July 2021

Wenxian Shi

Lei Li

Papers citing "Follow Your Path: a Progressive Method for Knowledge Distillation"

27 / 27 papers shown

Title
Towards Understanding Knowledge Distillation Mary Phuong Christoph H. Lampert 58 314 0 27 May 2021
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 101 1,042 0 23 Oct 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 65 1,847 0 23 Sep 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 103 833 0 25 Aug 2019
Knowledge Distillation via Route Constrained Optimization Xiao Jin Baoyun Peng Yichao Wu Yu Liu Jiaheng Liu Ding Liang Junjie Yan Xiaolin Hu 37 169 0 19 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.0K 93,936 0 11 Oct 2018
AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods Zhiming Zhou Qingru Zhang Guansong Lu Hongwei Wang Weinan Zhang Yong Yu 41 66 0 29 Sep 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 675 7,080 0 20 Apr 2018
CoT: Cooperative Training for Generative Modeling of Discrete Data Sidi Lu Lantao Yu Siyuan Feng Yaoming Zhu Weinan Zhang Yong Yu GAN 47 19 0 11 Apr 2018
Model compression via distillation and quantization A. Polino Razvan Pascanu Dan Alistarh MQ 67 722 0 15 Feb 2018
AMC: AutoML for Model Compression and Acceleration on Mobile Devices Yihui He Ji Lin Zhijian Liu Hanrui Wang Li Li Song Han 69 1,348 0 10 Feb 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 151 19,124 0 13 Jan 2018
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices Xiangyu Zhang Xinyu Zhou Mengxiao Lin Jian Sun AI4TS 120 6,830 0 04 Jul 2017
Generative Bridging Network in Neural Sequence Prediction Wenhu Chen Guanlin Li Shuo Ren Shujie Liu Zhirui Zhang Mu Li M. Zhou 40 10 0 28 Jun 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 492 129,831 0 12 Jun 2017
Deep Mutual Learning Ying Zhang Tao Xiang Timothy M. Hospedales Huchuan Lu FedML 128 1,647 0 01 Jun 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 421 4,444 0 18 Apr 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.1K 20,747 0 17 Apr 2017
Guided Policy Search as Approximate Mirror Descent William H. Montgomery Sergey Levine 54 125 0 15 Jul 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 168 8,067 0 16 Jun 2016
Smooth Imitation Learning for Online Sequence Prediction Hoang Minh Le Andrew Kang Yisong Yue Peter Carr 50 33 0 03 Jun 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 270 7,951 0 23 May 2016
Quantized Convolutional Neural Networks for Mobile Devices Jiaxiang Wu Cong Leng Yuhang Wang Qinghao Hu Jian Cheng MQ 73 1,163 0 21 Dec 2015
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.5K 192,638 0 10 Dec 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 248 19,523 0 09 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 940 149,474 0 22 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 989 99,991 0 04 Sep 2014