Teacher-Student Architecture for Knowledge Distillation: A Survey

8 August 2023

Xue Liu

Papers citing "Teacher-Student Architecture for Knowledge Distillation: A Survey"

42 / 142 papers shown

Title
Emotion Recognition in Speech using Cross-Modal Transfer in the Wild Samuel Albanie Arsha Nagrani Andrea Vedaldi Andrew Zisserman CVBM 64 271 0 16 Aug 2018
Teacher Guided Architecture Search P. Bashivan Mark Tensen J. DiCarlo 3DV 66 27 0 04 Aug 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 351 10,356 0 10 Jul 2018
Modality Distillation with Multiple Stream Networks for Action Recognition Nuno C. Garcia Pietro Morerio Vittorio Murino 71 184 0 19 Jun 2018
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 293 480 0 12 Jun 2018
Born Again Neural Networks Tommaso Furlanello Zachary Chase Lipton Michael Tschannen Laurent Itti Anima Anandkumar 80 1,034 0 12 May 2018
Label Refinery: Improving ImageNet Classification through Label Progression Hessam Bagherinezhad Maxwell Horton Mohammad Rastegari Ali Farhadi 62 190 0 07 May 2018
Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification Chenrui Zhang Yuxin Peng SSL 71 56 0 26 Apr 2018
Developing Far-Field Speaker System Via Teacher-Student Learning Jinyu Li Rui Zhao Zhuo Chen Changliang Liu Xiong Xiao Guoli Ye Jiawei Liu 46 56 0 14 Apr 2018
Learning Deep Representations with Probabilistic Knowledge Transfer Nikolaos Passalis Anastasios Tefas 63 412 0 28 Mar 2018
Learning to Adapt Structured Output Space for Semantic Segmentation Yi-Hsuan Tsai Wei-Chih Hung S. Schulter Kihyuk Sohn Ming-Hsuan Yang Manmohan Chandraker OOD SSeg 149 1,547 0 28 Feb 2018
Paraphrasing Complex Network: Network Compression via Factor Transfer Jangho Kim Seonguk Park Nojun Kwak 77 551 0 14 Feb 2018
Progressive Neural Architecture Search Chenxi Liu Barret Zoph Maxim Neumann Jonathon Shlens Wei Hua Li Li Li Fei-Fei Alan Yuille Jonathan Huang Kevin Patrick Murphy 114 1,994 0 02 Dec 2017
CyCADA: Cycle-Consistent Adversarial Domain Adaptation Judy Hoffman Eric Tzeng Taesung Park Jun-Yan Zhu Phillip Isola Kate Saenko Alexei A. Efros Trevor Darrell 147 3,005 0 08 Nov 2017
Topology Adaptive Graph Convolutional Networks Jian Du Shanghang Zhang Guanhang Wu José M. F. Moura S. Kar GNN 90 304 0 28 Oct 2017
Data-Free Knowledge Distillation for Deep Neural Networks Raphael Gontijo-Lopes Stefano Fenu Thad Starner 60 273 0 19 Oct 2017
Incremental Learning of Object Detectors without Catastrophic Forgetting K. Shmelkov Cordelia Schmid Alahari Karteek ObjD 82 520 0 23 Aug 2017
Large-Scale Domain Adaptation via Teacher-Student Learning Jinyu Li M. Seltzer Xi Wang Rui Zhao Jiawei Liu 142 141 0 17 Aug 2017
Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Net Guorui Zhou Ying Fan Runpeng Cui Weijie Bian Xiaoqiang Zhu Kun Gai 71 116 0 14 Aug 2017
DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer Yuntao Chen Naiyan Wang Zhaoxiang Zhang FedML 102 223 0 05 Jul 2017
Teacher-Student Curriculum Learning Tambet Matiisen Avital Oliver Taco S. Cohen John Schulman ODL 97 381 0 01 Jul 2017
Deep Mutual Learning Ying Zhang Tao Xiang Timothy M. Hospedales Huchuan Lu FedML 153 1,654 0 01 Jun 2017
Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning Werner Zellinger Thomas Grubinger E. Lughofer T. Natschläger Susanne Saminger-Platz OOD 105 578 0 28 Feb 2017
Knowledge Adaptation: Teaching to Adapt Sebastian Ruder Parsa Ghaffari J. Breslin CLL TTA 56 53 0 07 Feb 2017
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer Sergey Zagoruyko N. Komodakis 147 2,586 0 12 Dec 2016
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 374 7,561 0 02 Dec 2016
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data Nicolas Papernot Martín Abadi Ulfar Erlingsson Ian Goodfellow Kunal Talwar 94 1,020 0 18 Oct 2016
Semi-Supervised Classification with Graph Convolutional Networks Thomas Kipf Max Welling GNN SSL 662 29,156 0 09 Sep 2016
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,162 0 01 Jul 2016
Learning without Forgetting Zhizhong Li Derek Hoiem CLL OOD SSL 308 4,428 0 29 Jun 2016
Sequence-Level Knowledge Distillation Yoon Kim Alexander M. Rush 130 1,122 0 25 Jun 2016
R-FCN: Object Detection via Region-based Fully Convolutional Networks Jifeng Dai Yi Li Kaiming He Jian Sun ObjD 179 5,642 0 20 May 2016
Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks Nicolas Papernot Patrick McDaniel Xi Wu S. Jha A. Swami AAML 113 3,076 0 14 Nov 2015
Bidirectional LSTM-CRF Models for Sequence Tagging Zhiheng Huang Wenyuan Xu Kai Yu 186 4,033 0 09 Aug 2015
Cross Modal Distillation for Supervision Transfer Saurabh Gupta Judy Hoffman Jitendra Malik 120 538 0 02 Jul 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 528 62,377 0 04 Jun 2015
Bilinear CNNs for Fine-grained Visual Recognition Tsung-Yu Lin Aruni RoyChowdhury Subhransu Maji 133 1,876 0 29 Apr 2015
Object Detection Networks on Convolutional Feature Maps Shaoqing Ren Kaiming He Ross B. Girshick Xinming Zhang Jian Sun ObjD 106 409 0 23 Apr 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 364 19,733 0 09 Mar 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 319 3,898 0 19 Dec 2014
Understanding Deep Image Representations by Inverting Them Aravindh Mahendran Andrea Vedaldi FAtt 131 1,965 0 26 Nov 2014
Representation Learning: A Review and New Perspectives Yoshua Bengio Aaron Courville Pascal Vincent OOD SSL 278 12,458 0 24 Jun 2012