Graph-based Knowledge Distillation by Multi-head Attention Network

4 July 2019

Papers citing "Graph-based Knowledge Distillation by Multi-head Attention Network"

23 / 23 papers shown

Title
Improved Knowledge Distillation via Teacher Assistant Seyed Iman Mirzadeh Mehrdad Farajtabar Ang Li Nir Levine Akihiro Matsukawa H. Ghasemzadeh 92 1,075 0 09 Feb 2019
Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation Shiming Ge Shengwei Zhao Chenyu Li Jia Li CVBM 123 188 0 25 Nov 2018
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons Byeongho Heo Minsik Lee Sangdoo Yun J. Choi 55 526 0 08 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,770 0 11 Oct 2018
Self-supervised Knowledge Distillation Using Singular Value Decomposition Seunghyun Lee D. Kim B. Song 49 139 0 18 Jul 2018
Relational inductive biases, deep learning, and graph networks Peter W. Battaglia Jessica B. Hamrick V. Bapst Alvaro Sanchez-Gonzalez V. Zambaldi ... Pushmeet Kohli M. Botvinick Oriol Vinyals Yujia Li Razvan Pascanu AI4CE NAI 753 3,119 0 04 Jun 2018
Linguistically-Informed Self-Attention for Semantic Role Labeling Emma Strubell Pat Verga D. Andor David J. Weiss Andrew McCallum OffRL 82 380 0 23 Apr 2018
Non-local Neural Networks Xinyu Wang Ross B. Girshick Abhinav Gupta Kaiming He OffRL 289 8,905 0 21 Nov 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 692 131,526 0 12 Jun 2017
Deep Pyramidal Residual Networks Dongyoon Han Jiwhan Kim Junmo Kim 93 694 0 10 Oct 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 772 36,794 0 25 Aug 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 433 18,350 0 27 May 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 337 7,984 0 23 May 2016
Long Short-Term Memory-Networks for Machine Reading Jianpeng Cheng Li Dong Mirella Lapata AIMat RALM 99 1,120 0 25 Jan 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 193,878 0 10 Dec 2015
SSD: Single Shot MultiBox Detector Wen Liu Dragomir Anguelov D. Erhan Christian Szegedy Scott E. Reed Cheng-Yang Fu Alexander C. Berg ObjD BDL 229 29,816 0 08 Dec 2015
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 380 7,962 0 17 Aug 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 502 62,270 0 04 Jun 2015
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 1.8K 77,133 0 18 May 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 350 19,643 0 09 Mar 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,289 0 11 Feb 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 305 3,883 0 19 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.6K 100,348 0 04 Sep 2014