FitNets: Hints for Thin Deep Nets

19 December 2014

Adriana Romero

Nicolas Ballas

Samira Ebrahimi Kahou

Papers citing "FitNets: Hints for Thin Deep Nets"

50 / 748 papers shown

Title
Knowing What, Where and When to Look: Efficient Video Action Modeling with Attention Juan-Manuel Perez-Rua Brais Martínez Xiatian Zhu Antoine Toisoul Victor Escorcia Tao Xiang 50 19 0 02 Apr 2020
Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge Long Zhao Xi Peng Yuxiao Chen Mubbasir Kapadia Dimitris N. Metaxas 16 66 0 01 Apr 2020
Regularizing Class-wise Predictions via Self-knowledge Distillation Sukmin Yun Jongjin Park Kimin Lee Jinwoo Shin 29 276 0 31 Mar 2020
Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images Juil Sock Guillermo Garcia-Hernando Anil Armagan Tae-Kyun Kim 27 5 0 27 Mar 2020
Circumventing Outliers of AutoAugment with Knowledge Distillation Longhui Wei Anxiang Xiao Lingxi Xie Xin Chen Xiaopeng Zhang Qi Tian 26 62 0 25 Mar 2020
A Survey of Methods for Low-Power Deep Learning and Computer Vision Abhinav Goel Caleb Tung Yung-Hsiang Lu George K. Thiruvathukal VLM 15 92 0 24 Mar 2020
Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives Duo Li Qifeng Chen 153 19 0 24 Mar 2020
Distilling Knowledge from Graph Convolutional Networks Yiding Yang Jiayan Qiu Xiuming Zhang Dacheng Tao Xinchao Wang 166 226 0 23 Mar 2020
Efficient Crowd Counting via Structured Knowledge Transfer Lingbo Liu Jiaqi Chen Hefeng Wu Tianshui Chen Guanbin Li Liang Lin 29 64 0 23 Mar 2020
SuperMix: Supervising the Mixing Data Augmentation Ali Dabouei Sobhan Soleymani Fariborz Taherkhani Nasser M. Nasrabadi 21 98 0 10 Mar 2020
Knowledge distillation via adaptive instance normalization Jing Yang Brais Martínez Adrian Bulat Georgios Tzimiropoulos 21 23 0 09 Mar 2020
Distilling portable Generative Adversarial Networks for Image Translation Hanting Chen Yunhe Wang Han Shu Changyuan Wen Chunjing Xu Boxin Shi Chao Xu Chang Xu 83 83 0 07 Mar 2020
Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs Sangwoo Mo Minsu Cho Jinwoo Shin 30 212 0 25 Feb 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 47 1,214 0 25 Feb 2020
Residual Knowledge Distillation Mengya Gao Yujun Shen Quanquan Li Chen Change Loy 22 28 0 21 Feb 2020
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning Yeming Wen Dustin Tran Jimmy Ba OOD FedML UQCV 32 483 0 17 Feb 2020
Self-Distillation Amplifies Regularization in Hilbert Space H. Mobahi Mehrdad Farajtabar Peter L. Bartlett 40 229 0 13 Feb 2020
Subclass Distillation Rafael Müller Simon Kornblith Geoffrey E. Hinton 34 33 0 10 Feb 2020
Understanding and Improving Knowledge Distillation Jiaxi Tang Rakesh Shivanna Zhe Zhao Dong Lin Anima Singh Ed H. Chi Sagar Jain 27 129 0 10 Feb 2020
Feature-map-level Online Adversarial Knowledge Distillation Inseop Chung Seonguk Park Jangho Kim Nojun Kwak GAN 30 128 0 05 Feb 2020
Compact recurrent neural networks for acoustic event detection on low-energy low-complexity platforms G. Cerutti Rahul Prasad Alessio Brutti Elisabetta Farella 21 47 0 29 Jan 2020
Lightweight 3D Human Pose Estimation Network Training Using Teacher-Student Learning D. Hwang Suntae Kim Nicolas Monet Hideki Koike Soonmin Bae 3DH 30 39 0 15 Jan 2020
Resource-Efficient Neural Networks for Embedded Systems Wolfgang Roth Günther Schindler Lukas Pfeifenberger Robert Peharz Sebastian Tschiatschek Holger Fröning Franz Pernkopf Zoubin Ghahramani 34 47 0 07 Jan 2020
Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification Liuyu Xiang Guiguang Ding Jungong Han 30 281 0 06 Jan 2020
ZeroQ: A Novel Zero Shot Quantization Framework Yaohui Cai Z. Yao Zhen Dong A. Gholami Michael W. Mahoney Kurt Keutzer MQ 43 389 0 01 Jan 2020
AdderNet: Do We Really Need Multiplications in Deep Learning? Hanting Chen Yunhe Wang Chunjing Xu Boxin Shi Chao Xu Qi Tian Chang Xu 29 194 0 31 Dec 2019
SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning Albert Zhao Tong He Yitao Liang Haibin Huang Guy Van den Broeck Stefano Soatto 17 16 0 06 Dec 2019
Online Knowledge Distillation with Diverse Peers Defang Chen Jian-Ping Mei Can Wang Yan Feng Chun-Yen Chen FedML 33 297 0 01 Dec 2019
Blockwisely Supervised Neural Architecture Search with Knowledge Distillation Changlin Li Jiefeng Peng Liuchun Yuan Guangrun Wang Xiaodan Liang Liang Lin Xiaojun Chang 33 180 0 29 Nov 2019
Towards Oracle Knowledge Distillation with Neural Architecture Search Minsoo Kang Jonghwan Mun Bohyung Han FedML 43 44 0 29 Nov 2019
QKD: Quantization-aware Knowledge Distillation Jangho Kim Yash Bhalgat Jinwon Lee Chirag I. Patel Nojun Kwak MQ 26 64 0 28 Nov 2019
GhostNet: More Features from Cheap Operations Kai Han Yunhe Wang Qi Tian Jianyuan Guo Chunjing Xu Chang Xu 20 2,591 0 27 Nov 2019
DeepMimic: Mentor-Student Unlabeled Data Based Training Itay Mosafi E. David N. Netanyahu 24 6 0 24 Nov 2019
Towards Making Deep Transfer Learning Never Hurt Ruosi Wan Haoyi Xiong Xingjian Li Zhanxing Zhu Jun Huan 30 21 0 18 Nov 2019
Preparing Lessons: Improve Knowledge Distillation with Better Supervision Tiancheng Wen Shenqi Lai Xueming Qian 25 68 0 18 Nov 2019
Label-similarity Curriculum Learning Ürün Dogan A. Deshmukh Marcin Machura Christian Igel 23 21 0 15 Nov 2019
Collaborative Distillation for Top-N Recommendation Jae-woong Lee Minjin Choi Jongwuk Lee Hyunjung Shim 22 47 0 13 Nov 2019
Iteratively Training Look-Up Tables for Network Quantization Fabien Cardinaux Stefan Uhlich K. Yoshiyama Javier Alonso García Lukas Mauch Stephen Tiedemann Thomas Kemp Akira Nakamura MQ 27 16 0 12 Nov 2019
Deep geometric knowledge distillation with graphs Carlos Lassance Myriam Bontonou G. B. Hacene Vincent Gripon Jian Tang Antonio Ortega 27 39 0 08 Nov 2019
Active Subspace of Neural Networks: Structural Analysis and Universal Attacks Chunfeng Cui Kaiqi Zhang Talgat Daulbaev Julia Gusak Ivan Oseledets Zheng-Wei Zhang AAML 32 25 0 29 Oct 2019
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 49 1,034 0 23 Oct 2019
4-Connected Shift Residual Networks Andrew Brown Pascal Mettes M. Worring 3DPC 31 8 0 22 Oct 2019
VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition Mengjia Yan Mengao Zhao Zining Xu Qian Zhang Guoli Wang Zhizhong Su CVBM 26 91 0 11 Oct 2019
DiabDeep: Pervasive Diabetes Diagnosis based on Wearable Medical Sensors and Efficient Neural Networks Hongxu Yin Bilal Mukadam Xiaoliang Dai N. Jha 34 47 0 11 Oct 2019
Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data Subhabrata Mukherjee Ahmed Hassan Awadallah 26 25 0 04 Oct 2019
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks Xiaohan Ding Guiguang Ding Xiangxin Zhou Yuchen Guo Jungong Han Ji Liu 18 162 0 27 Sep 2019
Deep Model Transferability from Attribution Maps Mingli Song Yixin Chen Xinchao Wang Chengchao Shen Xiuming Zhang 27 54 0 26 Sep 2019
Compact Trilinear Interaction for Visual Question Answering Tuong Khanh Long Do Thanh-Toan Do Huy Tran Erman Tjiputra Quang-Dieu Tran 36 59 0 26 Sep 2019
Revisiting Knowledge Distillation via Label Smoothing Regularization Li-xin Yuan Francis E. H. Tay Guilin Li Tao Wang Jiashi Feng 25 90 0 25 Sep 2019
Extremely Small BERT Models from Mixed-Vocabulary Training Sanqiang Zhao Raghav Gupta Yang Song Denny Zhou VLM 14 53 0 25 Sep 2019