FitNets: Hints for Thin Deep Nets

19 December 2014

Adriana Romero

Nicolas Ballas

Samira Ebrahimi Kahou

Papers citing "FitNets: Hints for Thin Deep Nets"

50 / 748 papers shown

Title
On The Distribution of Penultimate Activations of Classification Networks Minkyo Seo Yoonho Lee Suha Kwak UQCV 18 4 0 05 Jul 2021
Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model Zhiqi Huang Fenglin Liu Xian Wu Shen Ge Helin Wang Wei Fan Yuexian Zou AuLLM 29 2 0 04 Jul 2021
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation Zhiwei Hao Jianyuan Guo Ding Jia Kai Han Yehui Tang Chao Zhang Dacheng Tao Yunhe Wang ViT 41 68 0 03 Jul 2021
Pool of Experts: Realtime Querying Specialized Knowledge in Massive Neural Networks Hakbin Kim Dong-Wan Choi 25 2 0 03 Jul 2021
Simple Distillation Baselines for Improving Small Self-supervised Models Jindong Gu Wei Liu Yonglong Tian 27 8 0 21 Jun 2021
Knowledge Distillation via Instance-level Sequence Learning Haoran Zhao Xin Sun Junyu Dong Zihe Dong Qiong Li 34 23 0 21 Jun 2021
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective Geoff Pleiss John P. Cunningham 28 24 0 11 Jun 2021
Knowledge distillation: A good teacher is patient and consistent Lucas Beyer Xiaohua Zhai Amelie Royer L. Markeeva Rohan Anil Alexander Kolesnikov VLM 52 287 0 09 Jun 2021
Privileged Graph Distillation for Cold Start Recommendation Shuai Wang Kun Zhang Le Wu Haiping Ma Richang Hong Meng Wang 12 28 0 31 May 2021
Fair Feature Distillation for Visual Recognition S. Jung Donggyu Lee Taeeon Park Taesup Moon 27 75 0 27 May 2021
Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation Lewei Yao Renjie Pi Hang Xu Wei Zhang Zhenguo Li Tong Zhang 89 38 0 27 May 2021
Divide and Contrast: Self-supervised Learning from Uncurated Data Yonglong Tian Olivier J. Hénaff Aaron van den Oord SSL 66 96 0 17 May 2021
Graph-Free Knowledge Distillation for Graph Neural Networks Xiang Deng Zhongfei Zhang 34 65 0 16 May 2021
Carrying out CNN Channel Pruning in a White Box Yuxin Zhang Mingbao Lin Chia-Wen Lin Jie Chen Feiyue Huang Yongjian Wu Yonghong Tian Rongrong Ji VLM 39 58 0 24 Apr 2021
Balanced Knowledge Distillation for Long-tailed Learning Shaoyu Zhang Chen Chen Xiyuan Hu Silong Peng 56 57 0 21 Apr 2021
Distill on the Go: Online knowledge distillation in self-supervised learning Prashant Shivaram Bhat Elahe Arani Bahram Zonooz SSL 24 28 0 20 Apr 2021
DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning Yuting Gao Jia-Xin Zhuang Xiaowei Guo Hao Cheng Xing Sun Ke Li Feiyue Huang 59 40 0 19 Apr 2021
Distilling Knowledge via Knowledge Review Pengguang Chen Shu Liu Hengshuang Zhao Jiaya Jia 155 424 0 19 Apr 2021
End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving Hengli Wang Peide Cai Rui Fan Yuxiang Sun Ming Liu 40 23 0 18 Apr 2021
Lottery Jackpots Exist in Pre-trained Models Yuxin Zhang Mingbao Lin Yan Wang Rongrong Ji Rongrong Ji 35 15 0 18 Apr 2021
MRI-based Alzheimer's disease prediction via distilling the knowledge in multi-modal data Hao Guan Chaoyue Wang Dacheng Tao 24 30 0 08 Apr 2021
Distilling and Transferring Knowledge via cGAN-generated Samples for Image Classification and Regression Xin Ding Z. J. Wang Zuheng Xu Z. Jane Wang William J. Welch 41 22 0 07 Apr 2021
SIMPLE: SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation Jiabin Zhang Zheng Zhu Jiwen Lu Junjie Huang Guan Huang Jie Zhou 3DH 36 14 0 06 Apr 2021
Learning from Self-Discrepancy via Multiple Co-teaching for Cross-Domain Person Re-Identification Suncheng Xiang Yuzhuo Fu Mengyuan Guan Ting Liu 37 22 0 06 Apr 2021
Content-Aware GAN Compression Yuchen Liu Zhixin Shu Yijun Li Zhe Lin Federico Perazzi S. Kung GAN 37 58 0 06 Apr 2021
Students are the Best Teacher: Exit-Ensemble Distillation with Multi-Exits Hojung Lee Jong-Seok Lee 25 8 0 01 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 27 990 0 31 Mar 2021
Complementary Relation Contrastive Distillation Jinguo Zhu Shixiang Tang Dapeng Chen Shijie Yu Yakun Liu A. Yang M. Rong Xiaohua Wang 27 77 0 29 Mar 2021
Distilling Object Detectors via Decoupled Features Jianyuan Guo Kai Han Yunhe Wang Han Wu Xinghao Chen Chunjing Xu Chang Xu 48 199 0 26 Mar 2021
Distilling a Powerful Student Model via Online Knowledge Distillation Shaojie Li Mingbao Lin Yan Wang Yongjian Wu Yonghong Tian Ling Shao Rongrong Ji FedML 27 47 0 26 Mar 2021
Universal Representation Learning from Multiple Domains for Few-shot Classification Weihong Li Xialei Liu Hakan Bilen SSL OOD VLM 32 84 0 25 Mar 2021
Learning Scene Structure Guidance via Cross-Task Knowledge Transfer for Single Depth Super-Resolution Baoli Sun Xinchen Ye Baopu Li Haojie Li Zhihui Wang Rui Xu 11 45 0 24 Mar 2021
ReCU: Reviving the Dead Weights in Binary Neural Networks Zihan Xu Mingbao Lin Jianzhuang Liu Jie Chen Ling Shao Yue Gao Yonghong Tian Rongrong Ji MQ 24 81 0 23 Mar 2021
Student Network Learning via Evolutionary Knowledge Distillation Kangkai Zhang Chunhui Zhang Shikun Li Dan Zeng Shiming Ge 24 83 0 23 Mar 2021
Compacting Deep Neural Networks for Internet of Things: Methods and Applications Ke Zhang Hanbo Ying Hongning Dai Lin Li Yuangyuang Peng Keyi Guo Hongfang Yu 23 38 0 20 Mar 2021
Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation Mingi Ji Seungjae Shin Seunghyun Hwang Gibeom Park Il-Chul Moon 13 120 0 15 Mar 2021
Adaptive Multi-Teacher Multi-level Knowledge Distillation Yuang Liu Wei Zhang Jun Wang 28 157 0 06 Mar 2021
There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge Francisco Rivera Valverde Juana Valeria Hurtado Abhinav Valada 28 72 0 01 Mar 2021
Siamese Labels Auxiliary Learning Wenrui Gan Zhulin Liu Chong Chen Tong Zhang 25 1 0 27 Feb 2021
Localization Distillation for Dense Object Detection Zhaohui Zheng Rongguang Ye Ping Wang Dongwei Ren W. Zuo Qibin Hou Ming-Ming Cheng ObjD 104 115 0 24 Feb 2021
Enhancing Data-Free Adversarial Distillation with Activation Regularization and Virtual Interpolation Xiaoyang Qu Jianzong Wang Jing Xiao 18 14 0 23 Feb 2021
AlphaNet: Improved Training of Supernets with Alpha-Divergence Dilin Wang Chengyue Gong Meng Li Qiang Liu Vikas Chandra 160 44 0 16 Feb 2021
Bridging the gap between Human Action Recognition and Online Action Detection Alban Main De Boissiere R. Noumeir 22 0 0 21 Jan 2021
GhostSR: Learning Ghost Features for Efficient Image Super-Resolution Ying Nie Kai Han Zhenhua Liu Chunjing Xu Yunhe Wang OOD 42 22 0 21 Jan 2021
Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation Lingyun Feng Minghui Qiu Yaliang Li Haitao Zheng Ying Shen 46 10 0 20 Jan 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 247 190 0 12 Jan 2021
Resolution-Based Distillation for Efficient Histology Image Classification Joseph DiPalma A. Suriawinata L. Tafe Lorenzo Torresani Saeed Hassanpour 40 36 0 11 Jan 2021
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 107 345 0 05 Jan 2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting Wangchunshu Zhou Tao Ge Canwen Xu Ke Xu Furu Wei LRM 16 15 0 02 Jan 2021
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 30 257 0 31 Dec 2020