FitNets: Hints for Thin Deep Nets

19 December 2014

Adriana Romero

Nicolas Ballas

Samira Ebrahimi Kahou

Papers citing "FitNets: Hints for Thin Deep Nets"

50 / 748 papers shown

Title
Decentralized Federated Learning via Mutual Knowledge Transfer Chengxi Li Gang Li P. Varshney FedML 26 106 0 24 Dec 2020
Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead Maurizio Capra Beatrice Bussolino Alberto Marchisio Guido Masera Maurizio Martina Mohamed Bennai BDL 64 140 0 21 Dec 2020
Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup Guodong Xu Ziwei Liu Chen Change Loy UQCV 23 39 0 17 Dec 2020
ISD: Self-Supervised Learning by Iterative Similarity Distillation Ajinkya Tejankar Soroush Abbasi Koohpayegani Vipin Pillai Paolo Favaro Hamed Pirsiavash SSL 27 44 0 16 Dec 2020
LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understanding Hao Fu Shaojun Zhou Qihong Yang Junjie Tang Guiquan Liu Kaikui Liu Xiaolong Li 54 58 0 14 Dec 2020
DE-RRD: A Knowledge Distillation Framework for Recommender System SeongKu Kang Junyoung Hwang Wonbin Kweon Hwanjo Yu 29 79 0 08 Dec 2020
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 45 288 0 06 Dec 2020
Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning Haohang Xu Xiaopeng Zhang Hao Li Lingxi Xie H. Xiong Qi Tian SSL 16 12 0 04 Dec 2020
Data-Free Model Extraction Jean-Baptiste Truong Pratyush Maini R. Walls Nicolas Papernot MIACV 15 181 0 30 Nov 2020
Bringing AI To Edge: From Deep Learning's Perspective Di Liu Hao Kong Xiangzhong Luo Weichen Liu Ravi Subramaniam 52 116 0 25 Nov 2020
EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications Minghui Qiu Peng Li Chengyu Wang Hanjie Pan Yaliang Li ... Jun Yang Yaliang Li Jun Huang Deng Cai Wei Lin VLM SyDa 39 20 0 18 Nov 2020
Ensemble Knowledge Distillation for CTR Prediction Jieming Zhu Jinyang Liu Weiqi Li Jincai Lai Xiuqiang He Liang Chen Zibin Zheng 36 56 0 08 Nov 2020
Distilling Knowledge by Mimicking Features G. Wang Yifan Ge Jianxin Wu 17 33 0 03 Nov 2020
Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets Kai Han Yunhe Wang Qiulin Zhang Wei Zhang Chunjing Xu Tong Zhang 24 87 0 28 Oct 2020
Comprehensive Attention Self-Distillation for Weakly-Supervised Object Detection Zeyi Huang Yang Zou V. Bhagavatula Dong Huang WSOD 28 120 0 22 Oct 2020
Combining Ensembles and Data Augmentation can Harm your Calibration Yeming Wen Ghassen Jerfel Rafael Muller Michael W. Dusenberry Jasper Snoek Balaji Lakshminarayanan Dustin Tran UQCV 32 63 0 19 Oct 2020
Regularizing Dialogue Generation by Imitating Implicit Scenarios Shaoxiong Feng Xuancheng Ren Hongshen Chen Bin Sun Kan Li Xu Sun 20 20 0 05 Oct 2020
Cross-Task Representation Learning for Anatomical Landmark Detection Zeyu Fu Jianbo Jiao M. Suttie J. A. Noble CVBM 30 2 0 28 Sep 2020
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 33 209 0 27 Sep 2020
MimicDet: Bridging the Gap Between One-Stage and Two-Stage Object Detection Xin Lu Quanquan Li Buyu Li Junjie Yan ObjD 29 52 0 24 Sep 2020
Densely Guided Knowledge Distillation using Multiple Teacher Assistants Wonchul Son Jaemin Na Junyong Choi Wonjun Hwang 27 111 0 18 Sep 2020
MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks Zhiqiang Shen Marios Savvides 33 63 0 17 Sep 2020
Compression of Deep Learning Models for Text: A Survey Manish Gupta Puneet Agrawal VLM MedIm AI4CE 22 115 0 12 Aug 2020
Unravelling Small Sample Size Problems in the Deep Learning World Rohit Keshari Soumyadeep Ghosh S. Chhabra Mayank Vatsa Richa Singh 45 33 0 08 Aug 2020
Prime-Aware Adaptive Distillation Youcai Zhang Zhonghao Lan Yuchen Dai Fangao Zeng Yan Bai Jie Chang Yichen Wei 21 40 0 04 Aug 2020
Differentiable Feature Aggregation Search for Knowledge Distillation Yushuo Guan Pengyu Zhao Bingxuan Wang Yuanxing Zhang Cong Yao Kaigui Bian Jian Tang FedML 25 44 0 02 Aug 2020
Distilling Visual Priors from Self-Supervised Learning Bingchen Zhao Xin Wen SSL 8 14 0 01 Aug 2020
Fully Dynamic Inference with Deep Neural Networks Wenhan Xia Hongxu Yin Xiaoliang Dai N. Jha 3DH BDL 34 39 0 29 Jul 2020
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge Chaoyang He M. Annavaram A. Avestimehr FedML 32 23 0 28 Jul 2020
Learning with Privileged Information for Efficient Image Super-Resolution Wonkyung Lee Junghyup Lee Dohyung Kim Bumsub Ham 33 134 0 15 Jul 2020
Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation Le Thanh Nguyen-Meidine Atif Bela M. Kiran Jose Dolz Louis-Antoine Blais-Morin Eric Granger 38 82 0 14 Jul 2020
Optical Flow Distillation: Towards Efficient and Stable Video Style Transfer Xinghao Chen Yiman Zhang Yunhe Wang Han Shu Chunjing Xu Chang Xu VGen 21 54 0 10 Jul 2020
Robust Re-Identification by Multiple Views Knowledge Distillation Angelo Porrello Luca Bergamini Simone Calderara 32 65 0 08 Jul 2020
Knowledge Distillation Beyond Model Compression F. Sarfraz Elahe Arani Bahram Zonooz 35 40 0 03 Jul 2020
Surrogate-assisted Particle Swarm Optimisation for Evolving Variable-length Transferable Blocks for Image Classification Bin Wang Bing Xue Mengjie Zhang 23 53 0 03 Jul 2020
Multi-fidelity Neural Architecture Search with Knowledge Distillation I. Trofimov Nikita Klyuchnikov Mikhail Salnikov Alexander N. Filippov Evgeny Burnaev 37 15 0 15 Jun 2020
Knowledge Distillation Meets Self-Supervision Guodong Xu Ziwei Liu Xiaoxiao Li Chen Change Loy FedML 49 280 0 12 Jun 2020
Dataset Condensation with Gradient Matching Bo Zhao Konda Reddy Mopuri Hakan Bilen DD 41 479 0 10 Jun 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 28 2,857 0 09 Jun 2020
Self-Distillation as Instance-Specific Label Smoothing Zhilu Zhang M. Sabuncu 22 116 0 09 Jun 2020
A Framework for Neural Network Pruning Using Gibbs Distributions Alex Labach S. Valaee 9 5 0 08 Jun 2020
ResKD: Residual-Guided Knowledge Distillation Xuewei Li Songyuan Li Bourahla Omar Fei Wu Xi Li 28 47 0 08 Jun 2020
Multi-view Contrastive Learning for Online Knowledge Distillation Chuanguang Yang Zhulin An Yongjun Xu 24 23 0 07 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 47 98 0 05 Jun 2020
Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels Marvin Klingner Andreas Bär Philipp Donn Tim Fingscheidt CLL 24 45 0 12 May 2020
Data-Free Network Quantization With Adversarial Knowledge Distillation Yoojin Choi Jihwan P. Choi Mostafa El-Khamy Jungwon Lee MQ 27 119 0 08 May 2020
Structure-Level Knowledge Distillation For Multilingual Sequence Labeling Xinyu Wang Yong-jia Jiang Nguyen Bach Tao Wang Fei Huang Kewei Tu 35 36 0 08 Apr 2020
Towards Efficient Unconstrained Palmprint Recognition via Deep Distillation Hashing Huikai Shao Dexing Zhong Xuefeng Du 38 9 0 07 Apr 2020
A Learning Framework for n-bit Quantized Neural Networks toward FPGAs Jun Chen Lu Liu Yong Liu Xianfang Zeng MQ 41 26 0 06 Apr 2020
FastBERT: a Self-distilling BERT with Adaptive Inference Time Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Haotang Deng Qi Ju 59 354 0 05 Apr 2020