Do Deep Nets Really Need to be Deep?

21 December 2013

Lei Jimmy Ba

Papers citing "Do Deep Nets Really Need to be Deep?"

50 / 379 papers shown

Title
Taurus: A Data Plane Architecture for Per-Packet ML Tushar Swamy Alexander Rucker M. Shahbaz Ishan Gaur K. Olukotun 23 82 0 12 Feb 2020
Lightweight 3D Human Pose Estimation Network Training Using Teacher-Student Learning D. Hwang Suntae Kim Nicolas Monet Hideki Koike Soonmin Bae 3DH 25 39 0 15 Jan 2020
PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning Dor Livne Kobi Cohen 29 50 0 14 Jan 2020
Resource-Efficient Neural Networks for Embedded Systems Wolfgang Roth Günther Schindler Lukas Pfeifenberger Robert Peharz Sebastian Tschiatschek Holger Fröning Franz Pernkopf Zoubin Ghahramani 34 47 0 07 Jan 2020
SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning Albert Zhao Tong He Yitao Liang Haibin Huang Mathias Niepert Stefano Soatto 17 16 0 06 Dec 2019
Online Knowledge Distillation with Diverse Peers Defang Chen Jian-Ping Mei Can Wang Yan Feng Chun-Yen Chen FedML 11 297 0 01 Dec 2019
Blockwisely Supervised Neural Architecture Search with Knowledge Distillation Changlin Li Jiefeng Peng Liuchun Yuan Guangrun Wang Xiaodan Liang Liang Lin Xiaojun Chang 31 179 0 29 Nov 2019
Preparing Lessons: Improve Knowledge Distillation with Better Supervision Tiancheng Wen Shenqi Lai Xueming Qian 25 68 0 18 Nov 2019
Self-training with Noisy Student improves ImageNet classification Qizhe Xie Minh-Thang Luong Eduard H. Hovy Quoc V. Le NoLa 88 2,364 0 11 Nov 2019
Domain Robustness in Neural Machine Translation Mathias Müller Annette Rios Gonzales Rico Sennrich 33 95 0 08 Nov 2019
Deep geometric knowledge distillation with graphs Carlos Lassance Myriam Bontonou G. B. Hacene Vincent Gripon Jian Tang Antonio Ortega 21 39 0 08 Nov 2019
Real-time Memory Efficient Large-pose Face Alignment via Deep Evolutionary Network Bin Sun Ming Shao Siyu Xia Y. Fu 3DH CVBM 17 2 0 25 Oct 2019
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 47 1,034 0 23 Oct 2019
Deep Learning at the Edge Sahar Voghoei N. Tonekaboni Jason G. Wallace H. Arabnia 15 41 0 22 Oct 2019
Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data Subhabrata Mukherjee Ahmed Hassan Awadallah 26 25 0 04 Oct 2019
On the Efficacy of Knowledge Distillation Ligang He Rui Mao 45 600 0 03 Oct 2019
Exascale Deep Learning to Accelerate Cancer Research Robert M. Patton J. T. Johnston Steven R. Young Catherine D. Schuman T. Potok ... Junghoon Chae L. Hou Shahira Abousamra Dimitris Samaras Joel H. Saltz 21 15 0 26 Sep 2019
Compact Trilinear Interaction for Visual Question Answering Tuong Khanh Long Do Thanh-Toan Do Huy Tran Erman Tjiputra Quang-Dieu Tran 36 59 0 26 Sep 2019
Extremely Small BERT Models from Mixed-Vocabulary Training Sanqiang Zhao Raghav Gupta Yang Song Denny Zhou VLM 14 53 0 25 Sep 2019
FEED: Feature-level Ensemble for Knowledge Distillation Seonguk Park Nojun Kwak FedML 31 41 0 24 Sep 2019
Adversarial Learning with Margin-based Triplet Embedding Regularization Yaoyao Zhong Weihong Deng AAML 28 50 0 20 Sep 2019
Extreme Low Resolution Activity Recognition with Confident Spatial-Temporal Attention Transfer Yucai Bai Qinglong Zou Xieyuanli Chen Lingxi Li Zhengming Ding Long Chen 18 3 0 09 Sep 2019
A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA Mohammad Farhadi Mehdi Ghasemi Yezhou Yang 22 27 0 05 Sep 2019
Knowledge Distillation for End-to-End Person Search Bharti Munjal Fabio Galasso S. Amin FedML 43 15 0 03 Sep 2019
Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations Bohan Zhuang Jing Liu Mingkui Tan Lingqiao Liu Ian Reid Chunhua Shen MQ 29 45 0 10 Aug 2019
Defending Against Adversarial Iris Examples Using Wavelet Decomposition Sobhan Soleymani Ali Dabouei J. Dawson Nasser M. Nasrabadi AAML 27 9 0 08 Aug 2019
Memory- and Communication-Aware Model Compression for Distributed Deep Learning Inference on IoT Kartikeya Bhardwaj Chingyi Lin A. L. Sartor R. Marculescu GNN 18 51 0 26 Jul 2019
Distilled Siamese Networks for Visual Tracking Jianbing Shen Yuanpei Liu Xingping Dong Xiankai Lu Fahad Shahbaz Khan Guosheng Lin 15 101 0 24 Jul 2019
Lifelong GAN: Continual Learning for Conditional Image Generation Mengyao Zhai Lei Chen Frederick Tung Jiawei He Megha Nawhal Greg Mori CLL 36 180 0 23 Jul 2019
Compact Global Descriptor for Neural Networks Xiangyu He Ke Cheng Qiang Chen Qinghao Hu Peisong Wang Jian Cheng 31 8 0 23 Jul 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 30 73 0 22 Jul 2019
BAM! Born-Again Multi-Task Networks for Natural Language Understanding Kevin Clark Minh-Thang Luong Urvashi Khandelwal Christopher D. Manning Quoc V. Le 21 228 0 10 Jul 2019
ReachNN: Reachability Analysis of Neural-Network Controlled Systems Chao Huang Jiameng Fan Wenchao Li Xin Chen Qi Zhu 31 78 0 25 Jun 2019
Scalable Syntax-Aware Language Models Using Knowledge Distillation A. Kuncoro Chris Dyer Laura Rimell S. Clark Phil Blunsom 35 26 0 14 Jun 2019
Interpretable Few-Shot Learning via Linear Distillation Arip Asadulaev Igor Kuznetsov Andrey Filchenkov FedML FAtt 11 1 0 13 Jun 2019
BasisConv: A method for compressed representation and learning in CNNs M. Tayyab Abhijit Mahalanobis 3DPC SSL 24 6 0 11 Jun 2019
Efficient Object Embedding for Spliced Image Retrieval Bor-Chun Chen Zuxuan Wu L. Davis Ser-Nam Lim 32 8 0 28 May 2019
Zero-shot Knowledge Transfer via Adversarial Belief Matching P. Micaelli Amos Storkey 19 228 0 23 May 2019
Play and Prune: Adaptive Filter Pruning for Deep Model Compression Pravendra Singh Vinay Kumar Verma Piyush Rai Vinay P. Namboodiri VLM 33 71 0 11 May 2019
A Review of Modularization Techniques in Artificial Neural Networks Mohammed Amer Tomás Maul 26 80 0 29 Apr 2019
A Large RGB-D Dataset for Semi-supervised Monocular Depth Estimation Jaehoon Cho Dongbo Min Youngjung Kim Kwanghoon Sohn MDE 3DV 33 47 0 23 Apr 2019
Feature Fusion for Online Mutual Knowledge Distillation Jangho Kim Minsung Hyun Inseop Chung Nojun Kwak FedML 26 91 0 19 Apr 2019
DocBERT: BERT for Document Classification Ashutosh Adhikari Achyudh Ram Raphael Tang Jimmy J. Lin LLMAG VLM 13 296 0 17 Apr 2019
Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillation Gakuto Kurata Kartik Audhkhasi 16 46 0 17 Apr 2019
Variational Information Distillation for Knowledge Transfer Sungsoo Ahn S. Hu Andreas C. Damianou Neil D. Lawrence Zhenwen Dai 58 609 0 11 Apr 2019
Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency Jia Li K. Fu Shengwei Zhao Shiming Ge 38 26 0 10 Apr 2019
Correlation Congruence for Knowledge Distillation Baoyun Peng Xiao Jin Jiaheng Liu Shunfeng Zhou Yichao Wu Yu Liu Dongsheng Li Zhaoning Zhang 63 507 0 03 Apr 2019
Benchmarking Approximate Inference Methods for Neural Structured Prediction Lifu Tu Kevin Gimpel BDL 33 17 0 01 Apr 2019
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks Raphael Tang Yao Lu Linqing Liu Lili Mou Olga Vechtomova Jimmy J. Lin 32 417 0 28 Mar 2019
Class-incremental Learning via Deep Model Consolidation Junting Zhang Jie Zhang Shalini Ghosh Dawei Li Serafettin Tasci Larry Heck Heming Zhang C.-C. Jay Kuo CLL 27 335 0 19 Mar 2019