Do Deep Nets Really Need to be Deep?

21 December 2013

Lei Jimmy Ba

Papers citing "Do Deep Nets Really Need to be Deep?"

50 / 379 papers shown

Title
Distilling Object Detectors with Feature Richness Zhixing Du Rui Zhang Ming-Fang Chang Xishan Zhang Shaoli Liu Tianshi Chen Yunji Chen ObjD 19 74 0 01 Nov 2021
RGP: Neural Network Pruning through Its Regular Graph Structure Zhuangzhi Chen Jingyang Xiang Yao Lu Qi Xuan Xiaoniu Yang 27 1 0 28 Oct 2021
PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations Louis Fortier-Dubois Gaël Letarte Benjamin Leblanc Franccois Laviolette Pascal Germain UQCV 19 0 0 28 Oct 2021
Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation A. Tavera Fabio Cermelli Carlo Masone Barbara Caputo 29 19 0 22 Oct 2021
An Economy of Neural Networks: Learning from Heterogeneous Experiences A. Kuriksha 27 7 0 22 Oct 2021
Augmenting Knowledge Distillation With Peer-To-Peer Mutual Learning For Model Compression Usma Niyaz Deepti R. Bathula 26 8 0 21 Oct 2021
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation Sumanth Chennupati Mohammad Mahdi Kamani Zhongwei Cheng Lin Chen 30 4 0 19 Oct 2021
Efficient and Private Federated Learning with Partially Trainable Networks Hakim Sidahmed Zheng Xu Ankush Garg Yuan Cao Mingqing Chen FedML 49 13 0 06 Oct 2021
Multilingual AMR Parsing with Noisy Knowledge Distillation Deng Cai Xin Li Jackie Chun-Sing Ho Lidong Bing W. Lam 27 18 0 30 Sep 2021
Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Network Takaaki Saeki Shinnosuke Takamichi Hiroshi Saruwatari 34 3 0 22 Sep 2021
A Studious Approach to Semi-Supervised Learning Sahil Khose Shruti Jain V. Manushree 18 0 0 18 Sep 2021
Comfetch: Federated Learning of Large Networks on Constrained Clients via Sketching Tahseen Rabbani Brandon Yushan Feng Marco Bornstein Kyle Rui Sang Yifan Yang Arjun Rajkumar A. Varshney Furong Huang FedML 59 2 0 17 Sep 2021
Secure Your Ride: Real-time Matching Success Rate Prediction for Passenger-Driver Pairs Yuandong Wang Hongzhi Yin Lian Wu Tong Chen Chunyang Liu 18 7 0 14 Sep 2021
On the Efficiency of Subclass Knowledge Distillation in Classification Tasks A. Sajedi Konstantinos N. Plataniotis 16 4 0 12 Sep 2021
FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Models Lan Zhang Dapeng Wu Xiaoyong Yuan FedML 38 48 0 08 Sep 2021
Compact representations of convolutional neural networks via weight pruning and quantization Giosuè Cataldo Marinò A. Petrini D. Malchiodi Marco Frasca MQ 21 4 0 28 Aug 2021
Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP Lifu Tu BDL 35 0 0 27 Aug 2021
Efficient training of lightweight neural networks using Online Self-Acquired Knowledge Distillation Maria Tzelepi Anastasios Tefas 13 6 0 26 Aug 2021
Supervised Compression for Resource-Constrained Edge Computing Systems Yoshitomo Matsubara Ruihan Yang Marco Levorato Stephan Mandt 19 58 0 21 Aug 2021
DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANs J. Nistal Stefan Lattner G. Richard 26 8 0 03 Aug 2021
Developing efficient transfer learning strategies for robust scene recognition in mobile robotics using pre-trained convolutional neural networks H. Baumgartl Ricardo Buettner 3DPC 54 3 0 23 Jul 2021
SAGE: A Split-Architecture Methodology for Efficient End-to-End Autonomous Vehicle Control Arnav V. Malawade Mohanad Odema Sebastien Lajeunesse-DeGroot M. A. Al Faruque 28 20 0 22 Jul 2021
Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies F. Xie Han Yuan Yilin Ning M. Ong Mengling Feng W. Hsu B. Chakraborty Nan Liu 32 84 0 21 Jul 2021
Mitigating severe over-parameterization in deep convolutional neural networks through forced feature abstraction and compression with an entropy-based heuristic Nidhi Gowdra R. Sinha Stephen G. MacDonell W. Yan 21 9 0 27 Jun 2021
Simple Distillation Baselines for Improving Small Self-supervised Models Jindong Gu Wei Liu Yonglong Tian 24 8 0 21 Jun 2021
Knowledge Distillation via Instance-level Sequence Learning Haoran Zhao Xin Sun Junyu Dong Zihe Dong Qiong Li 34 23 0 21 Jun 2021
We Can Always Catch You: Detecting Adversarial Patched Objects WITH or WITHOUT Signature Binxiu Liang Jiachun Li Jianjun Huang AAML 33 12 0 09 Jun 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
Data-Free Knowledge Distillation for Heterogeneous Federated Learning Zhuangdi Zhu Junyuan Hong Jiayu Zhou FedML 27 631 0 20 May 2021
What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory Rahul Parhi Robert D. Nowak MLT 38 70 0 07 May 2021
Performance Evaluation of Deep Convolutional Maxout Neural Network in Speech Recognition Arash Dehghani Seyyed Ali Seyyedsalehi 23 2 0 04 May 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 187 690 0 22 Apr 2021
Balanced Knowledge Distillation for Long-tailed Learning Shaoyu Zhang Chen Chen Xiyuan Hu Silong Peng 48 57 0 21 Apr 2021
Distill on the Go: Online knowledge distillation in self-supervised learning Prashant Shivaram Bhat Elahe Arani Bahram Zonooz SSL 22 28 0 20 Apr 2021
Knowledge Distillation as Semiparametric Inference Tri Dao G. Kamath Vasilis Syrgkanis Lester W. Mackey 40 31 0 20 Apr 2021
Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation Emilio Parisotto Ruslan Salakhutdinov 42 44 0 04 Apr 2021
Student Network Learning via Evolutionary Knowledge Distillation Kangkai Zhang Chunhui Zhang Shikun Li Dan Zeng Shiming Ge 22 83 0 23 Mar 2021
Compacting Deep Neural Networks for Internet of Things: Methods and Applications Ke Zhang Hanbo Ying Hongning Dai Lin Li Yuangyuang Peng Keyi Guo Hongfang Yu 21 38 0 20 Mar 2021
Membership Inference Attacks on Machine Learning: A Survey Hongsheng Hu Z. Salcic Lichao Sun Gillian Dobbie Philip S. Yu Xuyun Zhang MIACV 35 412 0 14 Mar 2021
Contrastive Semi-supervised Learning for ASR Alex Xiao Christian Fuegen Abdel-rahman Mohamed 26 20 0 09 Mar 2021
Deep Model Intellectual Property Protection via Deep Watermarking Jie Zhang Dongdong Chen Jing Liao Weiming Zhang Huamin Feng G. Hua Nenghai Yu 33 106 0 08 Mar 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 33 199 0 08 Mar 2021
Adaptive Multi-Teacher Multi-level Knowledge Distillation Yuang Liu Wei Zhang Jun Wang 28 157 0 06 Mar 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 29 14 0 21 Feb 2021
Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS Felix O. Olowononi D. Rawat Chunmei Liu 36 133 0 14 Feb 2021
Copycat CNN: Are Random Non-Labeled Data Enough to Steal Knowledge from Black-box Models? Jacson Rodrigues Correia-Silva Rodrigo Berriel C. Badue Alberto F. de Souza Thiago Oliveira-Santos MLAU 18 14 0 21 Jan 2021
Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation Lingyun Feng Minghui Qiu Yaliang Li Haitao Zheng Ying Shen 46 10 0 20 Jan 2021
Resolution-Based Distillation for Efficient Histology Image Classification Joseph DiPalma A. Suriawinata L. Tafe Lorenzo Torresani Saeed Hassanpour 40 35 0 11 Jan 2021
Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts Kunpeng Li Zizhao Zhang Guanhang Wu Xuehan Xiong Chen-Yu Lee Zhichao Lu Y. Fu Tomas Pfister 34 5 0 11 Jan 2021
Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead Maurizio Capra Beatrice Bussolino Alberto Marchisio Guido Masera Maurizio Martina Muhammad Shafique BDL 59 140 0 21 Dec 2020