v1v2 (latest)

An Overview of Neural Network Compression

5 June 2020

Papers citing "An Overview of Neural Network Compression"

50 / 157 papers shown

Title
Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control Devdhar Patel H. Siegelmann OffRL 100 0 0 11 Oct 2024
Towards Understanding Knowledge Distillation Mary Phuong Christoph H. Lampert 71 323 0 27 May 2021
Pruning neural networks without any data by iteratively conserving synaptic flow Hidenori Tanaka D. Kunin Daniel L. K. Yamins Surya Ganguli 180 650 0 09 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 889 42,463 0 28 May 2020
Improved Noisy Student Training for Automatic Speech Recognition Daniel S. Park Yu Zhang Ye Jia Wei Han Chung-Cheng Chiu Yue Liu Yonghui Wu Quoc V. Le 103 243 0 19 May 2020
Training with Quantization Noise for Extreme Model Compression Angela Fan Pierre Stock Benjamin Graham Edouard Grave Remi Gribonval Hervé Jégou Armand Joulin MQ 102 246 0 15 Apr 2020
Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural Networks Léopold Cambier Anahita Bhiwandiwalla Ting Gong M. Nekuii Oguz H. Elibol Hanlin Tang MQ 122 48 0 16 Jan 2020
Exact expressions for double descent and implicit regularization via surrogate random design Michal Derezinski Feynman T. Liang Michael W. Mahoney 72 78 0 10 Dec 2019
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 123 945 0 04 Dec 2019
What's Hidden in a Randomly Weighted Neural Network? Vivek Ramanujan Mitchell Wortsman Aniruddha Kembhavi Ali Farhadi Mohammad Rastegari 66 361 0 29 Nov 2019
Distilling Knowledge Learned in BERT for Text Generation Yen-Chun Chen Zhe Gan Yu Cheng Jingzhou Liu Jingjing Liu 61 28 0 10 Nov 2019
Understanding Knowledge Distillation in Non-autoregressive Machine Translation Chunting Zhou Graham Neubig Jiatao Gu 76 221 0 07 Nov 2019
NAT: Neural Architecture Transformer for Accurate and Compact Architectures Yong Guo Yin Zheng Mingkui Tan Qi Chen Jian Chen P. Zhao Junzhou Huang 130 84 0 31 Oct 2019
Variational Student: Learning Compact and Sparser Networks in Knowledge Distillation Framework Srinidhi Hegde Ranjitha Prasad R. Hebbalaguppe Vishwajith Kumar 18 18 0 26 Oct 2019
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 168 1,054 0 23 Oct 2019
On the Efficacy of Knowledge Distillation Ligang He Rui Mao 98 619 0 03 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 255 7,554 0 02 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 373 6,469 0 26 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 113 1,872 0 23 Sep 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 336 1,918 0 17 Sep 2019
Deep Equilibrium Models Shaojie Bai J. Zico Kolter V. Koltun 94 672 0 03 Sep 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 142 843 0 25 Aug 2019
Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks Ruihao Gong Xianglong Liu Shenghu Jiang Tian-Hao Li Peng Hu Jiazhen Lin F. Yu Junjie Yan MQ 74 459 0 14 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 689 24,557 0 26 Jul 2019
Similarity-Preserving Knowledge Distillation Frederick Tung Greg Mori 126 981 0 23 Jul 2019
And the Bit Goes Down: Revisiting the Quantization of Neural Networks Pierre Stock Armand Joulin Rémi Gribonval Benjamin Graham Hervé Jégou MQ 84 149 0 12 Jul 2019
Sparse Networks from Scratch: Faster Training without Losing Performance Tim Dettmers Luke Zettlemoyer 145 340 0 10 Jul 2019
Sharing Attention Weights for Fast Transformer Tong Xiao Yinqiao Li Jingbo Zhu Zhengtao Yu Tongran Liu 67 52 0 26 Jun 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 236 8,451 0 19 Jun 2019
Weight Agnostic Neural Networks Adam Gaier David R Ha OOD 67 242 0 11 Jun 2019
When Does Label Smoothing Help? Rafael Müller Simon Kornblith Geoffrey E. Hinton UQCV 210 1,955 0 06 Jun 2019
Mixed Precision Training With 8-bit Floating Point Naveen Mellempudi Sudarshan Srinivasan Dipankar Das Bharat Kaul MQ 66 69 0 29 May 2019
A Study of BFLOAT16 for Deep Learning Training Dhiraj D. Kalamkar Dheevatsa Mudigere Naveen Mellempudi Dipankar Das K. Banerjee ... Sudarshan Srinivasan Abhisek Kundu M. Smelyanskiy Bharat Kaul Pradeep Dubey MQ 88 347 0 29 May 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 167 18,193 0 28 May 2019
HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision Zhen Dong Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 90 528 0 29 Apr 2019
Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding Xiaodong Liu Pengcheng He Weizhu Chen Jianfeng Gao FedML 62 182 0 20 Apr 2019
Relational Knowledge Distillation Wonpyo Park Dongju Kim Yan Lu Minsu Cho 87 1,425 0 10 Apr 2019
ASAP: Architecture Search, Anneal and Prune Asaf Noy Niv Nayman T. Ridnik Nadav Zamir Sivan Doveh Itamar Friedman Raja Giryes Lihi Zelnik-Manor 70 103 0 08 Apr 2019
An Attentive Survey of Attention Models S. Chaudhari Varun Mithal Gungor Polatkan R. Ramanath 153 662 0 05 Apr 2019
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning Shaohui Lin Rongrong Ji Chenqian Yan Baochang Zhang Liujuan Cao QiXiang Ye Feiyue Huang David Doermann CVBM 56 510 0 22 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 117 375 0 18 Mar 2019
Stabilizing the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin 70 103 0 05 Mar 2019
Learning Implicitly Recurrent CNNs Through Parameter Sharing Pedro H. P. Savarese Michael Maire 79 70 0 26 Feb 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 89 314 0 15 Feb 2019
Improved Knowledge Distillation via Teacher Assistant Seyed Iman Mirzadeh Mehrdad Farajtabar Ang Li Nir Levine Akihiro Matsukawa H. Ghasemzadeh 100 1,081 0 09 Feb 2019
Compressing GANs using Knowledge Distillation Angeline Aguinaldo Ping Yeh-Chiang Alex Gain Ameya D. Patil Kolten Pearson Soheil Feizi GAN 57 85 0 01 Feb 2019
Convolutional Neural Networks with Layer Reuse Okan Kopuklu M. Babaee S. Hörmann Gerhard Rigoll 40 17 0 28 Jan 2019
A Survey of the Recent Architectures of Deep Convolutional Neural Networks Asifullah Khan A. Sohail Umme Zahoora Aqsa Saeed Qureshi OOD 119 2,310 0 17 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 247 1,659 0 28 Dec 2018
Training Deep Neural Networks with 8-bit Floating Point Numbers Naigang Wang Jungwook Choi D. Brand Chia-Yu Chen K. Gopalakrishnan MQ 73 503 0 19 Dec 2018