Rethinking Weight Decay For Efficient Neural Network Pruning

20 November 2020

Vincent Gripon

Papers citing "Rethinking Weight Decay For Efficient Neural Network Pruning"

50 / 60 papers shown

Title
Pruning Neural Networks at Initialization: Why are We Missing the Mark? Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin 55 240 0 18 Sep 2020
Pruning neural networks without any data by iteratively conserving synaptic flow Hidenori Tanaka D. Kunin Daniel L. K. Yamins Surya Ganguli 133 641 0 09 Jun 2020
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 255 1,047 0 06 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 258 387 0 05 Mar 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 96 274 0 03 Feb 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 144 617 0 11 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 354 42,299 0 03 Dec 2019
Rigging the Lottery: Making All Tickets Winners Utku Evci Trevor Gale Jacob Menick Pablo Samuel Castro Erich Elsen 162 600 0 25 Nov 2019
Deep geometric knowledge distillation with graphs Carlos Lassance Myriam Bontonou G. B. Hacene Vincent Gripon Jian Tang Antonio Ortega 37 39 0 08 Nov 2019
Sparse Networks from Scratch: Faster Training without Losing Performance Tim Dettmers Luke Zettlemoyer 114 339 0 10 Jul 2019
Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform? Xiaolong Ma Sheng Lin Shaokai Ye Zhezhi He Linfeng Zhang ... Deliang Fan Xuehai Qian Xinyu Lin Kaisheng Ma Yanzhi Wang MQ 66 92 0 03 Jul 2019
Importance Estimation for Neural Network Pruning Pavlo Molchanov Arun Mallya Stephen Tyree I. Frosio Jan Kautz 3DPC 73 879 0 25 Jun 2019
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers Ari S. Morcos Haonan Yu Michela Paganini Yuandong Tian 56 229 0 06 Jun 2019
Attention Based Pruning for Shift Networks G. B. Hacene Carlos Lassance Vincent Gripon Matthieu Courbariaux Yoshua Bengio 57 25 0 29 May 2019
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask Hattie Zhou Janice Lan Rosanne Liu J. Yosinski UQCV 49 386 0 03 May 2019
The State of Sparsity in Deep Neural Networks Trevor Gale Erich Elsen Sara Hooker 125 758 0 25 Feb 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 69 311 0 15 Feb 2019
Rethinking the Value of Network Pruning Zhuang Liu Mingjie Sun Tinghui Zhou Gao Huang Trevor Darrell 36 1,471 0 11 Oct 2018
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 227 1,196 0 04 Oct 2018
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks Yang He Guoliang Kang Xuanyi Dong Yanwei Fu Yi Yang AAML VLM 60 963 0 21 Aug 2018
PCAS: Pruning Channels with Attention Statistics for Deep Network Compression Kohei Yamamoto K. Maeno 41 33 0 14 Jun 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 200 3,457 0 09 Mar 2018
Compressing Neural Networks using the Variational Information Bottleneck Bin Dai Chen Zhu David Wipf MLT 49 181 0 28 Feb 2018
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers Jianbo Ye Xin Lu Zhe Lin Jianmin Wang 67 408 0 01 Feb 2018
Learning to Prune Filters in Convolutional Neural Networks Qiangui Huang S. Kevin Zhou Suya You Ulrich Neumann VLM 77 177 0 23 Jan 2018
Learning Sparse Neural Networks through $L_0$ Regularization Christos Louizos Max Welling Diederik P. Kingma 335 1,144 0 04 Dec 2017
NISP: Pruning Networks using Neuron Importance Score Propagation Ruichi Yu Ang Li Chun-Fu Chen Jui-Hsin Lai Vlad I. Morariu Xintong Han M. Gao Ching-Yung Lin L. Davis 70 799 0 16 Nov 2017
Deep Rewiring: Training very sparse deep networks G. Bellec David Kappel Wolfgang Maass Robert Legenstein BDL 123 277 0 14 Nov 2017
NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm Xiaoliang Dai Hongxu Yin N. Jha DD 75 235 0 06 Nov 2017
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 167 1,273 0 05 Oct 2017
Learning Efficient Convolutional Networks through Network Slimming Zhuang Liu Jianguo Li Zhiqiang Shen Gao Huang Shoumeng Yan Changshui Zhang 122 2,418 0 22 Aug 2017
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression Jian-Hao Luo Jianxin Wu Weiyao Lin 52 1,760 0 20 Jul 2017
Channel Pruning for Accelerating Very Deep Neural Networks Yihui He Xiangyu Zhang Jian Sun 196 2,521 0 19 Jul 2017
Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science Decebal Constantin Mocanu Elena Mocanu Peter Stone Phuong H. Nguyen M. Gibescu A. Liotta 148 626 0 15 Jul 2017
Bayesian Compression for Deep Learning Christos Louizos Karen Ullrich Max Welling UQCV BDL 139 479 0 24 May 2017
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon Xin Luna Dong Shangyu Chen Sinno Jialin Pan 148 504 0 22 May 2017
Structured Bayesian Pruning via Log-Normal Multiplicative Noise Kirill Neklyudov Dmitry Molchanov Arsenii Ashukha Dmitry Vetrov BDL 113 188 0 20 May 2017
Soft Weight-Sharing for Neural Network Compression Karen Ullrich Edward Meeds Max Welling 142 417 0 13 Feb 2017
Variational Dropout Sparsifies Deep Neural Networks Dmitry Molchanov Arsenii Ashukha Dmitry Vetrov BDL 115 827 0 19 Jan 2017
Training Sparse Neural Networks Suraj Srinivas Akshayvarun Subramanya R. Venkatesh Babu 118 207 0 21 Nov 2016
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 474 10,305 0 16 Nov 2016
Compact Deep Convolutional Neural Networks With Coarse Pruning S. Anwar Wonyong Sung 3DPC 54 55 0 30 Oct 2016
Semi-Supervised Classification with Graph Convolutional Networks Thomas Kipf Max Welling GNN SSL 573 28,964 0 09 Sep 2016
Pruning Filters for Efficient ConvNets Hao Li Asim Kadav Igor Durdanovic H. Samet H. Graf 3DPC 186 3,692 0 31 Aug 2016
Dynamic Network Surgery for Efficient DNNs Yiwen Guo Anbang Yao Yurong Chen 79 1,058 0 16 Aug 2016
Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures Hengyuan Hu Rui Peng Yu-Wing Tai Chi-Keung Tang 60 891 0 12 Jul 2016
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks Mohammad Rastegari Vicente Ordonez Joseph Redmon Ali Farhadi MQ 159 4,350 0 16 Mar 2016
Structured Pruning of Deep Convolutional Neural Networks S. Anwar Kyuyeon Hwang Wonyong Sung 85 747 0 29 Dec 2015
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.9K 193,426 0 10 Dec 2015
BinaryConnect: Training Deep Neural Networks with binary weights during propagations Matthieu Courbariaux Yoshua Bengio J. David MQ 181 2,984 0 02 Nov 2015