The State of Sparsity in Deep Neural Networks

25 February 2019

Papers citing "The State of Sparsity in Deep Neural Networks"

50 / 155 papers shown

Title
Token Pooling in Vision Transformers D. Marin Jen-Hao Rick Chang Anurag Ranjan Anish K. Prabhu Mohammad Rastegari Oncel Tuzel ViT 76 66 0 08 Oct 2021
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis Cheng-I Jeff Lai Erica Cooper Yang Zhang Shiyu Chang Kaizhi Qian ... Yung-Sung Chuang Alexander H. Liu Junichi Yamagishi David D. Cox James R. Glass 26 6 0 04 Oct 2021
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 92 54 0 01 Oct 2021
RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging Edouard Yvinec Arnaud Dapogny Matthieu Cord Kévin Bailly 28 15 0 30 Sep 2021
Training Deep Spiking Auto-encoders without Bursting or Dying Neurons through Regularization Justus F. Hübotter Pablo Lanillos Jakub M. Tomczak 16 3 0 22 Sep 2021
Layer-wise Model Pruning based on Mutual Information Chun Fan Jiwei Li Xiang Ao Fei Wu Yuxian Meng Xiaofei Sun 48 19 0 28 Aug 2021
M-FAC: Efficient Matrix-Free Approximations of Second-Order Information Elias Frantar Eldar Kurtic Dan Alistarh 13 57 0 07 Jul 2021
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot? Xiaolong Ma Geng Yuan Xuan Shen Tianlong Chen Xuxi Chen ... Ning Liu Minghai Qin Sijia Liu Zhangyang Wang Yanzhi Wang 30 63 0 01 Jul 2021
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity Shiwei Liu Tianlong Chen Zahra Atashgahi Xiaohan Chen Ghada Sokar Elena Mocanu Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu OOD 31 49 0 28 Jun 2021
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration Shiwei Liu Tianlong Chen Xiaohan Chen Zahra Atashgahi Lu Yin Huanyu Kou Li Shen Mykola Pechenizkiy Zhangyang Wang Decebal Constantin Mocanu 40 112 0 19 Jun 2021
A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness James Diffenderfer Brian Bartoldson Shreya Chaganti Jize Zhang B. Kailkhura OOD 31 69 0 16 Jun 2021
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better Gaurav Menghani VLM MedIm 23 366 0 16 Jun 2021
GANs Can Play Lottery Tickets Too Xuxi Chen Zhenyu Zhang Yongduo Sui Tianlong Chen GAN 24 58 0 31 May 2021
Effective Sparsification of Neural Networks with Global Sparsity Constraint Xiao Zhou Weizhong Zhang Hang Xu Tong Zhang 21 61 0 03 May 2021
Post-training deep neural network pruning via layer-wise calibration Ivan Lazarevich Alexander Kozlov Nikita Malinin 3DPC 18 25 0 30 Apr 2021
Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones? Franco Pellegrini Giulio Biroli 49 6 0 27 Apr 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 109 54 0 23 Apr 2021
Partitioning sparse deep neural networks for scalable training and inference G. Demirci Hakan Ferhatosmanoglu 20 11 0 23 Apr 2021
Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm Dongkuan Xu Ian En-Hsu Yen Jinxi Zhao Zhibin Xiao VLM AAML 31 56 0 18 Apr 2021
The Elastic Lottery Ticket Hypothesis Xiaohan Chen Yu Cheng Shuohang Wang Zhe Gan Jingjing Liu Zhangyang Wang OOD 23 34 0 30 Mar 2021
On the Robustness of Monte Carlo Dropout Trained with Noisy Labels Purvi Goel Li Chen NoLa 36 15 0 22 Mar 2021
$Function approximation by deep neural networks with parameters $\{0,\pm \frac{1}{2}, \pm 1, 2\}$$ Function approximation by deep neural networks with parameters $\{0,\pm \frac{1}{2}, \pm 1, 2\}$ A. Beknazaryan 18 5 0 15 Mar 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 33 64 0 11 Mar 2021
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy Lucas Liebenwein Cenk Baykal Brandon Carter David K Gifford Daniela Rus AAML 40 71 0 04 Mar 2021
Reduced-Order Neural Network Synthesis with Robustness Guarantees R. Drummond M. Turner S. Duncan 24 9 0 18 Feb 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 95 35 0 16 Feb 2021
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch Aojun Zhou Yukun Ma Junnan Zhu Jianbo Liu Zhijie Zhang Kun Yuan Wenxiu Sun Hongsheng Li 64 240 0 08 Feb 2021
SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks Enzo Tartaglione Andrea Bragagnolo Francesco Odierna Attilio Fiandrotti Marco Grangetto 40 18 0 07 Feb 2021
Pruning and Quantization for Deep Neural Network Acceleration: A Survey Tailin Liang C. Glossner Lei Wang Shaobo Shi Xiaotong Zhang MQ 150 675 0 24 Jan 2021
Neural Pruning via Growing Regularization Huan Wang Can Qin Yulun Zhang Y. Fu 37 144 0 16 Dec 2020
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Michael Carbin Zhangyang Wang 27 123 0 12 Dec 2020
Rethinking Weight Decay For Efficient Neural Network Pruning Hugo Tessier Vincent Gripon Mathieu Léonardon M. Arzel T. Hannagan David Bertrand 26 25 0 20 Nov 2020
Neural Network Compression Via Sparse Optimization Tianyi Chen Bo Ji Yixin Shi Tianyu Ding Biyi Fang Sheng Yi Xiao Tu 36 15 0 10 Nov 2020
Are wider nets better given the same number of parameters? A. Golubeva Behnam Neyshabur Guy Gur-Ari 27 44 0 27 Oct 2020
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win Utku Evci Yani Andrew Ioannou Cem Keskin Yann N. Dauphin 35 87 0 07 Oct 2020
Pruning Convolutional Filters using Batch Bridgeout Najeeb Khan Ian Stavness 28 3 0 23 Sep 2020
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot Jingtong Su Yihang Chen Tianle Cai Tianhao Wu Ruiqi Gao Liwei Wang J. Lee 14 85 0 22 Sep 2020
The Hardware Lottery Sara Hooker 27 203 0 14 Sep 2020
SparseRT: Accelerating Unstructured Sparsity on GPUs for Deep Learning Inference Ziheng Wang 40 66 0 26 Aug 2020
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights Shail Dave Riyadh Baghdadi Tony Nowatzki Sasikanth Avancha Aviral Shrivastava Baoxin Li 59 82 0 02 Jul 2020
Revisiting Loss Modelling for Unstructured Pruning César Laurent Camille Ballas Thomas George Nicolas Ballas Pascal Vincent 30 14 0 22 Jun 2020
Sparse GPU Kernels for Deep Learning Trevor Gale Matei A. Zaharia C. Young Erich Elsen 17 228 0 18 Jun 2020
On the Predictability of Pruning Across Scales Jonathan S. Rosenfeld Jonathan Frankle Michael Carbin Nir Shavit 25 37 0 18 Jun 2020
Directional Pruning of Deep Neural Networks Shih-Kang Chao Zhanyu Wang Yue Xing Guang Cheng ODL 21 33 0 16 Jun 2020
A Framework for Neural Network Pruning Using Gibbs Distributions Alex Labach S. Valaee 9 5 0 08 Jun 2020
Movement Pruning: Adaptive Sparsity by Fine-Tuning Victor Sanh Thomas Wolf Alexander M. Rush 32 468 0 15 May 2020
Ensembled sparse-input hierarchical networks for high-dimensional datasets Jean Feng N. Simon 19 4 0 11 May 2020
The Right Tool for the Job: Matching Model and Instance Complexities Roy Schwartz Gabriel Stanovsky Swabha Swayamdipta Jesse Dodge Noah A. Smith 38 168 0 16 Apr 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 232 383 0 05 Mar 2020
Sparse Networks from Scratch: Faster Training without Losing Performance Tim Dettmers Luke Zettlemoyer 20 334 0 10 Jul 2019