Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

31 January 2021

Dan Alistarh

Papers citing "Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks"

50 / 361 papers shown

Title
A Theoretical View on Sparsely Activated Networks Cenk Baykal Nishanth Dikkala Rina Panigrahy Cyrus Rashtchian Xin Wang 19 10 0 08 Aug 2022
ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity Xinchi Qiu Javier Fernandez-Marques Pedro Gusmão Yan Gao Titouan Parcollet Nicholas D. Lane FedML 55 67 0 04 Aug 2022
CrAM: A Compression-Aware Minimizer Alexandra Peste Adrian Vladu Eldar Kurtic Christoph H. Lampert Dan Alistarh 39 9 0 28 Jul 2022
Sparse Deep Neural Network for Nonlinear Partial Differential Equations Yuesheng Xu T. Zeng 33 5 0 27 Jul 2022
Trainability Preserving Neural Pruning Huan Wang Yun Fu AAML 22 17 0 25 Jul 2022
Dynamic Channel Selection in Self-Supervised Learning Tarun Krishna Ayush K. Rai Y. A. D. Djilali Alan F. Smeaton Kevin McGuinness Noel E. O'Connor SSL 46 2 0 25 Jul 2022
Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks Chuang Liu Xueqi Ma Yinbing Zhan Liang Ding Dapeng Tao Bo Du Wenbin Hu Danilo Mandic 44 29 0 18 Jul 2022
Context-sensitive neocortical neurons transform the effectiveness and efficiency of neural information processing Ahsan Adeel Mario Franco Mohsin Raza K. Ahmed 31 9 0 15 Jul 2022
SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning Zihao Ye Ruihang Lai Junru Shao Tianqi Chen Luis Ceze 78 93 0 11 Jul 2022
Understanding the effect of sparsity on neural networks robustness Lukas Timpl R. Entezari Hanie Sedghi Behnam Neyshabur O. Saukh 51 12 0 22 Jun 2022
0/1 Deep Neural Networks via Block Coordinate Descent Hui Zhang Shenglong Zhou Geoffrey Ye Li N. Xiu 43 7 0 19 Jun 2022
The State of Sparse Training in Deep Reinforcement Learning L. Graesser Utku Evci Erich Elsen Pablo Samuel Castro OffRL 17 34 0 17 Jun 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 86 27 0 17 Jun 2022
Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solution Xueying Ding Lingxiao Zhao Leman Akoglu OODD 41 22 0 15 Jun 2022
Deep Neural Network Pruning for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images A. Mahbod R. Entezari Isabella Ellinger O. Saukh 19 8 0 15 Jun 2022
A Stochastic Proximal Method for Nonsmooth Regularized Finite Sum Optimization Dounia Lakhmiri D. Orban Andrea Lodi 14 0 0 14 Jun 2022
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time Anand Subramoney Khaleelulla Khan Nazeer Mark Schöne Christian Mayr David Kappel 38 16 0 13 Jun 2022
A Theoretical Understanding of Neural Network Compression from Sparse Linear Approximation Wenjing Yang G. Wang Jie Ding Yuhong Yang MLT 41 7 0 11 Jun 2022
Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm Aidan Good Jia-Huei Lin Hannah Sieg Mikey Ferguson Xin Yu Shandian Zhe J. Wieczorek Thiago Serra 42 11 0 07 Jun 2022
Differentially Private Model Compression Fatemehsadat Mireshghallah A. Backurs Huseyin A. Inan Lukas Wutschitz Janardhan Kulkarni SyDa 24 13 0 03 Jun 2022
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees Jue Wang Binhang Yuan Luka Rimanic Yongjun He Tri Dao Beidi Chen Christopher Ré Ce Zhang AI4CE 31 11 0 02 Jun 2022
Spartan: Differentiable Sparsity via Regularized Transportation Kai Sheng Tai Taipeng Tian Ser-Nam Lim 34 11 0 27 May 2022
Compression-aware Training of Neural Networks using Frank-Wolfe Max Zimmer Christoph Spiegel Sebastian Pokutta 31 9 0 24 May 2022
Sharp asymptotics on the compression of two-layer neural networks Mohammad Hossein Amani Simone Bombari Marco Mondelli Rattana Pukdee Stefano Rini MLT 27 0 0 17 May 2022
An Inexact Augmented Lagrangian Algorithm for Training Leaky ReLU Neural Network with Group Sparsity Wei Liu Xin Liu Xiaojun Chen 20 10 0 11 May 2022
Federated Progressive Sparsification (Purge, Merge, Tune)+ Dimitris Stripelis Umang Gupta Greg Ver Steeg J. Ambite FedML 28 9 0 26 Apr 2022
Visual Attention Emerges from Recurrent Sparse Reconstruction Baifeng Shi Ya-heng Song Neel Joshi Trevor Darrell Xin Eric Wang 3DH 27 6 0 23 Apr 2022
Robust PCA Unrolling Network for Super-resolution Vessel Extraction in X-ray Coronary Angiography Binjie Qin Haohao Mao Yiming Liu Jun Zhao Yisong Lv Y. Zhu Song Ding Xu Chen 21 23 0 16 Apr 2022
End-to-End Sensitivity-Based Filter Pruning Z. Babaiee Lucas Liebenwein Ramin Hasani Daniela Rus Radu Grosu AAML 30 1 0 15 Apr 2022
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency Yanyang Li Fuli Luo Runxin Xu Songfang Huang Fei Huang Liwei Wang 47 3 0 06 Apr 2022
Optimizing the Consumption of Spiking Neural Networks with Activity Regularization Simon Narduzzi Siavash Bigdeli Shih-Chii Liu L. A. Dunbar 26 9 0 04 Apr 2022
Local Stochastic Factored Gradient Descent for Distributed Quantum State Tomography J. Kim Taha Toghani César A. Uribe Anastasios Kyrillidis 35 3 0 22 Mar 2022
Data-Efficient Structured Pruning via Submodular Optimization Marwa El Halabi Suraj Srinivas Simon Lacoste-Julien 22 18 0 09 Mar 2022
Dynamic ConvNets on Tiny Devices via Nested Sparsity Matteo Grimaldi Luca Mocerino A. Cipolletta A. Calimera 30 6 0 07 Mar 2022
Robust Training under Label Noise by Over-parameterization Sheng Liu Zhihui Zhu Qing Qu Chong You NoLa OOD 34 106 0 28 Feb 2022
Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets Tianlong Chen Xuxi Chen Xiaolong Ma Yanzhi Wang Zhangyang Wang 24 35 0 09 Feb 2022
Backpropagation Neural Tree Varun Ojha Giuseppe Nicosia BDL 36 14 0 04 Feb 2022
SPDY: Accurate Pruning with Speedup Guarantees Elias Frantar Dan Alistarh 47 33 0 31 Jan 2022
OptG: Optimizing Gradient-driven Criteria in Network Sparsity Yuxin Zhang Mingbao Lin Yonghong Tian Rongrong Ji Rongrong Ji 44 5 0 30 Jan 2022
Near-Optimal Sparse Allreduce for Distributed Deep Learning Shigang Li Torsten Hoefler 31 51 0 19 Jan 2022
Explainability Tools Enabling Deep Learning in Future In-Situ Real-Time Planetary Explorations Daniel Lundström Alexander Huyen Arya Mevada Kyongsik Yun Thomas Lu 6 1 0 15 Jan 2022
Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques JunKyu Lee L. Mukhanov A. S. Molahosseini U. Minhas Yang Hua Jesus Martinez del Rincon K. Dichev Cheol-Ho Hong Hans Vandierendonck 44 29 0 30 Dec 2021
Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks Kevin Lee Hunter Lawrence Spracklen Subutai Ahmad 28 20 0 27 Dec 2021
On Causal Inference for Data-free Structured Pruning Martin Ferianc A. Sankaran Olivier Mastropietro Ehsan Saboori Quentin Cappart CML 12 2 0 19 Dec 2021
How Well Do Sparse Imagenet Models Transfer? Eugenia Iofinova Alexandra Peste Mark Kurtz Dan Alistarh 27 38 0 26 Nov 2021
Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes Hugh Dance Brooks Paige GP 20 10 0 08 Nov 2021
Meta-Learning Sparse Implicit Neural Representations Jaehoon Lee Jihoon Tack Namhoon Lee Jinwoo Shin 29 44 0 27 Oct 2021
Differentiable Network Pruning for Microcontrollers Edgar Liberis Nicholas D. Lane 22 18 0 15 Oct 2021
Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations Xinyu Zhang Ian Colbert Ken Kreutz-Delgado Srinjoy Das MQ 37 11 0 15 Oct 2021
Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks Ghada Sokar Decebal Constantin Mocanu Mykola Pechenizkiy CLL 35 8 0 11 Oct 2021