Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

31 January 2021

Dan Alistarh

Papers citing "Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks"

50 / 361 papers shown

Title
Catapult Dynamics and Phase Transitions in Quadratic Nets David Meltzer Junyu Liu 31 9 0 18 Jan 2023
A Novel Sparse Regularizer Hovig Bayandorian 38 0 0 18 Jan 2023
ACQ: Improving Generative Data-free Quantization Via Attention Correction Jixing Li Xiaozhou Guo Benzhe Dai Guoliang Gong Min Jin Gang Chen Wenyu Mao Huaxiang Lu MQ 35 4 0 18 Jan 2023
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning Huan Wang Can Qin Yue Bai Yun Fu 37 20 0 12 Jan 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 44 6 0 09 Jan 2023
A Theory of I/O-Efficient Sparse Neural Network Inference Niels Gleinig Tal Ben-Nun Torsten Hoefler 33 0 0 03 Jan 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 41 643 0 02 Jan 2023
Sparse neural networks with skip-connections for identification of aluminum electrolysis cell E. Lundby Haakon Robinson Adil Rasheed I. Halvorsen J. Gravdahl 30 2 0 02 Jan 2023
Lifelong Reinforcement Learning with Modulating Masks Eseoghene Ben-Iwhiwhu Saptarshi Nath Praveen K. Pilly Soheil Kolouri Andrea Soltoggio CLL OffRL 37 20 0 21 Dec 2022
Dynamic Sparse Network for Time Series Classification: Learning What to "see'' Qiao Xiao Boqian Wu Yu Zhang Shiwei Liu Mykola Pechenizkiy Elena Mocanu Decebal Constantin Mocanu AI4TS 43 28 0 19 Dec 2022
Refiner: Data Refining against Gradient Leakage Attacks in Federated Learning Mingyuan Fan Cen Chen Chengyu Wang Ximeng Liu Wenmeng Zhou Jun Huang AAML FedML 34 0 0 05 Dec 2022
Distributed Pruning Towards Tiny Neural Networks in Federated Learning Hong Huang Lan Zhang Chaoyue Sun R. Fang Xiaoyong Yuan Dapeng Wu FedML 16 16 0 05 Dec 2022
Where to Pay Attention in Sparse Training for Feature Selection? Ghada Sokar Zahra Atashgahi Mykola Pechenizkiy Decebal Constantin Mocanu 40 17 0 26 Nov 2022
Efficient Incremental Text-to-Speech on GPUs Muyang Du Chuan Liu Jiaxing Qi Junjie Lai 24 1 0 25 Nov 2022
Spatial Mixture-of-Experts Nikoli Dryden Torsten Hoefler MoE 41 9 0 24 Nov 2022
Sparse Probabilistic Circuits via Pruning and Growing Meihua Dang Hoang Trung-Dung Guy Van den Broeck TPM 43 15 0 22 Nov 2022
LightDepth: A Resource Efficient Depth Estimation Approach for Dealing with Ground Truth Sparsity via Curriculum Learning Fatemeh Karimi Amir Mehrpanah Reza Rawassizadeh 29 1 0 16 Nov 2022
Efficient Traffic State Forecasting using Spatio-Temporal Network Dependencies: A Sparse Graph Neural Network Approach Bin Lei Shaoyi Huang Caiwen Ding Monika Filipovska GNN AI4TS 22 0 0 06 Nov 2022
Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions Shuhao Gu Bojie Hu Yang Feng CLL 44 13 0 03 Nov 2022
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar Saleh Ashkboos Torsten Hoefler Dan Alistarh MQ 33 905 0 31 Oct 2022
Sparsity in Continuous-Depth Neural Networks H. Aliee Till Richter Mikhail Solonin I. Ibarra Fabian J. Theis Niki Kilbertus 39 10 0 26 Oct 2022
Real-time Speech Interruption Analysis: From Cloud to Client Deployment Quchen Fu Szu-Wei Fu Yaran Fan Yu-Huan Wu Zhuo Chen J. Gupchup Ross Cutler 34 0 0 24 Oct 2022
Unlocking the potential of two-point cells for energy-efficient and resilient training of deep nets Ahsan Adeel A. Adetomi K. Ahmed Amir Hussain T. Arslan William A. Phillips 73 13 0 24 Oct 2022
Compressing multidimensional weather and climate data into neural networks La-mei Huang Torsten Hoefler AI4CE 51 31 0 22 Oct 2022
Pruning by Active Attention Manipulation Z. Babaiee Lucas Liebenwein Ramin Hasani Daniela Rus Radu Grosu 24 0 0 20 Oct 2022
Deep Differentiable Logic Gate Networks Felix Petersen Christian Borgelt Hilde Kuehne Oliver Deussen AI4CE 24 23 0 15 Oct 2022
CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models Denis Kuznedelev Eldar Kurtic Elias Frantar Dan Alistarh VLM ViT 27 11 0 14 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 36 47 0 13 Oct 2022
GMP*: Well-Tuned Gradual Magnitude Pruning Can Outperform Most BERT-Pruning Methods Eldar Kurtic Dan Alistarh AI4MH 38 14 0 12 Oct 2022
The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers Zong-xiao Li Chong You Srinadh Bhojanapalli Daliang Li A. S. Rawat ... Kenneth Q Ye Felix Chern Felix X. Yu Ruiqi Guo Surinder Kumar MoE 29 87 0 12 Oct 2022
SGD with Large Step Sizes Learns Sparse Features Maksym Andriushchenko Aditya Varre Loucas Pillaud-Vivien Nicolas Flammarion 47 56 0 11 Oct 2022
Advancing Model Pruning via Bi-level Optimization Yihua Zhang Yuguang Yao Parikshit Ram Pu Zhao Tianlong Chen Min-Fong Hong Yanzhi Wang Sijia Liu 56 68 0 08 Oct 2022
Over-the-Air Split Machine Learning in Wireless MIMO Networks YuZhi Yang Zhaoyang Zhang Yuqing Tian Zhaohui Yang Chongwen Huang C. Zhong Kai‐Kit Wong 31 23 0 07 Oct 2022
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning Weicong Liang Yuhui Yuan Henghui Ding Xiao Luo Weihong Lin Ding Jia Zheng-Wei Zhang Chao Zhang Hanhua Hu 45 26 0 03 Oct 2022
Green Learning: Introduction, Examples and Outlook C.-C. Jay Kuo A. Madni 78 71 0 03 Oct 2022
Model Zoos: A Dataset of Diverse Populations of Neural Network Models Konstantin Schurholt Diyar Taskiran Boris Knyazev Xavier Giró-i-Nieto Damian Borth 62 29 0 29 Sep 2022
A novel corrective-source term approach to modeling unknown physics in aluminum extraction process Haakon Robinson E. Lundby Adil Rasheed J. Gravdahl 28 5 0 22 Sep 2022
Extremely Simple Activation Shaping for Out-of-Distribution Detection Andrija Djurisic Nebojsa Bozanic Arjun Ashok Rosanne Liu OODD 172 152 0 20 Sep 2022
Self-Attentive Pooling for Efficient Deep Learning Fang Chen Gourav Datta Souvik Kundu Peter A. Beerel 82 6 0 16 Sep 2022
Efficient Quantized Sparse Matrix Operations on Tensor Cores Shigang Li Kazuki Osawa Torsten Hoefler 82 31 0 14 Sep 2022
Sparse deep neural networks for modeling aluminum electrolysis dynamics E. Lundby Adil Rasheed I. Halvorsen J. Gravdahl 29 14 0 13 Sep 2022
One-shot Network Pruning at Initialization with Discriminative Image Patches Yinan Yang Yu Wang Yi Ji Heng Qi Jien Kato VLM 34 4 0 13 Sep 2022
HammingMesh: A Network Topology for Large-Scale Deep Learning Torsten Hoefler Tommaso Bonato Daniele De Sensi Salvatore Di Girolamo Shigang Li Marco Heddes Jon Belk Deepak Goel Miguel Castro Steve Scott 3DH GNN AI4CE 34 20 0 03 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 40 109 0 31 Aug 2022
Generalization In Multi-Objective Machine Learning Peter Súkeník Christoph H. Lampert AI4CE 34 5 0 29 Aug 2022
Towards Sparsified Federated Neuroimaging Models via Weight Pruning Dimitris Stripelis Umang Gupta Nikhil J. Dhinagar Greg Ver Steeg Paul M. Thompson J. Ambite FedML 31 0 0 24 Aug 2022
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning Elias Frantar Sidak Pal Singh Dan Alistarh MQ 28 218 0 24 Aug 2022
Combining Compressions for Multiplicative Size Scaling on Natural Language Tasks Rajiv Movva Jinhao Lei Shayne Longpre Ajay K. Gupta Chris DuBois VLM MQ 50 4 0 20 Aug 2022
A Scalable, Interpretable, Verifiable & Differentiable Logic Gate Convolutional Neural Network Architecture From Truth Tables Adrien Benamira Tristan Guérand Thomas Peyrin Trevor Yap Bryan Hooi 42 1 0 18 Aug 2022
An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers Chao Fang Aojun Zhou Zhongfeng Wang MoE 35 53 0 12 Aug 2022