Accelerating Sparse Deep Neural Networks

16 April 2021

Paulius Micikevicius

Papers citing "Accelerating Sparse Deep Neural Networks"

50 / 51 papers shown

Title
SPAP: Structured Pruning via Alternating Optimization and Penalty Methods Hanyu Hu Xiaoming Yuan 48 0 0 06 May 2025
Efficient LLMs with AMP: Attention Heads and MLP Pruning Leandro Giusti Mugnaini Bruno Yamamoto Lucas Lauton de Alcantara Victor Zacarias Edson Bollis Lucas Pellicer A. H. R. Costa Artur Jordao 47 0 0 29 Apr 2025
Hardware/Software Co-Design of RISC-V Extensions for Accelerating Sparse DNNs on FPGAs Muhammad Sabih Abrarul Karim Jakob Wittmann Frank Hannig J. Teich 75 0 0 28 Apr 2025
Periodic Online Testing for Sparse Systolic Tensor Arrays C. Peltekis Chrysostomos Nicopoulos G. Dimitrakopoulos 52 0 0 25 Apr 2025
DyDiT++: Dynamic Diffusion Transformers for Efficient Visual Generation Wangbo Zhao Yizeng Han Jiasheng Tang Kai Wang Hao Luo Yibing Song Gao Huang Fan Wang Yang You 74 0 0 09 Apr 2025
MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures Jiayu Qin Jianchao Tan Kaipeng Zhang Xunliang Cai Wei Wang 45 0 0 19 Feb 2025
Advancing Weight and Channel Sparsification with Enhanced Saliency Xinglong Sun Maying Shen Hongxu Yin Lei Mao Pavlo Molchanov Jose M. Alvarez 54 1 0 05 Feb 2025
GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference Chao Zeng Songwei Liu Shu Yang Fangmin Chen Xing Mei Lean Fu MQ 42 0 0 23 Dec 2024
Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training Elia Cunegatti Leonardo Lucio Custode Giovanni Iacca 52 0 0 11 Nov 2024
Navigating Extremes: Dynamic Sparsity in Large Output Spaces Nasib Ullah Erik Schultheis Mike Lasby Yani Andrew Ioannou Rohit Babbar 35 0 0 05 Nov 2024
Efficient Sparse Training with Structured Dropout Andy Lo BDL 35 0 0 02 Nov 2024
Self-calibration for Language Model Quantization and Pruning Miles Williams G. Chrysostomou Nikolaos Aletras MQ 165 0 0 22 Oct 2024
OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition Stephen Zhang Vardan Papyan VLM 51 1 0 20 Sep 2024
S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training Yuezhou Hu Jun-Jie Zhu Jianfei Chen 43 0 0 13 Sep 2024
Realizing Unaligned Block-wise Pruning for DNN Acceleration on Mobile Devices Hayun Lee Dongkun Shin MQ 28 0 0 29 Jul 2024
Effective Interplay between Sparsity and Quantization: From Theory to Practice Simla Burcu Harma Ayan Chakraborty Elizaveta Kostenok Danila Mishin Dongho Ha ... Martin Jaggi Ming Liu Yunho Oh Suvinay Subramanian Amir Yazdanbakhsh MQ 44 6 0 31 May 2024
SparseDM: Toward Sparse Efficient Diffusion Models Kafeng Wang Jianfei Chen He Li Zhenpeng Mi Jun-Jie Zhu DiffM 68 8 0 16 Apr 2024
Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models Xudong Lu Qi Liu Yuhui Xu Aojun Zhou Siyuan Huang Bo-Wen Zhang Junchi Yan Hongsheng Li MoE 32 26 0 22 Feb 2024
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes Lucio Dery Steven Kolawole Jean-Francois Kagey Virginia Smith Graham Neubig Ameet Talwalkar 44 28 0 08 Feb 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 52 10 0 23 Dec 2023
MaxQ: Multi-Axis Query for N:M Sparsity Network Jingyang Xiang Siqi Li Junhao Chen Zhuangzhi Chen Tianxin Huang Linpeng Peng Yong-Jin Liu 16 0 0 12 Dec 2023
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 44 1 0 29 Nov 2023
REDS: Resource-Efficient Deep Subnetworks for Dynamic Resource Constraints Francesco Corti Balz Maag Joachim Schauer U. Pferschy O. Saukh 34 2 0 22 Nov 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 62 359 0 20 Jun 2023
Boost Vision Transformer with GPU-Friendly Sparsity and Quantization Chong Yu Tao Chen Zhongxue Gan Jiayuan Fan MQ ViT 30 23 0 18 May 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 30 41 0 07 Apr 2023
VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs Geonhwa Jeong S. Damani Abhimanyu Bambhaniya Eric Qin C. Hughes S. Subramoney Hyesoon Kim T. Krishna MoE 37 24 0 17 Feb 2023
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks Mahdi Nikdan Tommaso Pegolotti Eugenia Iofinova Eldar Kurtic Dan Alistarh 26 11 0 09 Feb 2023
Mixed Precision Post Training Quantization of Neural Networks with Sensitivity Guided Search Clemens J. S. Schaefer Elfie Guo Caitlin Stanton Xiaofan Zhang T. Jablin Navid Lambert-Shirzad Jian Li Chia-Wei Chou Siddharth Joshi Yu Wang MQ 31 3 0 02 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
Statistical guarantees for sparse deep learning Johannes Lederer 16 11 0 11 Dec 2022
On-device Training: A First Overview on Existing Systems Shuai Zhu Thiemo Voigt Jeonggil Ko Fatemeh Rahimian 34 14 0 01 Dec 2022
SNIPER Training: Single-Shot Sparse Training for Text-to-Speech Perry Lam Huayun Zhang Nancy F. Chen Berrak Sisman Dorien Herremans VLM 30 0 0 14 Nov 2022
SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance Edouard Yvinec Arnaud Dapogny Matthieu Cord Kévin Bailly 42 9 0 08 Jul 2022
DRESS: Dynamic REal-time Sparse Subnets Zhongnan Qu Syed Shakib Sarwar Xin Dong Yuecheng Li Huseyin Ekin Sumbul B. D. Salvo 3DH 18 1 0 01 Jul 2022
Minimum Variance Unbiased N:M Sparsity for the Neural Gradients Brian Chmiel Itay Hubara Ron Banner Daniel Soudry 21 10 0 21 Mar 2022
Dynamic N:M Fine-grained Structured Sparse Attention Mechanism Zhaodong Chen Yuying Quan Zheng Qu L. Liu Yufei Ding Yuan Xie 36 22 0 28 Feb 2022
Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks Kevin Lee Hunter Lawrence Spracklen Subutai Ahmad 23 20 0 27 Dec 2021
How Well Do Sparse Imagenet Models Transfer? Eugenia Iofinova Alexandra Peste Mark Kurtz Dan Alistarh 27 38 0 26 Nov 2021
Prune Once for All: Sparse Pre-Trained Language Models Ofir Zafrir Ariel Larey Guy Boudoukh Haihao Shen Moshe Wasserblat VLM 34 82 0 10 Nov 2021
NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM Connor Holmes Minjia Zhang Yuxiong He Bo Wu 37 18 0 28 Oct 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 30 39 0 10 Oct 2021
RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging Edouard Yvinec Arnaud Dapogny Matthieu Cord Kévin Bailly 28 15 0 30 Sep 2021
Group Fisher Pruning for Practical Network Compression Liyang Liu Shilong Zhang Zhanghui Kuang Aojun Zhou Jingliang Xue Xinjiang Wang Yimin Chen Wenming Yang Q. Liao Wayne Zhang 25 146 0 02 Aug 2021
1xN Pattern for Pruning Convolutional Neural Networks Mingbao Lin Yu-xin Zhang Yuchao Li Bohong Chen Rongrong Ji Mengdi Wang Shen Li Yonghong Tian Rongrong Ji 3DPC 33 40 0 31 May 2021
Post-Training Sparsity-Aware Quantization Gil Shomron F. Gabbay Samer Kurzum U. Weiser MQ 39 33 0 23 May 2021
Dual-side Sparse Tensor Core Yang-Feng Wang Chen Zhang Zhiqiang Xie Cong Guo Yunxin Liu Jingwen Leng 20 74 0 20 May 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 59 111 0 16 Feb 2021
Structured Pruning for Efficient ConvNets via Incremental Regularization Huan Wang Qiming Zhang Yuehai Wang Haoji Hu 3DPC 40 45 0 20 Nov 2018
Faster Neural Network Training with Approximate Tensor Operations Menachem Adelman Kfir Y. Levy Ido Hakimi M. Silberstein 29 26 0 21 May 2018