The State of Sparsity in Deep Neural Networks

25 February 2019

Papers citing "The State of Sparsity in Deep Neural Networks"

50 / 153 papers shown

Title
Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry Mohammed Adnan Rohan Jain Ekansh Sharma Rahul Krishnan Yani Andrew Ioannou 56 0 0 08 May 2025
Efficient Shapley Value-based Non-Uniform Pruning of Large Language Models Chuan Sun Han Yu Lizhen Cui Xiaoxiao Li 143 0 0 03 May 2025
TeleSparse: Practical Privacy-Preserving Verification of Deep Neural Networks Mohammad Maheri Hamed Haddadi Alex Davidson 74 0 0 27 Apr 2025
Dynamic Gradient Sparse Update for Edge Training I-Hsuan Li Tian-Sheuan Chang 68 1 0 23 Mar 2025
Poly-MgNet: Polynomial Building Blocks in Multigrid-Inspired ResNets Antonia van Betteray Matthias Rottmann Karsten Kahl 51 0 0 13 Mar 2025
Advancing Weight and Channel Sparsification with Enhanced Saliency Xinglong Sun Maying Shen Hongxu Yin Lei Mao Pavlo Molchanov Jose M. Alvarez 54 1 0 05 Feb 2025
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries Chris Kolb T. Weber Bernd Bischl David Rügamer 113 0 0 04 Feb 2025
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs Mohammad Mozaffari Amir Yazdanbakhsh Zhao Zhang M. Dehnavi 82 5 0 28 Jan 2025
Layer-Adaptive State Pruning for Deep State Space Models Minseon Gwak Seongrok Moon Joohwan Ko PooGyeon Park 30 0 0 05 Nov 2024
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models Philipp Mondorf Sondre Wold Barbara Plank 39 0 0 02 Oct 2024
HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning Tianyi Chen Xiaoyi Qu David Aponte Colby R. Banbury Jongwoo Ko Tianyu Ding Yong Ma Vladimir Lyapunov Ilya Zharkov Luming Liang 83 1 0 11 Sep 2024
AdapMTL: Adaptive Pruning Framework for Multitask Learning Model Mingcan Xiang Steven Jiaxun Tang Qizheng Yang Hui Guan Tongping Liu VLM 39 0 0 07 Aug 2024
Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments Angie Boggust Venkatesh Sivaraman Yannick Assogba Donghao Ren Dominik Moritz Fred Hohman VLM 58 3 0 06 Aug 2024
LPViT: Low-Power Semi-structured Pruning for Vision Transformers Kaixin Xu Zhe Wang Chunyun Chen Xue Geng Jie Lin Xulei Yang Min-man Wu Min Wu Xiaoli Li Weisi Lin ViT VLM 51 7 0 02 Jul 2024
Geometric sparsification in recurrent neural networks Wyatt Mackey Ioannis Schizas Jared Deighton David L. Boothe, Jr. Vasileios Maroulas 33 0 0 10 Jun 2024
Group Projected Subspace Pursuit for Block Sparse Signal Reconstruction: Convergence Analysis and Applications Roy Y. He Haixia Liu Hao Liu 25 2 0 01 Jun 2024
Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning E. Chimoto Jay Gala Orevaoghene Ahia Julia Kreutzer Bruce A. Bassett Sara Hooker VLM 42 4 0 29 May 2024
Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes Ruihao Gong Yang Yong Zining Wang Jinyang Guo Xiuying Wei Yuqing Ma Xianglong Liu 54 5 0 09 May 2024
The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth Victor Quétu Zhu Liao Enzo Tartaglione 49 4 0 27 Apr 2024
Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initialization Bailey J. Eccles Leon Wong Blesson Varghese 38 2 0 22 Apr 2024
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind Hongchuan Zeng Hongshen Xu Lu Chen Kai Yu 59 5 0 06 Apr 2024
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 52 123 0 26 Jan 2024
Stochastic Subnetwork Annealing: A Regularization Technique for Fine Tuning Pruned Subnetworks Tim Whitaker Darrell Whitley 33 0 0 16 Jan 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 91 0 0 12 Jan 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 52 10 0 23 Dec 2023
Performance Optimization of Deep Learning Sparse Matrix Kernels on Intel Max Series GPU Mohammad Zubair Christoph Bauinger 14 0 0 01 Nov 2023
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yuxin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 45 40 0 13 Oct 2023
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity Lu Yin You Wu Zhenyu Zhang Cheng-Yu Hsieh Yaqing Wang ... Mykola Pechenizkiy Yi Liang Michael Bendersky Zhangyang Wang Shiwei Liu 30 79 0 08 Oct 2023
Uncovering the Hidden Cost of Model Compression Diganta Misra Muawiz Chaudhary Agam Goyal Bharat Runwal Pin-Yu Chen VLM 38 0 0 29 Aug 2023
Quantifying lottery tickets under label noise: accuracy, calibration, and complexity V. Arora Daniele Irto Sebastian Goldt G. Sanguinetti 38 2 0 21 Jun 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 67 359 0 20 Jun 2023
Vision Transformers for Mobile Applications: A Short Survey Nahid Alam Steven Kolawole S. Sethi Nishant Bansali Karina Nguyen ViT 34 3 0 30 May 2023
Sparsifying Spiking Networks through Local Rhythms W. Olin-Ammentorp 24 0 0 30 Apr 2023
Sparsified Model Zoo Twins: Investigating Populations of Sparsified Neural Network Models D. Honegger Konstantin Schurholt Damian Borth 31 4 0 26 Apr 2023
STen: Productive and Efficient Sparsity in PyTorch Andrei Ivanov Nikoli Dryden Tal Ben-Nun Saleh Ashkboos Torsten Hoefler 34 4 0 15 Apr 2023
Distributed Neural Representation for Reactive in situ Visualization Qi Wu J. Insley V. Mateevitsi S. Rizzi M. Papka Kwan-Liu Ma 29 1 0 28 Mar 2023
Take 5: Interpretable Image Classification with a Handful of Features Thomas Norrenbrock Marco Rudolph Bodo Rosenhahn FAtt 42 7 0 23 Mar 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 36 3 0 21 Mar 2023
Gradient-Free Structured Pruning with Unlabeled Data Azade Nova H. Dai Dale Schuurmans SyDa 40 20 0 07 Mar 2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! Shiwei Liu Tianlong Chen Zhenyu Zhang Xuxi Chen Tianjin Huang Ajay Jaiswal Zhangyang Wang 32 29 0 03 Mar 2023
DSD $^2$ : Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free? Victor Quétu Enzo Tartaglione 32 7 0 02 Mar 2023
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization Riade Benbaki Wenyu Chen X. Meng Hussein Hazimeh Natalia Ponomareva Zhe Zhao Rahul Mazumder 21 26 0 28 Feb 2023
Full Stack Optimization of Transformer Inference: a Survey Sehoon Kim Coleman Hooper Thanakul Wattanawong Minwoo Kang Ruohan Yan ... Qijing Huang Kurt Keutzer Michael W. Mahoney Y. Shao A. Gholami MQ 36 101 0 27 Feb 2023
Can we avoid Double Descent in Deep Neural Networks? Victor Quétu Enzo Tartaglione AI4CE 20 3 0 26 Feb 2023
A Unified Framework for Soft Threshold Pruning Yanqing Chen Zhengyu Ma Wei Fang Xiawu Zheng Zhaofei Yu Yonghong Tian 85 19 0 25 Feb 2023
Considering Layerwise Importance in the Lottery Ticket Hypothesis Benjamin Vandersmissen José Oramas 37 1 0 22 Feb 2023
Less is More: The Influence of Pruning on the Explainability of CNNs David Weber F. Merkle Pascal Schöttle Stephan Schlögl Martin Nocker FAtt 34 1 0 17 Feb 2023
Bi-directional Masks for Efficient N:M Sparse Training Yuxin Zhang Yiting Luo Mingbao Lin Mingliang Xu Jingjing Xie Rongrong Ji Rongrong Ji 52 15 0 13 Feb 2023
Pruning Deep Neural Networks from a Sparsity Perspective Enmao Diao G. Wang Jiawei Zhan Yuhong Yang Jie Ding Vahid Tarokh 27 30 0 11 Feb 2023
What Matters In The Structured Pruning of Generative Language Models? Michael Santacroce Zixin Wen Yelong Shen Yuan-Fang Li 21 33 0 07 Feb 2023