Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

31 January 2021

Dan Alistarh

Papers citing "Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks"

50 / 361 papers shown

Title
Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained Environments Ibne Farabi Shihab Sanjeda Akter Anuj Sharma Mamba 54 0 0 13 May 2025
ICE-Pruning: An Iterative Cost-Efficient Pruning Pipeline for Deep Neural Networks Wenhao Hu Paul Henderson José Cano 32 0 0 12 May 2025
Sparsity is All You Need: Rethinking Biological Pathway-Informed Approaches in Deep Learning Isabella Caranzano Corrado Pancotti Cesare Rollo Flavio Sartori Pietro Liò P. Fariselli Tiziana Sanavia OOD UQCV 65 0 0 07 May 2025
Onboard Optimization and Learning: A Survey Monirul Islam Pavel Siyi Hu Mahardhika Pratama Ryszard Kowalczyk 26 0 0 07 May 2025
Switch-Based Multi-Part Neural Network Surajit Majumder Paritosh Ranjan Prodip Roy Bhuban Padhan OOD 79 0 0 25 Apr 2025
Periodic Online Testing for Sparse Systolic Tensor Arrays C. Peltekis Chrysostomos Nicopoulos G. Dimitrakopoulos 52 0 0 25 Apr 2025
Connecting Parameter Magnitudes and Hessian Eigenspaces at Scale using Sketched Methods Andres Fernandez Frank Schneider Maren Mahsereci Philipp Hennig 38 0 0 20 Apr 2025
The Binary and Ternary Quantization Can Improve Feature Discrimination Weizhi Lu Mingrui Chen Weiyu Li MQ 178 0 0 18 Apr 2025
PQS (Prune, Quantize, and Sort): Low-Bitwidth Accumulation of Dot Products in Neural Network Computations Vikas Natesh H. T. Kung MQ 213 0 0 12 Apr 2025
Efficient FPGA-accelerated Convolutional Neural Networks for Cloud Detection on CubeSats Angela Cratere M. Salim Farissi Andrea Carbone Marcello Asciolla Maria Rizzi Francesco DellÓlio Augusto Nascetti Dario Spiller 33 1 0 04 Apr 2025
Towards Symmetric Low-Rank Adapters Tales Panoutsos Rodrygo L. T. Santos Flavio Figueiredo 33 0 0 29 Mar 2025
Temporal Action Detection Model Compression by Progressive Block Drop Xiaoyong Chen Yong Guo Jiaming Liang Sitong Zhuang Runhao Zeng Xiping Hu 55 0 0 21 Mar 2025
Changing Base Without Losing Pace: A GPU-Efficient Alternative to MatMul in DNNs Nir Ailon Akhiad Bercovich Omri Weinstein 57 0 0 15 Mar 2025
Robust Dataset Distillation by Matching Adversarial Trajectories Wei Lai Tianyu Ding ren dongdong Lei Wang Jing Huo Yang Gao Wenbin Li AAML DD 62 0 0 15 Mar 2025
From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches Wei Ruan Tianze Yang Yue Zhou Tianming Liu Jin Lu MoMe 98 0 0 13 Mar 2025
LapSum -- One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection Łukasz Struski Michał B. Bednarczyk Igor T. Podolak Jacek Tabor BDL 62 0 0 08 Mar 2025
Energy-Latency Attacks: A New Adversarial Threat to Deep Learning H. B. Meftah W. Hamidouche Sid Ahmed Fezza Olivier Déforges AAML 48 0 0 06 Mar 2025
Beyond Worst-Case Dimensionality Reduction for Sparse Vectors Sandeep Silwal David P. Woodruff Qiuyi Zhang 59 0 0 27 Feb 2025
I Know What I Don't Know: Improving Model Cascades Through Confidence Tuning Stephan Rabanser Nathalie Rauschmayr Achin Kulshrestha Petra Poklukar Wittawat Jitkrittum Sean Augenstein Congchao Wang Federico Tombari 42 0 0 26 Feb 2025
Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs Shane Bergsma Nolan Dey Gurpreet Gosal Gavia Gray Daria Soboleva Joel Hestness 58 6 0 21 Feb 2025
E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation Boqian Wu Q. Xiao Shiwei Liu Lu Yin Mykola Pechenizkiy Decebal Constantin Mocanu M. V. Keulen Elena Mocanu MedIm 65 4 0 20 Feb 2025
CondensNet: Enabling stable long-term climate simulations via hybrid deep learning models with adaptive physical constraints Xin Wang Juntao Yang Jeff Adie Simon See Kalli Furtado Chen Chen T. Arcomano R. Maulik G. Mengaldo AI4CE 52 0 0 18 Feb 2025
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 58 0 0 17 Feb 2025
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries Chris Kolb T. Weber Bernd Bischl David Rügamer 115 0 0 04 Feb 2025
Symmetric Pruning of Large Language Models Kai Yi Peter Richtárik AAML VLM 73 0 0 31 Jan 2025
Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models J. P. Muñoz Jinjie Yuan Nilesh Jain Mamba 72 1 0 28 Jan 2025
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs Mohammad Mozaffari Amir Yazdanbakhsh Zhao Zhang M. Dehnavi 82 5 0 28 Jan 2025
Meta-Sparsity: Learning Optimal Sparse Structures in Multi-task Networks through Meta-learning Richa Upadhyay Ronald Phlypo Rajkumar Saini Marcus Liwicki 42 0 0 21 Jan 2025
UPAQ: A Framework for Real-Time and Energy-Efficient 3D Object Detection in Autonomous Vehicles Abhishek Balasubramaniam Febin P. Sunny S. Pasricha 3DPC 44 0 0 08 Jan 2025
Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic Yifei He Yuzheng Hu Yong Lin Tong Zhang Han Zhao FedML MoMe 70 19 0 08 Jan 2025
SlimGPT: Layer-wise Structured Pruning for Large Language Models Gui Ling Ziyang Wang Yuliang Yan Qingwen Liu 38 2 0 24 Dec 2024
A Comparative Study of Pruning Methods in Transformer-based Time Series Forecasting Nicholas Kiefer Arvid Weyrauch Muhammed Öz Achim Streit Markus Gotz Charlotte Debus AI4TS 77 0 0 17 Dec 2024
Is Oracle Pruning the True Oracle? Sicheng Feng Keda Tao Haoyu Wang VLM 70 0 0 28 Nov 2024
Individual Content and Motion Dynamics Preserved Pruning for Video Diffusion Models Yiming Wu Huan Wang Zhenghao Chen Dong Xu DiffM VGen 84 1 0 27 Nov 2024
FuseGPT: Learnable Layers Fusion of Generative Pre-trained Transformers Zehua Pei Hui-Ling Zhen Xianzhi Yu Sinno Jialin Pan M. Yuan Bei Yu AI4CE 89 0 0 21 Nov 2024
Change Is the Only Constant: Dynamic LLM Slicing based on Layer Redundancy Razvan-Gabriel Dumitru Paul-Ioan Clotan Vikas Yadav Darius Peteleaza Mihai Surdeanu 44 4 0 05 Nov 2024
Navigating Extremes: Dynamic Sparsity in Large Output Spaces Nasib Ullah Erik Schultheis Mike Lasby Yani Andrew Ioannou Rohit Babbar 35 0 0 05 Nov 2024
NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference Xuanlin Jiang Yang Zhou Shiyi Cao Ion Stoica Minlan Yu 50 8 0 02 Nov 2024
Inducing Semi-Structured Sparsity by Masking for Efficient Model Inference in Convolutional Networks David A. Danhofer 32 0 0 01 Nov 2024
Efficient Model Compression for Bayesian Neural Networks Diptarka Saha Zihe Liu Feng Liang BDL 31 0 0 01 Nov 2024
Learning and Unlearning of Fabricated Knowledge in Language Models Chen Sun Nolan Miller A. Zhmoginov Max Vladymyrov Mark Sandler KELM MU 35 1 0 29 Oct 2024
Computational Bottlenecks of Training Small-scale Large Language Models Saleh Ashkboos Iman Mirzadeh Keivan Alizadeh Mohammad Hossein Sekhavat Moin Nabi Mehrdad Farajtabar Fartash Faghri 26 0 0 25 Oct 2024
Harnessing Your DRAM and SSD for Sustainable and Accessible LLM Inference with Mixed-Precision and Multi-level Caching Jie Peng Zhang Cao Huaizhi Qu Zhengyu Zhang Chang Guo Yanyong Zhang Zhichao Cao Tianlong Chen 42 2 0 17 Oct 2024
Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learning Xinran Li Ling Pan Jun Zhang 20 1 0 11 Oct 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira K. Wense Lawrence E Hunter Matt Jones MoE 46 0 0 10 Oct 2024
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration Heming Xia Yongqi Li Jun Zhang Cunxiao Du Wenjie Li LRM 56 6 0 09 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 36 0 0 05 Oct 2024
EntryPrune: Neural Network Feature Selection using First Impressions Felix Zimmer Patrik Okanovic Torsten Hoefler 29 0 0 03 Oct 2024
Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement Gaurav Patel Christopher Sandino Behrooz Mahasseni Ellen L. Zippi Erdrin Azemi Ali Moin Juri Minxha TTA AI4TS 55 3 0 03 Oct 2024
Getting Free Bits Back from Rotational Symmetries in LLMs Jiajun He Gergely Flamich José Miguel Hernández-Lobato MQ 23 0 0 02 Oct 2024