v1v2v3 (latest)

SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

2 January 2023

Elias Frantar

Dan Alistarh

VLM

ArXiv (abs)PDF HTML HuggingFace (3 upvotes)Github (799★)

Papers citing "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot"

50 / 287 papers shown

Title
SD $^2$ : Self-Distilled Sparse Drafters Mike Lasby Nish Sinnadurai Valavan Manohararajah Sean Lie Yani Andrew Ioannou Vithursan Thangarasa 565 1 0 10 Apr 2025
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models Ruikang Liu Yuxuan Sun Manyi Zhang Haoli Bai Xianzhi Yu Tiezheng Yu C. Yuan Lu Hou MQ LRM 200 18 0 07 Apr 2025
AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design Yanbiao Liang Huihong Shi Haikuo Shao Zhongfeng Wang 119 0 0 07 Apr 2025
Compression Laws for Large Language Models Ayan Sengupta Siddhant Chaudhary Tanmoy Chakraborty 97 0 0 06 Apr 2025
Entropy-Based Block Pruning for Efficient Large Language Models Liangwei Yang Yuhui Xu Juntao Tan Doyen Sahoo Siyang Song Caiming Xiong Han Wang Shelby Heinecke AAML 110 0 0 04 Apr 2025
When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models Nan Zhang Eugene Kwek Yusen Zhang Ngoc-Hieu Nguyen Prasenjit Mitra Rui Zhang MQ LRM 263 0 0 02 Apr 2025
SQuat: Subspace-orthogonal KV Cache Quantization Hao Wang Ligong Han Kai Xu Akash Srivastava MQ 175 1 0 31 Mar 2025
STADE: Standard Deviation as a Pruning Metric Diego Coello de Portugal Mecke Haya Alyoussef Ilia Koloiarov Ilia Koloiarov Lars Schmidt-Thieme Lars Schmidt-Thieme 150 0 0 28 Mar 2025
Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models Runpeng Dai Run Yang Fan Zhou Hongtu Zhu 111 3 0 28 Mar 2025
Mobile-VideoGPT: Fast and Accurate Video Understanding Language Model Abdelrahman M. Shaker Muhammad Maaz Chenhui Gou Hamid Rezatofighi Salman Khan Fahad Shahbaz Khan 614 2 0 27 Mar 2025
As easy as PIE: understanding when pruning causes language models to disagree Pietro Tropeano Maria Maistro Tuukka Ruotsalo Christina Lioma 157 0 0 27 Mar 2025
Maximum Redundancy Pruning: A Principle-Driven Layerwise Sparsity Allocation for LLMs Chang Gao Kang Zhao Runqi Wang Jianfei Chen Liping Jing 127 0 0 24 Mar 2025
EfficientLLaVA:Generalizable Auto-Pruning for Large Vision-language Models Yinan Liang Xiping Hu Xiuwei Xu Jie Zhou Jiwen Lu VLM LRM 120 3 0 19 Mar 2025
Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Process Yuanze Li Shihao Yuan Haolin Wang Qizhang Li Ming-Yu Liu Chen Xu Guangming Shi Wangmeng Zuo 138 1 0 17 Mar 2025
PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing Cheng Deng Luoyang Sun Jiwen Jiang Yongcheng Zeng Xinjian Wu ... Haoyang Li Lei Chen Lionel M. Ni Jun Wang Jun Wang 612 2 0 15 Mar 2025
Towards Extreme Pruning of LLMs with Plug-and-Play Mixed Sparsity Chi Xu Gefei Zhang Yantong Zhu Luca Benini Guosheng Hu Yawei Li Zhihong Zhang 104 1 0 14 Mar 2025
Samoyeds: Accelerating MoE Models with Structured Sparsity Leveraging Sparse Tensor Cores Chenpeng Wu Qiqi Gu Heng Shi Jianguo Yao Haibing Guan MoE 106 4 0 13 Mar 2025
Wanda++: Pruning Large Language Models via Regional Gradients Yifan Yang Kai Zhen Bhavana Ganesh Aram Galstyan Goeric Huybrechts ... S. Bodapati Nathan Susanj Zheng Zhang Jack FitzGerald Abhishek Kumar 343 6 0 06 Mar 2025
Sparse Brains are Also Adaptive Brains: Cognitive-Load-Aware Dynamic Activation for LLMs Yiheng Yang Yujie Wang Chi Ma Lei Yu Emmanuele Chersoni Chu-Ren Huang 184 1 0 26 Feb 2025
Compressing Language Models for Specialized Domains Miles Williams G. Chrysostomou Vitor Jeronymo Nikolaos Aletras MQ 156 1 0 25 Feb 2025
Systematic Weight Evaluation for Pruning Large Language Models: Enhancing Performance and Sustainability Ashhadul Islam S. Belhaouari Amine Bermak 124 0 0 24 Feb 2025
Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing Qi Le Enmao Diao Ziyan Wang Xinran Wang Jie Ding Li Yang Ali Anwar 178 7 0 24 Feb 2025
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models Weilan Wang Yu Mao Dongdong Tang Hongchao Du Nan Guan Chun Jason Xue MQ 162 2 0 24 Feb 2025
LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint Qianli Ma Dongrui Liu Qian Chen Linfeng Zhang Jing Shao MoMe 668 3 0 24 Feb 2025
Delta Decompression for MoE-based LLMs Compression Hao Gu Wei Li Lujun Li Qiyuan Zhu Mark Lee Shengjie Sun Wei Xue Yike Guo MoE 182 10 0 24 Feb 2025
PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation Tao Fan Guoqiang Ma Yuanfeng Song Lixin Fan Kai Chen Qiang Yang 122 2 0 21 Feb 2025
Dynamic Low-Rank Sparse Adaptation for Large Language Models Weizhong Huang Yuxin Zhang Xiawu Zheng Teli Ma Aiyue Chen Yiwu Yao Rongrong Ji 308 3 0 21 Feb 2025
MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures Jiayu Qin Jianchao Tan Jianchao Tan Xunliang Cai Wei Wang 119 0 0 19 Feb 2025
EvoP: Robust LLM Inference via Evolutionary Pruning Shangyu Wu Hongchao Du Ying Xiong Shuai Chen Tei-Wei Kuo Nan Guan Chun Jason Xue 291 2 0 19 Feb 2025
DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs Minxuan Lv Zhenpeng Su Leiyu Pan Yizhe Xiong Zijia Lin ... Guiguang Ding Cheng Luo Di Zhang Kun Gai Songlin Hu MoE 178 1 0 18 Feb 2025
PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery Bowei He Lihao Yin Hui-Ling Zhen Xiaokun Zhang Mingxuan Yuan Chen Ma 235 0 0 18 Feb 2025
Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis Jiaqi Zhao Ming Wang Miao Zhang Yuzhang Shang Xuebo Liu Yaowei Wang Min Zhang Liqiang Nie MQ 349 3 0 18 Feb 2025
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models Jiaqi Zhao Miao Zhang Ming Wang Yuzhang Shang Kaihao Zhang Weili Guan Yaowei Wang Min Zhang MQ 153 1 0 18 Feb 2025
MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models Zhen Zhang Yue Yang Kai Zhen Nathan Susanj Athanasios Mouchtaris Siegfried Kunzmann Zheng Zhang 246 2 0 17 Feb 2025
An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning Cen-Jhih Li Aditya Bhaskara 183 0 0 17 Feb 2025
EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models Xingrun Xing Zheng Liu Shitao Xiao Boyan Gao Yiming Liang Wanpeng Zhang Haokun Lin Guoqi Li Jiajun Zhang LRM 393 6 0 10 Feb 2025
Identify Critical KV Cache in LLM Inference from an Output Perturbation Perspective Yuan Feng Junlin Lv Yuhang Cao Xike Xie S.Kevin Zhou 196 5 0 06 Feb 2025
M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference Nikhil Bhendawade Mahyar Najibi Devang Naik Irina Belousova MoE 232 0 0 04 Feb 2025
Progressive Binarization with Semi-Structured Pruning for LLMs Xinyu Yan Tianao Zhang Zhiteng Li Yulun Zhang Yulun Zhang MQ 240 4 0 03 Feb 2025
Symmetric Pruning of Large Language Models Kai Yi Peter Richtárik AAML VLM 180 2 0 31 Jan 2025
Brain network science modelling of sparse neural networks enables Transformers and LLMs to perform as fully connected Yingtao Zhang Diego Cerretti Jialin Zhao Wenjing Wu Ziheng Liao Umberto Michieli C. Cannistraci 273 1 0 31 Jan 2025
Merino: Entropy-driven Design for Generative Language Models on IoT Devices Youpeng Zhao Ming Lin Huadong Tang Qiang Wu Jun Wang 261 1 0 28 Jan 2025
GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments Yanyu Chen Ganhong Huang 196 0 0 28 Jan 2025
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs Mohammad Mozaffari Amir Yazdanbakhsh Zhao Zhang M. Dehnavi 213 11 0 28 Jan 2025
You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning Ayan Sengupta Siddhant Chaudhary Tanmoy Chakraborty 220 4 0 25 Jan 2025
Optimization Strategies for Enhancing Resource Efficiency in Transformers & Large Language Models Tom Wallace Naser Ezzati-Jivan Beatrice Ombuki-Berman MQ 133 1 0 16 Jan 2025
Tailored-LLaMA: Optimizing Few-Shot Learning in Pruned LLaMA Models with Task-Specific Prompts Danyal Aftab Steven Davy ALM 158 2 0 10 Jan 2025
iServe: An Intent-based Serving System for LLMs Dimitrios Liakopoulos Tianrui Hu Prasoon Sinha N. Yadwadkar VLM 687 0 0 08 Jan 2025
MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks Yifei Liu Zhihang Zhong Yifan Zhan Sheng Xu Xiao Sun 3DGS 270 9 0 29 Dec 2024
DecDEC: A Systems Approach to Advancing Low-Bit LLM Quantization Y. Park Jake Hyun Hojoon Kim Jae W. Lee MQ 218 1 0 28 Dec 2024