Fluctuation-based Adaptive Structured Pruning for Large Language Models

19 December 2023

Ming Tang

Papers citing "Fluctuation-based Adaptive Structured Pruning for Large Language Models"

30 / 30 papers shown

Title
SPAP: Structured Pruning via Alternating Optimization and Penalty Methods Hanyu Hu Xiaoming Yuan 48 0 0 06 May 2025
ReplaceMe: Network Simplification via Layer Pruning and Linear Transformations Dmitriy Shopkhoev Ammar Ali Magauiya Zhussip Valentin Malykh Stamatios Lefkimmiatis N. Komodakis Sergey Zagoruyko VLM 161 0 0 05 May 2025
Efficient LLMs with AMP: Attention Heads and MLP Pruning Leandro Giusti Mugnaini Bruno Yamamoto Lucas Lauton de Alcantara Victor Zacarias Edson Bollis Lucas Pellicer A. H. R. Costa Artur Jordao 47 0 0 29 Apr 2025
CUT: Pruning Pre-Trained Multi-Task Models into Compact Models for Edge Devices Jingxuan Zhou Weidong Bao Ji Wang Zhengyi Zhong 32 0 0 14 Apr 2025
Model Hemorrhage and the Robustness Limits of Large Language Models Ziyang Ma Zehan Li Lefei Zhang Gui-Song Xia Bo Du Liangpei Zhang Dacheng Tao 59 0 0 31 Mar 2025
ZeroLM: Data-Free Transformer Architecture Search for Language Models Zhen-Song Chen Hong-Wei Ding Xian-Jia Wang Witold Pedrycz 55 0 0 24 Mar 2025
Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Process Yuanze Li Shihao Yuan Haolin Wang Qizhang Li Ming-Yu Liu Chen Xu Guangming Shi Wangmeng Zuo 56 0 0 17 Mar 2025
A Sliding Layer Merging Method for Efficient Depth-Wise Pruning in LLMs Xuan Ding Rui Sun Yunjian Zhang Xiu Yan Yueqi Zhou Kaihao Huang Suzhong Fu Angelica I Aviles-Rivero Chuanlong Xie Yao Zhu 131 1 0 26 Feb 2025
Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing Qi Le Enmao Diao Ziyan Wang Xinran Wang Jie Ding Li Yang Ali Anwar 77 2 0 24 Feb 2025
Dynamic Low-Rank Sparse Adaptation for Large Language Models Weizhong Huang Yuxin Zhang Xiawu Zheng Yong-Jin Liu Jing Lin Yiwu Yao Rongrong Ji 97 1 0 21 Feb 2025
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 59 0 0 17 Feb 2025
Progressive Binarization with Semi-Structured Pruning for LLMs Xinyu Yan Tianao Zhang Zhiteng Li Yulun Zhang MQ 54 0 0 03 Feb 2025
Tailored-LLaMA: Optimizing Few-Shot Learning in Pruned LLaMA Models with Task-Specific Prompts Danyal Aftab Steven Davy ALM 49 0 0 10 Jan 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 119 1 0 18 Dec 2024
AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment Y. Fu Zhongzhi Yu Junwei Li Jiayi Qian Yongan Zhang Xiangchi Yuan Dachuan Shi Roman Yakunin Y. Lin 31 2 0 15 Nov 2024
Superficial Safety Alignment Hypothesis Jianwei Li Jung-Eun Kim 24 1 0 07 Oct 2024
MoDeGPT: Modular Decomposition for Large Language Model Compression Chi-Heng Lin Shangqian Gao James Seale Smith Abhishek Patel Shikhar Tuli Yilin Shen Hongxia Jin Yen-Chang Hsu 71 7 0 19 Aug 2024
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs Peijie Dong Lujun Li Dayou Du Yuhan Chen Zhenheng Tang ... Wei Xue Wenhan Luo Qi-fei Liu Yi-Ting Guo Xiaowen Chu MQ 50 4 0 03 Aug 2024
Greedy Output Approximation: Towards Efficient Structured Pruning for LLMs Without Retraining Jianwei Li Yijun Dong Qi Lei 32 5 0 26 Jul 2024
Reconstruct the Pruned Model without Any Retraining Pingjie Wang Ziqing Fan Shengchao Hu Zhe Chen Yanfeng Wang Yu Wang 50 1 0 18 Jul 2024
Large Language Model Pruning Hanjuan Huang Hao-Jia Song H. Pao 46 0 0 24 May 2024
HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models R. Sukthanker Arber Zela B. Staffler Aaron Klein Lennart Purucker Jorg K. H. Franke Frank Hutter ELM 43 3 0 16 May 2024
OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning Dan Qiao Yi Su Pinzheng Wang Jing Ye Wen Xie ... Wenliang Chen Guohong Fu Guodong Zhou Qiaoming Zhu Min Zhang MQ 35 0 0 09 May 2024
SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression Xin Wang Yu Zheng Zhongwei Wan Mi Zhang MQ 57 44 0 12 Mar 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 48 0 15 Feb 2024
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes Lucio Dery Steven Kolawole Jean-Francois Kagey Virginia Smith Graham Neubig Ameet Talwalkar 44 28 0 08 Feb 2024
Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methods Bo-Kyeong Kim Geonmin Kim Tae-Ho Kim Thibault Castells Shinkook Choi Junho Shin Hyoung-Kyu Song 62 30 0 05 Feb 2024
Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward Arnav Chavan Raghav Magazine Shubham Kushwaha M. Debbah Deepak Gupta 16 18 0 02 Feb 2024
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 36 193 0 15 Aug 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 319 2,232 0 22 Mar 2023