LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery

24 October 2023

Tianyi Chen

Papers citing "LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery"

24 / 24 papers shown

Title
Sparsity Outperforms Low-Rank Projections in Few-Shot Adaptation Nairouz Mrabah Nicolas Richet Ismail Ben Ayed Eric Granger BDL VLM 58 0 0 16 Apr 2025
LLM Inference Acceleration via Efficient Operation Fusion Mahsa Salmani I. Soloveychik 69 0 0 24 Feb 2025
Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression Xiaoyi Qu David Aponte Colby R. Banbury Daniel P. Robinson Tianyu Ding K. Koishida Ilya Zharkov Tianyi Chen MQ 70 1 0 23 Feb 2025
Dynamic Low-Rank Sparse Adaptation for Large Language Models Weizhong Huang Yuxin Zhang Xiawu Zheng Yong-Jin Liu Jing Lin Yiwu Yao Rongrong Ji 97 1 0 21 Feb 2025
EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models Xingrun Xing Zheng Liu Shitao Xiao Boyan Gao Yiming Liang Wanpeng Zhang Haokun Lin Guoqi Li Jiajun Zhang LRM 64 1 0 10 Feb 2025
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 119 1 0 18 Dec 2024
Mixture of Hidden-Dimensions Transformer Yilong Chen Junyuan Shang Zhengyu Zhang Shuaiyi Nie Tingwen Liu Shuohuan Wang Yu Sun Hua Wu Haifeng Wang MoE 78 0 0 07 Dec 2024
HESSO: Towards Automatic Efficient and User Friendly Any Neural Network Training and Pruning Tianyi Chen Xiaoyi Qu David Aponte Colby R. Banbury Jongwoo Ko Tianyu Ding Yong Ma Vladimir Lyapunov Ilya Zharkov Luming Liang 83 1 0 11 Sep 2024
$Mixed Sparsity Training: Achieving 4$\times$ FLOP Reduction for Transformer Pretraining$ Mixed Sparsity Training: Achieving 4 $\times$ FLOP Reduction for Transformer Pretraining Pihe Hu Shaolong Li Longbo Huang 33 0 0 21 Aug 2024
Flexora: Flexible Low Rank Adaptation for Large Language Models Chenxing Wei Yao Shu Ying Tiffany He Fei Richard Yu AI4CE 34 3 0 20 Aug 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 54 25 0 08 Jul 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 83 0 22 Apr 2024
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration Pengfei Wu Jiahao Liu Zhuocheng Gong Qifan Wang Jinpeng Li Jingang Wang Xunliang Cai Dongyan Zhao 25 1 0 18 Apr 2024
LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models Guangyan Li Yongqiang Tang Wensheng Zhang 49 5 0 15 Apr 2024
Why Lift so Heavy? Slimming Large Language Models by Cutting Off the Layers Shuzhou Yuan Ercong Nie Bolei Ma Michael Farber 42 3 0 18 Feb 2024
NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models Shengrui Li Junzhe Chen Xueting Han Jing Bai 24 6 0 15 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 48 0 15 Feb 2024
Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei Kaixuan Huang Yangsibo Huang Tinghao Xie Xiangyu Qi Mengzhou Xia Prateek Mittal Mengdi Wang Peter Henderson AAML 60 79 0 07 Feb 2024
Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward Arnav Chavan Raghav Magazine Shubham Kushwaha M. Debbah Deepak Gupta 16 18 0 02 Feb 2024
OTOv3: Automatic Architecture-Agnostic Neural Network Training and Compression from Structured Pruning to Erasing Operators Tianyi Chen Tianyu Ding Zhihui Zhu Zeyu Chen HsiangTao Wu Ilya Zharkov Luming Liang 21 3 0 15 Dec 2023
The Efficiency Spectrum of Large Language Models: An Algorithmic Survey Tianyu Ding Tianyi Chen Haidong Zhu Jiachen Jiang Yiqi Zhong Jinxin Zhou Guangzhi Wang Zhihui Zhu Ilya Zharkov Luming Liang 29 22 0 01 Dec 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 322 3,021 0 22 Mar 2023
OTOV2: Automatic, Generic, User-Friendly Tianyi Chen Luming Liang Tian Ding Zhihui Zhu Ilya Zharkov VLM MQ 45 31 0 13 Mar 2023
Sparsity-guided Network Design for Frame Interpolation Tian Ding Luming Liang Zhihui Zhu Tianyi Chen Ilya Zharkov 36 6 0 09 Sep 2022