LaCo: Large Language Model Pruning via Layer Collapse

17 February 2024

Papers citing "LaCo: Large Language Model Pruning via Layer Collapse"

22 / 22 papers shown

Title
Adaptive Layer-skipping in Pre-trained LLMs Xuan Luo Weizhi Wang Xifeng Yan 410 1 0 31 Mar 2025
Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Models Benyamin Jamialahmadi Parsa Kavehzadeh Mehdi Rezagholizadeh Parsa Farinneya Hossein Rajabzadeh A. Jafari Boxing Chen Marzieh S. Tahaei 80 0 0 06 Mar 2025
LESA: Learnable LLM Layer Scaling-Up Yifei Yang Zouying Cao Xinbei Ma Yao Yao L. Qin Zhongfu Chen Hai Zhao 135 0 0 20 Feb 2025
How Redundant Is the Transformer Stack in Speech Representation Models? Teresa Dorszewski Albert Kjøller Jacobsen Lenka Tětková Lars Kai Hansen 152 0 0 20 Jan 2025
Merging Feed-Forward Sublayers for Compressed Transformers Neha Verma Kenton W. Murray Kevin Duh AI4CE 108 0 0 10 Jan 2025
LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment Binrui Zeng Shezheng Song Xiaodong Liu Jie Yu Huijun Liu Jun Ma Xiaopeng Li Shasha Li Xinran Hong Yongtao Tang MQ 87 1 0 24 Dec 2024
GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference Chao Zeng Songwei Liu Shu Yang Fangmin Chen Xing Mei Lean Fu MQ 71 0 0 23 Dec 2024
FuseGPT: Learnable Layers Fusion of Generative Pre-trained Transformers Zehua Pei Hui-Ling Zhen Xianzhi Yu Sinno Jialin Pan Mingxuan Yuan Bei Yu AI4CE 223 3 0 21 Nov 2024
MoDeGPT: Modular Decomposition for Large Language Model Compression Chi-Heng Lin Shangqian Gao James Seale Smith Abhishek Patel Shikhar Tuli Yilin Shen Hongxia Jin Yen-Chang Hsu 127 13 0 19 Aug 2024
BlockPruner: Fine-grained Pruning for Large Language Models Longguang Zhong Fanqi Wan Ruijun Chen Xiaojun Quan Liangzhi Li 51 9 0 15 Jun 2024
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch Le Yu Yu Bowen Haiyang Yu Fei Huang Yongbin Li MoMe 104 320 0 06 Nov 2023
Baichuan 2: Open Large-scale Language Models Ai Ming Yang Bin Xiao Bingning Wang Borong Zhang Ce Bian ... Youxin Jiang Yuchen Gao Yupeng Zhang Guosheng Dong Zhiying Wu ELM LRM 165 745 0 19 Sep 2023
CMMLU: Measuring massive multitask language understanding in Chinese Haonan Li Yixuan Zhang Fajri Koto Yifei Yang Hai Zhao Yeyun Gong Nan Duan Tim Baldwin ALM ELM 92 262 0 15 Jun 2023
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Tim Dettmers M. Lewis Younes Belkada Luke Zettlemoyer MQ 93 653 0 15 Aug 2022
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers Z. Yao Reza Yazdani Aminabadi Minjia Zhang Xiaoxia Wu Conglong Li Yuxiong He VLM MQ 117 477 0 04 Jun 2022
CLUE: A Chinese Language Understanding Evaluation Benchmark Liang Xu Hai Hu Xuanwei Zhang Lu Li Chenjie Cao ... Cong Yue Xinrui Zhang Zhen-Yi Yang Kyle Richardson Zhenzhong Lan ELM 85 384 0 13 Apr 2020
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 117 593 0 25 Sep 2019
ChID: A Large-scale Chinese IDiom Dataset for Cloze Test Chujie Zheng Minlie Huang Aixin Sun 66 87 0 04 Jun 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 224 1,527 0 24 May 2019
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge Alon Talmor Jonathan Herzig Nicholas Lourie Jonathan Berant RALM 140 1,733 0 02 Nov 2018
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 126 1,676 0 27 Aug 2018
RACE: Large-scale ReAding Comprehension Dataset From Examinations Guokun Lai Qizhe Xie Hanxiao Liu Yiming Yang Eduard H. Hovy ELM 183 1,348 0 15 Apr 2017