Accurate LoRA-Finetuning Quantization of LLMs via Information Retention

v1v2 (latest)

Accurate LoRA-Finetuning Quantization of LLMs via Information Retention

8 February 2024

Yang Zhang

Xianglong Liu

Michele Magno

ArXiv (abs)PDF HTML Github (65★)

Papers citing "Accurate LoRA-Finetuning Quantization of LLMs via Information Retention"

9 / 9 papers shown

Title
Event-Priori-Based Vision-Language Model for Efficient Visual Understanding Haotong Qin Cheng Hu Michele Magno VLM 16 0 0 09 Jun 2025
Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retraining Deyu Cao Samin Aref MQ 80 0 0 14 Apr 2025
ORANSight-2.0: Foundational LLMs for O-RAN Pranshav Gajjar Vijay K. Shah AI4TS LRM VLM 131 4 0 07 Mar 2025
IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models Hang Guo Yawei Li Tao Dai Shu-Tao Xia Luca Benini MQ 122 2 0 29 Oct 2024
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization Xinhao Yao Hongjin Qian Xiaolin Hu Gengze Xu Wei Liu Jian Luan Bin Wang Yang Liu 114 1 0 03 Oct 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Mengzhao Chen Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Ping Luo MQ 154 35 0 10 Jul 2024
SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking Xingrun Xing Boyan Gao Zheng Zhang David A. Clifton Shitao Xiao Li Du Guoqi Li Jiajun Zhang 146 6 0 05 Jul 2024
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models Wei Huang Haotong Qin Yangdong Liu Yawei Li Qinshuo Liu Xianglong Liu Luca Benini Michele Magno Shiming Zhang Xiaojuan Qi MQ 133 19 0 23 May 2024
An empirical study of LLaMA3 quantization: from LLMs to MLLMs Wei Huang Xingyu Zheng Xudong Ma Haotong Qin Chengtao Lv Hong Chen Jie Luo Xiaojuan Qi Xianglong Liu Michele Magno MQ 130 42 0 22 Apr 2024

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.