QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

50 / 392 papers shown

Title
Alleviating the Fear of Losing Alignment in LLM Fine-tuning Kang Yang Guanhong Tao X. Chen Jun Xu 36 0 0 13 Apr 2025
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models Wenjin Ke Zhe Li D. Li Lu Tian E. Barsoum MQ 37 1 0 12 Apr 2025
AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation Tuhin Chakrabarty Philippe Laban C. Wu 34 1 0 10 Apr 2025
HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidance Jiazi Bu Pengyang Ling Yujie Zhou Pan Zhang Tong Wu Xiaoyi Dong Yuhang Zang Yuhang Cao Dahua Lin Jiaqi Wang 23 0 0 08 Apr 2025
Generative Large Language Model usage in Smart Contract Vulnerability Detection Peter Ince Jiangshan Yu Joseph K. Liu Xiaoning Du 37 0 0 07 Apr 2025
Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking Chris Samarinas Hamed Zamani ALM LRM 74 0 0 04 Apr 2025
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration Yuhang Li Ruokai Yin Donghyun Lee Shiting Xiao Priyadarshini Panda MQ 51 0 0 03 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 59 2 0 02 Apr 2025
Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots Erfan Shayegani G M Shahariar Sara Abdali Lei Yu Nael B. Abu-Ghazaleh Yue Dong AAML 78 0 0 01 Apr 2025
Benchmarking Systematic Relational Reasoning with Large Language and Reasoning Models Irtaza Khalid Amir Masoud Nourollah Steven Schockaert LRM 40 0 0 30 Mar 2025
Towards Symmetric Low-Rank Adapters Tales Panoutsos Rodrygo L. T. Santos Flavio Figueiredo 33 0 0 29 Mar 2025
Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models Hung-Yueh Chiang Chi-chih Chang N. Frumkin Kai-Chiang Wu Mohamed S. Abdelfattah Diana Marculescu MQ 164 0 0 28 Mar 2025
Explainable ICD Coding via Entity Linking Leonor Barreiros I. Coutinho Gonçalo M. Correia Bruno Martins 63 0 0 26 Mar 2025
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization Hao Mark Chen S. Hu Wayne Luk Timothy M. Hospedales Hongxiang Fan MoMe 72 0 0 16 Mar 2025
OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses Angela Lopez-Cardona Sebastian Idesis Miguel Barreda-Ángeles Sergi Abadal Ioannis Arapakis 48 0 0 13 Mar 2025
Validating LLM-as-a-Judge Systems in the Absence of Gold Labels Luke M. Guerdan Solon Barocas Kenneth Holstein Hanna M. Wallach Zhiwei Steven Wu Alexandra Chouldechova ALM ELM 224 0 0 13 Mar 2025
Prompt Inversion Attack against Collaborative Inference of Large Language Models Wenjie Qu Yuguang Zhou Yongji Wu Tingsong Xiao Binhang Yuan Heng Chang Jiaheng Zhang 76 0 0 12 Mar 2025
Leveraging Retrieval Augmented Generative LLMs For Automated Metadata Description Generation to Enhance Data Catalogs Mayank Singh Abhijeet Kumar Sasidhar Donaparthi Gayatri Karambelkar 43 0 0 12 Mar 2025
MoFE: Mixture of Frozen Experts Architecture Jean Seo Jaeyoon Kim Hyopil Shin MoE 185 0 0 09 Mar 2025
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning Zhibin Lan Liqiang Niu Fandong Meng Jie Zhou Jinsong Su VLM 69 2 0 04 Mar 2025
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 39 0 0 28 Feb 2025
FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework S M Sarwar AI4MH 46 0 0 27 Feb 2025
PiCO: Peer Review in LLMs based on the Consistency Optimization Kun-Peng Ning Shuo Yang Yu-Yang Liu Jia-Yu Yao Zhen-Hui Liu Yu Wang Ming Pang Li Yuan ALM 71 8 0 24 Feb 2025
Testing the limits of fine-tuning to improve reasoning in vision language models Luca M. Schulze Buschoff Konstantinos Voudouris Elif Akata Matthias Bethge Joshua B. Tenenbaum Eric Schulz LRM VLM Presented at ResearchTrend Connect \| VLM on 14 Mar 2025 126 0 1 24 Feb 2025
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning Raghav Singhal Kaustubh Ponkshe Rohit Vartak Lav R. Varshney Praneeth Vepakomma FedML 79 0 0 24 Feb 2025
Automatic Input Rewriting Improves Translation with Large Language Models Dayeon Ki Marine Carpuat 44 0 0 23 Feb 2025
Beyond Release: Access Considerations for Generative AI Systems Irene Solaiman Rishi Bommasani Dan Hendrycks Ariel Herbert-Voss Yacine Jernite Aviya Skowron Andrew Trask 62 1 0 23 Feb 2025
Fine-Tuning Qwen 2.5 3B for Realistic Movie Dialogue Generation Kartik Gupta VGen 48 0 0 22 Feb 2025
Dynamic Low-Rank Sparse Adaptation for Large Language Models Weizhong Huang Yuxin Zhang Xiawu Zheng Yong-Jin Liu Jing Lin Yiwu Yao Rongrong Ji 97 1 0 21 Feb 2025
How Much Do LLMs Hallucinate across Languages? On Multilingual Estimation of LLM Hallucination in the Wild Saad Obaid ul Islam Anne Lauscher Goran Glavas HILM LRM 122 1 0 21 Feb 2025
Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model Emre Can Acikgoz Jeremiah Greer Akul Datta Ze Yang William Zeng Oussama Elachqar Emmanouil Koukoumidis Dilek Hakkani-Tur Gokhan Tur LLMAG 108 3 0 20 Feb 2025
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 90 0 0 18 Feb 2025
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models J. Zhao Miao Zhang Hao Wu Yuzhang Shang Kaihao Zhang Weili Guan Yaowei Wang Min Zhang MQ 49 0 0 18 Feb 2025
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 56 0 0 17 Feb 2025
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation Vera Neplenbroek Arianna Bisazza Raquel Fernández 103 0 0 17 Feb 2025
SuperMerge: An Approach For Gradient-Based Model Merging Haoyu Yang Zheng Zhang Saket Sathe MoMe 127 0 0 17 Feb 2025
Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation Hieu Nguyen Zihao He Shoumik Atul Gandre Ujjwal Pasupulety Sharanya Kumari Shivakumar Kristina Lerman HILM 59 1 0 16 Feb 2025
LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits Zikai Zhou Qizheng Zhang Hermann Kumbong Kunle Olukotun MQ 264 0 0 12 Feb 2025
SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation Yixian Shen Qi Bi Jia-Hong Huang Hongyi Zhu Andy D. Pimentel Anuj Pathania 46 0 0 08 Feb 2025
DeepThink: Aligning Language Models with Domain-Specific User Intents Yang Li Mingxuan Luo Yeyun Gong Chen Lin Jian Jiao Yi Liu Kaili Huang LRM ALM ELM 59 0 0 08 Feb 2025
LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning Zhekai Du Yinjie Min Jingjing Li Ke Lu Changliang Zou Liuhua Peng Tingjin Chu Mingming Gong 186 1 0 05 Feb 2025
$deCIFer: Crystal Structure Prediction from Powder Diffraction Data using Autoregressive Language Models$ deCIFer: Crystal Structure Prediction from Powder Diffraction Data using Autoregressive Language Models Frederik L. Johansen Ulrik Friis-Jensen Erik B. Dam Kirsten M. Ø. Jensen Rocío Mercado Raghavendra Selvan 88 0 0 04 Feb 2025
Learning to Generate Unit Tests for Automated Debugging Archiki Prasad Elias Stengel-Eskin Justin Chih-Yao Chen Zaid Khan Joey Tianyi Zhou ELM 88 1 0 03 Feb 2025
Evaluation of Large Language Models via Coupled Token Generation N. C. Benz Stratis Tsirtsis Eleni Straitouri Ivi Chatzi Ander Artola Velasco Suhas Thejaswi Manuel Gomez Rodriguez 51 0 0 03 Feb 2025
Position: AI Scaling: From Up to Down and Out Yunke Wang Yanxi Li Chang Xu HAI 88 2 0 02 Feb 2025
Learning to Summarize from LLM-generated Feedback Hwanjun Song Taewon Yun Yuho Lee Jihwan Oh Gihun Lee Jason (Jinglun) Cai Hang Su 73 3 0 28 Jan 2025
Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs Nicolas Boizard Kevin El Haddad C´eline Hudelot Pierre Colombo 75 15 0 28 Jan 2025
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models Yibin Wang Haizhou Shi Ligong Han Dimitris N. Metaxas Hao Wang BDL UQLM 116 7 0 28 Jan 2025
Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models Jingwei Yi Yueqi Xie Bin Zhu Emre Kiciman Guangzhong Sun Xing Xie Fangzhao Wu AAML 62 65 0 28 Jan 2025
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs Mohammad Mozaffari Amir Yazdanbakhsh Zhao Zhang M. Dehnavi 80 5 0 28 Jan 2025