QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

50 / 407 papers shown

Title
Watermarking Makes Language Models Radioactive Tom Sander Pierre Fernandez Alain Durmus Matthijs Douze Teddy Furon WaLM 41 11 0 22 Feb 2024
GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence Kundan Krishna S. Ramprasad Prakhar Gupta Byron C. Wallace Zachary Chase Lipton Jeffrey P. Bigham HILM KELM SyDa 52 9 0 19 Feb 2024
LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 43 148 0 19 Feb 2024
WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More Yuxuan Yue Zhihang Yuan Haojie Duanmu Sifan Zhou Jianlong Wu Liqiang Nie MQ 34 42 0 19 Feb 2024
Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs Siyuan Wang Zhongyu Wei Yejin Choi Xiang Ren ReLM ELM LRM 16 21 0 18 Feb 2024
InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration Fali Wang Runxue Bao Suhang Wang Wenchao Yu Yanchi Liu Wei Cheng Haifeng Chen KELM 21 8 0 18 Feb 2024
Self-seeding and Multi-intent Self-instructing LLMs for Generating Intent-aware Information-Seeking dialogs Arian Askari Roxana Petcu Chuan Meng Mohammad Aliannejadi Amin Abolghasemi Evangelos Kanoulas Suzan Verberne 21 9 0 18 Feb 2024
CoLLaVO: Crayon Large Language and Vision mOdel Byung-Kwan Lee Beomchan Park Chae Won Kim Yonghyun Ro VLM MLLM 35 16 0 17 Feb 2024
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning Ming Li Lichang Chen Jiuhai Chen Shwai He Jiuxiang Gu Dinesh Manocha 29 52 0 15 Feb 2024
Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space Leo Schwinn David Dobre Sophie Xhonneux Gauthier Gidel Stephan Gunnemann AAML 51 38 0 14 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomáš Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 134 371 0 09 Feb 2024
Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector Haihui Yang Xiaojun Quan 3DV 29 2 0 07 Feb 2024
Flora: Low-Rank Adapters Are Secretly Gradient Compressors Yongchang Hao Yanshuai Cao Lili Mou 16 40 0 05 Feb 2024
Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward Arnav Chavan Raghav Magazine Shubham Kushwaha M. Debbah Deepak Gupta 16 18 0 02 Feb 2024
Institutional Platform for Secure Self-Service Large Language Model Exploration V. Bumgardner Mitchell A. Klusty W. V. Logan Samuel E. Armstrong Caylin D. Hickey Jeff Talbert Caylin Hickey Jeff Talbert 58 1 0 01 Feb 2024
Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling David Dukić Jan Šnajder 24 13 0 25 Jan 2024
Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators Yaniv Blumenfeld Itay Hubara Daniel Soudry 47 3 0 25 Jan 2024
In-context Learning with Retrieved Demonstrations for Language Models: A Survey an Luo Xin Xu Yue Liu Panupong Pasupat Mehran Kazemi RALM 34 55 0 21 Jan 2024
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 52 252 0 19 Jan 2024
Zero-shot Generative Large Language Models for Systematic Review Screening Automation Shuai Wang Harrisen Scells Shengyao Zhuang Martin Potthast Bevan Koopman Guido Zuccon 33 12 0 12 Jan 2024
Can AI Write Classical Chinese Poetry like Humans? An Empirical Study Inspired by Turing Test Zekun Deng Haoxia Yang Jun Wang 32 2 0 10 Jan 2024
Evaluating Language Model Agency through Negotiations Tim R. Davidson V. Veselovsky Martin Josifoski Maxime Peyrard Antoine Bosselut Michal Kosinski Robert West LLMAG 34 22 0 09 Jan 2024
MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation Zhongshen Zeng Pengguang Chen Shu Liu Haiyun Jiang Jiaya Jia ReLM ELM LRM 41 18 0 28 Dec 2023
Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation Jiayu Lin Rong Ye Meng Han Qi Zhang Ruofei Lai Xinyu Zhang Bo Zhao Xuanjing Huang Zhongyu Wei LRM 28 7 0 21 Dec 2023
Language Resources for Dutch Large Language Modelling Bram Vanroy MoE ALM 31 7 0 20 Dec 2023
Sparse is Enough in Fine-tuning Pre-trained Large Language Models Weixi Song Z. Li Lefei Zhang Hai Zhao Bo Du VLM 26 7 0 19 Dec 2023
Mixed Distillation Helps Smaller Language Model Better Reasoning Chenglin Li Qianglong Chen Liangyue Li Wang Caiyu Yicheng Li Zhang Yin Yin Zhang LRM 41 12 0 17 Dec 2023
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 36 13 0 13 Dec 2023
Dynamic Corrective Self-Distillation for Better Fine-Tuning of Pretrained Models Ibtihel Amara Vinija Jain Aman Chadha 32 0 0 12 Dec 2023
DYAD: A Descriptive Yet Abjuring Density efficient approximation to linear neural network layers S. Chandy Varun Gangal Yi Yang Gabriel Maggiotti 35 0 0 11 Dec 2023
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 40 32 0 11 Dec 2023
Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey Haotian Zhang S. D. Semujju Zhicheng Wang Xianwei Lv Kang Xu ... Jing Wu Zhuo Long Wensheng Liang Xiaoguang Ma Ruiyan Zhuang UQCV AI4TS AI4CE 29 4 0 11 Dec 2023
NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation Peter West Ronan Le Bras Taylor Sorensen Bill Yuchen Lin Liwei Jiang ... Khyathi Raghavi Chandu Jack Hessel Ashutosh Baheti Chandra Bhagavatula Yejin Choi VLM 26 10 0 10 Dec 2023
Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak Yanrui Du Sendong Zhao Ming Ma Yuhan Chen Bing Qin 26 15 0 07 Dec 2023
Run LoRA Run: Faster and Lighter LoRA Implementations Daria Cherniuk A. Mikhalev Ivan Oseledets AI4CE 22 1 0 06 Dec 2023
The Philosopher's Stone: Trojaning Plugins of Large Language Models Tian Dong Minhui Xue Guoxing Chen Rayne Holland Shaofeng Li Yan Meng Zhen Liu Haojin Zhu AAML 25 11 0 01 Dec 2023
Walking a Tightrope -- Evaluating Large Language Models in High-Risk Domains Chia-Chien Hung Wiem Ben-Rim Lindsay Frost Lars Bruckner Carolin (Haas) Lawrence AILaw ALM ELM 25 9 0 25 Nov 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 58 11 0 23 Nov 2023
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 36 43 0 20 Nov 2023
Inducing Political Bias Allows Language Models Anticipate Partisan Reactions to Controversies Zihao He Siyi Guo Ashwin Rao Kristina Lerman 21 2 0 16 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 32 42 0 16 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 25 27 0 15 Nov 2023
Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey Garima Agrawal Tharindu Kumarage Zeyad Alghami Huanmin Liu 37 81 0 14 Nov 2023
Interaction is all You Need? A Study of Robots Ability to Understand and Execute Kushal Koshti Nidhir Bhavsar 55 1 0 13 Nov 2023
BeLLM: Backward Dependency Enhanced Large Language Model for Sentence Embeddings Xianming Li Jing Li 45 12 0 09 Nov 2023
Chain of Images for Intuitively Reasoning Fanxu Meng Haotong Yang Yiding Wang Muhan Zhang LRM 36 7 0 09 Nov 2023
LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B Simon Lermen Charlie Rogers-Smith Jeffrey Ladish ALM 31 83 0 31 Oct 2023
Herd: Using multiple, smaller LLMs to match the performances of proprietary, large LLMs via an intelligent composer S. N. Hari Matt Thomson 32 0 0 30 Oct 2023
LLMs and Finetuning: Benchmarking cross-domain performance for hate speech detection Ahmad Nasir Aadish Sharma Kokil Jaidka Saifuddin Ahmed 32 3 0 29 Oct 2023
Punica: Multi-Tenant LoRA Serving Lequn Chen Zihao Ye Yongji Wu Danyang Zhuo Luis Ceze Arvind Krishnamurthy 44 34 0 28 Oct 2023