QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

50 / 392 papers shown

Title
Sequential LLM Framework for Fashion Recommendation Han Liu Xianfeng Tang Tianlang Chen Jiapeng Liu Indu Indu ... Roberto Fernandez Galan Michael D Porter Dongmei Jia Ning Zhang Lian Xiong AI4TS 35 1 0 15 Oct 2024
ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains Yein Park Chanwoong Yoon Jungwoo Park Donghyeon Lee Minbyul Jeong Jaewoo Kang KELM 61 1 0 13 Oct 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 149 1 0 11 Oct 2024
Disease Entity Recognition and Normalization is Improved with Large Language Model Derived Synthetic Normalized Mentions Kuleen Sasse Shinjitha Vadlakonda Richard Kennedy J. D. Osborne AI4CE 31 0 0 10 Oct 2024
Extracting and Transferring Abilities For Building Multi-lingual Ability-enhanced Large Language Models Zhipeng Chen Liang Song K. Zhou Wayne Xin Zhao Binghui Wang Weipeng Chen Ji-Rong Wen 68 0 0 10 Oct 2024
Q-VLM: Post-training Quantization for Large Vision-Language Models Changyuan Wang Ziwei Wang Xiuwei Xu Yansong Tang Jie Zhou Jiwen Lu MQ 32 1 0 10 Oct 2024
SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe Yuxin Xiao Shujian Zhang Wenxuan Zhou Marzyeh Ghassemi Sanqiang Zhao 124 0 0 07 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 73 6 0 06 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 36 0 0 05 Oct 2024
ARB-LLM: Alternating Refined Binarizations for Large Language Models Zhiteng Li Xinyu Yan Tianao Zhang Haotong Qin Dong Xie Jiang Tian Zhongchao Shi Linghe Kong Yulun Zhang Xiaokang Yang MQ 34 2 0 04 Oct 2024
Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization Zichen Miao Zhengyuan Yang Kevin Lin Ze Wang Zicheng Liu Lijuan Wang Qiang Qiu 48 3 0 04 Oct 2024
TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models Zefang Liu Yinzhu Quan 32 0 0 02 Oct 2024
Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal Ioannis Arapakis ALM 56 2 0 02 Oct 2024
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging Yiming Ju Ziyi Ni Xingrun Xing Zhixiong Zeng hanyu Zhao Siqi Fan Zheng Zhang MoMe 39 2 0 01 Oct 2024
PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization Yao Ni Shan Zhang Piotr Koniusz 163 2 0 25 Sep 2024
EventHallusion: Diagnosing Event Hallucinations in Video LLMs Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Zhiyu Tan Hao Li Jingjing Chen MLLM 61 18 0 25 Sep 2024
CAD: Memory Efficient Convolutional Adapter for Segment Anything Joohyeok Kim Joonhyeon Song Seohwan Yun Seongho Yoon Sangmin Lee 31 1 0 24 Sep 2024
PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs Jiahao Yu Yangguang Shao Hanwen Miao Junzheng Shi SILM AAML 71 4 0 23 Sep 2024
SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation Mingze Sun Chen Guo Puhua Jiang Shiwei Mao Yurun Chen Ruqi Huang 58 4 0 18 Sep 2024
Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models Haoran Ye Yuhang Xie Yuanyi Ren Hanjun Fang Xin Zhang Guojie Song LM&MA 37 1 0 18 Sep 2024
Prompt Obfuscation for Large Language Models David Pape Thorsten Eisenhofer Thorsten Eisenhofer Lea Schönherr AAML 38 2 0 17 Sep 2024
Leveraging Open-Source Large Language Models for Native Language Identification Yee Man Ng Ilia Markov 35 1 0 15 Sep 2024
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 48 1 0 12 Sep 2024
DiVA-DocRE: A Discriminative and Voice-Aware Paradigm for Document-Level Relation Extraction Yiheng Wu Roman Yangarber Xian Mao 33 0 0 07 Sep 2024
Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models Dipankar Srirag Aditya Joshi Jacob Eisenstein 46 1 0 31 Aug 2024
Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage Md. Rafi Ur Rashid Jing Liu T. Koike-Akino Shagufta Mehnaz Ye Wang MU SILM 43 3 0 30 Aug 2024
An Investigation of Warning Erroneous Chat Translations in Cross-lingual Communication Yunmeng Li Jun Suzuki Makoto Morishita Kaori Abe Kentaro Inui 65 1 0 28 Aug 2024
Personality Alignment of Large Language Models Minjun Zhu Linyi Yang Yue Zhang Yue Zhang ALM 67 5 0 21 Aug 2024
Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs Jiancheng Dong Lei Jiang Wei Jin Lu Cheng 44 1 0 18 Aug 2024
Enhancing Large Language Model-based Speech Recognition by Contextualization for Rare and Ambiguous Words Kento Nozawa Takashi Masuko Toru Taniguchi 43 1 0 15 Aug 2024
HateSieve: A Contrastive Learning Framework for Detecting and Segmenting Hateful Content in Multimodal Memes Xuanyu Su Yansong Li Diana Inkpen Nathalie Japkowicz VLM 83 2 0 11 Aug 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 38 9 0 10 Aug 2024
Learning to Rewrite: Generalized LLM-Generated Text Detection Wei Hao Ran Li Weiliang Zhao Junfeng Yang Chengzhi Mao DeLMO 56 3 0 08 Aug 2024
Conditioning LLMs with Emotion in Neural Machine Translation Charles Brazier Jean-Luc Rouas CVBM 41 2 0 06 Aug 2024
Examining Gender and Power on Wikipedia Through Face and Politeness Adil Soubki Shyne Choi Owen Rambow CVBM 33 0 0 05 Aug 2024
From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future Haolin Jin Linghan Huang Haipeng Cai Jun Yan Bo Li Huaming Chen 78 27 0 05 Aug 2024
Non Verbis, Sed Rebus: Large Language Models are Weak Solvers of Italian Rebuses Gabriele Sarti Tommaso Caselli Malvina Nissim Arianna Bisazza ReLM LRM 31 1 0 01 Aug 2024
ThinK: Thinner Key Cache by Query-Driven Pruning Yuhui Xu Zhanming Jie Hanze Dong Lei Wang Xudong Lu Aojun Zhou Amrita Saha Caiming Xiong Doyen Sahoo 75 14 0 30 Jul 2024
Model Agnostic Hybrid Sharding For Heterogeneous Distributed Inference Claudio Angione Yue Zhao Harry Yang Ahmad Farhan Fielding Johnston James Buban Patrick Colangelo 42 1 0 29 Jul 2024
Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning Yeongbin Seo Dongha Lee Jinyoung Yeo CLL KELM 90 1 0 24 Jul 2024
Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners Yifei Gao Jie Ou Lei Wang Fanhua Shang Jaji Wu MQ 49 0 0 22 Jul 2024
Impact of Model Size on Fine-tuned LLM Performance in Data-to-Text Generation: A State-of-the-Art Investigation Joy Mahapatra Utpal Garain 39 9 0 19 Jul 2024
Weighted Grouped Query Attention in Transformers Sai Sena Chinnakonduru Astarag Mohapatra 42 5 0 15 Jul 2024
Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systems Yunxiao Shi Xing Zi Zijing Shi Haimin Zhang Qiang Wu Min Xu 39 7 0 15 Jul 2024
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Yonghong Tian Wenqi Shao Peng Xu Jiahao Wang Peng Gao Kaipeng Zhang Ping Luo MQ 46 24 0 10 Jul 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 42 43 0 09 Jul 2024
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression Zhichao Xu Ashim Gupta Tao Li Oliver Bentham Vivek Srikumar 52 8 0 06 Jul 2024
The Impact of Quantization and Pruning on Deep Reinforcement Learning Models Heng Lu Mehdi Alemi Reza Rawassizadeh 39 1 0 05 Jul 2024
SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking Xingrun Xing Boyan Gao Zheng Zhang David A. Clifton Shitao Xiao Li Du Guoqi Li Jiajun Zhang 55 5 0 05 Jul 2024
Query-Guided Self-Supervised Summarization of Nursing Notes Ya Gao H. Moen S. Koivusalo M. Koskinen Pekka Marttinen 44 1 0 04 Jul 2024