QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

50 / 407 papers shown

Title
Pruning as a Domain-specific LLM Extractor Nan Zhang Yanchi Liu Xujiang Zhao Wei Cheng Runxue Bao Rui Zhang Prasenjit Mitra Haifeng Chen 26 9 0 10 May 2024
"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversations Preetam Prabhu Srikar Dammu Hayoung Jung Anjali Singh Monojit Choudhury Tanushree Mitra 39 8 0 08 May 2024
Optimizing Language Model's Reasoning Abilities with Weak Supervision Yongqi Tong Sizhe Wang Dawei Li Yifan Wang Simeng Han Zi Lin Chengsong Huang Jiaxin Huang Jingbo Shang LRM ReLM 42 8 0 07 May 2024
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving Yujun Lin Haotian Tang Shang Yang Zhekai Zhang Guangxuan Xiao Chuang Gan Song Han 90 76 0 07 May 2024
When LLMs Meet Cybersecurity: A Systematic Literature Review Jie Zhang Haoyu Bu Hui Wen Yu Chen Lun Li Hongsong Zhu 45 36 0 06 May 2024
Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs Jordan Dotzel Yuzong Chen Bahaa Kotb Sushma Prasad Gang Wu Sheng Li Mohamed S. Abdelfattah Zhiru Zhang 31 9 0 06 May 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Zhenyu Zhang Shuohuan Wang Weiran Xu Yu Sun Hua Wu CLL 42 4 0 29 Apr 2024
Building a Large Japanese Web Corpus for Large Language Models Naoaki Okazaki Kakeru Hattori Hirai Shota Hiroki Iida Masanari Ohi Kazuki Fujii Taishi Nakamura Mengsay Loem Rio Yokota Sakae Mizuki 55 7 0 27 Apr 2024
Ultrasound SAM Adapter: Adapting SAM for Breast Lesion Segmentation in Ultrasound Images Zhengzheng Tu Le Gu Xixi Wang Bo Jiang MedIm 46 3 0 23 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 84 46 0 23 Apr 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 57 22 0 22 Apr 2024
An empirical study of LLaMA3 quantization: from LLMs to MLLMs Wei Huang Xingyu Zheng Xudong Ma Haotong Qin Chengtao Lv Hong Chen Jie Luo Xiaojuan Qi Xianglong Liu Michele Magno MQ 59 38 0 22 Apr 2024
ColA: Collaborative Adaptation with Gradient Learning Enmao Diao Qi Le Suya Wu Xinran Wang Ali Anwar Jie Ding Vahid Tarokh 35 1 0 22 Apr 2024
AdvisorQA: Towards Helpful and Harmless Advice-seeking Question Answering with Collective Intelligence Minbeom Kim Hwanhee Lee Joonsuk Park Hwaran Lee Kyomin Jung 40 1 0 18 Apr 2024
KG-CTG: Citation Generation through Knowledge Graph-guided Large Language Models Avinash Anand Mohit Gupta Kritarth Prasad Ujjwal Goel Naman Lal Astha Verma R. Shah 56 7 0 15 Apr 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 52 19 0 15 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 46 8 0 13 Apr 2024
Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations Dayeon Ki Marine Carpuat 38 17 0 11 Apr 2024
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety Paul Röttger Fabio Pernisi Bertie Vidgen Dirk Hovy ELM KELM 60 31 0 08 Apr 2024
Enhancing Clinical Efficiency through LLM: Discharge Note Generation for Cardiac Patients Hyoje Jung Yunha Kim Heejung Choi Hyeram Seo Minkyoung Kim ... Soyoung Ko Byeolhee Kim Suyeon Kim Tae Joon Jun Young-Hak Kim 43 15 0 08 Apr 2024
Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning Teo Susnjak Peter Hwang N. Reyes A. Barczak Timothy R. McIntosh Surangika Ranathunga 70 22 0 08 Apr 2024
MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators Shenghai Yuan Jinfa Huang Yujun Shi Yongqi Xu Ruijie Zhu Bin Lin Xinhua Cheng Li-xin Yuan Jiebo Luo VGen 81 33 0 07 Apr 2024
Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement Zaid Khan B. Vijaykumar S. Schulter Yun Fu Manmohan Chandraker LRM ReLM 34 6 0 06 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 46 7 0 05 Apr 2024
Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models Wanyun Cui Qianle Wang MQ 42 2 0 03 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 64 73 0 03 Apr 2024
Efficiently Distilling LLMs for Edge Applications Achintya Kundu Fabian Lim Aaron Chew L. Wynter Penny Chong Rhui Dih Lee 42 6 0 01 Apr 2024
Token-Efficient Leverage Learning in Large Language Models Yuanhao Zeng Min Wang Yihang Wang Yingxia Shao 37 0 0 01 Apr 2024
Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMs Shu Yang Jiayuan Su Han Jiang Mengdi Li Keyuan Cheng Muhammad Asif Ali Lijie Hu Di Wang 37 5 0 30 Mar 2024
The Unreasonable Ineffectiveness of the Deeper Layers Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts 52 81 0 26 Mar 2024
Towards Enabling FAIR Dataspaces Using Large Language Models Benedikt T. Arnold Johannes Theissen-Lipp D. Collarana Christoph Lange Sandra Geisler Edward Curry Stefan Decker 34 1 0 18 Mar 2024
Are LLMs Good Cryptic Crossword Solvers? Abdelrahman Boda Daria Kotova Ekaterina Kochmar 24 3 0 15 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 46 2 0 14 Mar 2024
ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphs Preetam Prabhu Srikar Dammu Himanshu Naidu Mouly Dewan YoungMin Kim Tanya Roosta Aman Chadha Chirag Shah 46 6 0 12 Mar 2024
Development of a Reliable and Accessible Caregiving Language Model (CaLM) B. Parmanto Bayu Aryoyudanta Wilbert Soekinto Agus Setiawan Yuhan Wang Haomin Hu Andi Saptono Yong K Choi 32 0 0 11 Mar 2024
On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models Xinpeng Wang Shitong Duan Xiaoyuan Yi Jing Yao Shanlin Zhou Zhihua Wei Peng Zhang Dongkuan Xu Maosong Sun Xing Xie OffRL 41 16 0 07 Mar 2024
Preference optimization of protein language models as a multi-objective binder design paradigm Pouria A. Mistani Venkatesh Mysore 40 6 0 07 Mar 2024
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection Jiawei Zhao Zhenyu (Allen) Zhang Beidi Chen Zhangyang Wang A. Anandkumar Yuandong Tian 43 178 0 06 Mar 2024
Socratic Reasoning Improves Positive Text Rewriting Anmol Goel Nico Daheim Iryna Gurevych Iryna Gurevych LRM 39 4 0 05 Mar 2024
STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models Linhai Zhang Jialong Wu Deyu Zhou Guoqiang Xu 30 4 0 02 Mar 2024
Improving Socratic Question Generation using Data Augmentation and Preference Optimization Nischal Ashok Kumar Andrew S. Lan 33 8 0 01 Mar 2024
Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency Akila Wickramasekara F. Breitinger Mark Scanlon 52 8 0 29 Feb 2024
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning Xupeng Miao Gabriele Oliaro Xinhao Cheng Vineeth Kada Ruohan Gao ... April Yang Yingcheng Wang Mengdi Wu Colin Unger Zhihao Jia MoE 94 9 0 29 Feb 2024
Prediction-Powered Ranking of Large Language Models Ivi Chatzi Eleni Straitouri Suhas Thejaswi Manuel Gomez Rodriguez ALM 29 5 0 27 Feb 2024
On the Societal Impact of Open Foundation Models Sayash Kapoor Rishi Bommasani Kevin Klyman Shayne Longpre Ashwin Ramaswami ... Victor Storchan Daniel Zhang Daniel E. Ho Percy Liang Arvind Narayanan 26 54 0 27 Feb 2024
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation Sunghyeon Woo Baeseong Park Byeongwook Kim Minjung Jo S. Kwon Dongsuk Jeon Dongsoo Lee 65 2 0 27 Feb 2024
Investigating the Effectiveness of HyperTuning via Gisting Jason Phang 46 0 0 26 Feb 2024
A Comprehensive Evaluation of Quantization Strategies for Large Language Models Renren Jin Jiangcun Du Wuwei Huang Wei Liu Jian Luan Bin Wang Deyi Xiong MQ 32 31 0 26 Feb 2024
GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning Han Zou Qiyang Zhao Lina Bariah Yu Tian M. Bennis S. Lasaulce 101 12 0 26 Feb 2024
Fast Adversarial Attacks on Language Models In One GPU Minute Vinu Sankar Sadasivan Shoumik Saha Gaurang Sriramanan Priyatham Kattakinda Atoosa Malemir Chegini S. Feizi MIALM 43 34 0 23 Feb 2024