QLoRA: Efficient Finetuning of Quantized LLMs

23 May 2023

Tim Dettmers

Luke Zettlemoyer

Papers citing "QLoRA: Efficient Finetuning of Quantized LLMs"

50 / 407 papers shown

Title
HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models Yinghui He Yufan Wu Yilin Jia Rada Mihalcea Yulong Chen Naihao Deng LRM LLMAG 38 22 0 25 Oct 2023
LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking Zhenrui Yue Sara Rabhi Gabriel de Souza P. Moreira Dong Wang Even Oldridge LRM 46 36 0 25 Oct 2023
Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing Xinyu Hu Pengfei Tang Simiao Zuo Zihan Wang Bowen Song Qiang Lou Jian Jiao Denis Xavier Charles LRM 40 7 0 20 Oct 2023
Explainable Depression Symptom Detection in Social Media Eliseo Bao Souto Anxo Perez Javier Parapar 30 5 0 20 Oct 2023
Audio Editing with Non-Rigid Text Prompts Francesco Paissan Luca Della Libera Zhepei Wang Mirco Ravanelli Paris Smaragdis Cem Subakan DiffM 46 5 0 19 Oct 2023
JMedLoRA:Medical Domain Adaptation on Japanese Large Language Models using Instruction-tuning Issey Sukeda Masahiro Suzuki Hiroki Sakaji Satoshi Kodera LM&MA 27 7 0 16 Oct 2023
ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models Haoran Luo E. Haihong Zichen Tang Shiyao Peng Yikai Guo ... Guanting Dong Meina Song Wei Lin Yifan Zhu Luu Anh Tuan RALM 34 38 0 13 Oct 2023
LLaMA Rider: Spurring Large Language Models to Explore the Open World Yicheng Feng Yuxuan Wang Jiazheng Liu Sipeng Zheng Zongqing Lu LLMAG LRM 18 16 0 13 Oct 2023
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources Zhikai Li Xiaoxuan Liu Banghua Zhu Zhen Dong Qingyi Gu Kurt Keutzer MQ 32 7 0 11 Oct 2023
Selective Demonstrations for Cross-domain Text-to-SQL Shuaichen Chang Eric Fosler-Lussier 20 19 0 10 Oct 2023
MuseChat: A Conversational Music Recommendation System for Videos Zhikang Dong Bin Chen Xiulong Liu Paweł Polak Peng Zhang LRM 45 26 0 10 Oct 2023
SALMON: Self-Alignment with Instructable Reward Models Zhiqing Sun Songlin Yang Hongxin Zhang Qinhong Zhou Zhenfang Chen David D. Cox Yiming Yang Chuang Gan ALM SyDa 38 35 0 09 Oct 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 39 2 0 08 Oct 2023
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models Ming Jin Shiyu Wang Lintao Ma Zhixuan Chu James Y. Zhang ... Pin-Yu Chen Keli Zhang Yuan-Fang Li Shirui Pan Qingsong Wen AI4TS 50 358 0 03 Oct 2023
Making Retrieval-Augmented Language Models Robust to Irrelevant Context Ori Yoran Tomer Wolfson Ori Ram Jonathan Berant RALM LRM 19 185 0 02 Oct 2023
AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ Jonas Belouadi Anne Lauscher Steffen Eger 21 28 0 30 Sep 2023
Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models Antoine Louis Gijs van Dijck Gerasimos Spanakis ELM AILaw 30 35 0 29 Sep 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 34 93 0 27 Sep 2023
CPLLM: Clinical Prediction with Large Language Models Ofir Ben Shoham Nadav Rappoport LM&MA 34 26 0 20 Sep 2023
OpenChat: Advancing Open-source Language Models with Mixed-Quality Data Guan-Bo Wang Sijie Cheng Xianyuan Zhan Xiangang Li Sen Song Yang Liu ALM 27 231 0 20 Sep 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 51 33 0 23 Aug 2023
NUPES : Non-Uniform Post-Training Quantization via Power Exponent Search Edouard Yvinec Arnaud Dapogny Kévin Bailly MQ 24 6 0 10 Aug 2023
In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning Xiaochuang Han 25 19 0 08 Aug 2023
SimplyRetrieve: A Private and Lightweight Retrieval-Centric Generative AI Tool Youyang Ng Daisuke Miyashita Yasuto Hoshi Yasuhiro Morioka Osamu Torii Tomoya Kodama J. Deguchi RALM 15 9 0 08 Aug 2023
From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application? Rodrigo Pedro Daniel Castro Paulo Carreira Nuno Santos SILM AAML 41 51 0 03 Aug 2023
Generative Models as a Complex Systems Science: How can we make sense of large language model behavior? Ari Holtzman Peter West Luke Zettlemoyer AI4CE 32 14 0 31 Jul 2023
Scaling Sentence Embeddings with Large Language Models Ting Jiang Shaohan Huang Zhongzhi Luan Deqing Wang Fuzhen Zhuang LRM 44 40 0 31 Jul 2023
NLLG Quarterly arXiv Report 06/23: What are the most influential current AI Papers? Steffen Eger Christoph Leiter Jonas Belouadi Ran Zhang Aida Kostikova Daniil Larionov Yanran Chen Vivian Fresen AI4CE 29 4 0 31 Jul 2023
Three Bricks to Consolidate Watermarks for Large Language Models Pierre Fernandez Antoine Chaffin Karim Tit Vivien Chappelier Teddy Furon WaLM 19 47 0 26 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang Fahad Shahbaz Khan VLM 38 118 0 25 Jul 2023
Mini-Giants: "Small" Language Models and Open Source Win-Win Zhengping Zhou Lezhi Li Xinxi Chen Andy Li SyDa ALM MoE 29 6 0 17 Jul 2023
My3DGen: A Scalable Personalized 3D Generative Model Luchao Qi Jiaye Wu Annie N. Wang Sheng-Yu Wang Roni Sengupta 3DH 39 3 0 11 Jul 2023
Visual Instruction Tuning with Polite Flamingo Delong Chen Jianfeng Liu Wenliang Dai Baoyuan Wang MLLM 34 42 0 03 Jul 2023
Domain-specific ChatBots for Science using Embeddings Kevin G. Yager 32 8 0 15 Jun 2023
NF4 Isn't Information Theoretically Optimal (and that's Good) Davis Yoshida MQ 15 9 0 12 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric P. Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 51 3,854 0 09 Jun 2023
SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking Chris Cundy Stefano Ermon 16 10 0 08 Jun 2023
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning Baohao Liao Shaomu Tan Christof Monz KELM 23 29 0 01 Jun 2023
Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization Jeonghoon Kim J. H. Lee Sungdong Kim Joonsuk Park Kang Min Yoo S. Kwon Dongsoo Lee MQ 44 99 0 23 May 2023
Privacy-Preserving Prompt Tuning for Large Language Model Services Yansong Li Zhixing Tan Yang Liu SILM VLM 53 63 0 10 May 2023
Multi-step Jailbreaking Privacy Attacks on ChatGPT Haoran Li Dadi Guo Wei Fan Mingshi Xu Jie Huang Fanpu Meng Yangqiu Song SILM 50 322 0 11 Apr 2023
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference Tao Lei Junwen Bai Siddhartha Brahma Joshua Ainslie Kenton Lee ... Vincent Zhao Yuexin Wu Bo-wen Li Yu Zhang Ming-Wei Chang BDL AI4CE 30 55 0 11 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 162 585 0 06 Apr 2023
Commentary Generation from Data Records of Multiplayer Strategy Esports Game Zihan Wang Naoki Yoshinaga 18 0 0 21 Dec 2022
Controllable Citation Sentence Generation with Language Models Nianlong Gu Richard H. R. Hahnloser 30 2 0 14 Nov 2022
Learning New Tasks from a Few Examples with Soft-Label Prototypes Avyav Kumar Singh Ekaterina Shutova H. Yannakoudakis VLM 30 0 0 31 Oct 2022
Accelerating Transfer Learning with Near-Data Computation on Cloud Object Stores Arsany Guirguis Diana Petrescu Florin Dinu D. Quoc Javier Picorel R. Guerraoui 40 0 0 16 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 253 1,073 0 05 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 230 506 0 28 Sep 2022
Can large language models reason about medical questions? Valentin Liévin C. Hother Andreas Geert Motzfeldt Ole Winther ELM LM&MA AI4MH LRM 29 300 0 17 Jul 2022