LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models

4 April 2023

Lei Wang

Papers citing "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"

50 / 141 papers shown

Title
MoRe Fine-Tuning with 10x Fewer Parameters Wenxuan Tan Nicholas Roberts Tzu-Heng Huang Jitian Zhao John Cooper Samuel Guo Chengyu Duan Frederic Sala 34 0 0 30 Aug 2024
Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution Yucheng Ruan Xiang Lan Jingying Ma Yizhi Dong Kai He Mengling Feng LMTD 67 3 0 20 Aug 2024
NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models Cheng Lin Lujun Li Dezhi Li Jie Zou Wei Xue Yike Guo AI4TS 46 4 0 18 Aug 2024
CodeMirage: Hallucinations in Code Generated by Large Language Models Vibhor Agarwal Yulong Pei Salwa Alamir Xiaomo Liu 42 4 0 14 Aug 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 38 9 0 10 Aug 2024
Understanding the Performance and Estimating the Cost of LLM Fine-Tuning Yuchen Xia Jiho Kim Yuhan Chen Haojie Ye Souvik Kundu Cong Hao Nishil Talati MoE 37 22 0 08 Aug 2024
Enabling Contextual Soft Moderation on Social Media through Contrastive Textual Deviation Pujan Paudel Mohammad Hammas Saeed Rebecca Auger Chris Wells Gianluca Stringhini 75 1 0 30 Jul 2024
AI Act for the Working Programmer Holger Hermanns Anne Lauber-Rönsberg Philip Meinel Sarah Sterz Hanwei Zhang 42 1 0 23 Jul 2024
Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters Kartikeya Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... Paul N. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MoMe 33 0 0 22 Jul 2024
Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability Zhuoyan Xu Zhenmei Shi Yingyu Liang CoGe LRM 40 27 0 22 Jul 2024
InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification Yujia Hu Zhiqiang Hu C. Seah Roy Ka-wei Lee 34 0 0 16 Jul 2024
Low-Rank Interconnected Adaptation Across Layers Yibo Zhong Yao Zhou OffRL MoE 48 1 0 13 Jul 2024
SBoRA: Low-Rank Adaptation with Regional Weight Updates L. Po Yuyang Liu Haoxuan Wu Tianqi Zhang W. Yu Zeyu Jiang Kun Li 38 1 0 07 Jul 2024
BlockLLM: Memory-Efficient Adaptation of LLMs by Selecting and Optimizing the Right Coordinate Blocks A. Ramesh Vignesh Ganapathiraman I. Laradji Mark W. Schmidt 40 1 0 25 Jun 2024
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs Ashwinee Panda Berivan Isik Xiangyu Qi Sanmi Koyejo Tsachy Weissman Prateek Mittal MoMe 45 13 0 24 Jun 2024
Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning Ziyu Zhao Leilei Gan Guoyin Wang Yuwei Hu Tao Shen Hongxia Yang Kun Kuang Fei Wu MoE MoMe 39 11 0 24 Jun 2024
Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking Yuwei Zhang Tong Xia Jing Han Yu Wu Georgios Rizos Yang Liu Mohammed Mosuily Jagmohan Chauhan Cecilia Mascolo AI4CE 41 6 0 23 Jun 2024
RankAdaptor: Hierarchical Dynamic Low-Rank Adaptation for Structural Pruned LLMs Changhai Zhou Shijie Han Shiyang Zhang Shichao Weng Zekai Liu Cheng Jin 45 1 0 22 Jun 2024
Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning Brandon Huang Chancharik Mitra Assaf Arbelle Leonid Karlinsky Trevor Darrell Roei Herzig 49 13 0 21 Jun 2024
RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning Haoyu Wang Tianci Liu Ruirui Li Monica Cheng Tuo Zhao Jing Gao 29 7 0 16 Jun 2024
FouRA: Fourier Low Rank Adaptation Shubhankar Borse Shreya Kadambi N. Pandey Kartikeya Bhardwaj Viswanath Ganapathy Sweta Priyadarshi Risheek Garrepalli Rafael Esteves Munawar Hayat Fatih Porikli 42 6 0 13 Jun 2024
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning Hanqing Wang Zeguan Xiao Shuo Wang Guanhua Chen Guanhua Chen 44 19 0 13 Jun 2024
Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devices Ruiyang Qin Dancheng Liu Zheyu Yan Zhaoxuan Tan Zixuan Pan Zhenge Jia Meng Jiang Ahmed Abbasi Jinjun Xiong Yiyu Shi 59 11 0 06 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 36 4 0 05 Jun 2024
Multi-layer Learnable Attention Mask for Multimodal Tasks Wayner Barrios SouYoung Jin 39 0 0 04 Jun 2024
Low-Rank Adaption on Transformer-based Oriented Object Detector for Satellite Onboard Processing of Remote Sensing Images Xinyang Pu Feng Xu 32 3 0 04 Jun 2024
OccamLLM: Fast and Exact Language Model Arithmetic in a Single Step Owen Dugan Donato Manuel Jimenez Beneto Charlotte Loh Zhuo Chen Rumen Dangovski Marin Soljacic LRM 37 1 0 04 Jun 2024
OLoRA: Orthonormal Low-Rank Adaptation of Large Language Models Kerim Büyükakyüz AI4CE 23 5 0 03 Jun 2024
QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation Zhuo Chen Rumen Dangovski Charlotte Loh Owen Dugan Di Luo Marin Soljacic MQ 30 8 0 31 May 2024
Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models Xinxi Zhang Song Wen Ligong Han Felix Juefei Xu Akash Srivastava Junzhou Huang Hao Wang Molei Tao Dimitris N. Metaxas DiffM 36 5 0 31 May 2024
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors Vijay Lingam Atula Tejaswi Aditya Vavre Aneesh Shetty Gautham Krishna Gudur Joydeep Ghosh Alexandros G. Dimakis Eunsol Choi Aleksandar Bojchevski Sujay Sanghavi 44 9 0 30 May 2024
OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning Pengxiang Li Lu Yin Xiaowei Gao Shiwei Liu 34 7 0 28 May 2024
Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentation Ya Lu Jishnu Jaykumar Yunhui Guo Nicholas Ruozzi Yu Xiang VLM ISeg 58 4 0 28 May 2024
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters Klaudia Bałazy Mohammadreza Banaei Karl Aberer Jacek Tabor 43 17 0 27 May 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 49 34 0 27 May 2024
PLUG: Revisiting Amodal Segmentation with Foundation Model and Hierarchical Focus Zhaochen Liu Limeng Qiao Xiangxiang Chu Tingting Jiang 37 2 0 25 May 2024
Sparse Matrix in Large Language Model Fine-tuning Haoze He Juncheng Billy Li Xuan Jiang Heather Miller MoE 27 3 0 24 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 47 45 0 17 May 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 37 5 0 28 Apr 2024
Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications Charith Chandra Sai Balne S. Bhaduri Tamoghna Roy Vinija Jain Aman Chadha 40 12 0 21 Apr 2024
Privacy Preserving Prompt Engineering: A Survey Kennedy Edemacu Xintao Wu 49 18 0 09 Apr 2024
ReFT: Representation Finetuning for Language Models Zhengxuan Wu Aryaman Arora Zheng Wang Atticus Geiger Daniel Jurafsky Christopher D. Manning Christopher Potts OffRL 38 58 0 04 Apr 2024
Accelerating Transformer Pre-training with 2:4 Sparsity Yuezhou Hu Kang Zhao Weiyu Huang Jianfei Chen Jun Zhu 65 7 0 02 Apr 2024
Octopus: On-device language model for function calling of software APIs Wei Chen Zhiyuan Li Mingyuan Ma LLMAG 27 14 0 02 Apr 2024
MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning Ahmed A. Agiza Marina Neseem Sherief Reda MoE 48 12 0 29 Mar 2024
Dual-Personalizing Adapter for Federated Foundation Models Yiyuan Yang Guodong Long Taoshu Shen Jing Jiang Michael Blumenstein 94 19 0 28 Mar 2024
Evaluating the Efficacy of Prompt-Engineered Large Multimodal Models Versus Fine-Tuned Vision Transformers in Image-Based Security Applications Fouad Trad Ali Chehab MLLM 34 2 0 26 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 318 0 21 Mar 2024
SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation Yanming Liu Xinyue Peng Jiannan Cao Le Dai Xingzu Liu Mingbang Wang Weihao Liu SyDa 44 2 0 11 Mar 2024
Materials science in the era of large language models: a perspective Ge Lei Ronan Docherty Samuel J. Cooper 45 18 0 11 Mar 2024