LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models

4 April 2023

Lei Wang

Papers citing "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"

50 / 141 papers shown

Title
SRLoRA: Subspace Recomposition in Low-Rank Adaptation via Importance-Based Fusion and Reinitialization Haodong Yang Lei Wang Md Zakir Hossain 7 0 0 18 May 2025
Memory-Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation Fei Wu Jia Hu Geyong Min Shiqiang Wang 17 0 0 16 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun Luo Yue Gao MoE 46 2 0 05 May 2025
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients Yezhen Wang Zhouhao Yang Brian K Chen Fanyi Pu Bo-wen Li Tianyu Gao Kenji Kawaguchi 43 0 0 03 May 2025
Enhancing ML Model Interpretability: Leveraging Fine-Tuned Large Language Models for Better Understanding of AI Jonas Bokstaller Julia Altheimer Julian Dormehl Alina Buss Jasper Wiltfang Johannes Schneider Maximilian Röglinger 24 0 0 02 May 2025
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Dinesh Manocha Qinghua Lu Shanshan Ye Jing Jiang OODD 189 1 0 02 May 2025
Graph-Based Spectral Decomposition for Parameter Coordination in Language Model Fine-Tuning Hanlu Zhang Yumeng Ma Shuo Wang Guiran Liu B. Zhu 50 2 0 28 Apr 2025
Task-conditioned Ensemble of Expert Models for Continuous Learning Renu Sharma Debasmita Pal Arun Ross OOD KELM 174 0 0 11 Apr 2025
LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation Juzheng Zhang Jiacheng You Ashwinee Panda Tom Goldstein MoMe 53 1 0 10 Apr 2025
Beyond Progress Measures: Theoretical Insights into the Mechanism of Grokking Zihan Gu Ruoyu Chen Hua Zhang Yue Hu Xiaochun Cao 39 0 0 04 Apr 2025
Data Poisoning in Deep Learning: A Survey Pinlong Zhao Weiyao Zhu Pengfei Jiao Di Gao Ou Wu AAML 39 0 0 27 Mar 2025
Serial Low-rank Adaptation of Vision Transformer Houqiang Zhong Shaocheng Shen Ke Cai Zhenglong Wu Jiangchao Yao Yuan Cheng Xuefei Li Xiaoyun Zhang Li-Na Song Qiang Hu 47 0 0 22 Mar 2025
Aligning Vision to Language: Text-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoning Junming Liu Siyuan Meng Yanting Gao Song Mao Pinlong Cai Guohang Yan Yirong Chen Zilin Bian Botian Shi Ding Wang 57 1 0 17 Mar 2025
Learning to Inference Adaptively for Multimodal Large Language Models Zhuoyan Xu Khoi Duc Nguyen Preeti Mukherjee Saurabh Bagchi Somali Chaterji Yingyu Liang Yin Li LRM 49 1 0 13 Mar 2025
AdaptSR: Low-Rank Adaptation for Efficient and Scalable Real-World Super-Resolution Cansu Korkmaz Nancy Mehta Radu Timofte 67 0 0 10 Mar 2025
Digital Player: Evaluating Large Language Models based Human-like Agent in Games J. T. Wang Kai Wang Shaojie Lin Runze Wu Bihan Xu ... Zhipeng Hu Z. Fan Le Li Tangjie Lyu Changjie Fan LLMAG ELM AI4CE 55 1 0 28 Feb 2025
HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture Taiqiang Wu Chenchen Ding Wenyong Zhou Yuxin Cheng Xincheng Feng Shuqi Wang Chufan Shi Z. Liu Ngai Wong 58 0 0 27 Feb 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 79 2 0 24 Feb 2025
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment Chenghao Fan Zhenyi Lu Sichen Liu Xiaoye Qu Wei Wei Chengfeng Gu Yu-Xi Cheng MoE 162 0 0 24 Feb 2025
The Curse of Depth in Large Language Models Wenfang Sun Xinyuan Song Pengxiang Li Lu Yin Yefeng Zheng Shiwei Liu 75 4 0 09 Feb 2025
Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble Atsushi Nitanda Anzelle Lee Damian Tan Xing Kai Mizuki Sakaguchi Taiji Suzuki AI4CE 67 1 0 09 Feb 2025
LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning Zhekai Du Yinjie Min Jingjing Li Ke Lu Changliang Zou Liuhua Peng Tingjin Chu Mingming Gong 186 1 0 05 Feb 2025
RandLoRA: Full-rank parameter-efficient fine-tuning of large models Paul Albert Frederic Z. Zhang Hemanth Saratchandran Cristian Rodriguez-Opazo Anton van den Hengel Ehsan Abbasnejad 96 1 0 03 Feb 2025
CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models Guanduo Chen Yutong He Yipeng Hu Kun Yuan Binhang Yuan 54 0 0 03 Feb 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 43 4 0 28 Jan 2025
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics Kai He Rui Mao Qika Lin Yucheng Ruan Xiang Lan Mengling Feng Min Zhang LM&MA AILaw 93 154 0 28 Jan 2025
A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models Qinggang Zhang Shengyuan Chen Yuanchen Bei Zheng Yuan Huachi Zhou Zijin Hong Junnan Dong Hao-Heng Chen Yi-Ju Chang Xiao Huang 3DV 70 8 0 21 Jan 2025
SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training Tianjin Huang Ziquan Zhu Gaojie Jin Lu Liu Zhangyang Wang Shiwei Liu 44 1 0 12 Jan 2025
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models Xiaolin Hu Xiang Cheng Peiyu Liu Wei Liu Jian Luan Bin Wang Y. Liu 42 1 0 31 Dec 2024
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 88 4 0 31 Dec 2024
Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN Pengxiang Li Lu Yin Shiwei Liu 70 4 0 18 Dec 2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 94 1 0 08 Dec 2024
Training-Free Bayesianization for Low-Rank Adapters of Large Language Models Haizhou Shi Yibin Wang Ligong Han Huan Zhang Hao Wang UQCV 83 0 0 07 Dec 2024
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning Kaustubh Ponkshe Raghav Singhal Eduard A. Gorbunov Alexey Tumanov Samuel Horváth Praneeth Vepakomma 71 2 0 29 Nov 2024
Towards a Middleware for Large Language Models Narcisa Guran Florian Knauf Man Ngo Stefan Petrescu Jan S. Rellermeyer 73 1 0 21 Nov 2024
Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation Ayan Sengupta Vaibhav Seth Arinjay Pathak Natraj Raman Sriram Gopalakrishnan Tanmoy Chakraborty BDL 30 2 0 07 Nov 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 100 2 0 06 Nov 2024
Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models Junjiao Tian Chengyue Huang Z. Kira 44 1 0 03 Nov 2024
On the Crucial Role of Initialization for Matrix Factorization Bingcong Li Liang Zhang Aryan Mokhtari Niao He 28 1 0 24 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Dynamic Adaptive Rank Space Exploration for Efficient Sentiment Analysis with Large Language Models Hongcheng Ding Fuzhen Hu Xuanze Zhao Zixiao Jiang Shamsul Nahar Abdullah Deshinta Arrova Dewi 21 0 0 22 Oct 2024
MoRE: Multi-Modal Contrastive Pre-training with Transformers on X-Rays, ECGs, and Diagnostic Report Samrajya Thapa Koushik Howlader Subhankar Bhattacharjee Wei le MedIm 28 1 0 21 Oct 2024
Parameter-Efficient Fine-Tuning of State Space Models Kevin Galim Wonjun Kang Yuchen Zeng H. Koo Kangwook Lee 31 4 0 11 Oct 2024
One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation Fabian Paischer Lukas Hauzenberger Thomas Schmied Benedikt Alkin Marc Peter Deisenroth Sepp Hochreiter 37 4 0 09 Oct 2024
Hyperbolic Fine-tuning for Large Language Models Menglin Yang Aosong Feng Bo Xiong Jihong Liu Irwin King Rex Ying 23 1 0 05 Oct 2024
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint? Xi Chen Kaituo Feng Changsheng Li Xunhao Lai Xiangyu Yue Ye Yuan Guoren Wang 47 7 0 02 Oct 2024
Efficient Noise Mitigation for Enhancing Inference Accuracy in DNNs on Mixed-Signal Accelerators Seyedarmin Azizi Mohammad Erfan Sadeghi M. Kamal Massoud Pedram 27 2 0 27 Sep 2024
On the Feasibility of Fully AI-automated Vishing Attacks João Figueiredo Afonso Carvalho Daniel Castro Daniel Gonçalves Nuno Santos 27 2 0 20 Sep 2024
Frequency-Guided Spatial Adaptation for Camouflaged Object Detection Shizhou Zhang Dexuan Kong Yinghui Xing Yue Lu Lingyan Ran Guoqiang Liang Hexu Wang Yanning Zhang 38 5 0 19 Sep 2024
Fine-tuning and Prompt Engineering with Cognitive Knowledge Graphs for Scholarly Knowledge Organization Gollam Rabby Sören Auer Jennifer D'Souza A. Oelen 148 2 0 10 Sep 2024