AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning

18 March 2023

Papers citing "AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning"

50 / 83 papers shown

Title
FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA Seanie Lee Sangwoo Park Dong Bok Lee Dominik Wagner Haebin Seong Tobias Bocklet Juho Lee Sung Ju Hwang FedML 9 0 0 19 May 2025
AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections Xin Yu Yujia Wang Jinghui Chen Lingzhou Xue 17 0 0 18 May 2025
Efficient Fine-Tuning of Quantized Models via Adaptive Rank and Bitwidth Changhai Zhou Yuhua Zhou Qian Qiao Weizhong Zhang Cheng Jin MQ 27 0 0 02 May 2025
TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts Pradip Kunwar Minh Vu Maanak Gupta Mahmoud Abdelsalam Manish Bhattarai MoE MoMe 187 0 0 29 Apr 2025
A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning Jieming Bian Yuanzhe Peng Lei Wang Yin Huang Jie Xu FedML 65 0 0 29 Apr 2025
Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management Hang Zhang Jiuchen Shi Yixiao Wang Quan Chen Yizhou Shan Minyi Guo 36 0 0 19 Apr 2025
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models Wenjin Ke Zhe Li D. Li Lu Tian E. Barsoum MQ 43 1 0 12 Apr 2025
Adaptive Additive Parameter Updates of Vision Transformers for Few-Shot Continual Learning Kyle Stein A. Mahyari Guillermo Francia III Eman El-Sheikh CLL 67 0 0 11 Apr 2025
LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation Juzheng Zhang Jiacheng You Ashwinee Panda Tom Goldstein MoMe 53 1 0 10 Apr 2025
Teaching pathology foundation models to accurately predict gene expression with parameter efficient knowledge transfer Shi Pan Jianan Chen Maria Secrier 28 0 0 09 Apr 2025
SPF-Portrait: Towards Pure Portrait Customization with Semantic Pollution-Free Fine-tuning Xiaole Xian Zhichao Liao Qingyu Li Wenyu Qin Pengfei Wan Weicheng Xie Long Zeng L. Shen Pingfa Feng DiffM 61 0 0 01 Apr 2025
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Yong Li Bo Liu Sheng Huang Zhe Zhang Xiaotong Yuan Richang Hong 46 0 0 31 Mar 2025
Not All LoRA Parameters Are Essential: Insights on Inference Necessity Guanhua Chen Yutong Yao Ci-Jun Gao Lidia S. Chao Feng Wan Derek F. Wong 39 0 0 30 Mar 2025
Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generation Minho Park S. Park Jungsoo Lee Hyojin Park Kyuwoong Hwang Fatih Porikli Jaegul Choo Sungha Choi 39 0 0 28 Mar 2025
Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data Zhiyuan Ma Xinyue Liang Rongyuan Wu Xiangyu Zhu Zhen Lei Lei Zhang 78 0 0 27 Mar 2025
MSPLoRA: A Multi-Scale Pyramid Low-Rank Adaptation for Efficient Model Fine-Tuning Jiancheng Zhao Xingda Yu Zhen Yang MoE 62 1 0 27 Mar 2025
Coeff-Tuning: A Graph Filter Subspace View for Tuning Attention-Based Large Models Zichen Miao Wei Chen Qiang Qiu 92 1 0 24 Mar 2025
Hiding Images in Diffusion Models by Editing Learned Score Functions Haoyu Chen Yunqiao Yang Nan Zhong Kede Ma DiffM 68 0 0 24 Mar 2025
SALT: Singular Value Adaptation with Low-Rank Transformation Abdelrahman Elsayed Sarim Hashmi Mohammed Elseiagy Hu Wang Mohammad Yaqub Ibrahim Almakky OOD 55 0 0 20 Mar 2025
VenusFactory: A Unified Platform for Protein Engineering Data Retrieval and Language Model Fine-Tuning Y. Tan Chen Liu Jingyuan Gao Banghao Wu Mingchen Li ... Lingrong Zhang Huiqun Yu Guisheng Fan Liang Hong Bingxin Zhou 58 1 0 19 Mar 2025
SurgicalVLM-Agent: Towards an Interactive AI Co-Pilot for Pituitary Surgery Jiayuan Huang Runlong He Danyal Z. Khan E. Mazomenos Danail Stoyanov Hani J. Marcus Matthew J. Clarkson Mobarakol Islam LM&Ro 59 0 0 12 Mar 2025
Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs Umberto Cappellazzo Minsu Kim Stavros Petridis 57 0 0 09 Mar 2025
Integrating Frequency-Domain Representations with Low-Rank Adaptation in Vision-Language Models Md Azim Khan A. Gangopadhyay Jianwu Wang Robert F. Erbacher VLM 59 0 0 08 Mar 2025
Med-LEGO: Editing and Adapting toward Generalist Medical Image Diagnosis Yitao Zhu Yuan Yin Jiaming Li Mengjie Xu Zihao Zhao Honglin Xiong Sheng Wang Qian Wang MedIm 75 0 0 03 Mar 2025
LoR2C : Low-Rank Residual Connection Adaptation for Parameter-Efficient Fine-Tuning Jiancheng Zhao Xingda Yu Yuxiang Zhang Zhen Yang OffRL 45 0 0 01 Mar 2025
HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture Taiqiang Wu Chenchen Ding Wenyong Zhou Yuxin Cheng Xincheng Feng Shuqi Wang Chufan Shi Zhengwu Liu Ngai Wong 58 0 0 27 Feb 2025
GeoEdit: Geometric Knowledge Editing for Large Language Models Yujie Feng Liming Zhan Zexin Lu Yongxin Xu Xu Chu Yasha Wang Jiannong Cao Philip S. Yu Xiao-Ming Wu KELM 53 0 0 27 Feb 2025
Predicting Through Generation: Why Generation Is Better for Prediction Md. Kowsher Nusrat Jahan Prottasha Prakash Bhat Chun-Nam Yu Mojtaba Soltanalian Ivan Garibay O. Garibay Chen Chen Niloofar Yousefi AI4TS 65 0 0 25 Feb 2025
SECURA: Sigmoid-Enhanced CUR Decomposition with Uninterrupted Retention and Low-Rank Adaptation in Large Language Models Yuxuan Zhang CLL ALM 73 1 0 25 Feb 2025
Pastiche Novel Generation Creating: Fan Fiction You Love in Your Favorite Author's Style Xueran Han Yuhan Liu Mingzhe Li Wei Liu Sen Hu Rui Yan Zhiqiang Xu Preslav Nakov 69 0 0 24 Feb 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 85 2 0 24 Feb 2025
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning Raghav Singhal Kaustubh Ponkshe Rohit Vartak Lav R. Varshney Praneeth Vepakomma FedML 79 1 0 24 Feb 2025
R-LoRA: Random Initialization of Multi-Head LoRA for Multi-Task Learning Jinda Liu Yi-Ju Chang Yuan Wu 59 0 0 24 Feb 2025
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning Yujie Feng Xujia Wang Zexin Lu Shenghong Fu Guangyuan Shi Yongxin Xu Yasha Wang Philip S. Yu Xu Chu Xiao-Ming Wu CLL KELM 41 1 0 22 Feb 2025
Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis Xuben Wang Yan Hu Wenyu Du Reynold Cheng Benyou Wang Difan Zou 61 1 0 17 Feb 2025
Superpose Singular Features for Model Merging Haiquan Qiu You Wu Quanming Yao MoMe 48 0 0 15 Feb 2025
HADL Framework for Noise Resilient Long-Term Time Series Forecasting Aditya Dey Jonas Kusch Fadi Al Machot AI4TS 41 0 0 14 Feb 2025
LoCA: Location-Aware Cosine Adaptation for Parameter-Efficient Fine-Tuning Zhekai Du Yinjie Min Jingjing Li Ke Lu Changliang Zou Liuhua Peng Tingjin Chu Mingming Gong 196 1 0 05 Feb 2025
RandLoRA: Full-rank parameter-efficient fine-tuning of large models Paul Albert Frederic Z. Zhang Hemanth Saratchandran Cristian Rodriguez-Opazo Anton van den Hengel Ehsan Abbasnejad 111 2 0 03 Feb 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 50 1 0 28 Jan 2025
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models Xiaolin Hu Xiang Cheng Peiyu Liu Wei Liu Jian Luan Bin Wang Y. Liu 47 1 0 31 Dec 2024
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning Yehonathan Refael Jonathan Svirsky Boris Shustin Wasim Huleihel Ofir Lindenbaum 47 3 0 31 Dec 2024
FineGates: LLMs Finetuning with Compression using Stochastic Gates Jonathan Svirsky Yehonathan Refael Ofir Lindenbaum 77 0 0 17 Dec 2024
Adaptive Rank, Reduced Forgetting: Knowledge Retention in Continual Learning Vision-Language Models with Dynamic Rank-Selective LoRA Haodong Lu Chongyang Zhao Jason Xue Lina Yao Kristen Moore Dong Gong VLM KELM CLL 90 4 0 01 Dec 2024
COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection Jinqi Xiao S. Sang Tiancheng Zhi Jing Liu Qing Yan Linjie Luo Bo Yuan Bo Yuan VLM 103 1 0 26 Nov 2024
AutoMixQ: Self-Adjusting Quantization for High Performance Memory-Efficient Fine-Tuning Changhai Zhou Shiyang Zhang Yuhua Zhou Zekai Liu Shichao Weng MQ 66 0 0 21 Nov 2024
StreamAdapter: Efficient Test Time Adaptation from Contextual Streams Dilxat Muhtar Yelong Shen Yuqing Yang Xiaodong Liu Yadong Lu ... Feng Sun Xueliang Zhang Jianfeng Gao Weizhu Chen Qi Zhang TTA 67 0 0 14 Nov 2024
Towards More Accurate Fake Detection on Images Generated from Advanced Generative and Neural Rendering Models Chengdong Dong Vijayakumar Bhagavatula Zhenyu Zhou Ajay Kumar 36 0 0 13 Nov 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 49 3 0 24 Oct 2024
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 58 2 0 23 Oct 2024