Asymmetry in Low-Rank Adapters of Foundation Models

v1v2 (latest)

Asymmetry in Low-Rank Adapters of Foundation Models

26 February 2024

Kristjan Greenewald

Haitz Sáez de Ocáriz Borde

Rickard Brüel-Gabrielsson

Marzyeh Ghassemi

Mikhail Yurochkin

ArXiv (abs)PDF HTML

Papers citing "Asymmetry in Low-Rank Adapters of Foundation Models"

15 / 15 papers shown

Title
Noise Consistency Regularization for Improved Subject-Driven Image Synthesis Yao Ni Song Wen Piotr Koniusz A. Cherian 25 0 0 06 Jun 2025
PoLAR: Polar-Decomposed Low-Rank Adapter Representation Kai Lion Liang Zhang Bingcong Li Niao He 62 0 0 03 Jun 2025
CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning Jiangpeng He Zhihao Duan Fengqing M Zhu CLL 31 1 0 30 May 2025
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models Wenjin Ke Zhe Li D. Li Lu Tian E. Barsoum MQ 103 3 0 12 Apr 2025
DeLoRA: Decoupling Angles and Strength in Low-rank Adaptation Massimo Bini Leander Girrbach Zeynep Akata 228 1 0 23 Mar 2025
Quantum-PEFT: Ultra parameter-efficient fine-tuning Toshiaki Koike-Akino F. Tonin Yongtao Wu Frank Zhengqing Wu Leyla Naz Candogan Volkan Cevher MQ 227 5 0 07 Mar 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 230 4 0 24 Feb 2025
Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA Shuangyi Chen Yuanxin Guo Yue Ju Harik Dalal Ashish Khisti 115 2 0 03 Feb 2025
Decentralized Low-Rank Fine-Tuning of Large Language Models Sajjad Ghiasvand Mahnoosh Alizadeh Ramtin Pedarsani ALM 154 2 0 26 Jan 2025
LoRA vs Full Fine-tuning: An Illusion of Equivalence Reece Shuttleworth Jacob Andreas Antonio Torralba Pratyusha Sharma 150 19 0 28 Oct 2024
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization Xinhao Yao Hongjin Qian Xiaolin Hu Gengze Xu Wei Liu Jian Luan Bin Wang Yang Liu 124 1 0 03 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 155 22 0 02 Oct 2024
Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead Rickard Brüel-Gabrielsson Jiacheng Zhu Onkar Bhardwaj Leshem Choshen Kristjan Greenewald Mikhail Yurochkin Justin Solomon 172 9 0 17 Jun 2024
Improving LoRA in Privacy-preserving Federated Learning Youbang Sun Zitao Li Yaliang Li Bolin Ding 91 81 0 18 Mar 2024
SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules Xiangyu Chen Jing Liu Ye Wang Pu Wang Matthew Brand Guanghui Wang T. Koike-Akino 104 7 0 18 Mar 2024

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.