LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation

10 April 2025

Papers citing "LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation"

21 / 21 papers shown

Title
FISH-Tuning: Enhancing PEFT Methods with Fisher Information Kang Xue Ming Dong Xinhui Tu Tingting He 166 0 0 05 Apr 2025
HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning Chunlin Tian Zhanying Shi Zhijiang Guo Li Li Chengzhong Xu 63 43 0 30 Apr 2024
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 73 53 0 20 Nov 2023
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning Yun Luo Zhen Yang Fandong Meng Yafu Li Jie Zhou Yue Zhang CLL KELM 154 305 0 17 Aug 2023
LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition Chengsong Huang Qian Liu Bill Yuchen Lin Tianyu Pang Chao Du Min Lin MoMe 90 210 0 25 Jul 2023
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models Zhiqiang Hu Lei Wang Yihuai Lan Wanyu Xu Ee-Peng Lim Lidong Bing Xing Xu Soujanya Poria Roy Ka-wei Lee ALM 92 260 0 04 Apr 2023
Editing Models with Task Arithmetic Gabriel Ilharco Marco Tulio Ribeiro Mitchell Wortsman Suchin Gururangan Ludwig Schmidt Hannaneh Hajishirzi Ali Farhadi KELM MoMe MU 175 487 0 08 Dec 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 457 6,222 0 05 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 837 12,893 0 04 Mar 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 241 4,392 0 27 Oct 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 217 5,513 0 07 Jul 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 388 10,301 0 17 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 533 4,032 0 18 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 219 4,247 0 01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Armen Aghajanyan Luke Zettlemoyer Sonal Gupta 90 562 1 22 Dec 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 739 41,894 0 28 May 2020
AdapterFusion: Non-Destructive Task Composition for Transfer Learning Jonas Pfeiffer Aishwarya Kamath Andreas Rucklé Kyunghyun Cho Iryna Gurevych CLL MoMe 129 848 0 01 May 2020
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 215 1,516 0 24 May 2019
Experience Replay for Continual Learning David Rolnick Arun Ahuja Jonathan Richard Schwarz Timothy Lillicrap Greg Wayne CLL 114 1,158 0 28 Nov 2018
Gradient Episodic Memory for Continual Learning David Lopez-Paz MarcÁurelio Ranzato VLM CLL 117 2,711 0 26 Jun 2017
Learning without Forgetting Zhizhong Li Derek Hoiem CLL OOD SSL 288 4,400 0 29 Jun 2016