Multi-Head Adapter Routing for Cross-Task Generalization

7 November 2022

Nicolas Le Roux

Papers citing "Multi-Head Adapter Routing for Cross-Task Generalization"

21 / 21 papers shown

Title
GenKnowSub: Improving Modularity and Reusability of LLMs through General Knowledge Subtraction Mohammadtaha Bagherifard Sahar Rajabi Ali Edalat Yadollah Yaghoobzadeh KELM 29 0 0 16 May 2025
Generative Modeling of Individual Behavior at Scale Nabil Omi Lucas Caccia Anurag Sarkar Jordan T. Ash S. Sen 76 0 0 24 Feb 2025
One Arrow, Many Targets: Probing LLMs for Multi-Attribute Controllable Text Summarization Tathagato Roy Rahul Mishra 16 0 0 02 Nov 2024
AdaptGCD: Multi-Expert Adapter Tuning for Generalized Category Discovery Yuxun Qu Yongqiang Tang Chenyang Zhang Wensheng Zhang 31 0 0 29 Oct 2024
Glider: Global and Local Instruction-Driven Expert Router Pingzhi Li Prateek Yadav Jaehong Yoon Jie Peng Yi-Lin Sung Joey Tianyi Zhou Tianlong Chen MoMe MoE 33 1 0 09 Oct 2024
Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models Yuncheng Yang Yulei Qin Tong Wu Zihan Xu Gang Li ... Yuchen Shi Ke Li Xing Sun Jie Yang Yun Gu ALM OffRL MoE 54 0 0 28 Aug 2024
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks Yang Li Shaobo Han Shihao Ji MoE 27 11 0 24 May 2024
Simple Drop-in LoRA Conditioning on Attention Layers Will Improve Your Diffusion Model Joo Young Choi Jaesung R. Park Inkyu Park Jaewoong Cho Albert No Ernest K. Ryu AI4CE 35 4 0 07 May 2024
Multi-Head Mixture-of-Experts Xun Wu Shaohan Huang Wenhui Wang Furu Wei MoE 34 12 0 23 Apr 2024
Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet? E. Razumovskaia Ivan Vulić Anna Korhonen 46 6 0 04 Mar 2024
Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy? Nader Asadi Mahdi Beitollahi Yasser H. Khalil Yinchuan Li Guojun Zhang Xi Chen MoMe 43 8 0 23 Feb 2024
Learning to Route Among Specialized Experts for Zero-Shot Generalization Mohammed Muqeeth Haokun Liu Yufan Liu Colin Raffel MoMe 37 34 0 08 Feb 2024
OrchMoE: Efficient Multi-Adapter Learning with Task-Skill Synergy Haowen Wang Tao Sun Kaixiang Ji Jian Wang Cong Fan Jinjie Gu 18 1 0 19 Jan 2024
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 32 13 0 22 Nov 2023
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 26 14 0 15 Nov 2023
Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning Ted Zadouri Ahmet Üstün Arash Ahmadian Beyza Ermics Acyr Locatelli Sara Hooker MoE 35 88 0 11 Sep 2023
Subspace-Configurable Networks Dong Wang O. Saukh Xiaoxi He Lothar Thiele OOD 35 0 0 22 May 2023
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 130 100 0 24 May 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Cer VLM LRM 137 277 0 15 Oct 2021
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP Qinyuan Ye Bill Yuchen Lin Xiang Ren 214 180 0 18 Apr 2021