Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models

16 August 2024

Papers citing "Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models"

26 / 26 papers shown

Title
Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models Andy Zhou MoMe 120 0 0 13 Mar 2025
Generate to Discriminate: Expert Routing for Continual Learning Yewon Byun Sanket Vaibhav Mehta Saurabh Garg Emma Strubell Michael Oberst Bryan Wilder Zachary Chase Lipton 131 0 0 31 Dec 2024
Rehearsal-Free Modular and Compositional Continual Learning for Language Models Mingyang Wang Heike Adel Lukas Lange Jannik Strötgen Hinrich Schütze KELM CLL 63 15 0 31 Mar 2024
Continual Learning with Pre-Trained Models: A Survey Da-Wei Zhou Hai-Long Sun Jingyi Ning Han-Jia Ye De-Chuan Zhan CLL KELM 75 74 0 29 Jan 2024
Continual Learning: Applications and the Road Forward Eli Verwimp Rahaf Aljundi Shai Ben-David Matthias Bethge Andrea Cossu ... Joost van de Weijer Bing Liu Vincenzo Lomonaco Tinne Tuytelaars Gido M. van de Ven CLL 77 47 0 20 Nov 2023
TIES-Merging: Resolving Interference When Merging Models Prateek Yadav Derek Tam Leshem Choshen Colin Raffel Joey Tianyi Zhou MoMe 105 293 0 02 Jun 2023
AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages Shamsuddeen Hassan Muhammad Idris Abdulmumin Abinew Ali Ayele N. Ousidhoum David Ifeoluwa Adelani ... Hailu Beshada Balcha S. Chala Hagos Tesfahun Gebremichael Bernard Opoku Steven Arthur 56 89 0 17 Feb 2023
Progressive Prompts: Continual Learning for Language Models Anastasia Razdaibiedina Yuning Mao Rui Hou Madian Khabsa M. Lewis Amjad Almahairi VLM KELM CLL 92 135 0 29 Jan 2023
CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning James Smith Leonid Karlinsky V. Gutta Paola Cascante-Bonilla Donghyun Kim Assaf Arbelle Yikang Shen Rogerio Feris Z. Kira CLL VPVLM VLM 74 283 0 23 Nov 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 71 132 0 31 Oct 2022
How Relevant is Selective Memory Population in Lifelong Language Learning? Vladimir Araujo Helena Balabin J. Hurtado Alvaro Soto Marie-Francine Moens CLL KELM 91 7 0 03 Oct 2022
Memory Population in Continual Learning via Outlier Elimination J. Hurtado Alain Raymond-Sáez Vladimir Araujo Vincenzo Lomonaco Alvaro Soto D. Bacciu 50 9 0 04 Jul 2022
Entropy-based Stability-Plasticity for Lifelong Learning Vladimir Araujo J. Hurtado Alvaro Soto Marie-Francine Moens CLL 57 15 0 18 Apr 2022
Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning Jesujoba Oluwadara Alabi David Ifeoluwa Adelani Marius Mosbach Dietrich Klakow 64 152 0 13 Apr 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 116 976 1 10 Mar 2022
Learning to Prompt for Continual Learning Zifeng Wang Zizhao Zhang Chen-Yu Lee Han Zhang Ruoxi Sun Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VPVLM KELM VLM 84 769 0 16 Dec 2021
Selective Replay Enhances Learning in Online Continual Analogical Reasoning Tyler L. Hayes Christopher Kanan CLL 35 20 0 06 Mar 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 213 4,238 0 01 Jan 2021
Efficient Meta Lifelong-Learning with Limited Memory Zirui Wang Sanket Vaibhav Mehta Barnabás Póczós J. Carbonell CLL KELM 62 76 0 06 Oct 2020
Episodic Memory in Lifelong Language Learning Cyprien de Masson dÁutume Sebastian Ruder Lingpeng Kong Dani Yogatama CLL KELM 126 285 0 03 Jun 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.4K 94,511 0 11 Oct 2018
Memory-based Parameter Adaptation Pablo Sprechmann Siddhant M. Jayakumar Jack W. Rae Alexander Pritzel Adria Puigdomenech Badia Benigno Uria Oriol Vinyals Demis Hassabis Razvan Pascanu Charles Blundell ODL OOD VLM 65 101 0 28 Feb 2018
HDLTex: Hierarchical Deep Learning for Text Classification Kamran Kowsari Donald E. Brown Mojtaba Heidarysafa K. Meimandi M. Gerber Laura E. Barnes VLM 42 411 0 24 Sep 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 221 2,630 0 23 Jan 2017
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables Chris J. Maddison A. Mnih Yee Whye Teh BDL 153 2,529 0 02 Nov 2016
Character-level Convolutional Networks for Text Classification Xiang Zhang Jiaqi Zhao Yann LeCun 240 6,100 0 04 Sep 2015